Horse Racing and the CIA

John Mauldin | Thoughts from the Frontline
October 1, 2004

This week we explore in depth an essay on how we think from that bastion of investment analysis, the US Central Intelligence Agency. We look at the rookie mistakes made even by pros. I describe one of my own more embarrassing rookie moments and how it highlights an investment principle that the best performing professionals always keep in mind. I finish with a note on what you can give me (and your friends) for my birthday. It should make for a lively letter.

One of the things I try to get my kids to notice is how professional athletes all have a routine when they perform. Watch a golf pro or a baseball player. Before they get ready to hit, they have a routine, which is always the same. They have narrowed their thoughts down to a few simple steps.

Your Pre-Game (Investment) Routine

What happens to a golfer if he is thinking about his next meeting when he is in the middle of his backswing? It's hard enough to hit that small ball straight when you are totally focused. Getting distracted when a 98 mile per hour baseball is coming at you won't help your batting average. Thinking about something stressful when you need to be focused is a prescription for problems.

The pre-swing (or pre-race or pre-whatever) routine is crucial to solid performance. And this week, that fact was brought out to me in spades.

I have probably done at least 100 interviews on radio and TV this year, as well as lots of speeches. Not really realizing it, I have developed a routine over the years that serves me well. While not really all that glib in person (I think I do much better in print where I can edit my thoughts), I do seem to be able to not embarrass myself. Except last Wednesday.

The producer for the early morning CNBC Squawk Box called me on Monday and asked me to be on their show Wednesday morning. The topic was to be what the bond market may be telling us about the economy and Fed policy. Pretty standard stuff, and long time readers know that I have a few opinions. This is a topic with which I am pretty familiar. Normally, I would sit down about 30 minutes before the interview, review my thoughts and research and then step up and swing away.(Assuming, of course, that I have a more than reasonable familiarity with the subject, although that has not always kept me from having an opinion.)

However, I varied my pre-interview routine. I must confess that I have watched Mark Haines ask a tough question or two from time to time, so I decided to spend a little extra time the day before re-reading a lot of papers and looking at data. A little time turned into four hours. As we will see in a few pages, the CIA paper offers an opinion on what happens when we get too much information. That is the paper I should have been reading! I will save that insight for later, but for now let's go on to how I compounded that problem with a second (and what turned out to be far more serious) break in my pre-interview routine.

I enjoy coffee every morning, but rarely drink coffee with any caffeine in it. Over a period of a week or so, my body reacts quite negatively to too much caffeine. Half a cup here or there is no problem, but I normally stick to de-caf.

Having been out of town all last week, I worked quite late Tuesday night and had to get up quite early to drive across town to the studio. You guessed it, I decided to have a little caffeine: two mugs worth, thank you. By 8 am my time, I was feeling quite awake.

We went to the studio room, which is a small, dark place. No monitors, just bright lights glaring at you. The closest analogy is to the rooms you se on TV where they interview suspects. You look into a dark place between the lights (you can't even see the camera) and talk into it like the other party is sitting there. You have an earpiece with a dis-embodied voice and away you go.

I was ready. I had my opening down and then we would go to questions, which is probably what I do best. Then it hit. About two minutes before the show, I got the mother of all caffeine rushes. Literally, my hand started shaking. My jaws started to feel funny. I was thinking that someone had put a controlled substance into my coffee. I started having multiple swing thoughts. I took my eye off the ball.

At that moment, Mark came into my ear and asked me the first question. I forgot all but my first planned sentence. After that, rather than having a few main points at the top of my head which I could confidently tick off, every bit of research from the previous day all rushed to the front of my mind, competing to get out. Could I sort through them quickly and recover. Heck, no. I was worried they could see my hand shake. Did I want to look like some nervous rookie? I stuttered, paused and dropped a point or two. I in fact did get nervous. After a minute or two, Mark finally helped me with a softball question and I was kind of able to get back on track. Not my finest interview, although I have been assured it was not "all" that bad. However, no one used the word good, either. Oh, well. (I had a 30 minute radio interview two hours later and I could not sit still, walking around the whole interview.)

Your Most Embarrassing Investment Moment

Successful investors have a pre-investment routine before they "hit" an investment. Thorough research, in-depth due diligence and thoughtful analysis about how an investment fits into their overall portfolio is all part of it. In my experience, there is a strong correlation between the amount of work and research done before an investment is made and the success of the investor. Good research does not guarantee the performance of any particular investment, but it does help you avoid more of the bad ones. This goes for whether the time horizon for the investment is 30 minutes or 30 years.

Successful investors even develop contingency plans for what to do when the special investment or situation comes along that requires a little quicker action. The goal is to not let your emotions rule your investment routine.

It is at precisely such moments when the excitement of the opportunity, the thrill of the big score, comes our way and our emotions are running ahead of our thought process that we need to fall back on our pre-investment routine.

Maybe that emotional lunge will result in a ball straight down the middle. Maybe. However, my most embarrassing investment moments, the ones talked about late at night by my fellow professional, and I bet yours, too, are when we make that knee-jerk emotional snap judgment. We look back and say, "If I had stuck to my pre-investment routine, I would have found the flaw before I lost my money."

You don't want to end up in a small room with Mark Haines talking into your ear. Develop your routine and stick to it.

Do You Really Need More Information?

Jim Williams, the founder of the Williams Inference Center (www.williamsinference.com) recently sent me a pile of fascinating research which I am wading through. One of the first articles I read was an essay by Richards J. Heuer, Jr. entitled "Do You Really Need More Information?" It was published in a book called "Inside CIA's Private World: Declassified Articles from the Agency's Internal Journal 1955-1992."

Buried among articles on how (and how not to!) to spy is this rather straightforward piece on what to do with the information you get and the problems with objective and accurate analysis that are caused by our human thought process. The essay is quite timely, even though it was written in the spring of 1979. While reading the critique, one could not help but wish that it would be required reading at the CIA today. Perhaps we could have avoided a few problems. But that is a topic for someone else. Our beat today is thinking about money.

(All quotes are from the article. If you are interested in the complete article, you can go to http://www.cia.gov/csi/books/19104/ and click on chapter 5.)

I am guilty. Mea Culpa. I am constantly researching, looking at (sometimes obscure) data, trying to discern patters and trends. But what to do with all of it? How do we filter it into useful and investable ideas?

"This article challenges the often implicit assumption that lack of information is the principal obstacle to accurate intelligence estimates....Once an experienced analyst has the minimum information necessary to make an informed judgment, obtaining additional information generally does not improve the accuracy of his estimates. Additional information does, however, lead the analyst to become more confident in his judgment, to the point of overconfidence."

Horse Racing and the CIA

Heuer describes a study done about betting on horse races. They took 8 professional handicappers (someone who sets the betting odds based on calculations of the outcome of a contest, especially a horse race) and asked them to rank 80 different pieces of data about a horse race as to what they thought was most important. Do you factor in the jockey's record as well as the recent record of the horse? The weather? The competition? How much weight is the horse carrying? What is the length of the race? There are scores of variables.

Then the handicappers were given what they felt was the five most important pieces of data and asked to project the winners for a race (actual names and races were not given, so as to not bias the projections). They were also asked to rank their confidence about their predictions.

Now it gets interesting. They were then given 10, 20 and 40 pieces of what they individually considered to be the most important information. Three of the handicappers actually showed less accuracy as the amount of information increased, two improved their accuracy, and three were unchanged. But as a group, their accuracy did not improve and in fact was slightly down.

But with each increase in information, their confidence went up. In fact, by the end, their confidence has in fact doubled. If they had actually been at the track and betting, would they have doubled their bets as they became more confident? Human nature says "yes, they would." But that confidence would not have made them any better predictors. They just doubled their bets which magnified their gains or losses. Think of it like adding leverage to your stock portfolio.

"A series of experiments to examine the mental processes of medical doctors diagnosing illness found little relationship between thoroughness of data collection and accuracy of diagnosis." Another study was done with psychologists and patient information and diagnosis. Again, increasing knowledge yielded no better results but significantly increased confidence.

The inference is clear and quite important: "Experienced analysts have an imperfect understanding of what information they actually use in making judgments. They are unaware of the extent to which their judgments are determined by a few dominant factors, rather than by the systematic integration of all available information. Analysts use much less available information than they think they do."

How can this be? Heuer notes that individuals tend to: "...overestimates the importance he attributes to factors that have only a minor impact on his judgment, and underestimates the extent to which his decisions are based on a very few major variables... Possibly our feeling that we can take into account a host of different factors comes about because although we remember that at some time or other we have attended to each of the different factors, we fail to notice that it is seldom more than one or two that we consider at any one time."

As I wrote in Bull's Eye Investing, "The two most common biases are overoptimism and overconfidence . For instance, when teachers ask a class who will finish in the top half, on average around 80 percent of the class think they will! Not only are people overly optimistic, but they are overconfident as well. People are surprised more often than they expect to be. For instance, when you ask people to make a forecast of an event or a situation, and to establish at what point they are 98 percent confident about their predictions, we find that the correctness of their predictions ranges between 60 and 70 percent! What happens when we are only 75 percent sure or are playing that 50-50 hunch?"

Let's take one quick example where we think "knowledge" makes for a better investment. Again from my book:

"First, a researcher takes a deck of 52 cards and holds one card up. Watchers pay a dollar for the chance to win $100 if that card is picked out of the deck. Keep in mind the expected payout is 1/52 - 100 = 1.92. Las Vegas would quickly go broke with such odds. Then they are asked if they would like to sell their chances: roughly 80 percent would sell if they could, asking for an average price of $1.86. If you could get such a price, it would be a reasonable sell. For someone who could buy all 52 chances, it would be a good purchase or arbitrage. He would make a quick 3.18 percent.

"Now it gets interesting. The next time, someone is allowed to pick a card out of the deck and offered the same chance, but now he has a personal attachment to the card because he touched it. Only about 60 percent of those who picked cards were willing to sell their chances, and they wanted an average price of just over $6. And when this same trick was performed at MBA schools the average sale price has been over $9.

"I know this card. I have studied it. I have a personal involvement with the card; therefore it is worth more," thinks the investor. Of course, it is worth no more than in the first case, but the psychology of "owning" the card makes investors value it more."

Is it just MBA students? Studies show that institutional investors exhibit quite significant biases about industries they know, as well as thinking that their own home country stock market will outperform the stock market of another country.

Knowledge makes us confident. And the more knowledge we have, evidently the more confident we become, even though our accuracy may not be enhanced.

"How" Heuer asks, " Can this happen to smart people like us?"

There are four types of additional information that an analysts might receive.

Additional detail about variables already in our analysis. Information on additional variables. "Occasionally, in situations where there are known gaps in our understanding, a single report concerning some new and previously unconsidered factor will have a major impact on our judgments. Such a report would fall into either of the next two categories of new information: Information concerning the level of value attributed to variables already included in the analysis, and... Information concerning which variables are most important and how they relate to each other.

And that brings us to the heart of the matter: a discussion of the types of new information which is the basis for distinguishing two types of analysis: data-driven analysis and conceptually-driven analysis.

In data driven analysis, "...accuracy depends primarily upon the accuracy and completeness of the available data. If one makes the reasonable assumption that the analytical model is correct and the further assumption that the analyst properly applies this model to the data, then the accuracy of the analytical judgment depends entirely upon the accuracy and completeness of the data."

On the other hand, "Conceptually driven analysis is at the opposite end of the spectrum from data-driven analysis. The questions to be answered do not have neat boundaries, and there are many unknowns. The number of potentially relevant variables and the diverse and imperfectly understood relationships among these variables involve the analyst in enormous complexity and uncertainty. There is little tested theory to inform the analyst concerning which of the myriad pieces of information are most important and how they should be combined to arrive at probabilistic judgments.

"In the absence of any agreed-upon analytical schema, analysts are left to their own devices. They interpret information with the aid of mental models that are largely implicit rather than explicit. Assumptions concerning political forces and processes in the subject country may not be apparent even to the analyst. Such models are not representative of an analytical consensus. Other analysts examining the same data may well reach different conclusions, or reach the same conclusions but for different reasons. This analysis is conceptually driven, because the outcome depends at least as much upon the conceptual framework employed to analyze the data as it does upon the data itself."

There are some investment decisions that are data driven. Having the right data, the right research is key.

But more often than not, what we think of as data "proofs" for a certain viewpoint depend more upon our perceptions than the actual data. What happens is that we develop mental models. We take in information and process it according to our mental models, which are subject to our personal biases. The accuracy of any particular judgment depends almost exclusively upon the accuracy of your mental model, because there is virtually no other basis for our judgment. If the accuracy of our mental model is the key to accurate judgment, it is necessary to consider how this mental model gets tested against reality and how it can be changed so that we can improve the accuracy of our judgment.

How can we deal with this bias?

"Information that is consistent with our existing mindset is perceived and processed easily. However, since our mind strives instinctively for consistency, information that is inconsistent with our existing mental image tends to be overlooked, perceived in a distorted manner, or rationalized to fit existing assumptions and beliefs. Thus, new information tends to be perceived and interpreted in a way that reinforces existing beliefs.

"...If we are to penetrate to the heart and soul of the problem of improving analysis, we must somehow penetrate and affect the mental processes of the individuals who do the analysis. This strategy is to focus on improving the mental models employed by the analyst to interpret his data.

"...All involve confronting the analyst with alternative ways of thinking. The objective is to identify the most fundamental analytical assumptions, then to make these assumptions explicit so that they may be critiqued and re-evaluated.

AND THIS IS KEY: "...The analyst should then try to disprove, rather than prove, each of the alternatives. He or she should try to rebut rather than confirm hypotheses...It is especially important for the analyst to seek information that, if found, would disprove rather than bolster his own arguments. One key to identifying the kinds of information that are potentially most valuable is for the analyst to ask himself what it is that could make him change his mind. Adoption of this simple tactic would do much to avoid intelligence surprises."

Let's take a few real world examples. I am, and I presume many of my readers are, bearish on then dollar. We read Stephen Roach and Bill Gross and find comforting confirmation for our views. "Look at the trade deficit," we tell ourselves, "the over-indebtedness of the US consumer and government and on and on." Two years ago I wrote about the Fed study which shows that currencies drop about 30% on average when trade deficits get to 5%. We are now at 5.7% and the dollar is down a mere 12% or so on a trade weighted basis. What happened? Why is the deficit up?

Because it is not cut and dried. There are all sorts of factors at play. If there were not the dollar would already be at 1.5 to the euro. What is running counter to the theory? China and Japan, along with the rest of Asia, are buying our debt. Andy Kessler's book Running Money suggests it might be because the margin surplus we have in the states is more important than old economy models of trade deficits has a point. Many of you dismissed the idea because it is so clearly contrary to Monetary and Austrian models. But he makes a point in that global trade and markets are not like they were 50 years ago. Does he persuade me to think the dollar is going up? No, but it may partly explain why it is not in free fall.

Further, old economic models all suggest that if the dollar drops then our trade deficit will improve. In the long run, that is true. But it might not be in the short run.

Let's say China does let the Yuan float and the competitive currency devaluation practiced throughout Asia begins to wane, as they all let their currencies rise. Does that mean that our products will start to sell better? Perhaps not.

Think about a foreign consumer. China floats the Yuan. The dollar starts to drop. Wouldn't he be better off to wait another 6 months for the dollar to drop another 10% or 20% before he buys our products? You could actually see the deficit rise as foreigners hold off their purchases.

In some ways, that is precisely what we have already seen. Where is the rush from Europe to "buy American" when the euro is up almost 50%? There are other reasons put forth as to why the dollar might strengthen.

If we are going to make investment decisions, we MUST test our assumptions. But right now, we are talking about our personal thought models. We will cover the dollar at some later point.

(This week's Outside the Box, which you will get Monday evening, has a fascinating exchange of mail between Gross and Roach about deficits and the dollar.)

Who do we share our opinions and views with to "test" them? Most likely, it is people who are close to our fundamental mindset. But these are the individuals least likely to actually bring up a different viewpoint.

I force myself to read people with which I know I will not agree, and I work hard to keep an open mind. Perhaps that is why I am so "Muddle Through." When I do get a conviction, it may be wrong, but I have hopefully thought about it. And you should do the same.

Repeating a key point above, "...The analyst should then try to disprove, rather than prove, each of the alternatives. He or she should try to rebut rather than confirm hypotheses...It is especially important for the analyst to seek information that, if found, would disprove rather than bolster his own arguments."

What happens is that we create a mental picture of what the world looks like. You can call it a world view. Heuer suggests it is like a mosaic. All the pieces of data combine to make a picture or a mosaic which we can use to get an insight. Except that:

"Such insights suggest that the picture formed by the so-called mosaic is not a picture of reality, but only our self-constructed mental image of a reality we can never perceive directly. We form the picture first and only then do we fit in the pieces. Accurate estimates depend at least as much upon the mental model we use in forming that picture as upon the accuracy and completeness of the information itself."

If we believe there are weapons of mass destruction, then the data points that way. If we believe the dollar is going down, the markets up, gold up or down or whatever, then that is what the data says.

Again, as we close, take this point away and ponder how it applies not only to your investment thinking, but to every part of your world view:

"Information that is consistent with our existing mindset is perceived and processed easily. However, since our mind strives instinctively for consistency, information that is inconsistent with our existing mental image tends to be overlooked, perceived in a distorted manner, or rationalized to fit existing assumptions and beliefs. Thus, new information tends to be perceived and interpreted in a way that reinforces existing beliefs.

I can't tell you how many times I have traveled overseas and someone says after getting to know me, "You're not what I thought someone from Texas (white male, conservative, Republican, etc.) would be like." But in some places it takes some time. Those existing mindsets can be very tough to change.

Birthdays, Houston and Other Distractions

On Monday I hit the big Double Nickel. For some reason, (probably connected to all the human psychological biases, especially the over-optimism and denial I write about) I have never felt like I was getting all that old. Comfortably middle-aged, maybe.

But 55 is no longer in the middle. It is on the other side of middle and this birthday is giving me pause. But not for very long, I hope. There are so many good things happening, life is too much fun (at least as long as I don't think about -gasp- being 55) and all seven of the kids are getting together in an hour. Heck, I might just decide to do another 55 years.

My daughter Tiffani (who works with me) decided that I should point out since this letter is free, I should suggest an appropriate birthday present.

"Why don't you tell them to get a copy of Bull's Eye Investing and give it to friend?" (You have bought your own copy, haven't you?) "Or tell them to click on http://www.2000wave.com/tellfriend.asp with a note to make sure they tell why they love it so much...." Assuming of course, that you do.

You can read about what some call "The Most Important Book of 2004" and why you should get Bull's Eye Investing by going to www.absolutereturns.net or buy it a 32% discount from www.amazon.com/bullseye.

I will be in Houston next Saturday speaking at 11 AM to the Houston Investors Association at Melcher Hall on the University of Houston Campus. You can click on http://houstoninvestors.com/Location.htm to get directions.

On Friday night I will be doing a book signing at 6:30 to around 8 at Murder by the Book (the name of the bookstore) at 2432 Bissonnet Street (713-524-8597). The store is one of the nation's oldest and largest mystery specialty books stores. I have pondered the relationship between murder, mystery, secular bear market and investments, and hope to have a connection by next week. Drop by and help me solve the puzzle if you are in the area. (www.murderbooks.com)

Sadly, the youthful Rangers simply could not run the table. Anaheim over-powered us, and we must now wait till next year. But two years ago we ended the season 45 games out, and to be "in the hunt" in the last week is a much better feeling.

Next year will be the year for the Rangers. It will also be the year I get my weight down to 185 while bench pressing 185. And I get control of my travel, work and writing schedule. Hey, it could happen.

Your sometimes it's good to be blindly optimistic analyst,

Scroll down for comments.