Why the average investor is bamboozled by recession forecasts

I have a problem.  I cannot teach my research methods course in one short article - -not even the undergraduate version.

Since the topic is so important I want to do it right by conveying the most important features.  While readers of this article may not be able to write a critique of various research findings, you might develop a strong instinct for what is right.

Background

I have served as a referee for scholarly journals, as a methods prof at top schools, and most significantly, as a gatekeeper judging investment research.  You might be surprised to learn that much of the work at big-time firms would not pass muster by these standards.  The criteria for marketing and sales are different from peer review and actually putting your money on the line.

How to spot suspicious results

An expert can spot bogus results.  Suppose you assign a group of students to flip a coin 200 times and write down the results.  Some of them actually do it, while others fake it.  I could spot the fakes in a heartbeat, and so could you with a little practice.  Most people do not understand what random data looks like, so they do not know how to fake it.  They do not understand that "streaks" happen.  By the way, the IRS knows -- no problem for us honest folks!  Read here to learn more.

In the investment world, this is common with trading systems.  As the resident prof and gatekeeper for my trading firm in the late '80s, we got to see plenty of trading systems.  By way of background, the 1987 crash was preceded by extremely oversold conditions, a situation that many thought was right to buy.  Put premium was at unprecedented levels, and put selling was prevalent.  Even the strongest and most experienced traders sold a few puts into that fateful Friday close before the crash.  On Monday,  October 19th, 1987 there was no escape from the worst day in market history.

Very smart, very successful traders had spotted a buy signal that had been profitable for many years.  When people came around to sell trading systems, how many of them recommended a "buy" on that day?  If you are guessing "ZERO" then you are correct.  Even though the "syndrome" of successful indicators would probably resume working for another 20 years, no system developer would admit it.  It was almost as if they had a specific binary variable -- exclude any signal on a Friday in October of odd numbered years.  I am intentionally exaggerating this, of course, but it is essential.  Anyone with a computer can find something plausible that has the same effect and is more difficult to spot.  Some of the current bogus research uses this very approach -- too many variables for too few events.

Examples of Strong Research

Great research starts with a hypothesis.

My favorite professor in grad school tricked us, something that I chronicled in this article.  If you take the time to read it, you will see the error of modern "experts" who start with data instead of following the scientific method.

This same prof had a great study of innovation in public health agencies.  He discovered that agencies that had great motivation to innovate but lacked money produced little innovation.  Agencies that had plenty of bucks but a leader who was a dinosaur also did not innovate.  It took both.  He wrote a great analysis of statistical interaction.  Note the relevance of the theory, his brilliant starting point.  He did not collect some data and then define a "syndrome."

My dad solved many problems the same way --- fuel, oxygen, ignition.  It was amazing how many problems he solved this way, as I wrote in this article.  The story is amusing, but Dad had a solid lesson, which included some thoughts about respect and learning something from everyone.

Implications for Current Recession Forecasts

There is plenty of room for debate.  In my careful, months-long survey of recession forecasting I concluded that there were questions about each method.  There is a healthy discussion on this subject.  I hope that I have helped to encourage this, and I will continue to contribute.  I hope readers will understand that this is one of many important themes.  I see the concern as important but not urgent, so I am going at a careful pace.

Let us split the candidates into two groups, those that have a long-term record, and those who only started with the last recession -- or even later.

Long-term record

I still invite new nominations on this, but I think that Bob Dieli, the Leading Economic Indicators, and the ECRI are the only sources with such a record.

The LEI is a moving target.  The early roots were promising, and for years it was maintained by the government through the Commerce Department.  Some authors who wish to disparage the current findings still incorrectly refer to it as a government indicator.  A more telling criticism is that the components keep changing, with a complete repainting of the data history.  If you really want to know the record of this indicator, you need an "as reported" history.

The ECRI remains a mystery, but one that is closer to a solution every day.  A worldwide team of very bright and talented people have succeeded in reverse engineering the components of the index and also the mysterious translation to a growth index with a complex calculation.  I do not want to spoil the emerging story, but it  will not take long.  There are several key questions:

1. Is the published history based on the current index?  If not, doesn't this seem like false advertising?  If there are changes, what were they?  (I do not ask this lightly.  The team can replicate the old history almost perfectly, but not the current recession call -- the very thing we are all wondering about).
2. Does the method rely heavily upon commodity prices?  Didn't these spike in the first part of 2011 for reasons that many attribute to speculation, the Fed or other non-fundamental factors?  (Don't get me started on this!)  If so, would not a reversion to regular growth levels just be "normal" and not an indication of economic collapse?
3. Why did your story switch so dramatically?  One moment it was the idea that an era of slow growth made us more vulnerable.  In a heartbeat it switched to a 100% recession call with no effective policies for government leaders.  What happened?  Did the methods change?

Bob Dieli has done the best over many decades.  Like every model developer,  Bob is always questioning both his approach and his conclusions.  As a reviewer and critic, I am doing the same.  Unlike the ECRI, the Dieli method is open to review, and I will take this up in the next article.

Here is the key concept:  Most recession forecasters mistakenly look for weakness.  That tells you that the current economy is performing below trend, but it is not a signal of a new recession.

Bob's method looks for cycle peaks (and troughs), which is how the NBER defines recessions (and the end of recessions).  If you do not understand this, go back to the first article in this series.

The short-term candidates

There are several recession forecasters who do not have a long-term record in real time.  Essentially, they are doing research that tries to explain the past, sometimes without following a scientific method.  Since the statistical procedure and commentary are fancy, it may seem persuasive.

This is the illusion that comes from using too many variables on too few cases.  In fact, the fewer variables the better!  Especially when there are only a few cases to predict, using a long laundry list of binary indicators is a red flag!  You will overfit (over-explain) the few past cases and be forced to keep revising your indicators over time.

Hint for Careful Readers

Those who want to avoid being bamboozled can ask a simple question:

When someone tells me about a syndrome or Aunt Gertrude's symptoms, have they done the following:

1. Provided a hypothesis, stated in advance;
2. Explained the reason for the proposed statistical interaction;
3. Demonstrated a real-time record of recession forecasting; and
4. Maintained the same exact criteria over time (no fudging, changing, suggesting of Plan B or exceptions).

If not, taking up my old roles, I would not publish the article nor would I invest in the system.  It is sad that so many individual investors think that something that looks perfect is the best choice.  If only they had seen more such examples, they would have better instincts.

I hope this article will help.