There is a type of research that is especially dangerous for individual investors. The researcher takes current data and makes a statement like one of the following:
- If you avoided the ten worst trading days over the last five years....
- If you missed the ten best trading days over the last five years....
- If you threw out the ten strongest earnings reports....
- If you threw out the ten worst earnings reports...
- If you avoided the last five recessions....
- If you threw out the companies with the strongest stock performance...
- If you threw out the companies with the weakest stock performance...
This type of analysis is pretty easy to do for anyone with a computer and a data set.
Many trading systems do backfitting, avoiding the recessions or downturns. It is easy to find an indicator that gave a definitive signal when looking at past data. The problem is that such systems, lacking rigorous development of hypotheses, failing to use out-of-sample data, and willing to accept an insufficient number of cases, usually produce post-diction rather than robust predictive models.
The Current Example
Barry Ritholtz at The Big Picture highlights an interesting situation posited by Mike Panzer. Mike reports that a small number of stocks have powered the Nasdaq higher. Mike concludes as follows:
Finally, 13 out of 100 stocks -- 13% -- are responsible for two-thirds of the overall advance.
While this heavy lifting by a small number of shares does not mean the index can't go higher still, history suggests rallies that lack widespread participation sometimes lack long-term staying power.
What to Conclude?
We are troubled by this facile conclusion, which was reported without comment by various pundits. We enthusiastically endorse the more analytical questions raised by Barry:
What might this mean?
Are Technicals Waning as a Positive Influence? I'm not exactly sure -- What I'd like to see is how past rallies have moved forward in terms of leadership.
Is it unusual to have 13 stocks in the NDX’s 100 account for 67% of the aggregate advance? Is this unusually narrow? When has this occurred, early or late in a run?
I don't know the answers to these, but I am curious . . .
In scholarly research one would not start with a conclusion, but with a hypothesis. It might go something like this.......
When fewer than fifteen stocks in the Nasdaq account for two-thirds of recent gains of X percentage, we define the leadership as "narrow." Looking back on the Y number of cases fitting this description, we note that stock returns over the following Z days were as follows (table included).
Even with such a statement there are issues about how the definition of narrowness was determined, whether there were enough cases, and whether the researcher really generated the proposition and then tested or just used all of the data to identify key parameters. This approach would at least provide some comparison. Ideally, the relevant data would be provided to other researchers to check the selection of parameters.
In the absence of such data, one is left with questions. The indices are weighted by capitalization. What is the performance of the rest of the index? What is the market cap percentage of the top stocks? Markets often look for leadership. Is a gain by some of the top stocks indicative of success or of failure? If other stocks are lagging, is it possible that the market will later show strength in the laggards?
A Final Word
Like Barry, we do not pretend to know the answers to these questions. In our effort at "A Dash" to raise the standard of Street research, we try to highlight certain problems. Often these are conclusions that are readily embraced by those who seek support for what they already believe.
Some research is driven by conclusions, not by hypotheses. It is not scientific.
As usual, the discriminating investor, insisting upon strong research methods, can gain a contrarian advantage.
