False hope - Most trading strategies are not tested rigorously enough
(l'articolo completo, pubblicato in The Economist, 21 febbraio 2015, è disponibile qui)
Let me tell you about the perfect investment offer. Each week you will receive a share recommendation from a fund manager, telling you whether the stock’s price will rise or fall over the next week. After ten weeks, if all the recommendations are proved right, then you should be more than willing to hand over your money for investment. After all, there will be just a one-in-a-thousand chance that the result is down to luck.
Alas, this is a well-known scam. The promoter sends out 100,000 e-mails, picking a stock at random. Half the recipients are told that the stock will rise; half that it will fall. After the first week, the 50,000 who received the successful recommendation will get a second e-mail; those that received the wrong information will be dropped from the list. And so on for ten weeks. At the end of the period, just by the law of averages, there should be 98 punters convinced of the manager’s genius and ready to entrust their savings.
As a paper published last year in the Journal of Portfolio Management argued, this is a classic example of the misuse of statistics. Conduct enough tests on a bunch of data—run through half a million genetic sequences to find a link with a disease, for example—and there will be many sequences that appear meaningful. But most will be the result of chance.
[...] Financial research is highly prone to statistical distortion. Academics have the choice of many thousands of stocks, bonds and currencies being traded across dozens of countries, complete with decades’ worth of daily price data. They can backtest thousands of correlations to find a few that appear to offer profitable strategies.
[...] The authors’ conclusions are stark. “Most of the empirical research in finance, whether published in academic journals or put into production as an active trading strategy by an investment manager, is likely false. This implies that half the financial products (promising outperformance) that companies are selling to clients are false.”
For the academics, the lesson is simple. Much more rigorous analysis will be needed in future to reduce the number of “false positives” in the data. As for clients of the investment industry, they need to be much more sceptical about the brilliant trading strategies that fund managers try to sell them.