Can You Really Beat The Market?

A new book reopens the debate over whether it's possible to predict the movement of stock prices

Andrew W. Lo of the Massachusetts Institute of Technology remembers the excitement of reading A Random Walk Down Wall Street when he was in high school in the 1970s. In it, Princeton University economist Burton G. Malkiel made the case that stock prices are unpredictable--as random as the lurching of a drunk. Says Lo: "That was one of the first books that got me interested in economics. It was an eye-opener."

An eye-opener, maybe, but hardly a guiding light. Today, as a finance professor at MIT's Sloan School of Management, Lo is working to prove that Malkiel and his fellow random-walkers are wrong--that shrewd investors can beat the market after all. He and collaborator A. Craig MacKinlay of the University of Pennsylvania's Wharton School have even written a book called A Non-Random Walk Down Wall Street. It was published in March--two months before Malkiel came out with a thoroughly revised seventh edition of his own book, which has sold more than 500,000 copies.

FAIR REWARDS. Random-walk theory says that investors can't beat the stock market because news travels too rapidly. When a new bit of information emerges, investors react to it almost instantly, bidding a stock's price up or down until it reaches a new equilibrium. Therefore, the only things that the market hasn't taken into account are things that haven't happened yet. Those events are, by definition, random. You can count on making money in the long run only because prices generally trend upward, in line with economic growth.

But markets don't know everything, say the authors of A Non-Random Walk Down Wall Street. People who devote enough time, money, and brainpower can beat the market by finding undervalued companies or discovering persistent price patterns, say Lo and MacKinlay. Their profits are "simply the fair reward to breakthroughs in financial technology," they argue.

This is no sterile academic debate. Investors are taking sides every time they select a place to put their money. Random-walkers buy index funds. Disbelievers in the random walk are more likely to play the market--whether successfully, like Warren E. Buffett, or unsuccessfully, like your average small-time day trader.

When Malkiel wrote the first edition of A Random Walk Down Wall Street in 1973, he was the one challenging the conventional wisdom. At the time, random-walk theory was a scholarly notion unfamiliar to the average investor. Malkiel infuriated Wall Street brokerage firms by writing that their expensive advice was pretty much worthless. Index funds that match market averages didn't even exist in 1973; he wrote that they should. Here's a quote from the first edition: "Whenever below-average performance on the part of any mutual fund is noticed, fund spokesmen are quick to point out, `You can't buy the averages.' It's time the public could." Malkiel was ahead of his time: Today, about 20 cents of every retail dollar going into mutual funds goes into index funds.

SLOW RETREAT. By the time Lo and MacKinlay began collaborating, random-walk theory was catching on with the general public--and remained a shibboleth of academic finance. Their book recounts how in 1986, when they presented their first academic paper statistically rejecting the random walk, their discussant--"a distinguished economist and senior member of the profession"--assured them that they must have made a programming error. Ever since, random-walkers in academia have been forced into a slow but steady retreat.

Surprisingly, perhaps, Lo and MacKinlay actually agree with Malkiel's advice to the average investor. If you don't have any special expertise or the time and money to find expert help, they say, go ahead and purchase index funds. Where Lo and MacKinlay part company is over Malkiel's insistence that even the top investment professionals can't do better than garden-variety index funds, because any edge is wiped out by the costs of research and extra trading. Malkiel points out that a Standard & Poor's 500-stock index fund with annual expenses equaling 0.2% of assets has outperformed 90% of actively managed mutual funds over the last 3, 5, and 10 years. What's more, funds that beat the S&P over one time scale may not beat it over another.

True, acknowledge Lo and MacKinlay. But they insist that the best pros can still beat the market by analyzing the data and exploiting the subtle regularities that they discover. Evidence of that, they say, is that Wall Street firms continue to pour money into supercomputers and PhDs.

One big difference, of course, is that you can't (for now, anyway) protect the profits from a successful trading strategy by patenting it. Others will soon duplicate your strategy and, by so doing, take the profit out of it. The January effect, in which small stocks consistently rose in January, disappeared when it started getting too much publicity. Random-walkers say this process occurs so quickly that extra profits are a mirage. Not so, Lo and MacKinlay argue. Says MacKinlay: "As different trading strategies emerge, new anomalies can appear, or past anomalies can reappear."

Where are today's exploitable anomalies? Lo and MacKinlay argue that fast computers, chewing on newly available, tick-by-tick feeds of market-transactions data, can detect regularities in stock prices that would have been invisible as recently as five years ago. One example: "clientele bias," in which certain stocks are popular with investors who have certain trading styles. A case in point that doesn't take a supercomputer to detect is day traders' current enthusiasm for Internet stocks. Lo says that day traders tend to overreact to news--whether that news is positive or negative--so it should be possible to profit by taking the opposite side of their trades.

DIFFIDENCE. Lo is even open-minded about technical analysis--that is, calling turns in the market by studying patterns in price charts such as "head and shoulders formations" and "resistance levels." To Malkiel and most other professors, technical analysis is about as scientific as palm-reading. In his book, Malkiel calls it "anathema to the academic world." Yet Lo, a bona fide academic, is working with some of his MIT colleagues on computer software that identifies some of the chartists' favorite formations in a consistent way. His next step is to determine whether these patterns have any predictive value. Says Lo: "It could be that technical analysis summarizes in a very compact way the influences of supply and demand."

But Lo is hardly advocating that you turn over your savings to chartists. Or to fundamental stock-pickers, for that matter. At times in their book, he and MacKinlay sound just as diffident about beating the market as Malkiel, their elder. "Indeed," they write, "although there are probably still only a few ways to make money reliably, the growing complexity of financial markets has created many more ways to lose it, and lose it quickly." That's a sentiment that random and nonrandom walkers can agree on wholeheartedly.