Mining Profits From Microdata

New software exploits second-by-second market trends

People scoffed in 1985 when Richard B. Olsen assembled a team of physicists and statisticians to study second-by-second price data from financial markets. In the currency market at the time, price quotations weren't even being routinely stored. Stock markets at least recorded every trade, or "tick." But practically the only use for used ticker tapes was as confetti for parades. Conventional wisdom said that trading data at the tick-by-tick level was meaningless noise. "We were just laughed at," remembers Olsen, chairman of a Zurich-based market forecasting firm called Olsen & Associates Ltd.

Today, economic orthodoxy has swung around to Olsen's point of view: Academics as well as traders recognize that potentially lucrative information is embedded in the seemingly chaotic movement of prices from moment to moment. Statistical analysis of tick data may reveal price trends that are predictable for short stretches of time. Fast-moving traders with low commissions can cash in on those mini-moves. "Market microstructure is now one of the most active research areas in economics and finance," says a new book, The Econometrics of Financial Markets, by John Y. Campbell of Harvard University, Andrew W. Lo of Massachusetts Institute of Technology, and A. Craig MacKinlay of the University of Pennsylvania's Wharton School.

NUMBER CRUNCHERS. The revolution has been made possible by wider availability of tick data from various markets, as well as powerful computers for crunching those numbers. New database programs and statistical analysis software have helped as well. Says Gregory Kipnis, CEO of Invictus Partners LLC, a New York-based investment manager specializing in this kind of trading: "There isn't a single firm of note that isn't building tick databases and modeling the data."

Olsen has perhaps the grandest vision. He says the study of high-frequency market data demolishes the idea that current prices quickly reflect all new information. In fact, he says, information is digested in different ways by different types of market players--ones with a short time horizon vs. ones with a long horizon, for instance. Prices are determined by the way these heterogeneous players react to an outside event, and then react to each other's reactions, and so on. Anyone who can forecast those ripples can make money, he says. "In my view, the theory of finance is in a most dramatic transition, comparable to what happened in physics in the 1920s."

Olsen's foreign-exchange forecasting model is like a Swiss watch with hundreds of gears and springs--formulas representing such factors as the varying rhythms of trading in Tokyo, London, and New York and the behaviors of different players from day traders to corporate treasurers to central bankers. Olsen claims that his currency-trading model can reliably earn 4% to 8% above the risk-free interest rate. However, a fund set up to prove that claim was shut down because "we didn't want to compete with our customers," Olsen says. Germany's Munich-based Hypo-Bank says it has earned an 8.9% annual return using the Olsen model since August 1996. A joint venture between Olsen and the Netherlands' Rabobank Nederland, which manages $400 million, has been operating for half a year, but the principals won't disclose their results.

Unlike the academically inclined Olsen, most high-frequency traders couldn't care less about theory. They simply use statistical software to sift through tick-by-tick data looking for correlations between prices of different assets. When the assets move far enough out of their normal alignment, it's a signal to sell the high-priced one and buy the cheap one. Kipnis, of Invictus, says he used to trade on the spread between the stocks of J.P. Morgan and Bankers Trust New York Corp. when he ran the analytic and proprietary trading division of Morgan Stanley & Co. in the mid-1980s. Today, software from companies such as Leading Market Technologies Inc. of Cambridge, Mass., makes it possible to dig up some of the less obvious correlations.

It's hard to score big on a single trade, but a player with low enough transaction costs can get rich slowly by making a slew of similar transactions. The strategies can quickly cease to be profitable when correlations change or more traders catch on to them. So traders must constantly be searching for new opportunities to stay ahead of the pack.

That's why many people who study high-frequency data use it for purposes other than to forecast prices. For instance, an investor who needs to sell a big block of shares in General Motors could examine tick-by-tick data to see when the bid-ask spread is tightest, and use other clues to estimate how much "depth" there is to the buying interest. Having that knowledge allows the investor to tell whether his or her trades are being executed at competitive prices, says Michael C.L. Adam, executive chairman of London-based Inventure Ltd., which sells a program called Ranger that lets traders combine current and historical data. Ranger was developed by Paul Tudor Jones II, the legendary trader.

Financial houses have little choice but to probe for advantages in the tall grass of tick-by-tick data. Heightened competition has narrowed profit margins in just about every kind of trading. Take the case of foreign exchange, a $1 trillion-a-day business that's dominated by perhaps 20 major banks. Even that small number may be too many for the market to sustain. The winners will be those that have the best information about the state of the market so they can quote a price that appeals to the customer yet leaves a dram of profit. Where will that information come from? A computer model that combines tick-data analysis with traders' observations to supply up-to-the-second intelligence about market conditions around the world.

"ANY IDIOTS OUT THERE"? In other words, eyeballing the yen-mark exchange rate on a Reuters screen won't cut it anymore. A bank that makes a market in foreign exchange can't afford to have less information than its rivals possess. "You don't want to be the fourth bank that a customer calls," says one London-based consultant, who asked not to be identified. "They've already scoped the quotes from the first three and they just want to see if there are any idiots out there" who will make the mistake of quoting too good a price.

As the forex market goes, so go stocks, bonds, commodities, and derivatives. Information is increasingly the name of the game in finance, not only for traders but for their customers as well. And as in currency trading, the best place to find unique, exploitable, information is in the tick-by-tick market data.

Eventually, of course, even this brave new frontier will be overrun as larger and larger numbers of players start running high-frequency data through sophisticated econometric models. "My opinion is that the game will get harder," says Norman Packard, president and co-founder of Santa Fe (N.M.)-based Prediction Co. But for now, there's a lot of money to be made off a microscopic view of the market. Tick, tick, tick.

Before it's here, it's on the Bloomberg Terminal.