When mutual fund managers analyze Microsoft (MSFT ), they compare its 2005 profits not just with those of other software companies but also with its own earnings in 2004 and 2003. They do this because they want to know whether Bill Gates and Steve Ballmer are making the right moves, regardless of how the rest of the software industry is performing.
Can the same approach be used when analyzing the funds themselves? A study by Marcin Kacperczyk of the University of British Columbia's Sauder School of Business and Clemens Sialm and Lu Zheng of the University of Michigan's Ross School of Business indicates yes. In the paper, Unobserved Actions of Mutual Funds, the authors devised a way to benchmark a fund manager against himself. They also found that whether a fund beats or lags its individual benchmark has strong predictive value for performance. The results don't mean that existing evaluation methods, such as comparing funds with competitors and indexes, should be tossed aside, but the findings give investors insights that the others don't.
The study analyzed more than 2,500 diversified domestic equity funds over 20 years from the beginning of 1984 through the end of 2003 (http://papers.ssrn.com/sol3/papers.cfm?abstract-id=676103). For each fund, the researchers calculated the performance of its previous published portfolios. They then calculated the difference between the fund's actual performance and its performance if the manager had made no changes.
Think about it this way: To analyze, say, the $38 billion Fidelity Low Priced Stock Fund (FLPSX ), the researchers would take the fund's June 30 semiannual report, which includes a snapshot of its portfolio on that date, and enter all the fund's stock and cash holdings into a spreadsheet. In subsequent months they would track how that portfolio in the spreadsheet did, vs. the actual recorded performance of the fund. (They also would subtract the fund's management expenses from the spreadsheet portfolio's performance.)
If fund manager Joel Tillinghast did no trading at all, the spreadsheet of stocks and the fund itself would behave identically. But that rarely, if ever, happens. The question is: Are the stocks Tillinghast buys better than the ones he sells? If he's making smart trades, then the fund will outperform its old portfolio. If he's making unwise moves, then it will lag. In Tillinghast's case, the analysis demonstrates his skill: The fund outperformed its old portfolios, as measured by each semiannual report, by an average of two percentage points a year.
The researchers also discovered that this sort of analysis has some predictive value. For each year of data, the authors ranked the funds by their performance gap and then divided them into 10 equal groups. They found that on average the managers in the top group in any given year beat the market by an annualized 1.2 percentage points over the next five years, while those in the bottom group lagged 2.2 points. In comparison, a recent study of Morningstar's (MORN ) ratings found that funds rated five stars in 2002 subsequently beat the market by just 0.1 point and one-star funds lagged by 1.9 points.
So what are the study's top-ranked funds? Co-author Sialm is reluctant to reveal them. "We don't say that each individual fund in the top-performing group will perform better but that the group as an aggregate will," he says. Still, choosing from among the best is better than from the worst. Of those, the best in 2003 included: Smith Barney Small Cap Growth, which outperformed its old portfolio by 21.2 points; Boston Company Small Cap Tax-Sensitive Equity (SDCEX ), 21; American Century New Opportunity II (TWNTM ), 16.6; Pimco PEA Opportunity Fund (POPCX ), 15.9; RS Investment Smaller Company Growth (RSSGX ), 15.3; and Salomon Brothers Capital, 12.3. The study data ended in 2003, but the performance edge, by the authors' reckoning, should carry through 2008.
One of the surprising results was that the average gap for all 2,500 funds was close to zero. "We thought that trading costs would hurt the average fund," says Sialm, "but some funds systematically create value for shareholders by trading favorably." For instance, the rapidly trading CGM Focus Fund (CGMFX ) -- which has a 327% turnover ratio, meaning it changes its portfolio more than three times a year -- has an average 12.4 percentage point positive gap for its history covered by the study (just 1999 to 2003). If you look at the strategic shifts that Ken Heebner has made in the portfolio, you can understand why. He went from emphasizing raw materials to real estate, homebuilders, and energy stocks, and seemed to be always in the right place at the right time. Meanwhile, another rapid trader, ABN Amro Veredus Aggressive Growth Fund (AVEIX ), has suffered from moving too soon into technology during the bear market, with a -3.37 percentage point gap for the 2001-03 period. The fund continues to lag behind other small-cap growth funds.
Generally, the funds that trade the most will have the widest performance gaps. Yet Sialm warns that this sort of analysis has limitations. "If a fund's original portfolio of holdings happened to be really great or really awful, the performance gap won't accurately reflect the manager's skills," says Sialm. After all, if a manager had great stocks to begin with, any changes could make him look bad. A manager with an awful portfolio will look like a genius after dumping his dogs.
The study doesn't account for cash flows into the fund, which can dilute the performance of existing stocks, or outflows, which can force managers to sell stocks. Looking at a 20-year average may not help either, Sialm says, because funds' asset size, managers, and analysts change over time. Still, those problems exist with any rating system.
You could analyze one of your own funds like this, though it's easier with compact portfolios rather than those that hold hundreds of stocks. Take the Oakmark Select Fund (OAKLX ) and enter its 20 holdings into an online portfolio tracker (available at businessweek.com, morningstar.com, or finance.yahoo.com). Remember to include the cash, too. At the end of each quarter or six months, calculate the return minus a quarter or half of its expenses and compare it with the fund's actual return. That way you'll see whether the manager is earning his keep. An analysis using the BusinessWeek Portfolio Tracker Tool of Oakmark's holdings from Dec. 31, 2005, through Mar. 3 showed a slight lag of 0.8 percentage point after deducting a portion of its expenses, a cause for neither panic nor praise. You'd have to track that gap for two or three years to get a better read on the fund's management.
By Lewis Braham