Everyone loves a winner -- while it lasts.

Photographer: Victor J. Blue/Bloomberg

The S&P 500's Dubious Anniversary

Barry Ritholtz is a Bloomberg View columnist. He founded Ritholtz Wealth Management and was chief executive and director of equity research at FusionIQ, a quantitative research firm. He blogs at the Big Picture and is the author of “Bailout Nation: How Greed and Easy Money Corrupted Wall Street and Shook the World Economy.”
Read More.
a | A

The Standard & Poor's 500 Index is market-capitalization weighted, meaning that companies with higher stock-market valuations have a bigger influence on the index. There has been a cottage industry of criticism about this structure. Recently, it has led to a new world of fundamental indexing and so-called smart beta, or seeking ways of using the components of the index to outperform the index itself.

The S&P 500 wasn't always set up this way. This is a good time to think about the subject, because today marks the anniversary of the index's overhaul, giving us the structure we have now.

According to the S&P 500 2001 Directory (hat tip to Jason Zweig’s This Day in Financial History), the benchmark index for large cap U.S. stocks looked very different than it did before this day in 1988.

Back then the index was made up of:

400 industrial stocks

40 utilities

40 financials

20 transportations 

This seems like it was an effort to mimic Dow Jones, which had three big indexes -- the Industrial Average, the Transportation Average and the Utilities Average.  According to the Dow Jones Industrial Average Fact Sheet, the Industrial Average was designed “to represent large and well-known U.S. companies, cover[ing] all industries with the exception of Transportation and Utilities.” Having been around since the late-19th century, it had become the benchmark for measuring the U.S. stock market.

Here's what the S&P 500 looks like today:

The shift toward a broader, market-cap-weighted index was designed to avoid some of the fundamental flaws in the Industrial Average, which is price weighted, meaning companies with higher stock prices have greater influence. This leads to all manner of absurdities, as we have discussed before:

Companies with higher stock prices such as Visa and Goldman Sachs have a 9.7 percent and 6.7 percent weight, respectively, while lower-priced stocks, such as Cisco Systems and General Electric, are merely 1.05 percent and 0.91 percent, respectively. Why Goldman Sachs, with an $84 billion capitalization, matters more to the Dow than General Electric, with a $257 billion capitalization, is rather mystifying. A high-priced, smaller company carrying more weight than a lower-priced, bigger company makes no sense.

Shifting from price-weighted indexes to market cap-weighted indexes was a huge improvement, but it wasn't without its own problems. 

We can see this in two examples: The booms and busts of the “Nifty Fifty” in the 1960s and early 1970s and the tech giants in late 1990s and early 2000s.

After World War II there was a 20-year bull market from 1946-1966. The Nifty Fifty were the must-own, one-decision names for both institutional money managers and retail investors. Then came the bear market that ran from 1966-1982; indexes in real terms lost 75 percent of their value.

The S&P wasn't yet cap-weighted, so we only know the impact abstractly. But the Nifty-Fifty underperformed, even relative to their indexes.

Next, let's jump to the end of a bull market to see what happens to a cap-weighted index. The results, as the dot-com era showed us, are none too pretty.

When markets go up 30 percent a year, casual investors tend to catch the fever. When this group of unsophisticated buyers all pile into the same well-known mega-cap companies, regardless of valuations, the net result isn't a surprise -- a rapidly inflating bubble.

This is characterized by a self-reinforcing cycle. When we have more buyers of these stocks, it increases the size of their market cap, which sends the S&P 500 higher. At the same time, money managers who are obligated to match the index, send ever-more capital to the shares of the biggest cap companies. This tends to lead naive individual investors to chase the flashy names even more.

The cycle will repeat -- but not forever.

Once the inevitable market top occurs and the reversal begins, what began life as a virtuous cycle turns into a vicious one. The reversal of the money flows out of the biggest cap stocks can be brutal. The index buyers race to catch up (or down), exerting more downward pressure.

Companies that were tech darlings -- Cisco, Intel, Microsoft, Qualcomm -- have yet to reach their late-1990s highs and have been a huge drag on the S&P 500 for much of the past 15 years.

Given these flaws, it's no surprise that analysts have tried to find a better way to create index weighting. Rob Arnott, chief executive officer of Research Affiliates, is the person credited with the best alternative to date: Weighting indexes based on a company’s fundamental footprint relative to the economy. Arnott, along with colleagues Jason C. Hsu and Philip Moore, in an article titled “Fundamental Indexation,” found that creating an index using any one of a number of fundamental factors -- such as earnings, revenue growth, dividend yield, price-to-book ratio -- offered a variety of improvements, not the least of which was performance that exceeded indexes based on market-cap weighting. Arnott discussed this in our Masters in Business interview (iTunes).

There has been lots of buzz about smart beta and its promise of better performance. Regardless of your thoughts on the topic, the flaws in market-cap weighting are there for all to see.

This column does not necessarily reflect the opinion of Bloomberg View's editorial board or Bloomberg LP, its owners and investors.

To contact the author on this story:
Barry L Ritholtz at britholtz3@bloomberg.net

To contact the editor on this story:
James Greiff at jgreiff@bloomberg.net