While extreme market concentration and high valuations often signal a coming correction, historical precedents suggest that diversification remains the only reliable hedge against technological upheaval.

The Specter of Concentration

One of the primary benefits of index investing is supposed to be broad diversification. However, we are currently witnessing the most extreme level of index concentration in U.S. market history, dating back to 1927. Today, just seven stocks make up roughly 36% of the S&P 500. When a handful of companies wield such disproportionate influence, their individual fortunes dictate the direction of the entire market. If these giants stumble, the impact on the average investor’s portfolio is substantial.

This is a scenario we have seen before. In July 2000, a single company, Nortel Networks, accounted for 36% of the entire Canadian market index. When Nortel eventually crashed and became worthless, it dragged the broader Canadian market down with it. While the U.S. market today is not quite as concentrated as Canada was then, the parallels are striking. We are seeing a massive surge in capital expenditure and earnings growth concentrated in a tiny sliver of the economy—specifically, those companies positioned to lead the AI revolution.

The Utility of Productive Bubbles

Whether we are currently in an 'AI bubble' is only knowable in hindsight. However, the pattern of rapid infrastructure spending followed by a painful price correction is an age-old feature of financial markets. From the railroads of the 1840s to the fiber optic cables of the late 1990s, technological revolutions often spark speculative manias. Investors pile into the new technology, driving prices to unsustainable heights, which eventually leads to a crash.

Crucially, these bubbles are not entirely destructive. High stock prices facilitate the massive investment required to deploy revolutionary technologies into the economy. While investors may suffer when the bubble pops, the infrastructure remains. The redundant railway tracks and unused fiber optic cables of previous eras eventually paved the way for economic golden ages. In this sense, 'productive bubbles' are often a net positive for society, even if they are financially devastating for the individuals who bought at the peak.

Valuation vs. Concentration

To understand the current risk, we must distinguish between market concentration and market valuation. Concentration measures how much of the market's value is held by a few firms, while valuation—often measured by the Shiller CAPE ratio—measures how much investors are paying for future earnings. Historically, the relationship between concentration and future returns is noisy and statistically weak. Many international markets, such as Switzerland or Taiwan, have operated with much higher concentration than the U.S. for decades while still delivering positive returns.

Valuation, however, is a more reliable, if imperfect, omen. When the CAPE ratio is high, future ten-year returns tend to be lower. We saw this in the U.S. after 1999 and in Japan after 1989. In the Japanese case, the market reached such extreme valuations that, when adjusted for inflation, it has still not fully recovered thirty-five years later. While high valuations do not guarantee an immediate crash, they do suggest that investors should moderate their expectations for the 'rocket ship' returns seen in recent years.

The Value of the Free Lunch

The most effective defense against the aftermath of a bubble is diversification, often called the only 'free lunch' in investing. During the U.S. 'lost decade' from 2000 to 2013, when the broader market was flat, value stocks and small-cap value stocks actually performed quite well. Similarly, during the Japanese collapse of 1989, a globally diversified investor would have fared fine as the U.S. market took the torch of dominance. Diversification ensures that while you will always own the losers, you will also always own the winners.

The challenge of diversification is primarily behavioral. It requires the discipline to hold assets that are currently underperforming while the rest of the world is chasing a single, high-flying sector. It means accepting that you won't capture the absolute peak of a speculative mania in exchange for the security of not being wiped out when the cycle turns. By maintaining a disciplined, globally diversified portfolio and tempering expectations, investors can navigate the uncertainty of the AI era without needing to predict exactly when the music will stop.