Two weeks ago, weather forecasters warned of a blizzard on its way to dump two feet of snow in Philadelphia. Word quickly spread; friends and family warned me to make an emergency grocery store run and shore away a week's worth of supplies. On the night before the blizzard, I cracked under the peer pressure and headed to the Fresh Grocer. When I arrived, I was surprised to count over 100 people in line at the checkout. With dozens more already scrambling the aisles, I estimated a 30 minute wait at least...all for a carton of eggs and Cheez-Its? I decided to turn right around and head home. The next morning, I walked to work in about 10 inches of light powder, which was cleared by that evening.
What happened? Herd behavior can make a big impact on people's decisions around uncertainty. When we hear about hundreds of our friends making the same decision - like buying groceries "just in case" of a blizzard - it becomes pretty difficult to be the contrarian. "How could they be wrong?" we ask ourselves. In many cases, how could they? Herds are often right - just think of a stampede of people running away from a burning building.
As it turns out, herds are only correct under certain conditions. In general, the "wisdom of crowds" holds true in uncertain situations only when the crowd is diverse and decentralized, and when each member makes his decision independent of everyone else. For example, suppose we want to find how many jelly beans are inside a jar without counting. If we randomly select 100 people, equally aggregate and average their guesses (without each of them knowing what the other has guessed), we'll get a pretty reasonable estimate for the number of jelly beans.
During Philadelphia's Great Blizzard of 2017, the crowd's decisions were not independent of each other. Family members convinced other family members to buy supplies. Friends convinced other friends of the magnitude of the impending storm. Pretty soon, a bubble of opinion formed: this was going to be one hell of a blizzard.
Bubbles of opinions aren't exclusive to snowstorms. Economists observe bubble effects in stock and housing markets, though herding behavior among investors is very hard to detect and even harder to arbitrage. Most people did not see the 2008 mortgage bubble coming, and fewer still were able to successfully profit off of the bubble's implosion. Another example is the famous "tech bubble" of the early 2000's. Alan Greenspan warned of "irrational exuberance" in the markets as early as 1996, but the bubble didn't burst until 2000.
As Nate Silver describes in The Signal and the Noise, stock market behavior exists in a sort of "purgatory" between randomness and predictability. Index returns (in the Dow Jones, for example) are correlated from day-to-day, yet a basic momentum strategy of buying yesterday's winners and selling yesterday's losers is unprofitable after accounting for transaction costs. Even if the market reveals seemingly predictable statistical patterns, exploiting these patterns for excess return is extraordinarily difficult. Simply looking at performance for actively-managed mutual funds tells the same story; only 19% of them beat their benchmarks in 2016.
It's no question that markets occasionally get out of whack. Like my situation in the grocery store, investors herd into certain securities or strategies, potentially depressing or inflating prices and creating opportunities for the brave contrarian. But blizzards don't happen every day. Fisher Black believed markets are still efficient as much as 90% of the time. Or, as Lasse Pedersen put it, "markets are inefficient enough that money managers can be compensated for their costs through the profits of their trading strategies, and efficient enough that the profits after costs do not encourage additional active investing."
My takeaway is that understanding (and outperforming) the stock market is difficult, as it's mostly stuck between randomness and predictability, efficiency and inefficiency, and a rock and a hard place.
References:
1) The Wisdom of Crowds by James Surowiecki
2) The Signal and the Noise by Nate Silver