People drink during good times. They drink when times are bad. Bernhard Stroh knew this. He also learned how to build a fortune. In 1850, he left Germany with $150 and his family’s beer recipe.

As recently as the 1980s, the Strohs controlled the country’s third largest brewing company. They were worth about $700 million. But today, their once-great fortune is a fiscal train wreck. They’re no longer rich. Rampant spending, high debt loads and poor investment decisions sank their wealth.

Unfortunately, stories about a wealthy family’s fall are as common as pimples on a teen. Time journalist, Chris Taylor, references data from the Williams Group wealth consultancy. He says 70 percent of wealthy families lose their fortune after one generation. A whopping 90 percent lose their fortune by the third generation.

Victor Haghani knows what it’s like to see fortunes ravished. He’s one of the founding partners of Long-Term Capital Management. It was a hedge fund that earned scorching returns–until it didn’t. From 1994-1998, investors in the fund quadrupled their money. Its managers used leverage and state-of-the art algorithms to earn eye-popping returns. But one year later, their fund collapsed. Some say it almost destroyed America’s financial system. Roger Lowenstein detailed the story in his book, When Genius Failed.

Haghani must have learned a lot from this experience. In 2013, he delivered a TED talk titled, Where Are All The Billionaires and Why Should We Care? He says most people don’t build the amount of wealth that they should. One of the main reasons is that they speculate. They take unnecessary risks. They believe they can see the future, or they think they can hire someone with a working crystal ball.

Haghini uses wealthy families to prove his point. In 1900, there were about 4000 millionaires in the United States. Many of them had far more than $1 million. But if each of those families had invested $1 million, and if they earned a return equal to the U.S. stock market, each family would have had $30 billion by 2012. By August 31, 2018, each family would have had $79 billion.

Haghini notes, however, that those families had children. As a result, those 4000 families in 1900 would have spawned about 120,000 families today. If the families’ fortunes equaled the return of the U.S. stock market, each of the 120,000 families would now be worth about $2.63 billion.

That’s a lot of billionaires. But according to Business Insider, there were just 563 billionaires in the United States in 2017. And virtually none of them derived their fortunes from their family’s millions in 1900. So, what happened to those 120,000 families?

They fell victim to human error. Many of them spent too much. They didn’t adequately plan for their futures. In many cases, they made poor investment decisions and they failed to diversify.

Building wealth is simple. But that doesn’t make it easy. We can start by spending less than we make. If we invest in a portfolio of low-cost index funds, we’ll beat the returns of most professional investors. Even Victor Haghani now recommends index funds. He joins Warren Buffett and a myriad of economic Nobel Prize winners. Such a portfolio beats most hedge funds. It also beats plenty of high-profile investors, such as President Donald Trump.

But here’s the hardest part. We’re human. We often can’t resist tinkering with our money. We try to time the market, fearing market drops and dire economic forecasts. We also chase past returns. Investors often buy what’s hot, after it has risen. They often sell past winners, after they have fallen. Bubbles like the 1990s craze or the Bitcoin dream wreak havoc on our wealth.

Investing is simple. But it isn’t easy. Human emotions get in the way. Descendants of millionaires from the 1900s might know this better than most. If we can sideline fear and greed, we might learn from their mistakes.