What a relief! The stock market is coming to life, with the Dow and S&P 500 hitting 4-year highs last week. For many investors, the wounds from the 2000-01 crash are healing nicely.

And most of us have learned not to take the market for granted. But that doesn’t mean we’ve given up all our self-destructive investing habits.

In fact, legendary investor Warren Buffett thinks some bad practices are getting worse. His major concern is investment costs.

Many people, he argues, are now giving up a whopping 20 percent of their investment gains to “frictional costs” – layer upon layer of trading commissions, advisers’ fees and other expenses that can be avoided.

That was one of the chief messages last week in Buffett’s annual letter to shareholders at Berkshire Hathaway, the company he runs. The section is titled “How to Minimize Investment Returns.”

How could costs possibly chew up a fifth of your gains? Easy: Earn 10 percent a year and pay 2 percentage points in expenses.

The typical actively managed mutual fund that invests in stocks, for example, charges about 1.3 percent a year to pay for stock-pickers and other expenses. Buy it through a brokerage account that charges an additional 1 percent a year for giving you investment advice and you’re well above Buffett’s 20 percent figure, assuming the 10 percent annual return.

Earn only 5 percent a year – about what the S&P 500 did last year – and 2.3 percentage points of friction slashes your gains in half.

But surely all the help you get from the fund managers and other investment advisers is paying off by adding value to your holdings, isn’t it? Not necessarily, Buffett says:

“True, by buying and selling that is clever or lucky, investor A may take more than his share of the pie at the expense of investor B. And, yes, all investors ‘feel’ richer when stocks soar. But an owner can exit only by having someone take his place. If one investor sells high, another must buy high. For owners as a whole, there simply is no magic … that will enable them to extract wealth from their companies beyond that created by the companies themselves.

“Indeed, owners must earn less than their businesses earn because of ‘frictional’ costs. And that’s my point: These costs are now being incurred in amounts that will cause shareholders to earn far less than they historically have.”

By hiring pros, you can hope to become an “investor A,” who gets more than his share of the pie. But how do you know that an adviser who delivered above-average returns wasn’t just lucky?

If the market returned 10 percent a year and you gave up 2 points to fees, your advisers would have to earn you 12 percent just so you could match the gains of investors who didn’t pay all those fees. That’s tough to do.

Any investment with a higher fee is justified only if it can beat the simple indexer – beat it, that is, after all frictional costs are taken into account.