Rising fees have chased millions of people away from banks and into prepaid debit cards.

In just a handful of years, prepaid cards have become the fastest growing payment method in the U.S. Just this week, American Express became the first mainstream financial company to offer a prepaid card.

But the cards have problems of their own. Complex fee schedules. Few of the consumer protections afforded to bank and credit card customers. No ability to build credit history.

Consumer advocates are raising concerns and demanding more oversight and at least one state is investigating prepaid card issuers. The Consumer Financial Protection Bureau is expected to step up oversight of the industry when it launches in July.

“People are using prepaid cards as checking accounts and the government ought to regulate it similarly,” says Suzanne Martindale, staff attorney for Consumers Union, a nonprofit advocacy group that is concerned about unfair prepaid card fees.

Even so, Americans spent $140 billion using prepaid cards in 2009, according to the latest data available from the Federal Reserve. That’s a 21.5 percent increase each year over four years. The amount of money loaded onto the cards is expected to reach $552 billion in 2012 from $330 million three years ago, according to the Mercator Advisory Group, a research firm.

Prepaid cards have gone mainstream by catering to the ranks of the unbanked — people who don’t have a bank account. Nearly one in five Americans are unbanked, a 2009 government report found, and the number is growing.

Prepaid cards can be used to pay bills or buy merchandise in the same places a bank-issued debit card can be used. So it’s no wonder prepaid is the fastest growing method of payment over the last five years. This year, the IRS issued tax refunds on prepaid cards to about 600,000 bank account-free households. Social security payments for the unbanked have been loaded onto prepaid cards since 2008. And a growing number of small companies pay employees using the cards. On Tuesday, American Express joined the fray. The card giant launched a prepaid card in an effort to expand its customer base.

Most new prepaid card customers are seeking refuge from new and escalating fees, consumer advocates say. Among them: $3 to print an account summary at a Bank of America ATM; $12 a month for checking accounts with a balances below $1,500 at Chase and Bank of America; overdraft fees of $35 that most banks charge.