After the Sept. 11 attacks, there were grim questions about the future of the shaken, dust-covered neighborhoods around the World Trade Center. Would residents flee uptown or to the suburbs? Would the epic job of rebuilding lower Manhattan be too much to bear? Who would want to live so close to a place associated with such horror?

As it turns out, plenty of folks.

Census figures released last week show that the number of people living near ground zero has swelled by about 23,000 since 2000, making it one of the fastest-growing places in the city.

Virginia Lam, a publicist and former City Hall operative who moved into a newly converted residential building on Wall Street in 2006, said the site is a source of inspiration, rather than fear or gloom.

“It’s pretty amazing,” she said of the new towers rising from the 16-acre hole created by the attacks. “I feel like, being a New Yorker who was here on 9/11, and who has worked for the Fire Department and for the city, I think it is always in the back of my mind, but it’s not something that dominates my thinking. I go about living my life.”

About 45,750 people now live in the part of Manhattan south of Chambers Street, which encompasses ground zero. That is more than twice as many as there were during the last census.

There was also significant growth a little farther uptown. In all, 82,137 people were counted as living south of Canal Street, 15 blocks north of the trade center. That is an increase of 43 percent from 2000 in an area that includes the Financial District, Battery Park City, a section of tenement-packed Chinatown and the celebrity-studded streets of TriBeCa, which is short for “Triangle Below Canal.”

The change around Wall Street has been especially remarkable given the area’s history as a financial hub, rather than a residential district.

One by one, bank headquarters have moved elsewhere, and millions of square feet of office space have been converted to homes — a change spurred partly by government incentives intended to help revive downtown after Sept. 11.

“It’s astounding,” said Julie Menin, head of the local community board. “Many people thought after 9/11 that people wouldn’t remain in Lower Manhattan. Not only did people stay, they came in droves.”

New Yorkers have been well aware of the change. In the shell-shocked months after the attacks, the Financial District became a ghost town when workers left for the day. Now it is teeming with people around the clock.

Grocery stores have opened. Three new schools have opened up in four years. Briefcase-carrying stockbrokers now share sidewalk space with kids in strollers. A string of new apartment towers has been built along West Street, a short distance from the trade center site.

The area isn’t finished growing, either. Near the foot of the Brooklyn Bridge, work was recently completed on the tallest apartment tower in the Western Hemisphere, a 76-story, 900-unit skyscraper designed by the architect Frank Gehry.

Growth downtown has far exceeded the rate elsewhere in the city. The 2010 census put the city’s population at 8.18 million, up 2.1 percent from 2000.