Tourists line up to enter the World Trade Center Gallery and Tribute Center adjacent to ground zero in Lower Manhattan. Iran's president, rebuffed in his request to lay a wreath at the site, said Thursday that he would not fight the decision.

New York ? Almost everyone agrees Iranian President Mahmoud Ahmadinejad doesn’t belong at ground zero.

So who gets access these days to the 16-acre pit where the World Trade Center once anchored the Manhattan skyline, a slice of the city that many regard as hallowed ground?

Construction workers. The families of victims. The occasional journalist. And not too many others, in stark contrast to the days immediately after Sept. 11 when the smoldering site was overrun with celebrities, politicians and even Playboy playmates.

Amid the chaos after the twin towers fell, rescue workers and cleanup crews mingled with a parade of well known visitors: Muhammad Ali, Robert De Niro, cast members from “The Sopranos,” Martha Stewart.

U.N. Secretary-General Kofi Annan and other world leaders came. Almost half of the Senate arrived en masse.

Within a month, the city was turning down hundreds of requests to visit the site and began asking celebrities to avoid the area as the treacherous search for remains continued.

Six years later, Police Commissioner Ray Kelly said a proposed ground zero visit by Ahmadinejad during next week’s U.N. General Assembly had no chance. Police cited ongoing construction and security concerns, and the Iranian president, who is under Secret Service protection while in the U.S., was told to steer clear.

“We have communicated our concerns to the Iranian Mission,” Kelly said. “I am sure they will abide by our statement … Our position is that he will not be permitted to go.”

Mayor Michael Bloomberg said the city would provide protection for all guests of the United Nations without requiring that they pass a “litmus test for views.” He said of Ahmadinejad: “I personally find what this guy has said abhorrent, and I think it would be inappropriate to have him visit.”

Some objected to Ahmedinejad’s visit on political grounds.

“To have the leader of the greatest state sponsor of terrorism in the world visit the site of the most heinous terrorist attack on America would be an affront to the victims and families of 9/11 and to all who lived through that day,” said Abraham Foxman, national director of the Anti-Defamation League.

The sentiment was echoed by the State Department, where deputy spokesman Tom Casey called the idea of a visit by Ahmedinejad, who advocates the destruction of Israel, “rather appalling and the height of hypocrisy.” New York-based presidential candidates Hillary Clinton and Rudy Giuliani also expressed their opposition to the suggested visit.

Ahmadinejad told CBS’ “60 Minutes” Thursday in Iran, “Well, over there, local officials need to make the necessary coordinations. If they can’t do that, I won’t insist.”

Ahmadinejad said he was baffled that Americans would be angry if he placed a wreath at the site of the worst terror attack on American soil.

“Why should it be insulting?” he asked Scott Pelley of “60 Minutes.”

“We obviously are very much against any terrorist action and any killing. And also we are very much against any plots to sow the seeds of discord among nations. Usually you go to these sites to pay your respects. And also to perhaps to air your views about the root causes of such incidents.”

Pelley told him the American people believed his country exported terrorism, and would be offended if had a “photo op” at ground zero.

“Well, I’m amazed. How can you speak for the whole of the American nation?” the Ahmadinejad countered. “You are representing a media and you’re a reporter. The American nation is made up of 300 million people. There are different points of view over there.”