Momentum against the International Freedom Center continued to build yesterday as several prominent politicians shot down a last-ditch effort to keep the controversial museum at Ground Zero.

Former Mayor Rudy Giuliani emphatically added his voice to the chorus of no's, stating his opposition at the finish line of a race honoring Sept. 11 victims.

"They should change the whole concept and scrap those plans and start from the beginning and focus it on Sept. 11," Giuliani said. "I think it's a mistake the way it's become too complicated."

The former mayor expressed his opposition at the fourth annual Firefighter Stephen Siller Tunnel to Towers Run, honoring Siller, an off-duty firefighter, who was killed Sept. 11 after he ran carrying his gear through the Brooklyn-Battery Tunnel to the World Trade Center to join in the rescue effort.

Giuliani's no vote came two days after a former rival, Sen. Hillary Rodham Clinton, told The Post she wants the project canned for failing to listen to the 9/11 families, who fear the museum site will host anti-American exhibits.

"I cannot support the IFC," said Clinton. "I am troubled by the serious concerns family members and first responders have expressed to me."

Clinton's opposition was echoed by her new rival, Jeanine Pirro, the Republican Westchester County district attorney who is running for Clinton's Senate seat.

"Ground Zero has to reflect the reality of the heroes who died there," Pirro said. "It is about the people who were victimized as a result of the hatred of America."

Manhattan lawyer Ed Cox, also seeking the GOP's Senate nod, said constructing the IFC "on such hallowed ground would be completely inappropriate."

Former Yonkers Mayor John Spencer, also a would-be GOP Senate candidate, said the IFC was in the hands of "anti-American radicals."

Attorney General Eliot Spitzer, the all-but-certain Democratic candidate for governor next year, repeatedly refused to say where he stood on the IFC, claiming he still doesn't know enough to take a firm stand.

"The attorney general has only read about the project. He hasn't seen the design or received any specific information about it," said Spitzer spokesman Darren Dopp. "He's hesitant to comment without having studied the overall plan."