Panetta, speaking to an audience of military officers at the National Defence University, said relations with Pakistan were difficult because elements of the government had links with the Haqqani network, which is staging attacks on US-led troops in neighbouring Afghanistan, and Lashkar-e-Taiba (LeT), blamed for orchestrating attacks in India.
"What makes this complicated is that they have relationships with the Haqqanis, and the Haqqani tribe are going across the border and attacking our forces in Afghanistan," he said at the Washington institution, which is tasked with providing training in national and international security for US and foreign officials.
"It's pretty clear that there's a relationship there," said Panetta, who appeared along with Secretary of State Hillary Clinton at the event. "There's a relationship with LeT. And, you know, this is a group that goes into India and threatens attacks there. It has conducted attacks there," said Panetta, who recently stepped down as CIA director to take over at the Pentagon.
The Pakistanis were also sometimes withholding visas for some US embassy personnel assigned to the country, he added. But he said there were vital interests at stake that meant Washington had to stay committed to the partnership. "And yet there is no choice but to maintain a relationship with Pakistan. Why? Because we're fighting a war there.
"Because we are fighting al Qaeda there, and they (Pakistanis) do give us, you know, some co-operation in that effort," Panetta said. "They do represent an important force in that region. Because they do happen to be a nuclear power that has nuclear weapons and we have to be concerned about what happens with those nuclear weapons."
He added that there were going to be "ups and downs" in the relationship with Pakistan. Asked about reports that Islamabad likely allowed Chinese engineers to examine a US helicopter that crash landed during the raid that killed Osama bin Laden in his Pakistani hideout in May, the US defence chief declined to confirm or deny the accounts. "I'm not going to comment because it does relate to classified intelligence," Panetta said.