Witnesses said police beat those in the camp with truncheons after officials had failed to persuade them to board buses to move them from an affluent part of Cairo to another site.
The Interior Ministry said the Sudanese died in a stampede at the camp, where 3,500 Sudanese live in squalid conditions. It said 75 police officers were injured.
Pools of blood were visible on the pavement as men in the camp fought back with sticks and hurled bottles at the police, who fired water cannon to try to disperse them, witnesses said.
Police swept into the camp to break up a three-month sit-in to demand that they be moved to another country.
An Interior Ministry source said 20 protesters died and a refugee spokesman said the toll could be higher. The Health Ministry said 50 Sudanese were injured. The figures could not immediately be confirmed.
Egyptian television showed footage of several injured policemen in a hospital.
About 4,000 police ringed the site, near the offices of the UN refugee agency, where the Sudanese were protesting what they said was poor treatment since they fled Sudan's lengthy civil war.
The UN High Commissioner for Refugees (UNHCR) has said it is prepared to help Sudanese in Egypt but cannot arrange for all of them to resettle in another country because many are looking for a better life and are not refugees fleeing a conflict.
Reuters witnesses said there were about six unconscious Sudanese, some of them young children, lying on the ground.
A doctor who examined a girl aged about four who was brought to him after being found unconscious said: "She's dead."
The UNHCR called the deaths a tragedy but a Sudanese official said security forces were entitled to end the sit-in at the camp.
"There is no justification for such violence and loss of life," High Commissioner Antonio Guterres said in a statement.
A UNHCR spokesman said the agency had urged Egyptian authorities to deal with the situation peacefully. Egyptian police had not told UNHCR officials they would try to move the protesters on Friday, Astrid Van Genderen Stort said.
"The Egyptian government was within its rights to re-establish its control," said Sudanese presidential adviser Mahjoub Fadl in comments carried by Egypt's official Middle East News Agency (MENA).
The protesters said they wanted the UNHCR to arrange for them to be flown out of Egypt.
"Most Sudanese refugees have been subjected to violence in Egypt. We don't want to be here any more," said one Sudanese protester who gave his name as Wilson.
Hundreds of Sudanese picked up by the police were being held in two camps run by the security forces, who were checking their identities, representatives from among the protesters said.
Sudan's two-decade north-south civil war made 4 million people homeless and a separate conflict in the western Darfur region has produced a further 2 million refugees.
A January peace agreement ended the north-south civil war but many Sudanese say it is not safe to return home as the deal is fragile.