Two American tanks remained in position on the edge of the Old City.
According to a five-point peace plan drawn up by top Shiite cleric Grand Ayatollah Ali al-Sistani and signed by Sadr late Thursday, all armed elements must disarm and leave Najaf and Kufa, never to return.
Iraqi police would assume responsibility for law enforcement and foreign troops, in this case, the Americans, must withdraw.
Militiamen began secreting their weapons in Najaf early Friday following a call from Sadr to disarm. No armed Mehdi Army fighters were visible on the streets of either Najaf or its twin city of Kufa by late afternoon.
Sistani was granted a 24-hour cease fire to broker a truce to more than three weeks of fighting between Sadr's Mehdi Army and US-led Iraqi government troops, reports AFP.
The Reuters adds that Shiite fighters also left the holiest shrine in the Iraqi city of Najaf on Friday and began turning in their weapons, after tens of thousands of pilgrims celebrated a peace agreement that ended a bloody rebellion.
"We pray today that Najaf will recover. The military operations have only brought destruction," said Kassem Hameed, a 52-year-old oil worker from the southern city of Basra.
Sistani arrived in Iraq on Wednesday after three weeks in London for heart treatment. The uprising had erupted as he left his adopted home in Najaf.
Religious authorities locked the doors of the Imam Ali mosque after the Mehdi Army militia of radical cleric Moqtada al-Sadr left. The fighters had defied US military firepower and the interim Iraqi government for three weeks.
Iraq's most revered cleric, Grand Ayatollah Ali al-Sistani, made a dramatic return to Najaf on Thursday and persuaded Sadr to accept a peace deal to halt the fighting, after a day of violence in which 110 Iraqis were killed and 501 wounded.
Militants tossed AK-47 assault rifles and mortar launchers into wooden carts being pushed around near the shrine. Mosque loudspeaker announcements in Sadr's name gave the order.
Meanwhile, the Al Arabiya television said Sadr's representatives had handed over the keys to the mosque, Iraq's holiest Shiite shrine.
A Reuters correspondent there said Iraqi police took control of the area around the mosque, as envisaged under the deal.
Several Mehdi militants refused to give up their guns while some US troops, who were supposed to leave the southern city in line with the peace deal, were seen nearby.
By mid-afternoon, the narrow streets around the mosque were relatively quiet, destroyed and blackened buildings, a testament to the fierce fighting that killed hundreds and drove world oil prices to record highs.
"We will support whatever Ayatollah Sistani and Sayyed Moqtada have agreed. But we will still slit the throats of the Americans," said one militiaman, Hussein Taama.
"I will keep this warm and wait for Sayyed Moqtada's order", said another militant holding an AK-47 rifle which he said was his personal weapon that he would not give up.
The Najaf uprising has been a stark reminder to the interim government and the United States, which led the war to depose Saddam Hussein last year, of the huge hurdles ahead in Iraq.
While fighting may have ended for now in Najaf, elsewhere it had not. Insurgents attacked US troops with hand grenades in Baghdad on Friday, wounding 12 soldiers, the US military said.
President George W. Bush acknowledged for the first time on Thursday he had erred over post-war conditions in Iraq, the New York Times reported. It quoted him as saying in an interview that he made "a miscalculation of what the conditions would be".