A heavy contingent of police, paramilitary rangers and soldiers was deployed in major cities including Lahore, Karachi, Peshawar, Quetta and Multan. In Rawalpindi, Maulana Ahmad Ludhianvi, who leads Ahle Sunnat Wal Jamaat (ASWJ) organisation said: "We know how to fight against enemies of Islam and we are fighting against them."
In Quetta, up to 2,000 activists gathered to chant anti-Shia slogans.
Local ASWJ leader Ramzan Mengal asked the government to declare Pakistan a Sunni state.
Similar scenes were seen in Peshawar, where some 4,000 people answered ASWJ's call at the Shobha Bazaar, while protesters in Lahore called for the government to restrict Shiite processions and confine them to their mosques.
In the largest city Karachi, 15,000 people attended an ASWJ rally, shouting anti-Shia slogans.
All shops, restaurants and petrol stations in the city remained closed and roads were deserted amid a heavy deployment of police and paramilitary troops, an AFP reporter said.
Police have used shipping containers to block certain roads in the capital Islamabad and Rawalpindi, while the approaches to the diplomatic enclave, which houses foreign embassies, were sealed.
Umar Hayat Lalika, regional police chief for Rawalpindi told reporters that gatherings in the city have been banned and police would stop any attempts to hold rallies. He said police had arrested 24 suspects including a few police officials, adding they were being interrogated.