The militants, led by the jihadist Islamic State of Iraq and the Levant (ISIL), seized the towns of Rawa and Ana after taking the Al-Qaim border crossing on Saturday, residents said. They then gunned down 21 local notables in Rawa and Ana in two days of violence, according to officers and doctors.
The government said its forces had made a "tactical" withdrawal from the towns, control of which allows the militants to open a strategic route to neighbouring Syria where they also hold swathes of countryside along the Euphrates river valley.
ISIL aims to create an Islamic state incorporating both Iraq and Syria, where the group has become a major force in the rebellion against President Bashar al-Assad.
Washington wants Arab states to bring pressure on Iraq's leaders to speed up government formation, which has made little headway since April elections.
While American leaders have stopped short of calling for Prime Minister Nuri al-Maliki to step down - arguing it is up to Iraqis to choose their own leaders - they have left little doubt that they feel the Shia premier has squandered the opportunity to rebuild his country since US troops withdrew in 2011.
"We gave Iraq the chance to have an inclusive democracy. To work across sectarian lines, to provide a better future for their children," President Barack Obama told CNN Friday.
"Unfortunately what we've seen is a breakdown of trust."
The seizure of Al-Qaim leaves just one of three official border crossings with Syria in federal government hands. The third is controlled by Kurdish forces.
Anti-government fighters already hold areas of the western desert province of Anbar which abuts the Syrian border, after taking all of one city and parts of another earlier in the year.
Elsewhere, government forces launched an air strike on the militant-held city of Tikrit, killing at least seven people, residents said, as the defence ministry announced air strikes on the northern city of Mosul.
The insurgents also clashed with security forces and pro-government tribal fighters in Al-Alam east of Tikrit, with militants killing the women's affairs adviser to the provincial governor.
The fighting came as US Secretary of State John Kerry arrived in Cairo on a trip to the Middle East and Europe, with Washington aiming to unite Iraq's fractious leaders and repel the militants.
"We must urge Iraq's leaders to rise above sectarian considerations... and speak to all people," Kerry said in Cairo, while also adding that Washington is not responsible for the current crisis.
Kerry later travelled to Jordan, and will also visit Brussels and Paris, where Washington is also expected to push for greater efforts to cut off funding to ISIL.
"First and foremost, we are urging countries that have diplomatic dealings with Iraq and that are in the region to take that threat as seriously as we do," a senior State Department official said.
"Second, we are underscoring the need for Iraqi leaders to expedite their government formation process and to come together around a new government that is inclusive."
The official, speaking on condition of anonymity, also noted that "a lot of the funding and support that has over a long period of time fuelled extremism inside Iraq has flowed into Iraq from its neighbours."
While Kerry is also expected to travel to Iraq for his second visit since taking over as secretary of state in early 2013, it was not known when he would do so.
Washington initially favoured Maliki when he first became premier in 2006 as he was seen to be cracking down on Shia militias while reaching out to Sunni leaders. But he has since made what critics say are increasingly sectarian moves, triggering US calls for him to represent all Iraqis, particularly minority Sunni Arabs and Kurds.
Obama has offered to send up to 300 military advisers to Iraq, but has so far not backed air strikes as requested by Baghdad.
UN aid agencies are rushing supplies to Iraq to help more than one million displaced people.