"It will be an honour and a privilege to serve this great international institution," Sharma said at a news conference outside the Ugandan capital Kampala where the 53-nation grouping is holding its biennial summit.
On Thursday, Commonwealth foreign ministers overcame objections from some members such as Sri Lanka to suspend India's arch rival Pakistan for failing to restore democratic freedoms.
The loose federation of mostly British former colonies represents nearly a third of the world's population from some of its poorest to some of its richest. Sharma, who will take up his post on April 1, 2008, is the first secretary general from an Asian Commonwealth country in more than 40 years.
The appointment is also timely, as India - the largest member state in the 53-nation grouping - prepares to host the next Commonwealth Games in 2010. Leaders at the summit, which began on Friday and runs until Sunday, were discussing issues such as global trade and climate change ahead of next month's crunch international climate change conference in Bali.