The officers, who said they had taken over the island nation two days ago, are holed up in the barracks near the airport of the capital Antananarivo.
Some of them were behind the coup that brought President Andry Rajoelina to power last year.
The unrest came on the day of a referendum on a new constitution.
The vote is seen as a way of legitimising Mr Rajoelina’s rule of Madagascar, which has been beset by instability for several years.
He has been diplomatically isolated since coming to power in March 2009 and has ignored attempts by regional mediators to broker a consensus with the opposition.
But it’s not surprising they may be taking some action now because this referendum on the proposed new constitution is the last chance really to challenge Andry Rajoelina’s right to be in power. If people approve the constitution then he will have a larger amount of legitimacy than he has up to now.
It’s unclear how big this military rift is – what’s worth noting is that since Mr Rajoelina has been in power he has been giving important posts in the government to members of the military.
Traditionally the Malagasy military has stayed out of political trouble – when in 2002 the then President Marc Ravalomanana and former President Didier Ratsiraka were fighting it out, the military stayed neutral in that – so them taking part in the recent coup was actually quite a surprise.
Correspondents say roadblocks have been set up in the surrounding area and schools have been evacuated but ministers claim that negotiations with the mutineers have begun.
On Wednesday, one of the coup bid leaders, Col Charles Andrianasoavina, said Malagasies were tired of waiting for a resolution to the political crisis that has seen the country lose much of its foreign aid.
If the new constitution is passed it would allow Mr Rajoelina to stay in power as long as it takes to organise an election.
It would also lower the age limit for presidential candidates from 40 to 35 years, meaning the 36-year-old to stand if he wanted.
Mr Rajoelina, who has said he will not run for president, organised huge rallies in support of a yes vote.
He is a former DJ and mayor of the capital city and rose to power on a wave of popular support.
But some analysts say his failure to end leadership squabbles has eroded some of his popularity.
In the run-up to the referendum there were many demonstrations against it – and all three of the main opposition groups, each led by an ex-president, called for a boycott.
Source: BBC