But Sultani Makenga said 100 fighters would remain at Goma’s airport.
His spokesman has told Reuters news agency that a handover ceremony will be held in the strategic city on Friday.
At a crisis meeting over the weekend, regional leaders said the fighters must leave Goma before any negotiations could be held with them.
Since the M23 rebels mutinied and deserted from the army in April, some 500,000 people have fled their homes in the ensuing unrest.
Both Uganda and Rwanda strongly deny UN accusations that they are backing the M23.
Correspondents say it has been a confusing couple of days in Goma, with contradictory comments from the M23.
On Tuesday it was reported after talks in Uganda that Brig Gen Makenga had agreed to withdraw his forces to a 20km (12-mile) buffer zone around Goma.
At the same time the movement’s political leader said this would not happen unless Congolese army troops in the region disarmed – and he also gave a long list of other conditions, which included the release of political prisoners.
However, Brig Gen Makenga said on Wednesday that he had now ordered his fighters to gradually withdraw, first from smaller towns around Goma captured in the last week.
They would move to nearby Kibati, about 20km north of Goma, in the coming days, he said.
“We will leave one company of M23 here at the airport… 100 military [fighters],” the rebel commander said.
There are reports that the movement’s political wing may also remain in the city.
M23 spokesman Lt Col Vianney Kazarama told Reuters that a handover ceremony would take place on Friday in Goma, where a UN peacekeeping contingent is based.
Following the rebel capture of Goma, the UN has warned of a growing humanitarian crisis in the mineral-rich region.
Aid officials said the fighting has made camps for people displaced by earlier conflicts inaccessible, with food and medicines running short.
Some five million people died during the 1997-2003 DR Congo conflict, which drew in several regional countries.