The protesters formed a 3km-long (two miles) human chain from the parliament building to the seat of the government.
The rally was a culmination of a week of opposition protests.
Talks between the opposition and the ruling coalition led by the moderate Islamist Ennahda party have so far failed to achieve a major breakthrough.
“The government must fall,” chanted the protesters as they marched from the parliament building to the Kasbah in central Tunis.
“Ali Larayedh, the Tunisian people will not be humiliated,” they said, referring to the current prime minister.
The protest was organised by the National Salvation Front – a coalition of opposition groups.
The NSF accuses the government of failing to rein in radical Islamists and improve the economy.
Mr Larayedh’s cabinet denies the allegations. It has been pressing the opposition to accept its roadmap, which envisages legislative and presidential elections.
The Ennahda party was elected after the overthrow of President Zine el-Abidine Ben Ali in January 2011.
It has been facing mass street protests after the 25 July killing of MP Mohamed Brahmi, the leader of the small left-wing Popular Movement party.
In February, the first Islamist-led government was brought down after prominent secular opposition leader Chokri Belaid was also assassinated.
The authorities blame both killings on militant Salafists.