It is the first major demonstration on the streets of Addis Ababa since 2005 when hundreds of protesters were killed in violence.
Demonstrators shouted slogans calling for freedom and justice.
Ethiopia’s governing party, the EPRDF, maintains strict control over public life in the East African country.
The protests were organised by the opposition Semayawi (blue) party.
Party chairman Yilekal Getachew told Reuters that as well as the release of the prisoners, the demonstrators also wanted action on unemployment, inflation and corruption in Ethiopia.
Ethiopia’s Prime Minister Hailemariam Desalegn took office in September 2012 following the death of Meles Zenawi.
Tolerating dissent?
The BBC’s Africa analyst Richard Hamilton says the fact that the rally was given permission to be held at all, and passed off peacefully, suggests the new prime minister may be more tolerant of dissent than his predecessor.
Human rights groups argue that the Ethiopian government has used a 2009 anti-terrorism law as a pretext to imprison members of the opposition and the press.
The Committee to Protect Journalists says Ethiopia is close to replacing Eritrea as the African country with the most journalists behind bars.