The seven-carriage train pulled into a new station in the city after travelling the 75-km (47-mile) route without any cargo.
The line was completed last year at a cost of $165 million (£105m).
The authorities hope to open up new trade routes and link Afghanistan to its neighbours’ rail networks.
The new line could also become a key supply route for Nato forces in Afghanistan and help take military equipment out when the international troops withdraw, starting in 2012.
The US has been shifting its supply lines into the north and away from the volatile route from Pakistan.
The first journey on Wednesday was intended to test the track and signals, before the formal opening of the project at which President Hamid Karzai is expected to attend.
“This is a matter of pride for us and a very important issue for Afghanistan,” said Deputy Public Works Minister Noor Gul Mangal, who was there to watch the train arrive in Mazar-e-Sharif.
He said the government planned to build another line into Turkmenistan, to the north-west.