A statement confirmed the move, which has been backed by Afghan President Hamid Karzai.
Both the US and Germany have been pushing for such a representation in an effort to kick-start negotiations.
The office is seen by some as a key step towards ending the 10-year-long conflict in Afghanistan.
The move was welcomed by the Afghan High Peace Council, which is seeking a negotiated end to the war, as “a gesture of good faith”.
But it still remains unclear if the insurgents, who claim to be winning the war, are prepared to engage in truly meaningful peace talks – and whether they could take place while international forces continue to kill Taliban fighters and commanders.
The establishment of an office is thought by some to be a critical step in reaching a political settlement to the 10-year long conflict. It would give the group an address where negotiators could meet. Establishing the authenticity of would-be negotiators from the Taliban has been a problem in the past.