Last week’s fare rises prompted protests from his Trinamul Congress party, a key ally of the federal government.
His party wants him to roll back the increase, but he refused to do so.
India’s state-owned railway operates 7,000 passenger trains and carries some 13 million passengers daily.
Mr Trivedi raised the fares in Wednesday’s railway budget by a maximum of 30 paisa ($0.006; £0.0038) per km, saying the network was passing through a “difficult phase”.
However, within hours of the announcement, Mamata Banerjee, the chief of the Trinamul Congress party, rejected the increase and sought the resignation of the minister.
Mr Trivedi stepped down on Sunday evening after speaking to Ms Banerjee.
He said that the Trinamul Congress party had been instrumental in making him a minister and accepted that he “must obey the party’s discipline”.
Ms Banerjee has already asked Prime Minister Manmohan Singh to replace Mr Trivedi with another party MP, Mukul Roy.
Mr Trivedi had said he had been advised to impose a big fare increase as there had been none in nearly a decade, but had opted for a modest rise instead, out of “concern for the common man”.
Railways officials say the jump in fuel prices over the last eight years has badly affected their bottom line