The blast in the town of Damboa, around 90 km (56 miles) south of the state capital Maiduguri, came just hours after a suspected suicide bomber detonated explosives outside the national police headquarters in the capital Abuja.
“The children were playing in a compound near the church when the blast went off, killing the four and wounding another,” Borno state police commissioner Jinjiri Abubakar told Reuters, adding that the blast appeared to have been caused by a bomb.
Boko Haram, a radical Islamist group which wants a wider application of strict Islamic law across northern Nigeria, is suspected of being behind almost daily attacks in Borno state for several months.
Police also said the group was believed to have been responsible for the attack on their headquarters in Abuja on Thursday, which killed several people.