At least 47,000 personnel will take part in the biennial Keen Sword exercise, which runs until 16 November.
It is taking place off Okinawa, north of islands both Japan and China claim.
The two sides are locked in a diplomatic row over the islands, known as Senkaku in Japan and and Diaoyu in China.
Japan controls the islands, which Taiwan also claims. The Japanese government’s acquisition of three of the islands from their private Japanese owner in September sparked both diplomatic and public protests in China.
Since then ships from both China and Taiwan have been sailing in and out of waters around the islands.
The Japanese coast guard on Sunday reported that four Chinese ships were once again spotted in the area.
The two countries have held three rounds of talks since the row erupted, Japan’s Kyodo news agency reports, with the latest taking place in Wuhan in China on Sunday and Monday.
The two sides agreed to hold more talks at vice-ministerial level, the report said.
A US statement gave few details of the exercises but said it aimed to “improve interoperability required to effectively defend Japan or respond to a crisis in the Asia-Pacific region”.
The drills were originally supposed to have seen US and Japanese troops carrying out a mock invasion of an uninhabited island off the southern coast of Japan, says the BBC’s Rupert Wingfield-Hayes in Tokyo.
In the end, it was decided that this was a bit too provocative to China, our correspondent adds.