The International Crisis Group said ”conflicting mandates” and ”lack of co-ordination” among Chinese agencies had stoked tension in the region.
Philippine and Chinese vessels remain locked in a two-week stand-off at a remote shoal which both sides claim.
Meanwhile annual joint military exercises between the US and the Philippines are continuing in the area.
While the South China Sea conflict has been a longstanding one, the number of maritime disputes has ”increased dramatically” in recent years, the report by the think-tank notes.
This has led to ”concerns that China, Vietnam and the Philippines are growing more assertive on this issue, endangering regional stability”, it says.
In China, many government agencies are using this issue to bolster their power and budget, it says.
Repeated proposals for ”a more centralised mechanism” have not been met, it adds, and the foreign ministry ”does not have the authority or resources to manage other actors”.
”More immediate conflict risks lie in the growing number of law enforcement and paramilitary vessels playing an increasing role in disputed territories without a clear legal framework,” it says.
Six countries claim competing sovereignty over areas in the South China Sea, parts of which are believed to contain significant deposits of oil and gas.
Along with China and the Philippines, they are Brunei, Malaysia, Vietnam and Taiwan.
China’s claim includes almost the entire South China Sea, well into what the UN Convention on the Law of the Sea recognises as the 200-mile-from-shore Exclusive Economic Zones of other claimants.
That has led to occasional flare-ups and to competition to occupy islands, reefs and sandbars.
With China asserting its claims more aggressively the US has been strengthening old friendships in the region, says the BBC’s John Sudworth reporting from the South China Sea on the exercises.
This week, three US ships are also in Vietnam for a five-day naval exchange involving salvage and disaster training. No live fire drills are planned.