AI error prompts last-minute halt to US operation against Chinese vessel
U.S. officials aborted an armed operation against a Chinese vessel this spring after discovering that intelligence used to support the mission had been hallucinated by an AI chatbot. Military aircraft were already in the air when the error was identified, CNN reported. The operation was halted before it could proceed, avoiding a potential conflict with China.
The intelligence report circulated during the war with Iran and alleged that the vessel was carrying components for a nuclear weapons programme. The claim was false. Its origin was an analyst within Special Operations Command who used an AI chatbot to combine open-source information with classified signals intelligence.
An incorrect cargo manifest entered command channels
The chatbot misidentified the ship’s cargo manifest. The analyst then used the same tool again to format the erroneous finding into an official-looking summary. That document was circulated through command channels, allowing an AI-generated error to gain the appearance of formal intelligence reporting.
The episode illustrates a central difficulty in deploying large language models in sensitive workflows. Their output can be fluent and authoritative in tone even when a factual premise is wrong. In this case, the claimed cargo was directly relevant to an armed operation, making the need to establish the provenance and reliability of each assertion especially acute.
Speed must not replace verification
The U.S. military is pursuing AI integration to accelerate decision-making and preserve an advantage over China. The Pentagon has described AI as a significant advantage for speeding its kill chain so commanders can respond in the right time. Yet the same acceleration can also move untested material through a system before it is challenged by human reviewers.
Jake Steckler, a research scholar at GovAI and a U.S. Army veteran, said service members need to understand the uncertainty inherent in LLMs. He highlighted targeting, intelligence analysis and operational planning as areas where decisions can lead to the use of force and therefore carry life-and-death consequences.
Safeguards rather than abandonment
Steckler did not argue that the incident should end military use of AI. He said the tools can be useful in appropriate contexts with suitable safeguards, while warning that prioritising adoption speed above everything else could cause incidents that reduce service members’ trust and ultimately slow adoption.
For organisations introducing generative AI into consequential processes, the practical implication is clear: AI output should not become an operational fact merely because it has been formatted into a polished report. Teams need defined human review, evidence checks and escalation controls before such analysis can affect high-impact decisions.