A flawed intelligence assessment generated with the help of artificial intelligence reportedly brought the US military close to intercepting a Chinese vessel in the Middle East during the war with Iran, raising concerns over the risks of relying on AI for military decision-making. According to reports from CNN, a US military intelligence report circulated this spring claimed that a Chinese ship in the region was carrying components linked to a nuclear weapons programme. The document was prepared by an analyst with the US Special Operations Command using an artificial intelligence chatbot, which incorrectly identified the nature of the vessel's cargo despite combining open-source information with classified signals intelligence. The analyst, working with intelligence reporting on the ship's manifest originating from US Special Operations Command Pacific in Hawaii, queried a chatbot about the information before using AI again to turn the findings into a standard intelligence report. As reported by CNN, the chatbot combined open-source intelligence with secret signals intelligence from government holdings before reaching its erroneous conclusion about the ship's cargo, though it remains unclear what cargo the algorithm mistakenly identified. A former senior US official familiar with AI systems used by military and intelligence analysts told CNN that "The internal tools are mostly just copies of the commercial stuff wearing lipstick."
The intelligence report triggered preparations for an operation to intercept the vessel, with military forces taking concrete steps toward action. As reported by CNN, four sources familiar with the situation said that US military personnel began preparing to intercept the ship, with two sources confirming that armed service members were preparing to board it, while military aircraft were already in the air as the operation was being planned. The preparations indicate the seriousness with which the US military was treating the intelligence assessment, with officials conducting an additional review just before the operation began. According to CNN, the analyst's use of AI was not limited to the initial assessment - the analyst then used AI a second time to turn the findings into a standard intelligence report, which was subsequently distributed to military officials. The document was circulated among military officials, who treated it as a routine, vetted assessment, highlighting how flawed AI intelligence can be mistakenly accepted as legitimate military intelligence.
The operation was halted only after officials took a closer look at the intelligence report and discovered that it had been produced with assistance from an AI chatbot. According to CNN reports, the AI-generated intelligence report was found to be flawed, leading to the cancellation of the interception operation. One source described the report as "entirely false" and said it "almost started a war," highlighting how artificial intelligence systems can inadvertently escalate tensions between nations. As reported by CNN, officials only uncovered the problem shortly before the operation was due to begin, when they examined the intelligence more closely and discovered that a Special Operations Command analyst had used an AI chatbot to produce it. The same source noted that a US military operation against a Chinese ship could have escalated into an armed confrontation between Washington and Beijing. The incident demonstrates how flawed AI intelligence can bring military forces to the brink of confrontation with other nations, particularly when such outputs are used in military targeting and accepted as routine intelligence assessments.
The reported incident has renewed concerns about AI "hallucinations," instances in which AI systems generate inaccurate or unsupported information, particularly when such outputs are used in military targeting. As reported by CNN, the near-interception of the Chinese vessel demonstrates how flawed AI intelligence can bring military forces to the brink of confrontation with other nations. A former senior US official familiar with AI systems used by military and intelligence analysts told CNN that "AI in targeting is definitely something that is ramping up and there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide." Intelligence officials have warned that inaccurate or corrupted data could cause AI systems to produce conclusions that lead to catastrophic decisions, including an attack on a Chinese vessel based on faulty information. According to CNN, the type of AI "hallucination" involved in the incident has not been an isolated occurrence within the intelligence community since such systems began spreading throughout the government. The danger is not limited to autonomous actions by algorithms, but equally serious risks arise when people make decisions based on inaccurate or misleading data generated by automated systems.
The incident comes as the US military is rapidly expanding its use of AI, with Defense Secretary Pete Hegseth announcing the "Artificial Intelligence Acceleration Strategy" in January, which aims to speed up AI use across military operations, intelligence and other work. As reported by CNN, the strategy calls for AI tools to be made available across the department, but the Chinese ship incident has highlighted one of the risks of this faster rollout: inaccurate AI-generated information reaching officials who may use it to make decisions during military operations. The Pentagon has expanded secure generative AI access to about three million defense workers, adding military versions of ChatGPT and Grok to the GenAI.mil portal. The US military and intelligence agencies are actively adopting artificial intelligence to analyze vast amounts of data, select targets, plan troop movements, manage budgets, and oversee logistics and supply chains. One source familiar with military policies noted that "AI in targeting is definitely something that is ramping up and there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide," while another source summed up the concern more simply: "AI allows you to get to a bad idea faster."
