AI can sound certain and still be wrong. That's the bit businesses need to remember.
Keep up with IT Club
Add IT Club as a preferred source in Google Search.
CNN reported that an AI-assisted US intelligence assessment misidentified cargo on a Chinese vessel before a planned operation was halted. The chronology is not independently corroborated, the true cargo is unknown and “almost started a war” is a source’s quote—not a verified finding. The business lesson is straightforward: the higher the consequence, the stronger the verification.
The practical lesson
AI can produce a polished, confident answer without knowing whether the consequences of being wrong are trivial or severe. Treat an answer as a starting point for judgement, not as proof.
- Name a human owner for consequential decisions.
- Check important claims against the original evidence, not another AI summary.
- Record what was checked and who approved the final action.
The reported incident
CNN reported on 18 September 2026 that, during the war with Iran, a US intelligence report circulated saying a Chinese ship in the Middle East was carrying components of a nuclear-weapons programme. CNN’s unnamed sources said the US military planned to intercept the vessel; two sources said armed personnel were preparing to board and two said military aircraft were airborne.
CNN said officials examined the report shortly before the planned operation and found that it had been prepared with AI assistance. A chatbot had inaccurately identified the cargo. The analyst had reportedly supplied intelligence about the ship’s manifest, then used AI again to package the result into a standard intelligence report.
The ship’s true cargo is unknown. CNN says it could not determine what the chatbot was, whether it was commercial or government-built, or independently verify the operational chronology. The Pentagon and US Special Operations Command Pacific did not respond to CNN’s requests for comment.
One source described the report as “entirely false” and said it “almost started a war”. Those are attributed source statements, not independently verified findings. The safe description is an alleged AI-assisted intelligence near-miss—not that AI tried to start a war.
Why confident output is dangerous
A model can answer “give me five ideas for Friday lunch” and “does this evidence prove our customer committed fraud?” in the same fluent style. It does not inherently supply the business judgement that separates those questions. Humans must provide that distinction through process, access controls and review.
In an SME, the same failure can look less dramatic but still be expensive: a customer is wrongly marked as owing money, a contract clause is invented, a security alert is misread, a supplier is called compliant without supporting evidence, or a financial figure is accepted because it looks professional.
Use proportional verification
The bigger the consequence, the stronger the verification. Brainstorming and first drafts may need a light review. Research, supplier comparisons and customer communications need source checks. Payments, contractual decisions, security changes, employee decisions, regulated advice and destructive IT actions need independent confirmation and an accountable human approver.
- 1Classify the decision before using AI: low, moderate or high consequence.
- 2For important claims, open the original document, record or system rather than trusting a generated summary.
- 3Require a second person or suitable specialist review before high-impact action.
- 4Keep an audit trail showing the evidence, reviewer and final decision.
- 5Limit the model’s permissions so a wrong answer cannot directly trigger an irreversible action.
Verification is not the same as rejecting AI. It can still be useful for research, analysis, identifying possibilities, challenging assumptions and producing first drafts. The control is knowing when its output needs independent confirmation.
Plain-English takeaway
Do not ask only, “Do we trust AI?” Ask, “What level of verification does this particular decision require?” AI can help make a decision; it cannot turn confidence into evidence.
Sources and further reading
CNN — US military had close call after using AI for false intelligence report (18 September 2026) →
The Independent — account of the reported incident (based on CNN’s reporting) →
Related IT Club article — Your AI Agent Did Something Illegal. Who Is Responsible? →
Plain-English Takeaway
AI can help you make a decision, but confidence is not evidence. Match the strength of your verification to the consequence of being wrong.
Frequently asked questions
Did AI nearly start a war?
That is not an established fact. CNN reported that one unnamed source said an AI-assisted report “almost started a war”. The report’s chronology has not been independently corroborated, and the story does not establish that AI acted independently or that war was imminent.
What does the reported incident actually establish?
CNN says unnamed sources described an AI-assisted assessment that inaccurately identified cargo on a Chinese vessel, followed by preparations to intercept it before the assessment was checked more closely. CNN could not establish the vessel’s true cargo or whether the chatbot was commercial or government-built.
How should a small business verify AI output?
Use stronger, independent checks as the stakes rise. A draft can receive a quick sense-check; a payment, contractual decision, security change, employee decision or regulated communication needs a named human owner and evidence from an appropriate source.
Related Articles
Your watch is becoming an AI meeting assistant. Does everyone else know?
Apple's new Audio Intelligence features show why workplace rules need to cover what a wearable can hear, process and summarise, not just what it looks like.
Read articleYour AI Did Exactly What You Asked. Just Not How You Expected.
If an AI agent has an objective, credentials and access to real systems, businesses should not assume it will stay within the boundaries they had in mind.
Read articleAI Literacy for Small Businesses: More Than Better Prompts
AI literacy is not just knowing how to write a clever prompt. It is knowing what an AI system can do, where it can fail, what information it may handle and when a person must take responsibility.
Read article