Why AI Output Still Needs Human Review
IT Club provides practical technology guidance, not legal advice. Laws, contractual obligations and regulatory requirements vary according to the organisation, sector, data, location and use case. Obtain appropriate legal, data-protection, employment or regulatory advice where required.
AI systems are trained to produce plausible text, not verified truth. They can invent sources, misquote real ones, produce confident arithmetic errors and give different answers to the same question. None of that makes them useless — it makes review a design decision. This guide sets out a risk-based way to decide how much checking each task needs.
The ways AI output goes wrong
- Hallucinations — fluent statements with no basis in any source
- Outdated information presented as current
- Plausible errors — almost-right details that survive a casual read
- Missing context — technically true answers that mislead in your situation
- Mathematical errors, including in simple percentage and currency work
- Invented sources, citations and case references
- False or misattributed quotations
- Biased output reflecting patterns in training data
- Overconfident wording that hides uncertainty
- Incomplete analysis that quietly drops half the question
- Inconsistent answers to the same prompt asked twice
Match the review to the risk
| Level | Applies to | Review required |
|---|---|---|
| Low-risk review | Internal drafts, brainstorming, formatting, tone | Read for spelling, tone and formatting before use |
| Normal business review | Customer communications, published content, analysis | Check accuracy, context, sources and suitability; verify every factual claim |
| High-risk review | Legal, financial, health, employment, safety, security and regulated decisions | Qualified human review, with the AI output treated as a starting draft only |
The rule of proportion
The more serious the consequence of an error, the less appropriate it is to rely on AI output without qualified human review.
A six-point verification method
- 1SOURCE — Can the underlying source be opened? A citation that cannot be opened and read is not a source.
- 2DATE — Is it current? Guidance, prices, law and product features all change; check the publication date, not the model's confidence.
- 3AUTHORITY — Is the source competent and authoritative for this claim? A forum post and a regulator's guidance are not equivalent.
- 4CONTEXT — Does it apply to this organisation and situation? Rules differ by country, sector, company size and contract.
- 5CALCULATION — Have figures been checked independently? Recalculate anything that feeds a decision, using a calculator or spreadsheet.
- 6APPROVAL — Who is accountable for final use? Every AI-assisted output that leaves the business should have a named human approver.
Making review stick in practice
Review fails when it is nobody's job. Build it into the workflow: templates that include a “checked by” line, a rule that AI-drafted customer messages are read in full before sending, and a habit of asking the AI itself to list its assumptions and the claims that need verification — which makes checking faster, though never optional. Treat every AI answer as a draft from a fast, well-read, occasionally overconfident junior colleague: useful, but not sign-off.
Plain-English Takeaway
Decide the review level before the task, not after: light checks for low-stakes drafting, full source-checking for anything factual or customer-facing, and qualified human review wherever an error could affect money, health, employment, safety or legal position.
Sources and further reading
- NCSC — Thinking about the security of AI systems
- ICO — Explaining decisions made with AI
- GOV.UK — Introduction to AI assurance
External guidance changes. Check the source itself for the current position before acting on it.