Learn
AI hallucination examples: the incidents that made case law
Hallucination stops being an abstract model property the day it produces a court ruling, a sanctions order, or a nine-figure market reaction. The incidents below are public, documented, and instructive – because each one failed in a way a specific boundary check is designed to catch.
A pattern to notice before the examples: in none of these cases was the model broken. Every system produced fluent, confident, well-formatted output. The failure was that nothing stood between that output and the people who relied on it.
Four well-documented cases, spanning customer service, law, marketing, and consulting:
Air Canada’s chatbot (2024)
The airline’s support chatbot invented a bereavement-fare policy – telling a passenger he could apply for the discount retroactively, contradicting the airline’s actual policy on the same website. A British Columbia tribunal ordered Air Canada to honor the hallucinated policy and pay damages, rejecting the argument that the chatbot was “a separate legal entity responsible for its own actions.” The precedent: your bot’s words are your words.
Mata v. Avianca (2023)
A New York lawyer submitted a brief citing six judicial opinions that did not exist – complete with plausible names, docket numbers, and quotations, all generated by ChatGPT. The court sanctioned the lawyers and put every filing attorney in America on notice. Fabricated references remain the most common professional-use hallucination.
Google Bard’s launch demo (2023)
In its first public promo, Bard claimed the James Webb Space Telescope took the very first picture of an exoplanet – it did not (that happened in 2004). Alphabet’s shares fell roughly 9% the day the error made headlines. One unverified factual claim, in one ad, at maximum visibility.
The Deloitte Australia report (2025)
A consulting report delivered to the Australian government was found to contain fabricated academic citations and a made-up court quotation, later attributed to generative-AI-assisted drafting. Deloitte agreed to refund part of the contract. The revision-and-review failure mode: AI-assisted text entered a professional deliverable without claim-level verification.
Each incident maps to a class of check that operates on the output, not on trust in the model. A policy answer that contradicts the airline’s own published fare rules is a groundedness failure – checkable against the source. Six invented cases are fabricated references – checkable against the material actually provided or retrieved. The JWST claim was false by common, verifiable knowledge – exactly what a fact-check judge with web grounding flags. And the consulting report is the revision problem: every updated draft needs its claims re-verified, which only happens reliably when checking is on the wire rather than in a workflow someone must remember to run.
The common thread: none of these organizations lacked smart people or review processes. They lacked a layer that checks every answer, every time, before it ships – which is precisely the job of an output guardrail.
On the wire, the layers stack cheapest-first. Deterministic grounding matches every figure and identifier in the answer against the source material – the Air Canada policy contradiction and any altered number fall here, at zero model cost. A fact-check judge flags claims the provided material cannot support – the fabricated citations and the unhedged JWST claim fall here. External grounding (retrieval over your documents, live web search, a second-model critic) covers claims that need outside evidence. In PREVENT mode the failing answer never reaches the user; in FIX mode it ships flagged, with a grounding report showing exactly which claims failed and why.
What is the most famous AI hallucination incident?
The two most cited are Air Canada’s chatbot inventing a bereavement-fare policy – which a tribunal forced the airline to honor in 2024 – and Mata v. Avianca (2023), where a lawyer filed six ChatGPT-fabricated case citations and was sanctioned.
Are companies legally liable for AI hallucinations?
The Air Canada tribunal said yes for chatbot statements on a company’s own site: the company is responsible for all information on its website, interactive or not. Liability elsewhere is still developing, but the direction is consistent – the deploying organization owns the output.
Could these incidents have been prevented?
Each maps to a standard boundary check: grounding against the airline’s actual policy documents, reference verification against real sources, fact-checking against common knowledge, and re-verification of revised drafts. None requires a better model – they require checks the model’s output must pass.
Do newer models hallucinate less?
Rates improve, but no model reaches zero – and teams simultaneously route more traffic to cheaper models where rates are higher. That is why the durable fix is verification at the boundary rather than hoping the next model generation retires the problem.
See the checks run on your own traffic.
One base-URL change puts every request behind the guardrail pipeline. We’re running a limited demo – sign up and we’ll get you in as soon as we can.