AI hallucinations

An AI hallucination is when an AI system—especially a chatbot—states something false, made-up, or nonsensical with total confidence. It's the defining reliability problem of the current AI era, and (fittingly for this site) a brand-new industrial-scale source of things-that-sound-true-but-aren't.

What it looks like

• Invented citations: confidently citing books, papers, court cases or articles that don't exist — a notorious failure mode (lawyers have been sanctioned for filing AI-invented case law);
• Fabricated facts: made-up dates, statistics, quotes, biographies and events, delivered as fact;
• Fake details about real things: a real person with an invented award; a real book with an invented plot;
• Confident wrongness: the hallmark — the fluent, authoritative tone is identical whether the model is right or completely inventing.

Notable real incidents

• Fake case law in a real courtroom: in the widely reported 2023 case Mata v. Avianca, a New York lawyer submitted a legal brief containing several court cases that didn't exist — invented by ChatGPT and never checked before filing. The court sanctioned the attorneys, and the case became one of the first famous examples of AI hallucination causing real professional consequences;
• Wrong in the launch demo: in February 2023, Google's Bard chatbot made a factual error about the James Webb Space Telescope during its own public promotional demo — widely reported at the time, and a reminder that no chatbot, however well-funded, is immune.

Why it happens

• They predict, they don't 'know': large language models generate the most plausible next words based on patterns in training data — they're not looking up facts in a database. Plausible and true usually overlap, but not always;
• No built-in truth check: a base model has no inherent mechanism to verify a claim against reality — fluency is not the same as accuracy;
• Gaps get filled: asked about something rare or nonexistent, the model generates something that fits the pattern rather than admitting it doesn't know — strikingly like human confabulation;
• Training data has its own errors: models learn from enormous amounts of text scraped from the real world — including its existing mistakes, jokes and misinformation, which can resurface as if they were settled fact;
• The confidence is a style, not a signal: models are trained to sound helpful and assured, so wrongness arrives dressed as expertise.

Protecting yourself

• Verify anything that matters — especially names, numbers, dates, quotes, citations, legal/medical/financial claims — against a primary source;
• Be extra suspicious of specifics: exact statistics, precise quotes and cited sources are the most common fabrications. If a citation exists, check that it exists;
• Ask for sources and then check them — don't trust that a cited link says what the AI claims (or exists at all);
• Prefer tools that show their work: systems that cite sources or search the live web are more checkable than a bare chat answer — but 'it gave me a link' still isn't the same as 'the link says what it claims';
• Treat AI as a fast, fallible assistant, not an oracle — brilliant for drafting and brainstorming, unreliable as a sole fact source;
• For anything high-stakes — legal, medical, financial or safety-related — treat an AI answer as a first draft to verify with a qualified professional, not a final answer;
• Notice your own bias: fluent, confident answers feel true — the same trap behind human misconceptions (spotting fakes).

The bigger picture

Hallucination is an active area of AI research, and newer systems (with retrieval, tool use, and better training) reduce it — but no current system eliminates it. The healthy stance is the same one this whole site preaches: confident and true are different things, whether the source is a rumor, your own memory, or a machine. See how false memories form and media literacy for the human side of the same problem.