What is Hallucination?

What is Hallucination?

Hallucination is when an AI model generates something that is factually incorrect, but presents it confidently as true.

This is arguably the most important limitation to understand about current AI. It's not a bug that's about to be fixed — it's a fundamental consequence of how these models work. Knowing about it protects you from being misled.

Why It Happens

Language models generate text by predicting the most plausible next token, given everything that came before. They're pattern-matching engines, not databases.

They don't "know" facts the way you look something up in a reference book. They've learned statistical patterns from training data. When asked about something — even something they were never trained on — they'll sometimes generate a plausible-sounding response regardless.

Models can abstain — say "I don't know" or flag uncertainty — and modern models do this more than earlier ones. But it's unreliable, and research points to part of the reason why: much post-training evaluates models like a test that rewards a guess over an admission of uncertainty, so confidently guessing often scores better than saying "I don't know" — even when the guess is wrong. That training incentive, not an inability to express uncertainty, is a big part of why hallucination persists.

This is why hallucinations are often confident, detailed, and wrong. The model isn't being deceptive — it's a product of how it was trained and evaluated, and the output happens to be false.

Common Examples

Fake citations: Ask a model to cite academic papers on a topic and it may generate convincing-looking citations — correct author names, plausible journal names, realistic years — that simply don't exist.

Incorrect facts: Ask about a historical event and it may give the wrong date, wrong people, wrong location — all presented as fact.

Confabulation: Ask about a specific person or company and it may generate plausible-sounding but fabricated details.

Code bugs: Generated code may look correct but contain subtle logic errors that only appear at runtime.

The more obscure the topic — especially things underrepresented in training data — the more likely hallucination becomes.

How to Protect Yourself

1. Verify important claims Never trust an AI's output on factual matters without checking against a reliable source. This is especially true for anything consequential: medical, legal, financial.

2. Ask for sources — then check them Models can suggest where to look, but always verify the source actually exists and says what the model claims.

3. Use AI for what it's good at Drafting, brainstorming, summarising content you provide, writing code, explaining concepts — these are lower-risk tasks. Generating facts is higher-risk.

4. Use grounding: RAG and web search Systems that ground the model in specific documents — retrieval-augmented generation (RAG), feeding the actual source material into the context — hallucinate less because the model can refer to the provided text instead of relying on memory. Web-search grounding works the same way: the model searches and reads real pages before answering, rather than generating purely from what it learned in training. Both still need verifying — grounding reduces hallucination, it doesn't eliminate it.

5. Notice confident nonsense If something sounds too neat, too specific, or suspiciously well-sourced — check it. Hallucinations often have the texture of being too authoritative.