Why LLMs hallucinate and 7 ways to reduce it
TL;DR
Models make things up because of statistical pressure in pretraining and because benchmarks reward guessing over saying 'I don't know'. Reduce it by allowing uncertainty, quoting before answering, verifying claims and keeping a human reviewer.
An LLM invents a contract clause that doesn't exist, cites a paper that was never written, or confidently states the wrong birth date for someone. This is called hallucination. It isn't a random bug that one update can patch; it's a consequence of how models are trained and evaluated. Understanding the cause helps you pick the right countermeasures instead of just telling the model "don't make things up".
Why do models make things up? The research view
The paper "Why Language Models Hallucinate" (Kalai, Nachum, Vempala, Zhang, 2025) gives two main reasons:
- From pretraining: the authors frame hallucinations as errors in a binary classification problem. If a model can't reliably tell true statements from plausible false ones, errors arise as ordinary statistical pressure, especially for facts that are rare in the data (such as the birthday of a little-known person).
- From scoring: most benchmarks grade right/wrong, so "I don't know" earns zero while a guess has a chance of being right. Models are therefore optimised to be "good test-takers": when unsure, they guess.
The paper's proposal: change how popular benchmarks are scored so they stop penalising expressions of uncertainty, rather than just adding separate hallucination tests.
Common types of hallucination
| Type | Example | Warning sign |
|---|---|---|
| Invented facts | Dates, figures, names | Very specific details with no source |
| Invented citations | Paper titles, URLs, legal document numbers | Can't be found when checked |
| Unfaithful to provided documents | A summary adds points not in the original | Can't point to the source sentence |
| Invented APIs/libraries | Calling a function that doesn't exist | Code doesn't run, import errors |
Mitigation techniques (from Anthropic's docs)
Basic
- Allow "I don't know": state explicitly in the prompt that if information is missing, it should say so. The docs say this can drastically reduce false information.
- Quote first, analyse second: for long documents (over about 20k tokens), ask for word-for-word quotes first and base the answer only on them.
- Verify with citations: every claim needs a supporting quote; if none can be found, the claim is retracted.
Advanced
- Review the reasoning: enable thinking and read the reasoning summary when an answer looks wrong.
- Best-of-N: run the same prompt several times and compare; where the answers contradict each other is where to be suspicious.
- Iterative refinement: feed the output into a follow-up turn and ask the model to check each point.
- Restrict knowledge sources: use only the provided documents, not general knowledge.
Use only the information in <docs>. For each claim, quote the supporting sentence.
If you can't find a supporting sentence, write "Not enough information" instead of guessing.Risks and caveats
- No technique eliminates it: Anthropic's docs stress these methods reduce, not remove, hallucinations. Critical information always needs human verification.
- Temperature 0 is not a cure: the model can still be consistently wrong.
- Allowing "I don't know" has a price: the refusal rate goes up. Measure both: the error rate and the abstention rate.
How to apply it
- Dev: For any document Q&A feature, require cited output and add code that checks the cited text actually exists in the source.
- Tech Lead: Design evals that penalise "confidently wrong" more than "don't know", in the spirit of the paper, so you don't accidentally reward guessing.
- PM: Design UX for uncertain answers: show sources, let users open the original passage, add a "report error" button.
Sources & references
- Why Language Models Hallucinate — Kalai, Nachum, Vempala, Zhang (arXiv)
- Reduce hallucinations — Claude Docs