AI Hallucinations: Why They Happen and How to Reduce Them

You ask an AI for the paper that proves your point. It hands back an author, a title, a year, and a link. The link goes nowhere, and the paper doesn't exist.

That's an AI hallucination: a confident statement that isn't supported by the source you gave it or by reality. Google Cloud defines it as incorrect or misleading results that AI models generate. You can't switch it off, but you can make it rarer and, more usefully, make it easy to catch.

The short version:

• Models hallucinate most on specific facts that appear rarely or nowhere in their training data: a date, a page number, a function name.
• Training and grading habits reward a confident guess over an honest "I don't know."
• The fix is a workflow, not a better prompt: give it the source, make it quote, let it abstain, open the links, and ask twice.

What AI hallucinations are

Two flavours show up in the research, and the difference matters for how you defend against them.

Contradicting reality. The answer sounds right and is false. A made-up citation is the classic case.

Contradicting what you gave it. You paste in a report and ask for a summary, and the summary includes a detail that isn't in the report. Anthropic's docs describe hallucination as text that is factually incorrect or inconsistent with the given context, which covers both.

The word itself is a loose metaphor. The authors of a 2025 paper on why language models hallucinate point out that it differs fundamentally from the human perceptual experience. Think of it as a plausible falsehood delivered without hesitation, not as the model seeing things.

Why does AI hallucinate

The paper is by Adam Tauman Kalai, Ofir Nachum, Santosh Vempala and Edwin Zhang, posted to arXiv on 4 September 2025. Three of the four authors list OpenAI as their affiliation. It gives a plain-language account with two stages.

Stage one: learning from text. A model learns the patterns of language from a huge body of writing. Some things have a pattern to learn, like spelling. Others are arbitrary facts with no pattern, like a stranger's birthday. When the model can't tell a true statement from a plausible one, errors follow as a natural statistical result. The authors note that models seldom get frequently repeated facts wrong, such as a famous person's birthday, and the trouble concentrates on facts that appear rarely.

Stage two: being graded. After training, models are tuned and tested. The authors argue that most tests score a correct answer as a point and a blank or "I don't know" as nothing, so guessing never costs anything. Their analogy is a student bluffing on an exam: models are always in test-taking mode, so a confident guess beats an honest blank.

Two smaller causes round it out:

• The model is a poor fit for the task. The paper's example is counting letters. It asked one model how many Ds are in "DEEPSEEK" and got "2" or "3" across ten tries. The authors tie this to models reading text in chunks rather than single characters.
• Bad data in. Training text contains errors, and models can repeat them. Google's page lists incomplete or biased training data and a lack of proper grounding as causes, and says that can extend to fabricating links to pages that never existed.

One more point from the paper matters later: the authors say their analysis applies to search-and-retrieval models too.

Examples from the research itself

The paper opens with a few simple tests. You can run the same kind on any assistant.

Two details are worth keeping. These were tested in May 2025, so newer models may do better on these exact questions. And in the dissertation test, none of the models searched the web, so the failures came from recall alone.

Notice what the birthday test shows. The instruction said to answer only if known, and the model answered anyway. Permission to say "I don't know" helps, but it is not a guarantee.

Where it bites

The pattern is consistent: the more specific and checkable the detail, the less you should trust it unverified.

The last row is our reading of the paper's logic, not a finding from it. The paper tested facts and letter counts, not code. But a method name in a small library has the same shape as the birthday question: one specific fact, little repetition, no pattern.

A six-step checklist to reduce hallucinations

Each step below is tied to the source that supports it. Where one isn't, we say so.

1. Give it the document and fence it in. Anthropic lists "external knowledge restriction" among its techniques: instruct the model to use only the provided documents and not its general knowledge. Google's grounding documentation describes the same idea, connecting output to verifiable sources so the model has less room to invent. Paste the text or attach the file, and say "answer only from this."

2. Make it quote first. For long documents, Anthropic suggests asking the model to extract word-for-word quotes before doing its task, which grounds the answer in the actual text. Then ask it to analyse using only those quotes. If it finds no relevant quote, it should say so.

3. Ask for sources, then open them. Anthropic's "verify with citations" pattern: have the model cite a quote for each claim, or, after the draft, find a supporting quote for each claim and retract any it can't back. That makes the answer auditable. The audit is your part: click the link, find the sentence. Google's warning about fabricated links is the reason a citation you haven't opened counts for nothing.

4. Let it say "I don't know." Anthropic lists this first and says the simple technique can sharply cut false information. Add a line such as "If the document doesn't say, answer: not stated." The paper explains why it needs saying: the authors argue grading rewards guessing. The birthday test above is the caution. It reduces guessing, it doesn't end it.

5. Narrow the question. Ask for one thing at a time, with the time period, region or definition spelled out, rather than "tell me everything about X." No vendor doc we read prescribes this, so treat it as reasoning rather than a tested result. It follows from the paper's logic: the more arbitrary specifics you demand in one answer, the more guesses it has to make, and the more places one can hide.

6. Cross-check: ask twice, then check a second source. Anthropic's "best-of-N" technique runs the same prompt several times and compares: inconsistencies across outputs can indicate hallucination. The paper's own examples fit. Three attempts at one birthday gave three different dates, and ten counting attempts did not agree. For anything that matters, confirm the answer in a second, independent source.

None of this removes the need to check what matters. Anthropic's page says these techniques significantly reduce hallucinations but don't eliminate them entirely, and tells you to validate critical information, especially for high-stakes decisions.

Where search and retrieval help, and where they don't

Search and retrieval are the strongest single lever, because they change the job. Instead of recalling from memory, the model reads something and answers from it. That is the idea behind retrieval-augmented generation.

Google Cloud's grounding overview lists the benefits: it reduces hallucinations, anchors responses to your data sources, and provides links to sources so you can audit the answer. Anthropic's web search tool returns a final response with cited sources, and its citations feature returns the exact passages that support each claim from documents you supply.

Four limits are worth knowing:

• The model decides whether to search. In Anthropic's tool, Claude determines when to search based on the prompt, and it answers directly for things like established facts. If you want sources, ask for them.
• A valid pointer is not a true claim. Anthropic says its citations are guaranteed to contain valid pointers to the provided documents. That tells you where to look. It doesn't tell you the sentence was read correctly. Open the passage.
• Search can fail quietly. The paper says search and reasoning are not panaceas: the grading incentive to guess still applies when search doesn't yield a confident answer, and search may not fix miscalculations like the letter-counting error.
• The source can be wrong. Retrieval inherits the quality of what it finds.

For a deeper look at tools that cite their sources and a one-minute routine for checking them, see our guide to AI source finders.

Keeping a setup that works

Most of the checklist is boilerplate you'd retype every session: the "answer only from this document" line, the quote-first step, the not-stated fallback. Once you have a version that works for your documents, the annoying part is keeping it.

Taku is an AI-native desktop workspace where you mirror an AI setup someone already got working, run it, remix it, and keep it. It doesn't stop a model from hallucinating, and no workspace does. What it can save you is re-describing a careful process each time, and the free app library shows what is there to mirror. Taku is in Beta, and the Mac app is available now.

Key points

• A hallucination is a confident statement not supported by your source or by reality.
• Research from OpenAI and Georgia Tech authors points to two causes: arbitrary, rarely seen facts the model can't verify, and grading that rewards guessing over abstaining.
• Citations, numbers, exact quotes and code APIs are where it bites most.
• Give it the document, make it quote, let it say "I don't know," open the links, and ask twice.
• Search and retrieval help, but the model chooses whether to search, a valid pointer isn't a verified claim, and the paper says search is not a cure-all.

FAQ

What is an AI hallucination?

A confident statement from an AI that isn't supported by the source you gave it or by reality. Google Cloud defines it as incorrect or misleading results that AI models generate.

Why does AI hallucinate?

A 2025 paper by researchers at OpenAI and Georgia Tech gives two reasons. Models can't reliably separate true from plausible on arbitrary facts that appear rarely, and most tests reward guessing over saying "I don't know."

How do I stop AI making things up?

You can't stop it entirely. You can reduce it: supply the document, restrict the model to it, ask for word-for-word quotes, allow "I don't know," and verify. Anthropic's docs say these techniques reduce hallucinations but don't eliminate them.

Does AI make up sources and citations?

Yes, it can. Google notes models can fabricate links to web pages that never existed. Open every link, and find the exact sentence that supports the claim.

Does web search fix hallucinations?

It helps a lot, but not completely. The model decides when to search, and the paper notes search doesn't fix every error type. Treat cited passages as pointers to check.

Are some AI hallucination examples easy to test myself?

Yes. Ask a tool for a birthday or a thesis title for someone not famous, then check. Or ask it to count a letter in a long word, and run the same prompt several times to see if the answers agree.