Textbook of AI

Hallucination refers to a large language model generating plausible-sounding but factually incorrect or fabricated information. An LLM asked about a historical event might confidently invent details; asked for citations, it might fabricate plausible-looking references to non-existent papers; asked for statistics, it might produce numbers that sound reasonable but are wrong. Unlike a search engine that retrieves information, an LLM generates text by predicting likely continuations, and there is no inherent mechanism distinguishing accurate recall from confident confabulation.

Hallucination arises because LLMs have no grounding in external reality. Their knowledge is encoded entirely in parameters, frozen at training time. They have no ability to verify facts, no external memory to consult, and no sense of what they do not know. When asked about topics beyond their training data or about specific facts they did not encode reliably, they fall back on producing plausible-sounding text—the only thing they know how to do. The training objective (predict the next token) does not directly penalise factually wrong but plausible tokens.

Mitigation approaches include: retrieval-augmented generation (RAG) grounds the model in fetched documents, providing external knowledge; citation generation forces the model to produce sources that can be checked; tool use lets the model call calculators, search engines, or databases for facts; fine-tuning with truthfulness objectives; prompting techniques that encourage uncertainty acknowledgement ("If unsure, say so"); and self-consistency checks. None fully eliminate the problem. Hallucination is one of the central obstacles to deploying LLMs in high-stakes domains, and addressing it remains an active research area.