Hallucination (AI)

Hallucination is the term used to describe when a language model generates plausible-sounding but completely fabricated, factually incorrect, or logically inconsistent content, while presenting it with absolute confidence. A hallucinating model might invent a fake academic citation, state an incorrect historical date, attribute a real quote to the wrong person, or confidently output broken code that looks syntactically perfect.

Hallucination arises from the fundamental architecture of generative AI. Models are trained to predict the next most statistically likely token based on their training data; they are not inherently tied to an external database of verified facts. When a model lacks strong training signals for a specific query, it relies on learned linguistic patterns to generate text that *feels* correct. Crucially, the model has no internal mechanism to distinguish between a fact it 'knows' and a sequence it is generating purely probabilistically.

Hallucination differs from simple incorrectness because the model typically fails to express appropriate uncertainty. A well-calibrated model might say, 'I am not entirely sure, but I believe the treaty was signed in 1847.' A hallucinating model states, 'The treaty was signed in 1847,' leaving the user unaware of the fabrication.

Reducing hallucination is the single largest hurdle to enterprise AI adoption. Techniques like Retrieval-Augmented Generation (RAG) attempt to ground the model by providing verified facts in the prompt context, while calibration training teaches the model to express doubt when it is unsure.