The Generation Mechanism

To understand hallucination, you need to understand how language models produce text. The process is the same whether the output is accurate or completely fabricated.

The model processes your input and produces a probability distribution over its vocabulary for the next position. Each token gets a probability score based on patterns in training data, not on factual validity. When you ask "Who wrote Hamlet?" the token "William" gets a high probability because the sequence appears overwhelmingly in training data. This produces a correct answer because the statistical pattern and the factual answer coincide.

But the mechanism is identical when it produces a wrong answer. When the training data is sparse, the model still generates a probability distribution. The selected tokens are assembled from thinner statistical signals, fragments of related patterns stitched together into something that reads as coherent but corresponds to nothing real.

The model cannot know the difference. There is no internal representation of "this is real" versus "this is fabricated." There is only "this token has a 0.73 probability" versus "this token has a 0.71 probability."

Why Hallucinations Look So Real

The most dangerous property of AI hallucinations is that they are indistinguishable from accurate output. A hallucinated court case looks exactly like a real court case citation. A fabricated research paper has a plausible title, plausible authors, and a plausible DOI format.

This happens because the model learned the format of accurate information from training data. It knows what a court citation looks like. It knows the structure of an academic reference. When it fabricates, it fabricates in the correct format.

Think of it this way: a skilled forger does not produce paintings that look obviously fake. They match the style, technique, and materials of the originals. ChatGPT is a text forger. It has learned the surface patterns of factual text so thoroughly that its fabrications carry all the markers of authenticity.

The Taxonomy of Hallucinations

Fabricated entities. The model invents things that do not exist: fake court cases, nonexistent research papers, companies that were never founded. This is the most dramatic form but also the easiest to catch if you verify.

Misattributed facts. Real facts attached to the wrong entity. A quote attributed to the wrong person. A discovery credited to a different institution. Harder to catch because the individual components are all real.

Temporal confabulation. Real information from different time periods combined into a single response. A company's 2019 revenue presented as current. A proposed policy described as enacted law.

Statistical fabrication. Plausible-looking numbers with no basis in real data. "Studies show that 73% of users experienced..." when no such study exists. Nearly impossible to detect without independent verification.

Coherent fiction. The most insidious type. A long, internally consistent narrative weaving real facts, half-truths, and inventions. The real facts provide anchoring credibility. The fabricated elements fill gaps. The overall narrative is compelling, logical, and partially made up.

Why It Can't Be Fully Fixed

Retrieval-augmented generation (RAG) helps by grounding some responses in verified documents. Fine-tuning on higher-quality data can shift probability distributions toward accuracy. But none of these techniques eliminate hallucination, because hallucination is not a bug. It is a consequence of how the system works.

Any system that generates text by predicting the most probable next token will sometimes generate sequences that don't correspond to reality. The honest framing is: hallucination rates can be lowered, but hallucination as a category of failure will exist for as long as the technology works the way it currently works.

Promising users that hallucination will be solved is like promising that a compass will eventually tell you the temperature. It is not what the instrument does.

Why the Industry Chose the Word "Hallucination"

A hallucination, in human psychology, is an aberration. It implies that the default state is accurate perception, and the hallucination is a departure from that default.

This framing is backwards when applied to language models. The default state is statistical prediction. Sometimes that prediction is accurate. Sometimes it is not. There is no "accurate default" that the model occasionally departs from.

A more honest term would be "confabulation" or simply "fabrication." The industry prefers "hallucination" because it sounds medical, technical, and temporary. It sounds like something that can be cured. That framing serves the companies. It does not serve the users.

The Verification Asymmetry

If ChatGPT generates a ten-paragraph research summary citing eight sources, verifying it requires checking each citation, each attributed claim, each statistic. This is essentially the same work you would have done to write the summary from scratch.

For tasks where the writing is the hard part (drafts, emails, creative content), this tradeoff works. For tasks where the research is the hard part (legal analysis, medical information, academic work), the tradeoff breaks down. You have not saved time. You have just moved the work from "research and write" to "read and verify."

Living With It

Treat every piece of factual output from ChatGPT as unverified. Not as probably right. Not as usually reliable. Unverified.

For tasks where checking is fast and stakes are low, use the tool freely. For tasks where checking is slow and stakes are high, the tool may cost you more time than it saves.

The model will never warn you when it is hallucinating, because it does not know when it is hallucinating. That awareness is your job.