---
title: "A second opinion on hallucination"
author: Dr Arman Ouveysi
date: 2026-07-07
room: Lab
tags: llm-behaviour, hallucination, medicine
canonical: https://drouveysi.com.au/essays/a-second-opinion-on-hallucination/
summary: "The field's most famous failure has the wrong name. Medicine has the right one, and the right name comes with a treatment plan attached."
---

# A second opinion on hallucination

I want to start with what a hallucination actually is, because the word has a day job, and I've met it there.

A man in alcohol withdrawal, three days after his last drink, picks at the bedsheets because he can see insects on them. He isn't describing insects. He is *seeing* them: a full sensory experience, arriving through the visual system, with nothing in the world causing it. That is a hallucination: **a perception without a stimulus**. It is an event that happens *to* a subject, in a sensory channel, and it is one of the more frightening things a brain can do to its owner. Curiously, it doesn't even always come with belief. There is a condition in which people with failing eyesight see vivid, elaborate scenes, and most of them know perfectly well the scenes aren't real. Hallucination with full insight. Hold onto that detail; it earns its keep later.

Now put that word next to the thing it supposedly names in my other field. A language model, asked for a reference, produces a paper that does not exist: plausible authors, a real-sounding journal, a page range, a DOI in perfect format. Nothing was perceived. There is no sensory channel for anything to arrive through. No experience happened to anyone. What happened is that a system produced a confident, fluent, false statement at the exact point where its knowledge ran out, without any awareness that it had done so.

That is not a hallucination. Medicine has a precise word for it, and the word is *confabulation*.

I want to make the case that this isn't pedantry. In medicine, the name on the front of the file is a treatment plan in disguise. Call a presentation the wrong thing and you don't just mislabel it, you manage it wrongly, because the name is what tells everyone downstream which playbook to open. The AI field picked its word casually, borrowed from an adjacent discipline in the early days when the failure was new and needed a memorable label, and the label stuck. It's now doing quiet damage to how people think about these systems, and the correct diagnosis, as it usually does, comes with better treatment.

So, confabulation. The textbook home of it is Korsakoff's syndrome, a memory disorder caused by thiamine deficiency, classically after years of heavy drinking. The machinery that lays down new memories is damaged, and something remarkable happens in the gap. Ask a patient with the classic presentation what they did yesterday, a day they spent entirely on the ward, and you may get a detailed, coherent, cheerfully delivered account of a trip to the market: who they saw, what they bought, the weather. None of it happened. And here is the part that rearranges how you think about honesty: they are not lying. There is no intent to deceive, no awareness of falsehood, no flicker of doubt. The literature's phrase for it is *honest lying*, which is the kind of oxymoron you only coin when you've run out of normal language. The patient's narrative machinery, asked for an account, produced the most plausible one available, and the part of the system that should have flagged the gap is the part that's broken.

Look at the clinical features of confabulation as a list, and tell me what it reminds you of. The false content is fluent and grammatically perfect. It is plausible: appropriate to context, the kind of thing that *could* have happened. It is built from real fragments, actual memories, recombined and dropped into the wrong time and place. Confidence is completely uncoupled from accuracy; the false account is delivered with the same ease as a true one, and often more smoothly, because nothing is being checked. There is no insight, and the deficit of insight is structural, not attitudinal: the monitoring system that would catch the error is the system that failed. And, critically, it gets worse under questioning. The literature distinguishes confabulation that arrives spontaneously from confabulation that is *provoked*: produced because a question demanded an answer the memory could not supply, and the machinery, unable to say nothing, supplied something.

Every item on that list transfers to the machine, and the last one transfers with a click you can almost hear. A language model is, mechanically, an engine for producing the most plausible continuation of a text. Plausibility is the only variable it optimises; truth is not a quantity it has access to, only a pattern that plausible text usually exhibits, because the training data was written in a world where facts are common. Where the training distribution actually contains the fact, plausible and true coincide, and we call the output knowledge. Where the distribution has a gap, the machinery does not stop, because stopping was never among its behaviours. It fills. Fluently, from real fragments (real author names, real journal titles, real DOI formats, assembled into a paper that never existed), with confidence generated by the same process that generates everything else. And it cannot notice the gap from the inside, for the same structural reason the Korsakoff patient can't: the noticing would have to be done by the very machinery that just failed. Then we put this system in a chat interface, where every message is a question demanding an answer, and we express surprise at the results. **We built a provoked confabulator and provoked it.**

Remember the failing-eyesight patients, the ones who hallucinate with full insight, who see the visions and know they aren't real. Even human hallucination often comes with a working reality-check. The machine's failure never does. That alone should have told us we'd filed it under the wrong word.

Here is where the misnomer stops being a naming dispute and starts costing something. Diagnosis determines management. If you call the failure *hallucination*, you inherit a folk model: the machine perceives, and sometimes it perceives wrongly, the way a mad or broken thing does. Two conclusions follow, both wrong. The first is that the fix is a better perceiver: more scale, more training, wait for the next model, the problem is temporary. The second is subtler and worse: that when the machine is *not* hallucinating, it is perceiving, that its normal mode is seeing truth and the failures are departures from that mode. Neither is right. The model does not have a truth mode that occasionally fails into fabrication. **Generation is the only mode it has.** Some of it lands on facts. The landing is what all the engineering is for.

![Forked diagram contrasting hallucination management with confabulation management](https://drouveysi.com.au/assets/articles/lab/a-second-opinion-on-hallucination/fig-01.svg)

Call it *confabulation* instead, and the management plan writes itself, because medicine has been managing confabulation for the better part of a century, and the playbook is sitting right there.

You do not interrogate a failing memory; you supply the record. Nobody on a ward quizzes a Korsakoff patient about their medication doses and acts on the answers. You read the chart. The engineering translation is retrieval and grounding: give the model the document and ask it to work from what's in front of it, rather than demanding recall from weights and hoping. It is the single most effective anti-confabulation intervention in both fields, and for the identical reason: it converts a memory task, which the system fails at, into a reading task, which it doesn't.

You make "I don't know" a safe thing to say. In cognitive assessment, an examinee who cannot produce the words *I don't know*, who fills every gap rather than admit one, is telling you something diagnostic. We built that examinee on purpose. A training process that rewards an answer over an abstention, at scale, manufactures a system for whom the gap-filling response is always the winning move. The fix is the same as it is in any clinic: reward the calibrated admission. Make the null response a first-class output instead of a failure state.

You surround the patient with external memory. A memory ward runs on whiteboards, diaries, orientation boards, labels on doors: cognition outsourced to the environment because the internal version can't be trusted. Retrieval systems, tool access, scratchpads, and structured working state are orientation boards. Nobody calls a whiteboard a defeat for the patient, and nobody should call scaffolding a defeat for the model.

![Diptych: an examination chair facing a blank light, and a record card dissolving into fragments.](https://drouveysi.com.au/assets/articles/lab/a-second-opinion-on-hallucination/fig-02.webp)

And you corroborate, always, because you never build a management plan on a single unreliable narrator. Collateral history is a reflex in medicine: ask the family, check the record, confirm the dose with the pharmacy. The engineering translations are verification layers, citation requirements, and outputs checked against sources rather than admired for their fluency.

Line those up against the modern anti-hallucination toolkit and the correspondence is close to exact. Grounding, abstention training, external memory, verification. The field converged on the confabulation playbook empirically, one expensive lesson at a time, apparently without noticing that a geriatric memory service had written the standard operating procedure decades earlier. We rediscovered ward protocol and called it AI engineering. I find this genuinely funny, in the way things are funny when they've cost a lot of money.

The objection I hear is that it's only a word, that everyone technical knows what's meant. Medicine knows better than most fields what *only a word* costs. A family tells you their grandmother is "confused", and whether the word that lands on the file is *delirium* or *dementia* decides whether tonight involves an urgent search for the infection that's killing her or a referral letter that arrives in three weeks. Same behaviour at the bedside. Different name, different playbook, different outcome. **Words route action**, and *hallucination* has routed a generation of users into treating a memory problem as a vision problem: waiting for the machine to see better, when the work was always to stop asking a gap to describe itself.

The machine is not seeing things that aren't there. It is telling you, with total sincerity, about a market it never visited, assembled from real stalls it read about somewhere else. Medicine has managed that patient for a hundred years, kindly and effectively, and it starts every single time in the same place: getting the name right on the front of the file.