Come back if it gets worse
Every GP consultation ends with a ritual: here is what I think, here is what would prove me wrong, here is when to return. It is called safety-netting, it is the best uncertainty engineering I know, and AI systems ship without it.
Every consultation I run ends the same way, with a ritual so worn into me that I barely hear myself performing it. Here is what I think is going on. Here is what I expect to happen next. Here is what would tell us I am wrong. And here is exactly when to come back. General practice calls this safety-netting, teaches it to every registrar as core craft, and treats a consultation without it as unfinished. It is the single most transferable idea I have carried from the consulting room into AI engineering, and almost nobody on the engineering side has heard of it.
To see why medicine needed to invent it, look at what general practice actually is: decision-making under irreducible uncertainty, at volume, with asymmetric stakes. Most of what walks through the door is undifferentiated. The serious and the trivial share opening scenes; the headache that is a headache and the headache that is a catastrophe can be indistinguishable on day one, and no amount of skill fully separates them at first contact, because the information that separates them has not happened yet. A profession facing that reality had two options. It could demand certainty before acting, and grind to a halt. Or it could build a discipline for being wrong safely. It chose the second, refined it for generations, and the result is a toolkit that maps onto the systems I build with almost embarrassing directness.
The first tool is the differential ranked by danger rather than likelihood. Watch a good clinician think and you will notice the ordering is strange: the first question is not "what is most probable" but "what is the worst thing this could be, and have I excluded it". The most likely answer waits its turn behind the most dangerous one. This inverts how almost every AI system is built, because models are, by construction, machines for producing the most probable continuation, and probability-ordered thinking optimises the average case while danger-ordered thinking optimises the worst one. In consequential domains you want both, in the right order: an explicit must-not-miss list, checked before the plausible answer is allowed to ship. The plausible answer is usually right. "Usually" is doing lethal work in that sentence, which is exactly why medicine refuses to lead with it.
The second tool is the red flag, and it is best understood as an engineering component: a short, cheap, deliberately over-sensitive checklist that runs before the expensive reasoning. Medicine's red flags are tuned for recall, not precision, on purpose, because the cost of a false alarm is an unnecessary scan and the cost of a miss is a funeral, and when error costs are that asymmetric, a well-designed gate is supposed to cry wolf at a calibrated rate. Translated: a fast, high-recall classifier guarding the pipeline, whose false positives buy escalation and whose thresholds encode the harm asymmetry explicitly. Most systems I see have nothing in this position. The model is the gate, the reasoning, and the verdict all at once, which is like staffing an emergency department entirely with philosophers.
The third tool is the safety-net itself, and it is the one I most want engineering to steal, so I will state it carefully. In medicine, no answer ships alone. The answer ships wrapped in its own falsification conditions: this is almost certainly X, and if Y or Z happens, that theory is wrong, and here is what to do about it. The diagnosis arrives with the terms of its own revision attached. Now look at how AI systems deliver their outputs: an answer, in a confident voice, full stop. No conditions under which the answer expires. No signs that would mean it was wrong. No route back. I have written elsewhere about why fluent confidence is the most dangerous property these systems have; safety-netting is the antidote in procedural form, because it converts an answer from a verdict into a monitored hypothesis. A system that says "here is my answer, and here is what would mean I am wrong, and here is what to do then" is categorically safer than a marginally more accurate system that says nothing, and the difference costs no model capability at all. It is pure design. We just have to decide answers are not finished until they carry their own escape hatch.
The fourth tool is the review appointment, which sounds administrative and is actually epistemology. "Start this, and let's review in two weeks" is not politeness; it is a test, run on the cheapest instrument medicine owns, which is time. The treatment is a probe, the review reads the result, and the diagnosis is upgraded or demoted accordingly. The loop closes. The engineering cousin of this is outcome follow-up, and it is startlingly rare: most deployed systems fire their answers into the void and never learn what happened next. A system that never finds out how its outputs landed cannot improve, cannot be audited against reality, and is, in the medical sense, practising without follow-up, which is the polite term for practising badly.
And the fifth tool is a grammar, which is the subtlest of them. Medicine has an entire native register for provisional conclusions: working diagnosis, impression, for review, query this, cannot exclude that. The language itself carries the confidence level, and every downstream reader knows how hard to lean on each phrase. Our systems mostly lack this register. Everything comes out in the same polished voice, the certain and the speculative typeset identically, and the reader is given no purchase on which sentences are doing the real work. Building a native provisional register into system outputs, one that survives all the way to the user instead of being sanded off by product polish, is dull work and would do more for safety than most of what gets funded under that word.
Underneath the five tools sits the actual lesson, the cultural one. Medicine is a profession that admits, structurally and in advance, that its practitioners will be wrong every single working day, and has responded not with paralysis and not with bravado but with procedure: rituals that make being wrong detectable, survivable and correctable. Engineering culture, in my experience of it, tends to oscillate between the two poles medicine rejected. Ship it and see, which is overconfidence. Or nothing ships until it is provably perfect, which is paralysis with rigour's badge pinned to it. Safety-netting is the third stance, and it is the mature one: ship the best answer available, wrapped in the conditions of its own revision, with the follow-up loop already booked.
People assume that when a doctor joins an AI company, the cargo is medical knowledge. It mostly is not; the models have the knowledge, more of it than I do. The cargo is epistemics: a century of hard-won, blood-priced procedure for acting under uncertainty without letting the uncertainty become harm. "Come back if it gets worse" sounds like a pleasantry on the way out the door. Read it as an engineer and it is a hypothesis, a monitoring plan, an escalation trigger and a scheduled loop closure, compressed into six words a frightened person can hold onto. It is the densest piece of safety engineering I know. I intend to keep saying it in both of my buildings, and I would like the machines to learn to say it too.
Prefer the full experience? Read this essay in the house. Machine-readable: markdown source.