Handling hallucinations & failure

Key takeaways A model will sometimes be confidently wrong — a hallucination that looks completely fine — and the service behind it will sometimes be slow or down. A real feature plans for all three: wrong, slow, and unavailable. Reduce bad output and add guardrails to catch it, fail safe with fallbacks instead of crashing, and put a human-in-the-loop before anything high-stakes or irreversible. Start from the model as a probabilistic component.

Once you accept that a model is a probabilistic component, the next job is engineering. The model will hand you a wrong answer that looks right, and the call will sometimes fail to come back at all. Hoping it won’t isn’t a plan. This lesson is about the structure you build around the model so that when it misbehaves — and it will — your feature degrades gracefully instead of doing something wrong or falling over.

Two kinds of failure

There are two completely different ways an AI feature fails, and you have to handle both.

Content failure. The call succeeds and returns text, but the content is wrong — a made-up fact, a bad number, an invented API. This is the dangerous one because nothing errors: the output is fluent and plausible, so it sails straight through unless you check it.
Operational failure. The call itself doesn’t come back — it times out, the service rate-limits you (an HTTP 429), or the provider has an outage. This is the ordinary kind of failure any network dependency has, and it announces itself with an error.

They need different defences. Content failure is fought with grounding, validation, and review; operational failure is fought with retries, timeouts, and fallbacks. A feature that handles only one of these is only half-built.

Reducing hallucination

The cheapest bad output to handle is the one that never happens. You can’t eliminate hallucination, but a few habits cut it sharply:

Ground the model in real data. Instead of asking the model what it “knows,” give it the relevant facts in the prompt and ask it to answer from those. This is the whole point of grounding and retrieval — the single biggest lever on accuracy.
Give it an out. Instruct it explicitly: “If the answer isn’t in the context, say you don’t know.” A model told it may decline is far less likely to invent something.
Ask for citations. Requiring the model to point at its source both improves grounding and gives you something to check.
Lower the temperature. For factual, structured tasks, less randomness means fewer creative wrong turns.
Keep tasks narrow. A small, well-specified job has less room to wander than a vague, sprawling one. Break big asks into checkable steps.

Catching bad output

Reduction isn’t proof. Assume some wrong output gets through and build a net for it before it reaches anything that matters.

Validate structure and values. If you asked for JSON, confirm it parses and has the right fields; if you asked for a number, confirm it’s in a sane range. See parsing and validating output.
Run sanity and consistency checks. Does the answer contradict the input? Is a total the sum of its parts? Cheap logic catches a lot.
Add guardrail checks. Screen for the specific failure modes that matter to your feature — off-topic replies, unsafe content, values that violate a business rule.
Never auto-act on unverified output. If the model’s answer triggers an action — especially through tool use and function calling — verify it before the action fires, not after.

Failing safe

When something is wrong or missing, the feature should fail safe — degrade to a sensible state rather than break or do the wrong thing.

Fallbacks and graceful degradation. Have a lesser-but-fine answer ready: a cached result, a sensible default, a plain “I’m not sure — here’s what I can tell you,” or a handoff to a non-AI path. A narrower correct answer beats a confident wrong one.
Human-in-the-loop for high stakes. When an action is expensive or irreversible — spending money, deleting records, sending something you can’t unsend — the model proposes and a person confirms. Automate the cheap, reversible actions; gate the ones you can’t take back.

The rule of thumb: the higher the cost of being wrong, the more the design should refuse to act on the model’s word alone.

Operational resilience

For the calls that don’t come back, borrow the standard playbook for any flaky network dependency:

Retry with backoff. Transient errors and 429s often clear on a second try; retry a few times with increasing delays rather than hammering the service.
Set timeouts. A slow call shouldn’t be allowed to hang your whole feature. Cap how long you’ll wait and treat the overrun as a failure.
Use a circuit breaker. If the service is clearly down, stop calling it for a while and serve the fallback — don’t pile requests onto a dead dependency.
Keep a fallback model or provider. A cheaper or alternate model can carry the load when your first choice is unavailable (choosing a model for a feature).
Serve cached or default responses. When all else fails, a stored or default answer keeps the feature alive.

Costs and limits shape all of this — see cost, latency, and limits.

UX of uncertainty

How you present uncertainty is part of handling it. The interface should be honest that an AI produced the answer and that it might be wrong.

Show sources. Let people see what the answer was grounded in so they can judge it.
Express uncertainty. When confidence is low, say so — a hedge is more trustworthy than false certainty.
Make bad answers easy to report. A one-click “this was wrong” both helps the user in the moment and feeds your evaluations and monitoring, turning real failures into the data that makes the feature better.

Quick check: a feature could take an expensive, irreversible action based on the model's output. What's the safest design?

Recap

Plan for all three failure shapes: confidently wrong, slow, and down.
Content vs. operational failure are different problems — wrong output that looks fine, versus a call that never returns — and you must handle both.
Reduce hallucination by grounding in real data, giving the model an out, asking for citations, lowering temperature, and keeping tasks narrow.
Catch what’s left with structure and value validation, sanity checks, and guardrails — and never auto-act on unverified output.
Fail safe: fall back and degrade gracefully, and put a human in the loop before high-stakes or irreversible actions.
Stay resilient with retries and backoff, timeouts, circuit breakers, a fallback model, and cached defaults — and be honest about uncertainty in the UX.

Next up: prompt injection & data safety.