Handling Refusals Gracefully in the UI
MultigridWhy a refusal is not an error, why it is hard to detect, and what to show when the model declines a request the user thought was ordinary.
The user asked something ordinary — a security question, a medical detail, a violent scene in a novel — and the model declined. Nothing failed. Rendering that as a red error box tells the user your product is broken, when what actually happened is that it decided not to.
A refusal is not an error
An error means the system could not complete the operation. A refusal means it completed the operation and the result is a decline. Those need different visual treatment, different copy, different logging and different metrics, and collapsing them is why refusal handling is usually bad.
The stakes are higher than the styling suggests, because false refusals are common on entirely benign traffic — security research, medicine, law, fiction involving conflict, anything touching a sensitive category without being about it. See why models decline harmless requests. So this is not an edge case reserved for users who deserved it. It is a regular outcome for ordinary users doing ordinary work.
Four sources, four different messages
| Source | Description |
|---|---|
| Provider input filter | The request never reached the model. Usually a distinct error code. The request as phrased cannot succeed, so 'try again' is wrong; rephrasing genuinely may work. |
| Provider output filter | The model generated something and it was blocked, sometimes mid-stream. The user may have already seen the first half. Retrying can produce a different sample that passes, so this one is legitimately retryable. |
| Model soft refusal | A normal, successful completion in which the model declines in prose. Status 200, finish reason 'stop', no signal anywhere in the response envelope. The most common source and the hardest to handle. |
| Your own guardrail | Your policy, your classifier, your system prompt. The only case where you can state the reason precisely, and therefore the only one where a specific, helpful message is fully within your control. |
A fifth case gets misfiled here constantly: a capability decline — “I can’t browse the web”. That is not a refusal, it is a limit, and it belongs to expectation-setting. If users are learning your feature’s boundaries from mid-conversation declines, the boundary is in the wrong place.
The detection problem
Soft refusals are the hard case precisely because nothing marks them. The response is a well-formed completion; the only thing that distinguishes it is the content. Three approaches, in increasing order of cost:
- Structured output. If the task returns JSON, add a refusal branch to the schema — an explicit
{ declined: true, category }variant. Now a refusal is a value your code can switch on rather than prose it has to interpret. By far the most robust option, and available whenever you control the output contract. - A classifier on the output. One cheap model call asking whether this text declines the request. Adds latency and cost to every response, so it is usually reserved for paths where mishandling a refusal is expensive.
- Pattern matching. Fragile, language-specific, and it will misfire on legitimate content that discusses refusal. Adequate as a signal for logging; not adequate as a control-flow decision.
Where you cannot detect it, the fallback is to design so that a refusal rendered as an ordinary answer is not a disaster — which mostly means not treating the model’s output as a value your code depends on without validating it.
What to show
- Neutral treatment, not error styling. No red, no warning triangle. This is an outcome, not a fault, and the user did not necessarily do anything wrong.
- Say who declined and whether rephrasing helps. “This request was blocked by a safety filter — rephrasing may help” and “we don’t allow this in Acme” are different statements with different next actions. Guessing between them is worse than being vague about the source but honest about the action.
- Do not forward the model’s lecture. A generated refusal often comes with a paragraph of moralising the user did not ask for. If you are declining, decline in your product’s voice, briefly.
- Offer the adjacent action. Refusals frequently sit next to something permitted — a general explanation instead of a specific one, a template instead of a filled document. A dead end with an alternative attached is a much smaller failure.
- Keep the input. Same rule as error handling: rephrasing is the recommended action, and rephrasing is impossible if you cleared the box.
- Provide a report path. False refusals are the highest-value feedback you can collect, because each one is a concrete example of your configuration being wrong, with the input attached.
Refusals are a metric, not an incident
Log refusals in their own bucket, keyed by feature and by prompt version. The refusal rate on ordinary traffic is a quality signal with an unusual property: it can change without you changing anything, because the provider updated the model or its safety layer. See model updates you were not told about.
The consequence is that this metric wants a trend alarm rather than a threshold. A step change in refusal rate on unchanged traffic is one of the clearest available signals that something upstream moved, and it is usually noticed weeks earlier than the equivalent drift in output quality, because a refusal is a discrete event and a slightly worse answer is not.
Segment it by feature before looking at the aggregate. A support assistant and a security-documentation assistant have completely different baseline refusal rates, and averaging them produces a number that alarms on nothing and explains nothing. The per-feature rate, on the other hand, has a useful property: you know what its traffic looks like, so a change in it is a change in the system rather than a change in who showed up.
Finally, treat a high steady refusal rate on a feature as a product finding rather than a model problem. It usually means one of three things: the feature is pointed at a domain the model is configured to be cautious about, in which case the model or its safety settings are the wrong choice; the system prompt is over-constrained and is declining things you would allow; or users have discovered a use you did not intend and are being turned away by a guardrail that is working correctly. Those have entirely different fixes, and the refusal log — with the inputs attached — is the only place that distinguishes them.