Behaviour and trust · I cannot decide yet
When the system should refuse
Sometimes the correct output is no output. Evaluation-backed abstention prevents harm when authority, evidence, or terrain forbids completion.
9 min read
Cover for When the system should refuseA blog-sourced medical diet request arrives in the Meal Companion chat. The model can always produce meals that sound compliant. Low-FODMAP labels, gentle language, appetising names. The trustworthy system abstains from medical interpretation. It returns structured deferral, an escalation path, safe partial scope only, logged as a first-class result. Abstention is a correct output when confidence in correctness is insufficient, authority is missing, or terrain is frontier. Teams that measure it without treating it as failure to minimize are the ones who avoid the incidents that false completion invites.
Helpful context: Refuse and defer covers conduct language and templates. Reliability is not a confidence score explains sufficiency gates. Evaluation as engineering backs abstention decisions with suites.
Abstention versus false completion
False completion feels helpful in the demo. Abstention feels slower until the first incident avoided. Architecture chooses the slower honest path on purpose. A hallucinated low-FODMAP plan, a guess on leftover preference, a silent constraint drop, and a fluent plan on missing vendor fields all perform completion without the evidence or authority to back it. Abstention replaces each with a structured response: "I can't interpret this diet. Plan around endorsed allergies until you confirm clinician rules." Or a targeted question, or a named refusal with reason, or a structured missing-fields branch for the human clerk.
| False completion | Abstention |
|---|---|
| Hallucinated low-FODMAP plan | "I can't interpret this diet. Plan around endorsed allergies until you confirm clinician rules." |
| Guess on leftover preference | Targeted question or stop |
| Silent constraint drop | Named refusal with reason |
| Fluent plan on missing vendor fields | Structured missing-fields branch for human clerk |
The cognitive unit Ch. 10The caller's response to insufficiency is a branch, not a number.
When to abstain
Abstention applies when authority is missing (medical interpretation, purchase, policy override), when sufficiency fails after restraint prepares the decision, when terrain is frontier with weak reversibility, high consequence, and weak evaluation coverage, when guards fail and full deliberation is blocked, or when determinacy is low and competent humans disagree. Each case produces structured output, not empty error pages. Users see what was withheld, why, what remains safe to do, and what input or authority unlocks the next step.
The difference between abstention and a generic error matters because generic errors train retry loops. Users click regenerate until fluency wins. Structured abstention trains appropriate reliance. Retry loops on medical frontier burn cost and eventually produce false completion. Log abstention class so analytics distinguish deliberate stop from infrastructure failure.
The same pattern appears in enterprise domains. In invoice intake, insufficient vendor data triggers abstention from auto-approval recommendation and returns structured missing fields with a branch for the human clerk, logged by abstain class rather than generic error. In a vendor tier dispute, low determinacy where competent humans disagree triggers abstention from picking a winner. The system presents both positions and names who must decide. Abstention here is representation of disagreement, not weakness. When a guard fails on frontier mention, the system abstains from full composition and offers scoped plan or asks a material question rather than silently dropping the guard.
Abstention in UX and metrics
Abstention copy is part of the architecture. Good abstention includes scope withheld (medical diet-specific rules), reason (authority boundary, not capability boast), safe partial scope (plan around endorsed allergies), unlock path (clinician constraints, human approval, missing fields), and escalation (when product supports handoff). Compare to weasel words in refuse and defer: "I'm not a doctor, but here's a plan anyway" is false completion wearing deferral clothing.
Dashboards that reward completion alone train harm. Tracking abstention rate by class, user outcomes after deferral, false completion catches in evaluation, and time-to-accepted-plan when abstention prevented replan reframes the metric correctly. Success includes stop. Product analytics that treat abstention as drop-off need reframing, because a deferral on medical frontier is successful conduct rather than funnel abandonment. Demos that never abstain mis-set buyer expectations. Showing one deferral on medical frontier in every serious demo means buyers who see completion also see the product's partnership with appropriate limits.
Evaluation hooks and caller responsibility
Abstention classes appear in evaluation suites with expected JSON shape, carrying the same rigor as success cases. Teams that golden-path test never learn whether abstention regressed. Test expected abstain on medical frontier, expected ask when material gap follows restraint, expected unsuccessful stop on impossible constraints, and no false completion when abstain was expected. Connect to a judge is evidence, not truth: abstention eval uses contract checking on stop shape and scope, not oracle scoring.
Contestability ends where abstention begins in one sense: contesting a judgment the system correctly refused to make is not possible. Contesting the scope of deferral, escalation path quality, and safe partial scope boundaries is. Users contest "you should have tried" with visible authority spec, not with longer apologies.
When ungoverned retrieval or chat contaminates context, abstention may be correct even if individual cognitive units pass suites. Least trustworthy input wins applies. Caller-side gates abstain when composition cannot trust inputs. Individual cognitive units may return abstention correctly while the caller composes anyway. Caller-side gates honor abstention from child calls and abstain at composition level when inputs fail four kinds of source classification.
Refuse and defer owns conduct language and templates. This page owns abstention as evaluated architectural result: gates, logging, metrics, suite expectations.
Common patterns
Each abstention record includes class, scope withheld, safe partial scope offered, and unlock path suggested. Incident review then distinguishes correct abstention from broken gates, and missing logs make abstention look like product flakiness. Batch jobs tempt false completion because no user waits in chat, but abstention still fires when authority or evidence is missing. Batch outputs without abstention classes produce silent harm at scale.
When downstream automation consumes cognitive outputs, abstention shape must be machine-readable. Downstream branches on abstain class, not parsed apologies. Contract-first abstention enables safe orchestration across services. Governance forums review abstention rates by class the way they review error rates. Rising medical deferrals may mean marketing reached frontier users. Rising missing-field abstention may mean upstream data quality failed. Abstention is diagnostic, not shameful.
Legal and clinical reviewers approve abstention templates the way they approve marketing copy. Weasel deferrals create liability. Structured deferrals with safe partial scope reduce it. After abstention, recovery paths remain available for safe partial scope. Users who receive deferral still need patch and contest on non-frontier parts of the plan.
When routing among models by cost, cheaper models may abstain more often on frontier terrain, and routing policy treats correct abstention as success rather than routing failure to override with expensive guessing. Support staff who know abstention classes and safe partial scopes do not manually complete frontier work the product correctly withheld. Support playbooks mirror abstention JSON shape. Escalation from abstention attaches the same evidence users see in product. Handoffs without evidence packets recreate opaque deferrals and fail review.
Define JSON shape for each abstention class: scope withheld, reason code, safe partial scope, unlock path, escalation link. Suites assert shape the same way they assert success payloads. Shape drift breaks downstream analytics and handoff workflows. Run quarterly drills where teams produce correct abstention JSON for three frontier scenarios under time pressure. Drills expose missing templates faster than doc review. Partner APIs consuming outputs need abstention classes documented, because partners that treat all non-200 responses as retryable errors will fight abstention unless shapes are explicit.
What to do next
Abstention normalizes the idea that success includes stop. The structured result, logged by class with scope withheld, safe partial scope offered, and unlock path visible, replaces the false binary of completion versus error. Evaluation suites, product analytics, and downstream APIs all treat abstention as a first-class outcome rather than a failure to minimize.
Read refuse and defer, contestability, and when the system must stop.
Read next: Least trustworthy input wins explains how abstention often traces to weak inputs in the chain.