Reliability is not a confidence score

If a cognitive unit's behaviour can only be known by measurement, a natural suggestion follows: let it return a confidence score. High scores proceed. Low scores trigger caution. Many systems implement this. It does not work, not because models are badly calibrated, but because there is nothing there to calibrate.

Reliability in Thoughtware is an operational decision assembled from multiple instruments: sufficiency, stability, determinacy, coverage, and accuracy. No single number from the cognitive unit, vendor benchmark theatre, or warm feeling from a fluent paragraph substitutes for that assembly.

When the Meal Companion enforces a confirmed cashew allergy in deterministic code, reliability is visible and repeatable. When AssessMealPracticality judges busy evenings, reliability depends on sufficiency of inputs, evaluation coverage for that case class, and historical accuracy on graded runs. No single float at the bottom of either response substitutes for that assembly.

Helpful context: Trust across time describes what reliability supports. Evaluation as engineering supplies the records. Confidence theatre names fake certainty.

Five questions wearing one coat

"How reliable is this answer?" sounds like one question. In Thoughtware it is five distinct questions that demand different remedies. Sufficiency asks whether the cognitive unit had the information needed to decide: a busy-evening judgment without calendar context fails sufficiency regardless of eloquence. Stability asks whether the same answer would appear if asked again under the same conditions, since some judgments are intentionally underspecified and require sampling or wrappers. Determinacy asks whether competent people agree on cases like this, because low determinacy caps how far automation may proceed without representing disagreement. Coverage asks whether this case class appears in evaluation suites, because unevaluated terrain is unevaluated, not proven. Accuracy asks, on covered cases, how often the judgment was right against ground truth or graded criteria.

From inside a single call, a cognitive unit can report sufficiency honestly. It cannot see human agreement rates, suite membership, or historical accuracy against ground truth. Those live outside the call by design.

A confidence score is disconnected from stability and evaluation records, not badly calibrated.

The cognitive unit · Ch. 10

When a model emits 0.82, it has not measured stability or consulted an evaluation record. It has produced a number in the register where such numbers usually appear. That is a linguistic act, not a measurement. Each dimension requires a different instrument and a different response when low.

Dimension	Instrument	If low
Sufficiency	cognitive unit report plus preconditions	Gather input or abstain
Stability	Resample or wrapper	Wrap in sampling or treat as underspecified
Determinacy	Human agreement on anchors	Lower authority, represent disagreement
Coverage	Suite membership test	Treat as unevaluated, extend suite
Accuracy	Evaluation record per class	Improve implementation or wrap

Remedies differ completely. A single number cannot tell which action to take. This is why reliability engineering belongs in architecture rather than in UI chrome.

Gates, caps, and ceilings

Reliability assembly uses three kinds of limit. Gates apply to sufficiency and stability: if either fails, the system does not proceed on this result alone. The caller branches, gathering input, abstaining, resampling, or escalating. Caps apply to coverage and accuracy, limiting how far a result may be trusted even when it sounds excellent. Unevaluated case classes do not drive consequential action without explicit acceptance of risk. Ceilings apply to determinacy: no cognitive unit exceeds human agreement rate for that judgment class. Low determinacy is a terrain fact that authority design must respect, not a prompt problem.

The Meal Companion allergy check has high determinacy once knowledge is endorsed. It belongs in code. Busy evening fit has patterned but fuzzy terrain. It belongs in a named cognitive units with suites, not in a confidence percentage.

Vendor benchmarks rarely map to a product's decision classes. A model leaderboard score does not tell you whether JudgeHouseholdFit meets a household's effort thresholds on evaluated runs. Reliability claims that name which judgment, which suite, which abstention policy, and which authority gates apply are engineering. Reliability claims without those specifics are confidence theatre migrating from UI to datasheet. Engineering reliability talk sounds less glamorous and ages better: this class, these conditions, this abstention rate, this deterministic shell for closed constraints.

What this looks like in the Meal Companion

Allergy enforcement in the Meal Companion is reliable because it is deterministic code with exact membership, always firing once knowledge is endorsed. That reliability is unrelated to whether the model feels confident about cashews in prose.

AssessMealPracticality on busy evenings is reliable only under evaluation. Sufficiency requires calendar and effort budget in working state. Coverage requires suite cases for busy-Tuesday shapes. Accuracy comes from graded runs against household thresholds, not from fluency. Marketing "ninety-nine percent accurate" without naming which decision class, under which conditions, with which abstention rate, is unreliable reliability talk. It optimises the demo, not the architecture.

Invoice and policy domains repeat the pattern at higher stakes. Extraction may be fluent while sufficiency fails on missing fields. Approval recommendations may sound authoritative while coverage is zero for the vendor tier in dispute. Reliability assembly forces those gaps visible.

Reliability, authority, and sufficiency in production

A reliable judgment can still overstep if authority was never granted. Reliability answers whether the decision was sound under stated conditions. Authority is granted answers whether the system was permitted to make or act on it. The Meal Companion may reliably assess meal practicality while still being forbidden to complete a grocery purchase. Conflating reliability with permission breaks trust even when evaluations are excellent.

Sufficiency works best when callers treat it as a branch input, not as decoration in logs. When AssessMealPracticality reports missing calendar context, the Meal Companion asks one targeted question or defers busy-evening fit rather than proceeding and apologising later. That pattern scales to enterprise workflows: an approval recommender that proceeds without vendor tier fields has failed sufficiency even if the narrative sounds confident. Wiring sufficiency to abstention prevents false completion from masquerading as reliability.

Teams sometimes resist removing confidence scores because stakeholders want a single KPI. The alternative is a small set of labeled states: sufficient and covered, sufficient but unevaluated, insufficient, unstable under resample. Those states map to actions. A float does not.

Reliability reviews happen when suites, models, or authority policies change, not when marketing requests a new accuracy badge. Each review lists judgment classes, coverage status, abstention rates, and deterministic gates that pass regardless of model output. Reliability regressions often arrive as silent drift: a wrapper change that removes sufficiency checks, a memory promotion path that bypasses endorsement, a new feature that branches on confidence scores again. The review catches those regressions before households experience them as "forgot" or "wrong allergy." Sharing review outputs with product and security keeps external claims aligned with internal gates. A datasheet that cites unevaluated terrain honestly ages better than a headline percentage that collapses on first exception queue.

What to do next

The first concrete step is identifying one production path that currently displays or branches on a model confidence score. Replacing it with sufficiency reporting from the cognitive unit plus caller-side gates tied to evaluation records makes reliability visible where it was previously performative. Adding one unevaluated case class label where coverage is genuinely missing, rather than hiding the gap, turns an uncomfortable truth into an honest architecture commitment.

See confidence theatre, abstention as a result, and paying for reliability.

Read next: Authority is granted.