Least trustworthy input wins

Allergy enforcement is deterministic. Meal judgments are evaluated. Library cognitive units are sealed. Then retrieval injects stale chat: "maybe cashew sensitivity?" without endorsement. The plan excludes cashews inconsistently. Sometimes strictly, sometimes not. The chain failed at its least trustworthy input, not at the strongest component. Trust composition is only as strong as the weakest governed input, and teams that instrument the middle of the pipeline while skipping seams discover incidents where retrieval meets knowledge, where human paste enters context, and where two evaluated cognitive units hand off without a joint suite.

Helpful context: Knowledge is endorsed, not retrieved hardens factual inputs. Four kinds of source classifies what fills each information hole. Reliability is not a confidence score assembles caller-side gates.

Rank inputs by governance

Typical trust ordering from strong to weak: deterministic code with tests (allergy enforcement), endorsed knowledge with provenance, sealed library cognitive units with evaluation records, cognitive units with suites and abstention, unguarded retrieval and raw chat and open web, and model priors with no grounding. Failure analysis always asks what the least trustworthy input was that influenced the output. If the answer is not available, hardening is not possible.

The chain is only as trustworthy as its least governed input.

The cognitive unit Pr. 26

Before hardening, classify what filled each gap. Four kinds of source names approved (endorsed knowledge), system (deterministic or governed records), observed (runtime input the caller supplied), and unaccountable (cannot be traced or verified). Provenance decides whether the cognitive unit may proceed, defer, or refuse. Mixing sources without naming them hides the weakest link. A standing allergy rule from the preference store is approved. A one-off chat comment is observed or unaccountable until promoted. A model guess with no citation is unaccountable. When sources combine, the unaccountable snippet cannot override approved allergy rules because fluent retrieval suggested it.

Pipelines and seams

The least trustworthy input often lives at seams. Two green suites can still fail at the join if handoff allows low-trust text to reach output without crossing endorsement or abstention gates. Common seam failures include retrieval proposing facts that downstream treats as endorsed, chat history injected as "memory" without write policy, human paste in an admin panel reaching model context unlabeled, two cognitive units each passing alone with no joint eval on the composed path, and a stale library version invoked beside current knowledge.

In the Meal Companion pipeline, InterpretWeek reads calendar API (moderate trust, validate schema), retrieves endorsed allergies (high trust after endorsement), and also retrieves "similar past chats" (low trust). If low-trust retrieval can override high-trust knowledge, cashew risk dominates. The fix is that retrieval may propose updates while knowledge store and code enforcement decide. Chat does not silently change the enforcement path. Library cognitive units are checked for version, applicability, and evaluation status before invocation. Libraries of cognition owns the inspection protocol.

The same pattern appears in invoice intake. OCR field is observed. Tax rule from finance is approved. Ledger total is system. Model guess on vendor tier is unaccountable until promoted. Auto-approval abstains when unaccountable sources remain in the path. Seam evaluation is as important as cognitive unit evaluation at composition boundaries. When two evaluated cognitive units hand off, joint cases that fail only at the seam reveal least trustworthy input faster than isolated tests alone. Example: allergy knowledge passes, retrieval passes, composed plan still drops cashew because a chat snippet entered between them. See unevaluated seams.

Caller-side gates and failure analysis

Reliability is not a confidence score belongs at the caller. Individual cognitive units may pass suites while composition fails. Caller responsibility includes labeling source kind per field before invocation, blocking promotion when unaccountable sources remain, abstaining when least trustworthy input would materially change output without endorsement, and logging which input dominated failure in incidents. When the system should refuse covers evaluation-backed abstention when gates fail.

For each production incident, the failure analysis lens asks the same questions: list all inputs that influenced output, rank by governance using source kinds, identify the weakest input that materially changed the result, and harden by endorsing, gating, abstaining, or removing. Repeat across incidents and patterns reveal architectural debt. If every incident traces to chat retrieval, stop tuning middle cognitive units and fix the seam. False familiarity often means a low-trust pattern matched familiar terrain while frontier facts sat in ungoverned context. The fix is guards plus source discipline, not longer prompts.

When correcting chains, persistence before regeneration preserves governed work. Regenerating from scratch re-introduces least trustworthy inputs across the entire plan. Local patch contains blast radius. When child cognitive units return conflicting source classifications, the caller abstains rather than picking the optimistic path.

Hardening patterns

When the weakest input is unguarded retrieval: block override paths, require propose-then-endorse, abstain when unaccountable text would change enforcement. When the weakest input is a stale library version: inspect before call, fail closed on version mismatch. When the weakest input is a seam handoff: add joint eval and caller gate. Callers label each field with source kind before invoking downstream cognitive units. Unlabeled paste behaves as unaccountable until promoted. Explicit labels let gates abstain when unaccountable input would dominate.

Observed facts from chat may propose promotion to endorsed knowledge. Until promotion completes, they do not override approved rules in enforcement paths. Four kinds of source plus least trustworthy input wins together prevent silent promotion through retrieval.

Runtime inspectability makes least trustworthy input analysis possible during live operation rather than waiting for post-hoc log archaeology. Operators debugging incidents need to see which inputs reached each call. Production logs record source kind for fields that changed outcomes on incidents, because logs without source kind force post-hoc guessing during review.

Pipeline inputs change as teams add retrieval, tools, and paste paths. Quarterly inventory of inputs with governance rank catches new weak sources before incidents. Least trustworthy input analysis is maintenance, not one-time architecture review. Tool calls return observed or unaccountable data depending on vendor. Treat vendor JSON as observed until validated. Tool output that skips endorsement gates becomes the new weakest input overnight when a tool is added without review.

New engineers inherit a diagram of input trust tiers for their service, because onboarding without trust tier docs repeats seam mistakes. Design docs for new features list expected source kinds for each new field, and reviewers reject designs that leave source kind implicit. Red team exercises that inject stale chat or bad retrieval into pipelines reveal weakest inputs faster than unit cost alone. When hardening lags, temporary caller-side abstention on known weak seams prevents harm while endorsement or retrieval fixes ship. Document temporary gates with expiry dates so they do not become permanent by accident. Review expiring gates in weekly ops meetings. Trust tier reviews in design docs complete the discipline.

What to do next

Least trustworthy input analysis converts pipeline trust from assumed to observable. The trust ordering, the source classification, the seam evaluation, and the caller-side gates together prevent the pattern where strong middle components mask weak inputs at composition boundaries. Each pipeline has a weakest link, and the architecture either identifies and hardens it or discovers it in production.

Read knowledge is endorsed, four kinds of source, and abstention as a result.

Read next: Persistence before regeneration explains how to preserve governed work when correcting chains.