Concept
Behaviour and trust
How a system should conduct itself once it can think.
Cover for Behaviour and trust18 concepts in this section.
All sections
Illustration for Cognitive postureHow the system shows up
9 minCognitive posture
Two systems with the same model can feel completely different. The difference is posture, the stable stance a system takes toward uncertainty, authority, and the people relying on it.
Illustration for Trust across timeConduct accumulated over time
8 minTrust across time
Trust is not a satisfaction score or a demo reaction. It is the accumulated expectation of how a system will behave across success, uncertainty, correction, and failure.
Illustration for Reliability is not a confidence scoreMeasured behaviour, not a score
8 minReliability is not a confidence score
A model's confidence number measures nothing useful. Reliability is repeatable correct behaviour under stated conditions, assembled from evidence, not emitted from a single call.
Illustration for Authority is grantedPermission is explicit
9 minAuthority is granted
Capability does not imply permission. Systems may act consequentially only within authority that was explicitly granted, with scopes, approval paths, and audit.
Illustration for Knowledge is endorsed, not retrievedFacts someone stands behind
8 minKnowledge is endorsed, not retrieved
Knowledge enters a system through endorsement, approval, scope, and provenance, not because the model once saw text or the user mentioned something in chat.
Illustration for Refuse and deferLegitimate non-answers
8 minRefuse and defer
Trustworthy systems must stop well. Refusal and deferral are success behaviours, clear, reasoned, and designed, not failures to be hidden.
Illustration for Recovery is part of trustRepair after a miss
8 minRecovery is part of trust
Trust is tested when things go wrong. How a system acknowledges error, preserves valid work, and repairs locally shapes reliance as much as a flawless first draft.
Illustration for Adaptability without unpredictabilityChange without surprise
8 minAdaptability without unpredictability
Users want systems that adapt to their week, not systems that surprise them each run. Adaptation should follow visible conduct rules, not caprice.
Illustration for Memory discipline in conductWhat the system remembers how
8 minMemory discipline in conduct
What the system treats as remembered versus contextual is behavioural design. Temporary instructions must not silently become durable beliefs.
Illustration for Confidence theatreNumbers need instruments behind them
9 minConfidence theatre
Scores, absolutes, and overfilled prose that signal certainty without evidence. Confidence theatre is the enemy of trustworthy conduct.
Illustration for Restraint prepares the decisionHold before you commit
9 minRestraint prepares the decision
Moving fast sometimes means not acting yet. Restraint gathers missing information and authority before false closure. Designed conduct that prepares judgment.
Illustration for Targeted questionsAsk what unlocks the call
9 minTargeted questions
Generic clarifying questions annoy users. A targeted question is the minimum ask whose answer changes the plan materially. It is a designed artifact, not improvisation.
Illustration for Challenge as conductPush back with grounds
9 minChallenge as conduct
Trustworthy systems sometimes push back. Naming impracticality or conflicting constraints is conduct, not error.
Illustration for When the system must stopKnow when to halt
9 minWhen the system must stop
Agent loops need designed stop conditions. Budget, success criteria, and escalation belong in architecture, not in infinite helpfulness or silent retry spirals.
Illustration for ContestabilityCallers can challenge results
9 minContestability
Users must be able to challenge outcomes without fighting the system. Visible rationale, alternatives, and correction paths replace opaque final answers.
Illustration for When the system should refuseI cannot decide yet
9 minWhen the system should refuse
Sometimes the correct output is no output. Evaluation-backed abstention prevents harm when authority, evidence, or terrain forbids completion.
Illustration for Least trustworthy input winsWeakest source sets the ceiling
8 minLeast trustworthy input wins
A chain of well-governed components fails at the weakest input. Design should identify and harden the least trustworthy source, often unendorsed context or unguarded retrieval.
Illustration for Persistence before regenerationKeep state before re-asking
8 minPersistence before regeneration
Regenerating from scratch destroys accepted work and trust. Correct locally, preserve partial progress, and replan only when material assumptions change.