Experience becomes expertise

Your team logs every Meal Companion run: requests, plans, corrections, costs, outcomes. After a month, someone says: "We have enough data. Let's promote a Busy Week strategy." Another says: "We should fine-tune on the logs." Both sound like learning. Only one path produces expertise. The other often promotes temporary noise into permanent policy.

Experience and expertise are different materials. Experience records what happened. Expertise extracts pattern only when promotion criteria pass. Confusing them is one of the most expensive mistakes in intelligence-native systems.

Helpful context: How expertise forms owns the expertise artifact shape. Evaluation as engineering and applicability conditions govern promotion. This page covers the pipeline between them.

Experience: capture without premature packaging

Each Meal Companion run can append to experience. The log includes the originating household request and calendar context, the strategy used (whether full loop or named pattern attempt), corrections the household made, cost and latency, outcome such as accepted plan or failure, and evaluation of which checks passed and which judgments were sound.

Experience is valuable because it feeds promotion decisions. It is not yet expertise. Raw history is not strategy. A folder of JSON without evaluation hooks is archive, not learning material.

Capture quality matters more than volume. Ten ungraded runs produce arguments in promotion meetings. One well-evaluated trajectory with correction provenance can settle a guard design dispute.

What good capture includes

Replay requires more than input-output pairs. Architects need enough context to reconstruct why a judgment looked reasonable at the time. They need correction provenance: who changed what, after which visible assumption failed. They need cost and latency to weigh compression tradeoffs honestly.

Missing evaluation hooks is the most common capture gap. Teams store plans without grading process quality. Promotion then debates vibes. Evaluation separates lucky outcomes from sound process before compression enters the conversation.

Experience also records guard outcomes when patterns were attempted. A failed guard is data. It tells you where surface similarity lied. It feeds false familiarity postmortems and guard version bumps.

Promotion: evaluate before compress

Promotion follows a disciplined sequence. The first step is capturing trajectories with enough context to replay judgments. Evaluation follows, separating process quality from lucky outcomes. When applicability conditions show stability across runs, the team proposes an artifact with strategy skeleton, guards, and failure modes. Approval and versioning create a library entry with a regression suite. Production monitoring then tracks fallback rates, guard failures, and misfires.

Skipping evaluation produces expertise that encodes last week's mood. Skipping approval produces expertise nobody will defend when it misfires. Skipping monitoring produces silent rot. Each missing step weakens the artifact in a different way, and the weakness typically surfaces at the worst possible moment: when a consequential week triggers the pattern.

Raw history is not strategy. Untested familiarity is not maturity.

Introduction to Thoughtware · Ch. 25

What must not promote

Temporary corrections must not become expertise without approval. "Avoid pasta because we had it several times recently" belongs in context for this run. "Busy evenings stay under twenty-five minutes" belongs in knowledge once endorsed. Busy Week Pattern v0.2 belongs in expertise once guards and evaluation support it.

Promoting pasta avoidance into a global strategy is how systems exhibit drift marketed as learning. Fluent adaptation nobody authorized still damages trust.

Household language can trigger premature promotion too. "Same as last week" is a user phrase, not a guard. The system may reuse context. It does not invoke expertise unless conditions pass. The distance between "same as last week" and a guard-checked pattern invocation is where most promotion mistakes occur.

Experience quality beats volume

More logs are not better experience. Capture must include evaluation hooks, context snapshots, and correction provenance. Otherwise promotion committees debate noise. One well-evaluated trajectory beats ten ungraded runs for expertise formation.

Data teams sometimes celebrate row counts. Architecture teams celebrate replay fidelity. The productive question is whether you can answer why Tuesday's meal changed after critique. Can you show which assumption the household contested? If not, experience is incomplete regardless of how many rows the database holds.

What this looks like in the household planner

Three consecutive weeks show two busy evenings, spinach deadlines, leftovers accepted, plans accepted with minor patches. Evaluation shows stable process. Guards are identifiable. Failures are rare. Corrections stay local.

The team proposes Busy Week Pattern v0.1. The regression suite includes weeks with one low-time evening that should fail the guard, weeks with medical mentions that should defer, and weeks with conflicting spinach constraints that challenge the skeleton.

Week four introduces a low-FODMAP mention. Experience logs the deferral. Expertise does not update to "always ask about FODMAP." That would overfit one trajectory. The guard adds a condition: unfamiliar medical restriction present leads to fail. The version bump documents the change. The loop between experience capture and expertise refinement is how the system genuinely learns, in the sense grounded verbs defines.

Experience without expertise is still valuable

Not every trajectory promotes. Experience supports debugging misfires, extending evaluation suites, auditing costs, and identifying closing candidates for deterministic code.

The pipeline does not force compression. It makes compression evidence-backed when warranted. Teams afraid of premature promotion sometimes never promote at all. They pay full deliberation cost forever while claiming they need more data. Stability criteria exist to end that stall. When evaluation shows stable process across enough runs and guards are identifiable, the evidence supports promotion.

Roles in the promotion meeting

Product often arrives with user anecdotes. Engineering arrives with latency charts. Evaluation arrives with graded trajectories. Architecture asks guard questions. Promotion succeeds when those inputs converge on an artifact with fields from how expertise forms.

When promotion fails, failure gets documented too. "Not yet stable" is a valid outcome. It is better than shipping expertise with a known missing axis. Documenting why promotion was deferred keeps the conversation honest and gives the team a clear list of what would change the outcome.

Monitoring after promotion

Promotion is not graduation forever. Production monitors guard failure rates, fallback frequency, correction patterns, and cost shift. Rising guard failures may signal environment shift. Stale expertise is a library maintenance task, not a crisis unless it goes unmonitored.

Experience logs after promotion feed the next version. v0.2 exists because v0.1 missed a material axis. The loop continues.

Anti-patterns in capture meetings

Several anti-patterns recur in capture discussions. "Fine-tune on everything" treats experience as training data without guard design. "Promote on anecdote" treats one delighted household as proof of pattern. "Row count envy" treats volume as maturity. "Prompt diff worship" treats any behavioural change as learning.

Each anti-pattern skips a step in the promotion sequence. Naming the skipped step ends most unproductive debates. If evaluation did not run, promotion waits. If guards were not proposed, promotion waits. If approvers were not identified, promotion waits. The sequence is the discipline.

Rolling back a bad promotion

Rollback must be as disciplined as promotion. When Busy Week Pattern v0.2 misfires in production, rollback means disabling invocation, preserving experience logs, opening a guard revision ticket, and communicating to support with the version number affected. Silent rollback that leaves copy promising learning destroys trust twice: once for the misfire, once for the hidden fix.

Previous artifact versions stay callable during investigation. Forensics may need to compare v0.1 and v0.2 behaviour on the same logged week. Experience archives make that comparison possible when capture was complete.

Sharing experience across teams

Experience logs become more valuable when shared with consistent schema. One squad's promotion decision does not depend on another squad's private spreadsheet. Libraries of experience are less formal than libraries of expertise, but the same hygiene applies: named fields, evaluation hooks, and access control for household data.

Cross-team sharing also prevents duplicate expertise. Two teams promoting similar busy-week strategies under different names will confuse substitution and telemetry. Central promotion review catches duplication early.

Assigning an experience steward when log volume grows prevents schema drift without requiring full platform ownership. Stewards watch for promotion proposals that lack evaluation attachments. Those proposals return to capture until grades exist.

What to do next

The most direct starting point is listing what the system logs today and asking whether judgments can be replayed, also final outputs. Where replay fidelity is missing, experience is incomplete, and promotion conversations will circle without converging.

Defining promotion criteria before the next "let's learn from logs" meeting gives the conversation structure: required stability, evaluation gates, approvers, and rollback path. The criteria make promotion a gate rather than a mood.

Applicability conditions covers guard design for promoted artifacts. False familiarity shows what happens when compression runs without adequate guards. How expertise forms defines the artifact shape promotion produces.

Read next: Applicability conditions.