Ecosystem

A demo agent produces a plausible weekly meal plan in a single session. Stakeholders applaud. Then production asks the questions demos skip: Where are endorsed allergies enforced? Who publishes shared judges? What runs before release when the model provider updates silently? Who approves grocery purchase? What audit trail exists when memory promotes? What regression catches busy-week failures that already happened once?

The agent loop is the visible motion. The ecosystem is everything that keeps judgment useful, honest, and improvable when the demo ends. Bare cognition inside a chat pane, without surrounding services, is unfinished Thoughtware. Parts that work in isolation but conflict in combination fail the same way.

This page maps what sits outside the agent loop but inside the system boundary. The Thoughtware system names the whole. Memory as architecture covers typed stores. Libraries of cognition packages reusable judgment. Evaluation as engineering turns quality into gates.

What the ecosystem holds

Agents and Skills compose judgments toward owned goals. They call cognitive units, read memory, invoke tools, and respect authority. The ecosystem permits, records, measures, and stabilizes that work. Memory stores hold typed forms with governance rather than one retrieval index. Library services publish, search, inspect, and substitute sealed packages. Evaluation pipelines run suites, regression, calibration, and release gates. Deterministic shell enforces exact work: validation, permissions, transactions. Authority and approval grant irreversible or sensitive actions. Observability supplies audit, cost attribution, and drift signals. Improvement workflows govern promotion versus silent drift.

Using the Weekly Meal Companion grounding scenario from the Thoughtware system, these layers take concrete shape:

Service	Role
Preference / knowledge store	Endorsed allergies, standing rules, purchase policy
Library catalog	Domain cognitive units, judge library, org AskTargetedQuestion
Grader jobs	JudgeWeekdayPracticality, variety judges, regression on busy-week failures
Deterministic shell	Allergy enforcement, plan patching, shopping list, idempotent actions
Purchase approval	Human gate before grocery order
Audit log	Plan version approvals, memory promotions, authority events
Eval dashboard	Release gates, calibration drift, known failure replay

The household sees plan and shopping list when architecture is sound, not scaffolding panels advertising models. See when AI stops being visible. Enterprise document workflows mirror the same layers: field-level cognitive units in libraries, extraction Skills, posting approval, ledger-backed system sources, posting regression sets. Different domain, same ecosystem shapes.

Retrospection, bridging, and evaluation

Three ecosystem functions disappear from casual diagrams but dominate trust.

Retrospection notices weakness after a step: critique finds a material gap, eval flags regression, observability catches drift. Without retrospection, agents loop blindly. Bridging fills a named gap under approval: missing authority, missing endorsed fact, missing observation that must not be guessed. Bridging is structured, not silent prompt expansion. Evaluation asks whether improvement is real, not demo fluency and not provider marketing. Promotion to libraries and expertise requires eval evidence.

Read retrospection, bridging, and approval and knowledge is endorsed, not retrieved. Source classification at boundaries, covered in four kinds of source, connects memory and cognitive units to ecosystem gates.

Parts that work in isolation but conflict in combination are unfinished Thoughtware.

Introduction to Thoughtware · Ch. 22

Ecosystem failure is often organisational rather than model-related. The memory team owns preference stores, the ML team owns prompts, platform owns observability, compliance owns audit, and none share a diagram. Posture promises challenge while UI hides alternatives. Libraries publish packages agents bypass in ad-hoc templates. Eval runs offline while production ships on demo applause. Coherence requires shared ownership of the outcome and a map everyone references. The ecosystem is the implementation of that map as services.

Growing the ecosystem without ceremony

Ecosystem diagrams without running cognitive units are ceremony. The minimum layers an outcome needs come first: memory typing for safety-critical facts with endorsement paths, a library publish path for repeated judgments with semantic versioning and suites, exact checks plus one grader lane before release, approval for irreversible or high-consequence actions, and audit sufficient to reconstruct promotions and authority grants. Growth comes from real use rather than from platform slides listing every possible service on day one. Expertise catalogs earn their place when compression is eval-backed. Meta-agents arrive when search and substitution rules exist.

Teams rarely stand up every service at once. A pragmatic sequence reflects maturity. At the prototype stage, an outcome lives on a map, one agent loop runs, working state has a schema, endorsed knowledge covers safety-critical facts, exact checks run in CI, and transcript serves audit. At the shared judgment stage, a domain library publish path exists, a grader covers one high-value cognitive unit, a regression set captures known failures, and approval gates irreversible actions. At the operational trust stage, a full eval dashboard tracks regression and calibration, substitution rules bind, expertise runs behind guards, observability covers promotion and source violations, and an improvement log ties to semantic versioning.

Skipping prototype discipline because "stores come later" produces demos that poison production expectations. Skipping shared judgment produces prompt forks that diverge silently across teams.

Observability, cost, and vendor resilience

Observability in Thoughtware is not generic uptime alone. Teams need cost-per-decision edges, source-kind violations, promotion events, guard failures, abstention rates, and regression diffs after model or library bumps. These signals feed improvement workflows of governed promotion, substitution, and rollback rather than uncontrolled prompt edits. From experimenting to governing describes how organisations mature ecosystem layers without freezing innovation.

Layer	Typical owner	Weekly touchpoint
Memory stores	Platform or product infra	Promotion audit sample
Library catalog	Architecture or ML platform	Publish queue review
Eval pipelines	Quality or ML eng	Regression triage
Deterministic shell	Backend squad	Exact check failures
Authority flows	Product + compliance	Approval latency review

The shared map is the coordination artifact. Without it, owners optimize local metrics that conflict globally.

Ecosystem services insulate product outcomes from provider churn. When the model API changes, regression suites and exact checks still run. When embeddings re-index, library semantic versioning and substitution rules still bind. When a calendar vendor updates fields, system source validation fails loudly rather than silently corrupting context. Teams that embed vendor logic inside agent prompts without ecosystem layers feel churn as emergency rewires. Teams with maps, libraries, and eval treat churn as a semantic versioning and regression problem with a known playbook. The difference between emergency and routine during a vendor migration is whether the judgment map existed before the migration began, because teams that built ecosystem layers discover that most of the work is remeasurement rather than redesign.

Cost attribution belongs in ecosystem observability rather than finance alone. Tagging cognitive unit calls, library versions, retrieval paths, and human escalations lets product leaders see cost per decision edge rather than an opaque monthly token bill. Drift signals like abstention rate spikes, guard failure increases, and promotion volume anomalies belong on the same dashboard as latency. Without cost and drift visibility, ecosystem investments look like overhead until an incident proves they were savings.

Approval services belong in availability monitoring too. Stuck approval queues block outcomes as surely as API outages, but teams often omit them from SLO dashboards until finance or compliance escalates. When two squads share one outcome, a single map owner and a joint eval calendar entry prevents the split ownership that reproduces ecosystem silos at the team boundary. Ecosystem maturity becomes visible in incident response: teams with typed memory and eval gates know which store to inspect, which suite to rerun, and which approval record to pull. Teams without ecosystem layers grep prompts.

What to do next

Drawing the ecosystem for one outcome and marking what is missing today versus demo-only is the highest-leverage first step, because the gaps become concrete engineering work rather than aspirational platform slides. Memory, library, and eval owners align around a shared map and shared release gates. Autonomy follows from catalog plus eval rather than prompt length or context window size. Promotion, approval, and source-kind violations belong in instrumentation alongside latency and cost.

See libraries of cognition, evaluation as engineering, and Thoughtware in practice for operating rhythms across ecosystem services.

Read next: The Thoughtware system.