Two budgets, cost and time

A Meal Planning Agent that runs five cognitive units on every evening, samples each one three times, and loops until critique passes can produce a good plan. It can also burn through a monthly inference budget in a week and miss a household's Tuesday deadline because nobody counted seconds alongside dollars. The system worked in demo. It failed in production because architecture treated only one constraint as real.

Every Thoughtware system carries two budgets: money (compute, model calls, wrappers, human review) and time (latency, deadline pressure, loop depth). Most teams watch only the first. The second shows up as user frustration, abandoned sessions, and agent loops that work in demos but never finish in production.

Helpful context: Evaluation as engineering supplies the suites that justify spend. Maturity moves settled work out of runtime explains compression that saves both budgets. Deterministic guards before cognition is the first lever that shrinks both at scale. This page opens the Economics track after evaluation replaces certainty.

Money and time are both finite

A deterministic allergy check runs in milliseconds and costs nothing per token. AssessMealPracticality on a busy Tuesday evening costs both: model inference and the household's patience. Wrappers that buy reliability through sampling, critics, and cascades multiply money and often multiply time in equal measure.

Architecture that treats only the provider bill as real will over-invest in rare prestige judgments and under-invest in front-of-funnel guards that save both budgets at once. A triple-sampled variety judge on a familiar week may polish a number nobody notices while Tuesday's plan arrives after dinner. The same pattern appears in invoice processing, where tolerance negotiation on a trivial mismatch pays full latency while the clerk waits on a threshold the system could have applied deterministically.

Budget	What it measures	What ignoring it causes
Cost	Tokens, wrappers, human review per edge	Latency spend treated as free
Time	Per-call latency, loop depth, deadlines	Money spent on redundant deliberation

Allocation before optimisation

Before cost per decision arithmetic becomes useful, someone must decide policy: how much of each budget each class of work deserves. High-consequence medical deferral may warrant triple sampling and a human bridge. Variety judgment on a familiar week may not. The policy decision is architectural, not financial. It belongs on the decision map before individual edges receive prices.

The Busy Week Pattern exists partly because compression saves both budgets: fewer cognitive unit calls, fewer loop iterations, faster acceptance when applicability guards pass. Compression without measurement is hope. Compression with eval is architecture. The discipline is naming budget allocation per path so that arguments happen against policy rather than against intuition.

Cost follows volume, and volume is highest at the front of a funnel where the work is comparatively mechanical.

The cognitive unit, Ch. 12

Policy tables over dashboard totals

Monthly provider totals answer "what did we spend?" They do not answer "what should we spend on busy-week practicality versus variety versus allergy enforcement?" Policy tables attach budget caps to decision classes and paths, making the architecture arguable in review rather than discoverable only in incident postmortems.

An illustrative policy sketch for Meal Companion shows how the two budgets interact across different consequence tiers:

Path	Cost policy	Time policy
Allergy enforcement	Deterministic, no token spend	Sub-second
Busy-week assessment	Mid-tier model, single run	Under thirty seconds per evening
Household fit on endorsed restrictions	Double-run judge	May extend total plan time by one minute
Full loop on unfamiliar terrain	Higher spend allowed	Hard deadline with abstain if exceeded

Numbers can move as traffic patterns shift and model prices change. The discipline is naming paths so arguments happen against policy, not against vibes. A team that lacks this table discovers budget failures in production when the household gives up waiting, not in design review when the constraint was still cheap to fix.

What this looks like in the household planner

Consider two paths for the same household request on a busy week. The unguarded path runs GenerateCandidates for every evening, assesses each with full deliberation, composes, critiques, and patches. Five cognitive units times seven evenings, no short-circuit. Cost is high. Time means the plan arrives after dinner.

The guarded path looks different at every stage. Schema validation and allergy enforcement run first, costing nothing and completing in milliseconds. A busy-evening flag routes Tuesday and Thursday to AssessMealPracticality with a mid-tier model. When the Busy Week Pattern guard passes, the agent anchors busy evenings first and skips regenerating candidates for evenings already accepted. Cost drops to a fraction. Time means the plan is ready before the household needs it.

The difference between these paths is budget allocation, not a cheaper model alone. Closed work and guards consume almost nothing, and open judgment concentrates where terrain is genuinely open. Architecture made both budgets visible and distributed them according to consequence rather than treating every evening identically.

Incoming requestGuards (cheap, fast)Open judgmentsWrappers where consequence justifies

Both budgets shrink when guards and closed work sit at the front of the funnel.

Instrument both budgets in review

Monthly engineering review works when provider cost trend and p95 latency to accepted outcome appear on the same slide. Spikes in one without the other diagnose different problems. A cost spike with flat latency may mean wrapper multiplication on a rare path. A latency spike with flat cost may mean loop depth or serial calls that guards should have prevented. Seeing both numbers together is what makes the conversation productive.

Household-facing deadlines belong in the architecture artifact. If Tuesday plan must arrive by 5pm, time budget policy declares what abstains or compresses when the agent loop approaches the cap. Without that declaration, the system runs unbounded loops and delivers a perfect plan to a household that already ordered takeout.

Loop depth as time budget

Agent loops multiply both budgets. Each iteration re-runs compose, critique, and sometimes candidate generation. A loop that terminates on average in two passes but occasionally runs six creates tail latency that users experience as unreliability even when mean cost looks acceptable.

Cap loop depth explicitly. Define abstain or defer when deadline or iteration cap hits. Loop compression when expertise guards pass is how mature systems honour time budget without abandoning quality on open terrain. The tail matters more than the mean because households remember the evening they waited four minutes, not the average wait of ninety seconds.

Review p95 and p99 latency to accepted outcome alongside mean. Agent loops that average two iterations but occasionally run six create the tail users remember. Compression and guard placement often improve tails more than mean cost optimisation. Pair tail metrics with loop depth histograms in monthly review so the team sees distribution shape, not a single number that hides production experience.

Delegation without losing the map

Teams sometimes split "cost owner" and "latency owner" across infra and product. That split works only when both read the same decision map. Infra optimizes provider tier globally. Product adds features that multiply evenings. Without a shared map, each team hits its local metric while the system misses household deadlines and burns budget on unguarded paths.

One architect or tech lead keeps the policy table current when edges are added, guards move, or expertise patterns promote. The table is the contract between budgets and design. When that ownership is absent, budget conflicts surface as incidents rather than as design conversations.

Human time as a third line

Human review, clerk exception handling, and household approval steps carry cost and latency often omitted from inference bills. Raising the autonomy rate (share of cases concluding without human touch) may save more than halving a mid-funnel template rate. Counting assisted cases separately from autonomous ones reveals the true budget picture. A clerk reviewing every assessment has not displaced the full human decision. That clerk time is cost and latency the household experiences even if the provider bill stays flat.

Finance can engage productively when human minutes attach to named decision classes the same way tokens do. Feature budget spikes without decision map updates usually mean new open judgments arrived unnamed. Decision budget spikes without feature movement may mean volume grew on expensive edges. Both signals are visible only when human time appears on the same page as token spend.

When budgets conflict

Sometimes the cheapest path is too slow and the fastest path is too expensive. Architecture resolves that tension with policy, not heroics. Busy-week paths may accept mid-tier models to meet deadline while unfamiliar-terrain paths may spend more and include a human bridge for safety. Documenting these tradeoffs on the decision map means product chooses consequence-aware defaults rather than forcing engineering to guess which budget yields when pressure arrives.

The connection to evaluation matters here. Paying for reliability and eval suites spend both budgets deliberately. Skipping eval saves money until failures externalize to support and trust erosion. Evaluation replaces certainty names the operating norm. Two budgets name the constraints that norm must respect. Without named constraints, eval spend has no ceiling and reliability arguments have no anchor.

What to do next

Listing both money and time costs of one end-to-end run through a production path reveals which edges run on the full population when they should not. Placing deterministic guards before expensive cognition is typically the fastest way to shrink both budgets on the highest-volume paths. Drafting a policy table linking decision classes to budget caps creates the shared artifact that makes per-edge pricing conversations productive rather than abstract.

See maturity moves work out, evaluation as engineering, and deterministic code.

Read next: Deterministic guards before cognition.