How decisions close

The dinner planner has been running for eighteen months. AssessMealPracticality still judges whether a candidate meal fits a busy evening, and it still costs a model call every time it is asked. Sitting in the cache in front of it, the same busy-Tuesday shape has come back with the same answer several hundred times, at no cost, with no variance.

Nobody has looked. The decision closed in the traffic and the architecture never heard about it, so the team is still paying monthly for an answer they already have on file, and still describing that answer as a judgment in their own documentation.

Closure is therefore not a verdict a team reaches about a decision. It happens case by case while the system does ordinary work, and it stays invisible until somebody reads the evidence, extracts the rule, and writes down what moved. The drift toward closure is automatic and every benefit from it is deliberate, which is why closing is a process a team operates rather than a milestone it reaches.

Helpful context: Open, closed, and closing owns the two questions that sort a decision, and closed decisions belong in code owns the bar for calling one settled and what a close commits you to. This page covers how a decision arrives at that bar over time. Contested decisions covers the state before any of this can start.

Why decisions drift one way

Open, closed, and closing establishes the asymmetry this page runs on: one of its two answers is a fact about the decision and holds still, while the other is a fact about knowledge and travels. It travels in one direction, because what a team knows about a decision class does not spontaneously decrease. So the interesting question is not whether a decision moves but what the movement looks like from inside a system that is trying to profit from it.

It looks like five states, and they are unequally hospitable. Contested work waits on a policy nobody has set. Open work has a fact of the matter and no writable rule, which is where cognition earns its place, and a cognitive unit that has been sitting there for three years doing useful work is not a failure of anything. Closing work is where answers stabilise and the same shapes of case start producing the same shapes of answer, which makes it the state where all the work of this page happens. Closed work is where the rule could be stated and nobody has written it, so it is short in a healthy system and long in a neglected one, and expensive throughout because the price is paid on every run. Then the rule executes in code.

The first of those transitions is unlike the rest, and skipping it is the error that produces the worst version of all this. Contested to open happens when somebody with standing settles what the organisation values, and no volume of accumulated cases produces it, because data cannot say what a household or a company ought to prefer. Every transition after that one is cases making a pattern visible. Which means a team caching answers to a preferential question is not maturing its architecture, it is storing quiet policy under the name of evidence, and contested decisions covers what the system owes people until the settlement arrives.

Loading diagram…

Caching is how a class closes, case by case

The mechanism that carries a class through those states is something most teams already run and read as something else. Put a cache in front of a cognitive unit, keyed on its semantic inputs, so that when a case arrives the cognitive unit is called and the answer is stored. Once the same case arrives again, the answer comes back from the store without a model being involved at all.

Consider what has happened to that second case. It is now answered deterministically, identically every time, in a microsecond, for nothing, with no variance whatsoever, which means every property that made the decision uncomfortable has gone and none of them came back. A cached case is therefore a closed case, and that is why closure looks gradual instead of sudden: it arrives one case at a time, per class rather than per cognitive unit, so a single cognitive unit can hold one class that has nearly closed beside another that is wide open.

Caching closes a decision. It is not an optimisation.

The cognitive unit · Ch. 6

Read that way, the hit rate on a class of cases becomes a direct measurement of how much of that class has already closed, which is why it is worth pulling out of the dashboard where it currently sits under performance and says nothing anybody acts on. A class running at eight percent hits is genuinely open, so leave it alone and let the cognitive unit do the work it exists for. A class running at seventy percent is most of the way to being a rule, which means somebody should go and look for the rule before the system spends another quarter paying to reconstruct it.

The instrument is only as good as the key, which is where this goes wrong in practice. If the key is too coarse, cases that differ in a way that matters collide, and the second one receives an answer computed for the first. The hit rate looks excellent while the system is quietly wrong, and wrong in the most awkward available way, which is consistently. A hit rate that improves suddenly after somebody simplified a cache key is not good news and deserves investigation rather than celebration.

The opposite error is more common and less alarming, because it costs money rather than correctness. Include a timestamp, a request identifier, or any field that varies per call without affecting the answer, and the rate sits near zero forever while the class closes with nobody noticing. Both failures have the same repair, which is to build the key from the inputs that would change the answer if they changed and nothing else. The cognitive unit's declaration already names those inputs, so the work is small and it is what makes the whole instrument trustworthy.

Reading the evidence out as a rule

Once the hit rate says a class has mostly closed, the material for the rule is already sitting in the cache and the evaluation suite. You have a body of cases with their answers, produced by something that was reasoning about them, and the only question left is whether the mapping from case to answer can be described. Frequently it can, and frequently the description is embarrassingly small, because a great deal of what looked like judgment turns out to be two comparisons and a threshold nobody had identified.

Extraction has a test attached, which is what stops it from being a guess dressed as a milestone. Run the candidate rule and the cognitive unit side by side over the suite. If the rule agrees with the cognitive unit on the cases the cognitive unit was right about, and disagrees mainly where it was wrong, then the rule is the better instrument and should take over. Inside AssessMealPracticality, the effort budget for a busy evening reached that point only after enough weeks of household corrections had made the threshold visible, and what authorised writing the number down was the side-by-side comparison rather than anyone's confidence that the number was correct.

That comparison is also the guard against the commonest way extraction goes wrong, which is doing it on request. A team asked to reduce model spend can write a threshold in an afternoon, and the threshold will look identical to an extracted one while resting on nothing. The evidence is what makes the difference, so a close performed without it has skipped the only step that could have said what the rule should be.

Where the rule does hold, the structure it lands in and the four things a close commits the team to belong to closed decisions belong in code. What matters here is the arrangement's continuing reading: because coverage is partial, the rate at which cases still reach the cognitive unit is itself an instrument, and a rule that covered nine cases in ten last month and covers eight this quarter is reporting that the incoming work changed shape while everybody was watching cost.

When the reading was wrong

A purely one-directional story would be dishonest, because decisions do occasionally move back, and the return path looks nothing like a system degrading. A rule producing a steady stream of complaints was quite possibly never closed at all. Somebody read agreement into the class, wrote the rule, and was wrong about the agreement, so the complaints are reporting a classification error rather than a defect. Where rules stop working supplies the signal that distinguishes the two, which is users saying the system does not understand their case, at a rate that stays flat however many exceptions get patched in.

The busy-evening threshold shows what that looks like. It ships, the complaints keep coming, and the reason turns out to be that unattended oven time is not the same imposition on an evening as active time at the hob, so active effort was never the whole of practicality. Moving the decision back into a cognitive unit is the repair, and it feels like travelling backwards along the lifecycle while being nothing of the kind, because the decision was never at the position somebody had placed it. What gets corrected is a person's reading, so the record of who read it and when is what makes the correction possible.

Genuine change in the world does the same thing occasionally and is far easier to diagnose, because a regulation changed or a new category of case appeared and there is a date attached. Slow misclassification is the hard one, since the tests still pass against the wrong definition of the judgment and every dashboard stays green. That is the asymmetry mistakes both ways describes, arriving at the end of the lifecycle rather than at the start of it.

A close nobody recorded did not happen

Every diagnosis in the previous section depends on knowing what was promoted, on what evidence, and when. Without that, a constant-rate complaint stream is indistinguishable from a model having a bad quarter, and the team spends its investigation in the wrong layer entirely. The record is what makes closure legible after the fact, which is the part of the process most often skipped, because at the moment of promotion the reasoning feels too obvious to write down.

The useful test for what belongs in the record is which question a later reader will be unable to answer without it. An endorsed knowledge entry with an approver and a date answers whether the twenty-five-minute budget was a household decision or a developer's estimate. The cache key definition answers whether last year's hit rate meant anything, and it is the item most often missing when an old number turns out to have been measuring collisions. The side-by-side evaluation report answers whether code replaced cognition on evidence or on a release schedule. Fallback telemetry answers whether the rule still covers what it covered when it shipped.

At the level of the whole system, maturity moves settled work out is the principle these records serve, and the crude instrument for whether it is happening is the cognitive unit count. A healthy system has fewer cognitive units doing more valuable work after two years, because decisions that turned out to be specifiable were specified and the cognitive units that owned them were retired or kept as fallbacks. A count that only grows means either the system is expanding into new territory, which should be visible as new cognitive units in new areas rather than accumulation in the old ones, or nothing is being learned and the team is operating a system without observing it.

Which is the point the lifecycle keeps making in different registers. Cost per decision reads the same drift as money and finds that the marginal cost of a decision falls as the system ages, and that only happens where somebody is running the loop described here. Nothing about it is automatic, and a team that has never once retired a cognitive unit has not started.

What to do next

The reading available this week is hit rate per class rather than per cognitive unit, because closure is a per-class property and any aggregate hides a class at ninety percent behind a class at eight. Two classes near seventy percent then need opposite responses, which is what makes the number worth reading rather than reporting. Falling complaints alongside a high rate means the rule is sitting in the cases you already have. Rising complaints alongside a high rate means the key is colliding, and extracting a rule there would harden the collision into code.

When selection hides responsibility covers the way a correct close can still cost trust. A household that sees outcomes without seeing which checks passed or which rule fired cannot tell a matured system from a confident one, so closure nobody can inspect arrives as opacity even when the architecture behind it improved. That page ends the Decisions track.

Read next: When selection hides responsibility.