Thoughtware

Software executes. Judgment does not

Software can produce and carry out an answer, but it cannot inherit responsibility for whether that answer was the right call. Open decisions therefore need an accountable owner.

8 min read

Cover for Software executes. judgment does not

In a review of a meal planner, one engineer says the model can decide whether a dinner fits a busy Tuesday. The demo supports the claim: the proposed meals look sensible. Another engineer asks what happens when the planner recommends a forty-minute dish to a household that has twenty minutes.

The first question is about capability. The second is about judgment. Software can execute an open decision as willingly as a closed one, but execution does not take responsibility for the decision's consequence. A system that has been given the ability to form answers still needs a design that says who owns the quality and limits of those answers.

Helpful context: What is judgment provides the test for open decisions. Where does a decision live? gives those decisions an address, while authority is granted separates responsibility for a recommendation from permission to act.

Closed decisions can be handed to code

Once the household has endorsed Leena's cashew allergy, every candidate containing cashews should be rejected. Once a recipe and party size are known, portions can be calculated. These are not lesser forms of intelligence. They are closed decisions: the relevant rule exists, the inputs determine the result, and a wrong outcome is a defect that a test can expose.

Conventional software was built around this arrangement. People decided what should happen, software made it happen reliably, and responsibility for the underlying choice remained visibly human. The machine could fail to apply the rule, but it was not making a call about whether the rule fit the situation.

Open decisions leave a responsibility behind

Busy-evening fit is different. A meal may technically take forty minutes but require only ten minutes of attention. Another may take twenty-five minutes yet demand constant coordination while the household is settling children after school. A competent person can explain why one is acceptable and the other is not, but no fixed rule exhausts the comparison.

Here the system is not simply executing an answer already present in the design. It is interpreting circumstances and weighing trade-offs. Giving a model that task can reduce human cognitive effort, but it cannot make the responsibility vanish. If the plan is persistently impractical, someone must therefore be able to identify the decision that failed, revise its standard or evidence, and show that the revision actually improves the result.

Calling the output “what the model said” prevents all three. It treats a vendor response as the owner of a product decision, even though the product chose the context, the policies, the available actions, and whether the response would be trusted.

Capability has increased while architectural visibility has weakened.

Introduction to Thoughtware · Ch. 2

Ownership needs an address

An owner is not merely a person who is blamed after an incident. It is a component or role with a defined decision to own. AssessMealPracticality, for example, can declare the evidence it receives, the verdict it produces, when it should abstain, and the evaluation cases that measure it. A failure then has a place to go.

Without that address, a team responds to the same complaint by changing the prompt, a UI toggle, a retrieval weight, and a default preference. If the complaint stops, nobody knows which intervention mattered; if it returns, the system has learned nothing. The product has capability but no accountable object to improve. That is why a named owner is an engineering requirement, not a post-incident formality.

StepNature of the workWhat owns it
Reject recipes with cashewsClosed executionDeterministic validation
Calculate portionsClosed executionDeterministic code
Decide whether a meal fits TuesdayOpen judgmentAssessMealPracticality
Balance the full weekOpen judgmentPlanning agent strategy
Place a grocery orderConsequential actionHousehold approval

Judgment is not authority

Ownership of a judgment also does not give the system permission to act. The planner may be able to recommend a grocery order and still lack authority to submit it. Because capability answers whether a system can do something while authority answers whether it may, a high-quality recommendation does not bridge that gap.

This distinction keeps responsibility legible on both sides. The cognitive unit owns the quality of its recommendation, while the household or another authorised role owns the decision to commit money or create an obligation. Combining them because the model can perform both is therefore how a useful assistant becomes an ungoverned actor.

What to change in a design review

For every open decision, ask one question before debating the model: “Who can we point to when this result is wrong?” The answer should be a named judgment with a contract and an evaluation suite, not “the assistant,” “the prompt,” or “the team.” If no answer exists, the product has moved judgment into software without bringing accountability with it.

Read next: What is judgment defines which decisions need that kind of ownership.

Ownership is visible before an incident

An owner should be evident in the design, not assigned after a complaint. A reviewer ought to be able to point to the decision, read the standard it is held to, identify the evidence it may use, and see the cases that would make the team revisit it. These are ordinary properties of a dependable software boundary, applied to work whose result cannot be fixed in advance.

That standard also makes delegation more honest. The product can propose a plan confidently when it owns the quality of the recommendation, and it can stop confidently when a purchase, a medical interpretation, or another consequential decision belongs elsewhere. Ownership is therefore not an extra layer of process. It is how assistance remains useful without quietly becoming unaccountable autonomy.

Ownership changes the release standard

A feature with an unowned judgment should not be treated as complete because a demo looks plausible. Its owner must be able to state the decision, show the cases that test it, and explain what happens when the evidence is insufficient. These are release conditions because they determine whether a failure can be repaired after users encounter it.

This does not place all responsibility on one component. The component owns the quality of its judgment; deterministic services own their exact rules; product owners define acceptable consequence; authorised people grant actions. Clear ownership distributes responsibility instead of concentrating it in “the model.”