Judgment as a Software Material

Software architecture is, in part, the discipline of deciding how different kinds of capability should be organised. Computation is placed into functions and services. State is given persistence and lifecycle. Storage is separated according to access patterns and durability requirements. Messaging connects components that should not depend on synchronous execution. Permissions define who or what may act. Transactions establish boundaries within which changes must remain consistent. Interfaces determine how capabilities are exposed to people and other systems. Each of these is more than a technology. It is a material from which software systems are composed.

The emerging material of cognitive software is not “AI” in the abstract. AI is too broad a description to provide useful architectural guidance. The more consequential capability is the ability to delegate bounded cognitive responsibilities to the running system. Software can increasingly be asked not only to execute a procedure, but to determine an appropriate conclusion or action when that procedure cannot be exhaustively specified in advance. In this sense, judgment becomes something architecture must learn to place, constrain, combine, and evaluate.

The distinction can be expressed simply. A deterministic component is given an input and a procedure: given X, execute specified procedure Y. If the specification is complete and the implementation is correct, the expected behaviour follows from that procedure. A cognitive component receives something different: an input together with a specification of what a good result should achieve, and it must determine the result. Its contract is closer to: given X and a specification of what good judgment means, determine Y.

This does not create a clean boundary between “non-intelligent” and “intelligent” software. Cognitive responsibility exists across a spectrum. Consider a sequence of common business decisions. Calculating VAT is predominantly computational: the relevant values and rules can usually be specified precisely. Assigning an expense to a category may involve classification, especially when descriptions are inconsistent. Determining what a customer complaint is actually about requires interpretation. Proposing an appropriate remediation introduces recommendation and the balancing of several considerations. Deciding whether the remediation can be executed automatically or requires human escalation introduces consequential judgment involving policy, uncertainty, authority, and risk.

The important change across this sequence is not simply an increase in complexity. It is a gradual reduction in our ability to prescribe the complete path from input to correct output. Where the desired result can be reliably expressed as a procedure, deterministic implementation remains preferable. Where the problem requires interpretation of incomplete, variable, or contextual information, the specification increasingly describes the quality and boundaries of the decision rather than every operation required to produce it.

This distinction also prevents an unhelpful architectural tendency: treating every opportunity for model use as an opportunity for cognitive delegation. Judgment is not valuable merely because it is possible. A language model can be asked to perform arithmetic, generate identifiers, decide access permissions, or reproduce rules that conventional software can execute exactly. Doing so may add variability where none is needed. Cognitive capability becomes useful when the problem itself contains meaningful uncertainty, interpretation, or contextual choice. Thoughtware therefore depends as much on knowing where not to use judgment as on knowing where to introduce it.

The suitability of a judgment for delegation also varies with the terrain in which that judgment occurs. Several factors are especially important. Pattern density describes how strongly the situation resembles cases for which useful patterns are available. Knowledge coverage concerns whether the system has access to the information necessary to reach a sound conclusion. Novelty captures how far a situation departs from familiar cases. Consequence describes the cost of a poor decision. Reversibility asks whether an incorrect action can be easily undone. Evaluability concerns how readily the quality of a decision can be assessed.

Together, these factors form what this paper calls the Judgment Terrain. They do not produce a universal formula for automation, but they provide a more useful way to reason about delegation than a binary question of whether a model is capable of performing a task. A highly familiar, well-grounded, reversible, and easily evaluated judgment may be suitable for substantial machine authority even if some uncertainty remains. A novel, poorly grounded, irreversible judgment with severe consequences may require human involvement despite impressive model performance. The same underlying cognitive capability can therefore justify very different levels of autonomy depending on the conditions surrounding its use.

This is why capability and authority must remain separate architectural concerns. A system may be capable of recommending a refund without being authorised to issue it. It may be capable of interpreting a medical document without being permitted to make a consequential decision from that interpretation. It may assess whether evidence supports an insurance claim while remaining required to escalate cases above a defined threshold. Delegating judgment does not imply delegating unlimited action.

Treating judgment as a software material therefore changes the central design question. The question is no longer simply whether AI should be used. It is which judgments should be delegated, where those judgments should reside, what information they should be allowed to use, how much authority they should possess, and how their quality will be established.

Once judgment becomes an architectural material, it requires something software architecture has always demanded of important responsibilities: locality. A judgment needs a place where its purpose, inputs, outputs, boundaries, and quality can be made explicit. That need leads to the smallest structural primitive of Thoughtware: the Cognitive Unit.