Thoughtware

The Intelligence Age

Cognition became cheap, and the scarce thing became judgment about cognition. This page explains what that shift asks of the people who build software, and why a stronger model leaves the hard part untouched.

11 min read

Cover for The Intelligence Age

A household opens a dinner-planning product and writes:

Plan dinners for four people this week. Tuesday and Thursday are busy. Use the spinach before Wednesday and avoid meals we ate last week.

The household is not describing an implementation. It is stating an outcome in the language of its own week. The work behind the request is different: the product must read the week, weigh constraints that pull against each other, compare non-equivalent options, and revise when one fails. A recipe site answers with links because it was built to search. A spreadsheet answers with empty columns because it was built to store. What has changed is that households increasingly expect the product itself to do some of the weighing.

That expectation gets read as a product trend, and reading it that way is how teams end up with demos. The capability that became cheap is cognition. What became scarce is judgment about cognition: which of those weighings a system may perform, on what information, under whose authority, and who answers for a wrong one. The Intelligence Age is defined by that scarcity rather than by the capability, which is why each new model release leaves the defining problem roughly where it found it.

Helpful context: this is the first page in the collection and assumes nothing. From computation to cognition takes the same shift inside the architecture, and software executes and judgment does not works out who carries the consequence. This page stays with the historical change and what it asks of builders.

Every age is named after what it made cheap

Ages of human work take their names from whatever became abundant in them. Land and cultivation supported settlement, mechanical power supported industry, and information supported the software industry most of us have spent our working lives inside. In each case the abundance did more than lower a price, because it changed what people attempted, which skills were worth having, and what the tools of the period were expected to do.

The lag matters because abundance creates a shortage, and the shortage is what the engineering of the period ends up managing. When information became abundant, attention became scarce, and feeds, ranking, notifications, and summaries were all designed around that shortage rather than around the data itself. None of that shortage was visible at the beginning. It became unmistakable once the abundance was ordinary.

AgeWhat became abundantWhat became scarce
AgriculturalLand and cultivationLabour and season
IndustrialMechanical powerCoordination and capital
InformationData and communicationAttention
IntelligenceCognition in softwareJudgment about cognition

Cheap cognition made judgment about cognition scarce

The capability now becoming abundant is cognition: software that interprets an ambiguous request, weighs incomplete evidence, compares options that are not equivalent, drafts a plan, criticises its own output, and revises it. That capability is uneven and frequently wrong and improving quickly, and none of that is what matters here, because impressiveness in a demonstration is not the property that makes a thing hard to build with. What matters is that cognition has become something systems are assembled from while the practice of assembling with it is still largely undescribed.

Because an answer, a draft, or a plan now costs seconds, the difficult questions all arrive after it: which answer should be trusted, which decision may be delegated and which may not, what information was allowed to influence the result, who is accountable for the consequence, and how anyone would notice if the behaviour quietly degraded after an update. A stronger model improves the raw answer and touches none of those. They are questions about standing rather than quality, and standing has to be arranged in the architecture before the answer is produced.

The scarcity shows up first in the work people do

Before any of this reaches architecture, it is visible in what people spend their day on. Work that used to begin with a blank page now begins with a draft, work that used to begin with a search now begins with a summary that has to be checked, and work that used to mean following a procedure now means reviewing a proposal and deciding whether it is sound.

The centre of effort moves from producing to judging, and that is a change in the skill being exercised rather than a reduction in the work. Reviewing a plausible weekly plan requires knowing what a good result looks like, noticing what has been quietly assumed, and being willing to reject something that reads well: spinach scheduled after Wednesday, a three-course recipe on a busy Tuesday, repetition the household explicitly asked to avoid.

Software that participates this way therefore cannot be assessed on whether it produced output, because the person on the other side has already stopped assessing it that way. It has to be assessed on whether the judgment it carried was sound, whether it knew what it did not know, and whether it stopped when the missing element was a decision belonging to a person. Those properties have to be designed in, which is a question about what cognition is treated as during the build.

Why cognition has to be treated as a material

The difference between a feature and a material is that a feature gets inserted into a product while a material is chosen for some parts of a structure and refused for others. That stance carries obligations, because materials have properties and characteristic failure modes, which means they need joints where they meet other materials, tolerances that say how much variation is acceptable, and inspection that catches the cases where a joint failed.

Refusal is the half the feature framing loses, and refusal is where the consequences sit. Products that assign a whole domain to either the machine or the person are almost always wrong at the edges, because an assignment made at that grain has no way to represent the cases falling outside it. Household dinner planning can be delegated generously until the week contains an unfamiliar medical restriction or a grocery purchase, and those two moments are the reason the design needs edges at all.

Treating cognition as material also means it can be named, bounded, versioned, evaluated, and swapped, which cognition as material develops once the vocabulary exists to say it precisely. The narrower point here is what the stance costs immediately: allocation becomes a design act performed one decision at a time, rather than a platform choice made once and announced.

Building with it breaks the old guarantees

For most of its history, software came with a set of guarantees nobody thought to name, because behaviour was specified in advance and the machine only carried it out. Output could be compared against an expected value, a wrong behaviour could usually be traced back to a responsible line, and the same request produced the same answer next week.

Cognitive software weakens all of them, because inputs arrive incomplete and several outputs may be acceptable, so the same request can be handled differently on two occasions for defensible reasons. Behaviour then depends on retrieved information, accumulated memory, instruction, model choice, and the arrangement of the steps, as much as on the code the team wrote.

Correctness gives way to adequacy, and adequacy has to be defined rather than assumed.

Introduction to Thoughtware · Ch. 1

Engineering discipline survives that, but it has to be rebuilt around different questions: where each judgment lives, what may influence it, what must remain exact, who is permitted to decide, who is accountable, and how improvement is distinguished from change. The last one bites soonest, because a model refresh can change how a request is interpreted without touching a line of the team's own code. A team that cannot say which of its judgments moved cannot say whether the upgrade was an improvement, and no release note repairs that.

What this looks like in one product

Information Age product

Returns recipes and rankings. The household still reads the week, weighs effort, notices the spinach deadline, remembers prior meals, and makes the plan.

Intelligence Age product

Returns a provisional week, keeps busy evenings light, makes assumptions available for correction, and returns consequential decisions to the household.

The household notices the fluency. What decides whether it still trusts the product in a month is something it cannot see: whether anyone inside the team can name the part of the system that judged a traybake reasonable on Tuesday, and say what that part should do when a medical diet lies outside its grounds.

A larger model does not answer that second question, because a larger model changes the quality of the conclusion without changing where it was reached. Structure answers it, and the structure is what the rest of this collection builds.

What to do next

The quickest test of whether an initiative has left demo territory is to ask four questions:

  1. What judgment is the product now expected to carry?
  2. Which exact work stays in deterministic code?
  3. What information may influence the judgment, and how will it be evaluated?
  4. Who retains authority when the result creates a consequence?

If the honest answer to the final question is the model, nobody owns the result, and the next incident will be handled by editing a prompt until the complaint stops. If the answer names an inspectable, evaluable part of the system and a human authority boundary, the team has started building for this age rather than demonstrating in it.

Naming the shift is the easy half. The harder half is saying what changes inside the build, which begins with separating the work software has always done from the work it has only recently been asked to do.

Read next: From computation to cognition takes this shift into the architecture.