The Agent Harness Is the Real Product

The model gets the credit because the model is the part that appears to think.

It writes the code, interprets the document, chooses a tool, and explains the result. When a new model arrives, demos make the improvement look almost magical. The same prompt suddenly produces a cleaner design or a better diagnosis. It is natural to conclude that the model is the product and everything around it is plumbing.

In a real AI system, that conclusion does not survive contact with production.

The model does not decide which repository it is allowed to edit. It does not guarantee that the tool result it just received is still valid. It does not preserve a task after a process disappears, prevent two workers from making the same payment, or prove that a code change passed the tests. It cannot know which parts of a million-token history are operational truth and which are abandoned hypotheses unless the surrounding system makes that distinction visible.

The model supplies capability. The harness turns capability into dependable work.

By harness, I mean the operating environment around the model: context assembly, tool contracts, permissions, state, orchestration, recovery, evaluation, and the interface through which a human can understand and steer the work. A strong model inside a weak harness is impressive and unreliable. A good harness can make several different models useful because it gives each of them the same disciplined path from request to evidence.

That is why I increasingly think the harness is the real AI product.

Intelligence Is Only One Layer

An agent can look complete from the outside. It receives a goal, acts, and returns a result. Inside that loop, however, several different systems are doing different jobs.

The model interprets and proposes. The context layer decides what the model can see. The tool layer decides what it can touch. The orchestration layer decides what happens next. The state layer records what survives. The policy layer decides what is allowed. The verification layer decides whether the result deserves to count.

Blurring these responsibilities into one prompt creates the illusion of simplicity. It also creates a system where every failure looks like “the model got confused.”

Sometimes the model did get confused. Often the harness gave it contradictory instructions, stale files, an ambiguous tool response, or no durable representation of the task. Sometimes the agent repeated an action because the tool contract had no idempotency key. Sometimes it declared victory because the harness treated a fluent final answer as completion evidence.

The distinction matters because model problems and system problems have different remedies. A larger model may improve judgment. It will not repair an unsafe permission boundary or invent a missing recovery protocol.

Context Is a Compiled Artifact

Many agent systems treat context as a bag. The user message goes in, then a system prompt, some retrieved documents, a few tool definitions, and as much history as the window will hold.

A mature harness treats context more like a compiled artifact.

It selects inputs from different sources, resolves precedence, removes irrelevant material, preserves critical constraints, and produces a deliberate view of the current task. That view should be appropriate to the step. A planner needs the goal, constraints, architecture, and available capabilities. A code reviewer needs the acceptance criteria, diff, relevant source, and test evidence. A deployment executor does not need the brainstorming transcript that produced the change.

This is not merely token optimization. Context determines behavior. Every unnecessary instruction is another opportunity for conflict. Every missing constraint is authority the model may infer for itself. Every stale artifact can pull the work toward a state that no longer exists.

The harness should therefore make context provenance visible. We should know where an instruction came from, when a retrieved fact was produced, which rules outrank which other rules, and what was deliberately excluded. If context is the model's runtime environment, context assembly is a build system.

Tools Need Semantics, Not Just Schemas

Giving a model a JSON schema is not the same as giving it a reliable tool.

A schema can tell the model that create_refund accepts an order ID and an amount. It does not necessarily explain whether the call is reversible, whether retrying it creates a second refund, which error means “nothing happened,” or which response means “the request is still processing.” Those semantics determine whether the agent can act safely.

Good tool contracts answer operational questions:

• Is this call read-only, reversible, or consequential?
• Can it be retried safely?
• What identifies the operation across retries?
• Which preconditions must still be true when execution begins?
• What evidence confirms the action actually completed?
• Which failures require a human decision instead of another attempt?

The best harnesses can enforce some of this outside the model. They attach idempotency keys, validate arguments, limit scopes, check current state, and turn inconsistent provider errors into a smaller set of explicit outcomes. They do not ask the model to remember the entire operational contract in prose every time.

A tool is dependable when the agent can reason about its effects before the call and the system can verify those effects afterward.

Permissions Should Follow the Step

The fastest way to make an agent feel powerful is to give it every available tool. It is also the fastest way to confuse capability with authority.

An agent researching a production incident may need logs, recent changes, and service topology. That does not mean it should inherit a deployment credential. A worker drafting a customer response may need the account history and approved policy. It does not need the ability to send the message.

The harness should issue authority to a bounded action, not permanently to a personality called “the agent.” Permissions can be narrow in resource, operation, environment, and time. The model can propose a broader action, but the harness decides whether the current step has earned it.

This creates a useful separation between intention and consequence. Planning can remain flexible because execution remains controlled. A strong model is free to notice that a deployment might solve the incident. It is not free to convert that observation into a production change without the required evidence and approval.

Prompt instructions help behavior. Enforced permissions limit blast radius. A production system needs both.

State Cannot Live Only in the Transcript

A transcript is a record of conversation, not a dependable state machine.

It contains guesses beside facts, replaced plans beside current ones, and tool output beside the model's interpretation of that output. It grows until it must be compacted. It is difficult for another worker to resume and expensive for a human to audit.

The harness needs durable representations of the things that matter: the accepted goal, current plan, completed actions, artifacts, approvals, failures, retry counts, and evidence required for completion. These records should be inspectable without reconstructing the model's private journey.

This becomes essential once work lasts longer than a chat turn. Processes fail. Credentials expire. humans pause a workflow and return tomorrow. A different model may take over. Without external state, continuity depends on a summary of a summary. With it, a new worker can resume from a checkpoint whose meaning is explicit.

This is the same principle behind the loop and goal workflow: persistence is valuable only when the loop knows what remains true between iterations.

Recovery Is Part of Intelligence

Agent demos are usually edited around the happy path. Production systems live in the other footage.

A search endpoint times out after returning partial data. A shell command succeeds but its output is truncated. A browser session loses authentication. A file changes between inspection and edit. The model reaches the right answer but the process dies before the result is recorded.

A good harness turns these from improvisations into states.

It knows which steps are safe to retry, which require reconciliation, and which should stop. It preserves artifacts before launching the next action. It distinguishes “the call failed” from “the outcome is unknown.” It can resume without repeating a consequential operation merely because the transcript did not contain a success message.

This changes how we evaluate agent quality. An agent that completes 92 percent of ideal runs may be less useful than one that completes 85 percent but recovers cleanly from interruption, exposes uncertainty, and never duplicates a side effect. Dependability is not the absence of failure. It is disciplined behavior when failure occurs.

Verification Must Have the Last Word

The model is very good at producing a story in which the task is finished. The harness has to demand stronger evidence.

For code, that can mean a clean diff, relevant tests, static analysis, and a comparison against the original request. For research, it can mean source coverage, contradiction checks, and a clear line between reported fact and inference. For an operational action, it can mean reading the resulting state from the system of record rather than trusting the tool's optimistic acknowledgement.

Verification should be allowed to reject the agent's narrative. That is the important part.

If the same context that produced the work also decides whether it is correct, mistakes become correlated. A separate review stage, deterministic check, or independent source gives the system a chance to disagree with itself. The composable AI systems approach makes verification a capability because “looks good” is not a stable contract.

The final answer should summarize the evidence. It should not substitute for it.

Model Portability Is a Harness Test

Teams often talk about avoiding model lock-in by hiding provider APIs behind an adapter. That is necessary and insufficient.

Two models can accept the same messages and behave differently around tool selection, long context, refusals, parallel calls, and ambiguity. A harness that genuinely supports more than one model needs evaluations, routing rules, normalized tool semantics, and a way to preserve the same policy boundaries even when model behavior changes.

The goal is not perfect interchangeability. Some capabilities will benefit from a specific model. The goal is to keep model choice from leaking into every product decision.

If a stronger model can be introduced behind one capability, tested against representative work, shadowed beside the current model, and rolled back without rebuilding the workflow, the harness has created real option value. If changing the model requires rewriting prompts, tools, state, UI, and policy at once, the adapter was cosmetic.

Build the Harness From the Failure Inward

The wrong way to start is to design every possible layer before the product has done useful work.

Begin with one consequential workflow. Trace the path from request to outcome. Find where ambiguity becomes action, where state could be lost, where retries could duplicate effects, and where the system currently accepts confidence in place of proof.

Then build the smallest piece of harness that contains that failure.

Add a narrow tool boundary. Record one durable checkpoint. Require one piece of completion evidence. Separate one approval from execution. Introduce one evaluation that measures behavior on work you actually care about.

The architecture should grow around observed risk and repeated responsibility. Otherwise the harness becomes a framework project and the product never learns what it needs.

The Model Is the Engine, Not the Vehicle

A frontier model can make an old harness feel new for a few weeks. It will plan better, call tools more accurately, and recover from some ambiguity through raw reasoning. Those gains are real.

They do not remove the need for an operating system around the work.

Users do not ultimately care that a model generated an intelligent sequence of tokens. They care that the research is grounded, the change is safe, the task can resume, the action happens once, and the system can explain why it believed the work was complete.

That experience belongs to the harness.

The model is where capability arrives. The harness is where capability becomes a product.