Published field note · Refinement note
· 4 min read
“The harness matters” is not a book thesis
Runtime choices matter, but the useful question is which contracts keep a trajectory controlled when proposals, effects, evidence, and the system itself can change.
For a proposed engineering book about agents that act on software and services, “the harness matters” is too weak a thesis. The field already knows that runtime, environment, tool, and evaluator choices matter. Here, a harness is the runtime around a model that supplies context, routes tool calls, and decides when to continue or stop. A different interface can change what a model manages to do. A different sandbox can change what it is allowed to do. A different grader can change what gets called success. A book-level thesis has to do more than name that territory: it needs a falsifiable mechanism that changes where engineers put control and what failure they should expect when that control is absent.
Recent work makes that hard to miss. The AI Harness Engineering preprint explicitly studies a model together with its harness and environment. Anthropic's managed-agents report describes one production architecture that separates sessions, harness workers, and sandboxes. The OpenAI Agents SDK runner documentation specifies one concrete loop for model calls, tool calls, handoffs, and stopping. These sources play different roles—a frontier research proposal, a named production report, and official documentation for one implementation. None hands us a universal architecture, and none proves the four-contract proposal below.
So the question I want to test is narrower: once a model proposal can touch the world, what must the surrounding system enforce so effects remain bounded, recoverable, and auditable?
The current proposal is four executable contracts. “Contract” here means an enforced interface or invariant, not a paragraph in a design document.
The transition contract gates who may propose an effect, authorize it, execute it, observe it, stop, and retry. The state contract persists what survives a timeout, how work and artifacts are identified, and which uncertain effect must be reconciled before resumption. The evidence contract prevents a completion transition until named observations justify “done” and required collateral state has also been checked. The change contract binds the run to the model, prompt, tool, policy, evaluator, memory, workflow, and harness versions that produced it, so a bad candidate can be quarantined or rolled back without rewriting history.
That sounds abstract until an effect escapes the process. Suppose an operations agent proposes a 50. Before dispatch, the orchestrator persists a pending-effect record and an operation identity. If the billing provider supports idempotency, the adapter sends that same identity as the provider's idempotency key, so a retry denotes the original logical credit rather than a second one. The billing API receives the request, but the client times out before the response arrives.
Now the contracts change behavior. The transition contract forbids both “done” and blind retry while the outcome is unknown. The state contract keeps the pending effect and request artifact across restart. The evidence contract requires a fresh billing-ledger lookup: if the credit exists, mark it verified; if it is absent, retry with the same provider key; if the provider cannot deduplicate and the ledger cannot settle the question, stop and escalate instead of risking a duplicate. The change contract identifies the policy and adapter version if this failure began after a rollout. The system has not made failure impossible. It has bounded the legal transitions, made uncertainty durable, and defined how uncertainty may be resolved.
Without those enforced distinctions, “the agent issued a credit” compresses several incompatible realities. It may mean the model wrote a persuasive sentence. It may mean a request left the process. It may mean the customer ledger changed. It may mean an evaluator inferred success from the transcript. Those are not the same event, and only some of them permit another transition.
That is the stronger thesis being tested: a useful harness is not merely consequential. It implements a control surface that gates effects, persists uncertainty, requires outcome evidence, and makes system changes attributable. A system can still be unsafe with all four contracts, which is why the counterexample matters; but a transcript-only loop cannot supply these guarantees by eloquence alone.
What this does not prove: this four-contract frame is my synthesis being tested, not an external standard or a source-reported result. Four is not a magic number. A system may combine session and harness ownership, embed evaluation in an environment, use deterministic software instead of a model, or need a responsibility this frame hides.
What would break it? I am looking for a concrete agent-system failure where every transition, durable state boundary, evidence rule, and component change is correctly specified, yet the failure still cannot be localized without adding or replacing a contract. What is that counterexample?
Open question
What I'm testing
What agent-system counterexample cannot be diagnosed cleanly through transition, state, evidence, or change—or needs a fifth contract?
Evidence boundary
Sources and boundaries
A linked source supports only its named role below. This note connects and tests ideas; the note itself is not evidence.
- frontier preprintAI Harness Engineering
Treats the model, harness, and environment as an evaluated system in a frontier preprint; it does not define a universal agent architecture.
- production reportScaling managed agents: sessions, harnesses, and sandboxes
Reports one vendor's separation of durable sessions, replaceable harness workers, and sandboxes; it is a named implementation, not an independent standard.
- official specOpenAI Agents SDK: Running agents
Documents loop, tool, handoff, and stopping behavior for one SDK; it does not establish outcome quality or the note's four-contract synthesis.