All field notes

Published field note · Refinement note

· 4 min read

“The harness matters” is not a book thesis

Runtime choices matter, but the useful question is which contracts keep a trajectory controlled when proposals, effects, evidence, and the system itself can change.

For a proposed engineering book about agents that act on software and services, “the harness matters” is too weak a thesis. The field already knows that runtime, environment, tool, and evaluator choices matter. Here, a harness is the runtime around a model that supplies context, routes tool calls, and decides when to continue or stop. A different interface can change what a model manages to do. A different sandbox can change what it is allowed to do. A different grader can change what gets called success. A book-level thesis has to do more than name that territory: it needs a falsifiable mechanism that changes where engineers put control and what failure they should expect when that control is absent.

Recent work makes that hard to miss. The AI Harness Engineering preprint explicitly studies a model together with its harness and environment. Anthropic's managed-agents report describes one production architecture that separates sessions, harness workers, and sandboxes. The OpenAI Agents SDK runner documentation specifies one concrete loop for model calls, tool calls, handoffs, and stopping. These sources play different roles—a frontier research proposal, a named production report, and official documentation for one implementation. None hands us a universal architecture, and none proves the four-contract proposal below.

Controlled timeout path from persisted identity to reconciliationRead the top row left to right. The state contract persists an operation identity and pending-effect record. The transition contract dispatches the credit once. A timeout leaves the effect unknown rather than failed. The evidence contract reconciles against the billing ledger and branches to three labeled outcomes: a found credit becomes verified, an absent credit with provider deduplication may be retried with the same key, and an unresolved or non-deduplicated outcome stops and escalates. A footer binds the run to its policy and adapter versions through the change contract. The figure is a proposed control path, not a proven sufficient standard.CREDIT EFFECT / CONTROLLED TIMEOUT PATHSTATEPERSISTID + pendingTRANSITIONDISPATCHsend onceSTATETIMEOUTeffect unknownEVIDENCERECONCILEread ledgerFOUNDmark verifiedABSENT + DEDUPEsame-key retryUNRESOLVEDstop / escalateCHANGE CONTRACT / BIND RUN TO POLICY + ADAPTER VERSION
The proposed contracts turn a timed-out credit into an explicit branch: reconcile the ledger, then verify, retry with the same key when the provider deduplicates, or stop and escalate.

So the question I want to test is narrower: once a model proposal can touch the world, what must the surrounding system enforce so effects remain bounded, recoverable, and auditable?

The current proposal is four executable contracts. “Contract” here means an enforced interface or invariant, not a paragraph in a design document.

The transition contract gates who may propose an effect, authorize it, execute it, observe it, stop, and retry. The state contract persists what survives a timeout, how work and artifacts are identified, and which uncertain effect must be reconciled before resumption. The evidence contract prevents a completion transition until named observations justify “done” and required collateral state has also been checked. The change contract binds the run to the model, prompt, tool, policy, evaluator, memory, workflow, and harness versions that produced it, so a bad candidate can be quarantined or rolled back without rewriting history.

That sounds abstract until an effect escapes the process. Suppose an operations agent proposes a 40customercredit.Policyauthorizesupto40 customer credit. Policy authorizes up to 50. Before dispatch, the orchestrator persists a pending-effect record and an operation identity. If the billing provider supports idempotency, the adapter sends that same identity as the provider's idempotency key, so a retry denotes the original logical credit rather than a second one. The billing API receives the request, but the client times out before the response arrives.

Now the contracts change behavior. The transition contract forbids both “done” and blind retry while the outcome is unknown. The state contract keeps the pending effect and request artifact across restart. The evidence contract requires a fresh billing-ledger lookup: if the credit exists, mark it verified; if it is absent, retry with the same provider key; if the provider cannot deduplicate and the ledger cannot settle the question, stop and escalate instead of risking a duplicate. The change contract identifies the policy and adapter version if this failure began after a rollout. The system has not made failure impossible. It has bounded the legal transitions, made uncertainty durable, and defined how uncertainty may be resolved.

Without those enforced distinctions, “the agent issued a credit” compresses several incompatible realities. It may mean the model wrote a persuasive sentence. It may mean a request left the process. It may mean the customer ledger changed. It may mean an evaluator inferred success from the transcript. Those are not the same event, and only some of them permit another transition.

That is the stronger thesis being tested: a useful harness is not merely consequential. It implements a control surface that gates effects, persists uncertainty, requires outcome evidence, and makes system changes attributable. A system can still be unsafe with all four contracts, which is why the counterexample matters; but a transcript-only loop cannot supply these guarantees by eloquence alone.

What this does not prove: this four-contract frame is my synthesis being tested, not an external standard or a source-reported result. Four is not a magic number. A system may combine session and harness ownership, embed evaluation in an environment, use deterministic software instead of a model, or need a responsibility this frame hides.

What would break it? I am looking for a concrete agent-system failure where every transition, durable state boundary, evidence rule, and component change is correctly specified, yet the failure still cannot be localized without adding or replacing a contract. What is that counterexample?

Open question

What I'm testing

What agent-system counterexample cannot be diagnosed cleanly through transition, state, evidence, or change—or needs a fifth contract?

Evidence boundary

Sources and boundaries

A linked source supports only its named role below. This note connects and tests ideas; the note itself is not evidence.

  • frontier preprint
    AI Harness Engineering

    Treats the model, harness, and environment as an evaluated system in a frontier preprint; it does not define a universal agent architecture.

  • production report
    Scaling managed agents: sessions, harnesses, and sandboxes

    Reports one vendor's separation of durable sessions, replaceable harness workers, and sandboxes; it is a named implementation, not an independent standard.

  • official spec
    OpenAI Agents SDK: Running agents

    Documents loop, tool, handoff, and stopping behavior for one SDK; it does not establish outcome quality or the note's four-contract synthesis.

Back to all field notes