An Inside Look

June 24, 2026

When we started running an agent unattended against our own codebase, we expected the model to be the variable that mattered most. A newer model would mean better PRs, better PRs would mean more autonomy, more autonomy would mean shipping faster. Model selection is what most public writing about agentic coding optimizes for: which model, which prompt, which sandbox.

That's not what we've learned. The model matters, but it isn't what makes the system trustworthy. The majority of our engineering time has gone into the layer around the model: how we seed context, how we encode the rules the agent has to follow, how we measure whether the agent's decisions are right, how we deploy skills across providers, how we schedule and recover work. None of those have anything to do with the model itself. They're the harness: the system that turns a non-deterministic step into a repeatable workflow.

This is the first post in Facts, Rules, Runtime, a five-part series on what we built. It introduces how we think about the system: the components, the layers, the contracts, and previews the four posts that follow.

Where the Harness Operates

A typical software development lifecycle has roughly eight stages: requirements gathering, design and decomposition, implementation, review, verification, release, monitoring, and iteration. Naming and ordering vary across teams; the stages themselves are universal.

An SDLC harness covers all of them. Requirements get captured as structured intent artifacts and reviewed across multiple models before they bind to specific systems and components. Design decomposes intent into typed work units with explicit dependencies and completion criteria. Implementation runs through a verification gate that won't advance without exit-code-zero proof.

The same loop keeps running after the model writes code. Review uses multi-model adjudication on the work the agent produced. Verification and release happen through the same typed structure that gated implementation. Monitoring captures decision-level signal: what classifications the agent made, what reviewers disagreed with, and code-survival data over weeks and months. Iteration folds that signal back into the inputs the next agent run reads.

The components and discipline in the rest of this post show up across all of these stages. The posts that follow document which components dominate at which stage.

What's in the System

When we explain what we built to a new engineer, we start with the same six-part decomposition every time. It's not a framework anyone gave us. It's the shape that emerged after we'd shipped enough versions of the harness to see which parts had to exist.

The Model

The non-deterministic step. The harness is built so any model can sit here: proprietary frontier models, open-weight models, even smaller models we run locally. Commands and skills are normalized so the same instruction reaches any model the same way. Today that normalization runs through two paths: OpenCode, an open-source coding-agent CLI, as one adapter and a custom abstraction layer for the rest. We treat the model as an interpretive component: non-deterministic, the most likely part of the system to change quarter-over-quarter. The rest of the harness is built so the model can change without rewriting anything else.

The Harness

The harness is the deterministic surface around the model. The schemas that validate the model's output. The manifests that deploy its skills. The ledgers that record its decisions. The routers that attach its context. The runtimes that schedule and supervise its work.

It's also everything outside the obvious product surfaces: the chat tool that triggers a run, the sandbox where the agent runs, and the review tool that captures acceptance. Those surfaces are useful for quick fixes and isolated PRs: chat starts the request, a sandbox runs the agent, and GitHub captures human judgment. That is not the same as operating a project lifecycle. A full SDLC harness still has to answer what work should exist, what counts as done, which decisions were wrong, and how the next run improves.

The right mental model is durable infrastructure. The harness is the part of the system you should be able to depend on regardless of which model is running today, which provider released a new version this week, or which prompt-engineering trick the community is currently excited about. When teams treat their agent setup as "configuration to tune," they stay stuck in the model-selection loop: every new model release becomes a re-tuning exercise. When they treat it as infrastructure to build, the model becomes one swappable component inside a system that doesn't rewrite itself every quarter.

Rules and Contracts

Rules and contracts are the prescriptive content the agent has to respect. They split into two distinct kinds and decompose at every level of the work: there are rules at the RFC level and rules at the task level and rules baked into the codebase as conventions; there are contracts on a specific task and contracts shared as reusable policies that gate many tasks at once.

Rules describe intent and constraint: the why, the not-this, and the how-we-do-things-here. RFC narratives describe what we're building and why. Work plan goals describe a milestone's purpose; non-goals say what's explicitly out of scope. Guide files are agent-facing instruction files, like a targeted AGENTS.md for a specific part of the codebase; they describe codebase conventions, architectural decisions, and the patterns we want followed. Style and security and naming rules live at the codebase level. Rules go through the model: they're how the agent knows what we're trying to accomplish, what to avoid, and how to fit new work into the existing system. Rules can be ambiguous and the system survives; the model interprets them, the human reviews the interpretation, the work proceeds.

Contracts describe completion: verifiable assertions about what "done" looks like. They're TDD-style tests, JSON schemas, OpenAPI specs, Protobuf definitions, type signatures, acceptance criteria written as executable statements. A named verification policy that resolves to a command with an expected exit code. An evidence_required artifact that has to exist on disk. A passes=true flag that the runner refuses to set unless every check returned the right exit code. Contracts go through the runtime, not the model.

The split is the load-bearing distinction in this series. Rules describe intent; contracts describe completion. The agent reads rules to know what to do; the execution layer reads contracts to know whether the work is done. Confusing the two, writing rules that look like contracts (vague tests), or contracts that need to be interpreted (which makes them rules in disguise), produces the failure modes most agentic-coding tooling runs into. A test in TDD is a contract: it states what passing means, deterministically. Work plan verifications work the same way. Post 2 documents the contract layer in detail; it's where "is this task done?" becomes mechanically checkable.

Evals and Learning

The feedback loop. Two metric families: decision accuracy and code lifetime.

The first is per-decision accuracy: every classification the agent makes, triage decisions, review verdicts, task-kind labels, gets recorded as an entry in an append-only ledger. The second is code lifetime: how long does a line the agent wrote actually survive in the codebase before getting rewritten or reverted. The harness records commit-SHA provenance in each task's report; turning that provenance into a lifetime metric is a data-science problem on top of git history: fitting survival curves on commits, identifying churn signatures, learning which kinds of changes age and which get reverted. The output feeds the system back: today as queryable history that the rest of the harness reads from; eventually as training data for the agent itself. The two together catch different failure modes: the first catches the agent classifying wrong; the second catches the agent classifying right but producing code that doesn't age.

The ledger is infrastructure. Same discipline as any other immutable event store: append-only, typed events, queryable. We don't edit it; we append to it; we read from it.

This is the same pattern observability engineering relies on: use the events your tools already produce. Most of the signal we need isn't coming from a separate adjudication system; it's emitted continuously by the engineering tools the team already uses. Every PR review comment, every approve/request-changes verdict, every force-pushed fix over an agent commit, every accepted suggestion: all of these are structured signal about whether the agent's work was right. Git history records the same thing at a slower cadence: which commits the agent authored, which lines survived, which were rewritten. Build logs and CI failures add another layer. We fold each of these into the typed ledger; the ledger becomes the queryable history of agent decisions and their consequences, drawn from event streams that already existed.

Post 3 documents all three measurement angles, accuracy, lifetime, and context effectiveness, end to end: what we capture, how we turn it into rules, and what happens after.

Context

What gets seeded into a session before the agent starts work. Skills (the procedural prompts the agent reads). Guides: small distilled facts and rules, many of them emerging from the data-science work on the ledger described above, formatted as context the agent reads directly. Model Context Protocol (MCP) server outputs (live data from other systems). Project memory (the agent's persistent context across sessions). Routing: the mechanism that decides which guides attach to which task based on the files the task touches.

The guides are small on purpose. A learned fact lives in a few paragraphs; we don't dump everything we know into every task. More context isn't always better. Past a threshold, context starts working against the agent: irrelevant files dilute the relevant ones; the agent's attention scatters across material it doesn't need; performance degrades. We've seen tasks that would have succeeded with three guides fail after seven were attached, because the extra four contained patterns that contradicted each other in subtle ways. We treat context as a resource with a budget: each task declares its guide_files and must_read paths explicitly, the harness tracks total seeded context size, and routing rules are scored on precision (does this guide actually apply?) rather than recall (could it possibly apply?). Too many files before a task starts is itself a bad signal: it poisons the context, and the agent's output reflects it.

Today routing is a typed table that the work plan generator consults; tomorrow it's executable glob rules. Post 4 covers the deployment substrate that this lives in.

Pipeline

The pipeline is the chain of typed artifacts that carry work through the lifecycle stages from the section above. An RFC captures design intent, written for stakeholders to read, bound to specific systems and components. The work plan compiles the RFC into work: a structured contract defining milestones, tasks, verifications, and execution state. A milestone is itself a contract: when these tasks complete, this capability is delivered. A task is an atomic unit of work with its own typed contracts, evidence requirements, and a deterministic verification gate. Multi-model review and review synthesis, a separate pass that consolidates multiple reviews into one verdict, adjudicate the work; periodic reconciliation checks compare the work plan, implementation reports, review artifacts, and actual codebase to detect drift.

Structurally, the pipeline is a workflow definition. The work plan that decomposes an RFC into tasks has the same shape as a workflow on Lemma, our workflow platform: typed stages, explicit dependencies, completion criteria at each step, deterministic transitions between them. The orchestration lineage operates here at the component level; the closing section returns to it at the runtime level.

The pipeline absorbs something most teams treat as separate from engineering: execution tracking. The work plan is the execution tracker for agentic work: typed, validated, machine-readable, executable. Every task is in there; every status transition is typed and validated; every completion criterion is machine-checkable; every dependency is explicit. When the long-running runtime runs, it walks the same work plan a developer would; when a developer wants to know "what's blocked," they query the same work plan the runtime queries. There's no second source of truth to drift out of sync.

This isn't a small change. It's the difference between a ticket that says "make this work" (interpretive, gets argued about, drifts between tracker and code) and a work plan task that names the files, the contracts, the dependencies, the verification policies, and the definition of done. The work most teams scatter between a project management tool and the codebase consolidates here: typed, versioned, queried by the runtime and the human from the same place. What happens to the project management tool is a separate question we'll get to.

The pipeline is where the other components compose. The model proposes the implementation; the harness orchestrates the run; the contracts gate completion; the evals capture decisions; the context seeds the task; the runtime claims the work. Walking through one task end-to-end shows how the components compose into a system.

How the Pieces Fit

A single task moves through the harness like this:

The model is one box. The harness is most of the diagram.

The shape repeats across every task: same contracts, same routing, same verification gate, same ledger writes, same review tooling capturing signal at the end. That repetition is what makes the loop legible. When a task fails, we know which box. When override rates climb, meaning humans are disagreeing with the agent more often, we know which canonical command is drifting. When a rule emerges from override patterns, it joins the contracts and ships to every future agent run via the deployment substrate.

Three Core Takeaways

These three didn't start as principles. They started as constraints: places where the harness misbehaved until we encoded the discipline. Consolidated here as the post's takeaway.

These three compound because they reinforce each other. Contracts produce verifiable signal; routing decisions get measured against that signal; signal flows back as new rules and new contracts. Once the loop is wired, the system improves without anyone needing to remember to maintain it.

What's in the Rest of This Series

The order isn't arbitrary. Each post builds on the components established in the ones before it.

This is Orchestration

Thread AI has been building workflow orchestration long before agents were a category. Lemma, our durable workflow platform, runs on the primitives every serious orchestration runtime depends on: typed activity contracts, deterministic replay, retry policies, lease-based work claiming, structured observability across long-running execution. Those aren't decorative features; they are what makes a runtime trustworthy when work can't be allowed to disappear mid-flight.

The harness in this series is the same engineering discipline applied to a different kind of activity. A traditional workflow orchestrates deterministic activities: functions whose behavior is reproducible given fixed inputs. The harness orchestrates a non-deterministic activity: a model call. The runtime semantics are otherwise unchanged. Tasks are claimed from a durable queue. Each task carries a typed contract. Failures retry under explicit policy. Every transition emits a structured event the system can replay or audit.

What changes is what happens inside the activity. The queue, the contract, the retry envelope, and the observability layer around it are the same orchestration primitives we have been shipping for years. Non-determinism inside the activity raises the standard they have to meet; it does not replace them.

This is why the series uses workflow, orchestration, and harness interchangeably: the words come from different communities, but the engineering discipline is one continuous lineage. Lemma's runtime is where Thread AI started; the harness is what that lineage looks like when the activity is a model call. Post 5 returns to the comparison directly.

Next post: Post 2: The Verification Layer, From Policy to Exit Code.