An Inside Look

July 8, 2026

This is the second post in Facts, Rules, Runtime, our series on the engineering system around agentic software development. Post 1: How We Built Our Agentic SDLC Harness made the basic distinction: the model is not the system. The harness is the system around the model. It also drew the line this post builds on: rules describe intent; contracts describe completion. Rules go through the model; contracts go through the runtime.

Verification is the first load-bearing layer in that harness. An agent can explain why it thinks a task is done: it summarizes the files it changed, lists the tests it says it ran, describes how the implementation matches the work plan. Those explanations are useful, and we read plenty of them. They're just not a completion signal.

The failure mode is subtle because the agent is often directionally right. It may have made the right edit, understood the intent, and run a relevant subset of checks. But an SDLC cannot depend on "directionally right." It needs to know whether the task satisfied the contract that downstream work will rely on.

A completion signal has to survive model upgrades, stale context, optimistic summaries, and provider differences. It needs to be something the runtime can evaluate without asking the same model to reinterpret the work.

In our harness, that signal is contract-shaped: a named policy, a concrete command, an expected exit code, and evidence recorded before the task can move forward. These are facts the runtime can check, not claims it has to trust.

The Work Plan is the Input/Output Contract

The most important verification object in our harness isn't an individual test command. It's the work plan: the machine-readable project plan the first post introduced, the artifact that compiles an RFC into milestones, tasks, dependencies, and state. Machine-readable is the part that matters here. Because the plan is data, the harness can inspect it with code instead of asking the model whether the work is done.

Every task inside the plan carries its contract in both directions. On the way in: what the task is, which files it may touch, what it depends on, and which checks it owes before it can advance. On the way out: what actually happened, written as data, not as a summary. One task, reduced to its contract:

The left side is what makes a task eligible to run. The right side is what the rest of the SDLC builds on: dependency unblocking, review, measurement. And that record is only worth building on if every field in it is a fact, something checked rather than narrated. That word needs a definition.

Exit Code is a Field, Not the Contract

An exit code is the smallest useful verification fact. It answers whether one command accepted or rejected the current workspace. That's necessary. It's not enough.

The harness also has to know that the task was well-formed enough to run, that the agent's report can be trusted, and that the resulting state change was legal. Each is validated separately, and each can fail on its own.

A task can have passing tests and still be malformed as SDLC work: it edited files outside its boundary, skipped required evidence, or set passes=true through the wrong path. Conversely, a task can fail verification and still produce valuable output, if the failure report is structured enough to show which layer went wrong.

The verification layer is not "run tests." It is "advance state only when the typed contract allows it."

Policies and Checks

There are two levels in the verification layer: policies and checks. A policy is reusable. It says that a class of work must prove something before it can advance. A check is task-local. It is the concrete instance of a policy, or a one-off verification requirement, for a specific task.

policy_id: api-contract-tests
check_type: contract
command: pnpm --filter <service> test:contract
expected_exit_code: 0
expected_artifact_patterns:
  - reports/contract/<service>.json
fail_message: API contract verification failed
remediation_hint: inspect the provider/consumer diff
  before changing handler code

check_id: contract-suite-passes
policy_ref: api-contract-tests
command: pnpm --filter payments-api test:contract
expected_exit_code: 0
assertions:
  - provider schema remains compatible
  - changed endpoint keeps documented behavior

That split is what made the layer measurable. Early versions of our harness let every task write verification from scratch, and what we got was a pile of one-off commands. They passed, but they didn't teach the system much. There was no clean way to ask which policy failed most often, which policy was too weak, which task kinds needed stricter gates, or which command families were flaky.

Tasks that only referenced abstract policy names failed the other way: the agent satisfied the form while leaving too much unresolved. "Run tests" is not enough. Which tests? From which directory? With which environment? What artifact proves the run happened?

The useful shape is both: a reusable policy, a task-local command, an expected runtime result, and recorded evidence.

Policies mostly come from standards you already have. A style guide entry is a rule until a lint policy enforces it. An architecture decision record is a rule until a dependency check makes its boundary mechanical. Infra conventions, toolchain versions, security baselines: each decomposes into a part a command can verify, which becomes a policy, and a part that stays judgment, which stays a guide and goes through the model. Codifying that split is most of the work of building this layer.

A useful policy is also more than a command. It explains failure. The weakest version of the same idea fits in one line (policy_id: tests, command: run tests, expected_exit_code: 0), and that's the problem: it gates and does nothing else.

The reusable policy above also tells the runtime what to expect and tells the next agent or engineer where to start when the check fails. A good policy is not only a gate; it's a reusable debugging instruction.

The Verification Contract

The verification layer is a contract between four things. The work plan declares the input shape and the required output shape. The executor, the protocol the agent runs inside, performs the work. The verification commands test the workspace. And the report writer validates the agent's semantic output and owns the state transition. Every report it writes is appended to a durable, attempt-numbered index; retries add history, they never rewrite it. That index feeds the typed ledger the first post introduced, and it's what the measurement layer in the next post reads.

This shape prevents the agent from being the only authority over completion. The task object carries passes as a boolean, not prose. The execution protocol says never to set passes=true unless all checks pass, and the report writer is the single path for completed, failed, or blocked outcomes.

That single-writer rule is easy to underestimate until you've watched completion state get corrupted through a side path. The failures it exists to prevent all look alike: the model or a helper script updates passes outside the report writer; a report claims checks passed without the command output that proves it; a retry overwrites failure context instead of appending a new attempt; a task is marked complete before the commit exists. The harness should make the correct path boring and the incorrect paths hard.

None of this is new if you've built workflow runtimes: a single writer for state and a typed payload that validates before a transition commits are the semantics durable orchestration engines have enforced for years. It's the same discipline behind Lemma, our workflow platform, applied to an activity that happens to be a model call. Post 5 returns to that lineage.

Verification Starts Before the Code

Verification does not start after implementation. The protocol begins by validating the work plan itself, the preflight step. If the plan is malformed, the run stops before implementation, because a malformed plan is an input failure, not an implementation failure, and the harness needs to know which one it's looking at.

Ambiguity Blocks Instead of Failing

After preflight, the executor picks exactly one eligible task and reloads its context from scratch. Only then does it pass through the ambiguity gate.

If a requirement is ambiguous, underspecified, contradictory, or technically blocked, the executor should not implement. It records a blocked report and exits. That distinction protects the learning loop: failed means a valid contract was attempted and did not pass; blocked means the contract itself needs human input, additional context, or an upstream fix.

The check loop itself needs no cleverness: run every required command exactly as written, compare exit codes, record results. What's strict is the boundary around it: the plan validates before task selection, the payload validates before state mutation, and a single writer owns every completion field. An example executor sketch appears in the minimum viable layer below.

Verification is TDD, Taken Seriously

There's a familiar version of this discipline, and it's worth naming directly. In test-driven development, you write the failing test before the implementation. The test is the contract: it states what passing means, mechanically, before anyone argues about whether the code is good. Green isn't an opinion.

Agentic verification is the same move, applied one level up. The required checks are written into the task when the work plan is generated, before the agent touches code. The agent's job is to make them pass, not to decide what passing means. It reads the checks as intent, and it doesn't own them.

The part that makes TDD honest is that you don't get to delete the test to turn the bar green. The same rule holds here.

Who Gets to Change a Contract

So a check can't be silently weakened. But contracts do need to change sometimes: a policy can be wrong, too strict, or aimed at the wrong artifact. The real question is who is allowed to change one, and whether the change is recorded.

The agent's only sanctioned move is to block. It doesn't implement against a contract it believes is broken, and it doesn't edit the contract to make the problem disappear. Blocking records the reason and hands the decision back to a human.

A human can then amend the work plan: relax an assertion, retarget a policy, split a task. That amendment is itself an input. It re-enters through the same preflight validation every work plan passes, and it lands as a recorded delta from the original intent rather than an invisible edit mid-run. The rule isn't "contracts never change." It's "contracts only change through a path the harness can see."

That visibility is what makes loosening safe to allow at all. A contract that gets relaxed over and over is a signal in its own right: maybe the policy was too strict, maybe that task kind needs a different gate. If the change is recorded, the measurement layer can find the pattern. If the agent could edit checks in place, the signal would vanish into the diff.

Designing for Both Outcomes

Once implementation runs, verification has two exits, success and failure; blocked happens earlier, at the ambiguity gate, before there's an implementation to judge. Both have to be designed. It's tempting to write only the success path; an agentic SDLC can't afford to.

Passing Now, Settling Later

A harness needs to distinguish immediate verification from full lifecycle settlement. We treat this as a two-tier completion model. Immediate verification answers whether the implementation phase satisfied the required checks. Lifecycle settlement asks whether the task completed the broader lifecycle: review, reconciliation, release readiness.

The immediate signal is passes. That's the right signal for dependency unblocking. If task B depends on task A's implementation contract, task B shouldn't have to wait for every later review and reconciliation step once task A's required checks have passed.

The obvious objection is cascade risk: if review later forces a rewrite of task A, wasn't task B built on sand? This is exactly why the dependency is on task A's implementation contract, not on task A's final diff. Task B depends on the interface A committed to, the schema, the signature, the behavior the checks pinned down. Review usually reshapes the implementation behind that interface, not the interface itself. When it does change the interface, that's what the periodic reconciliation pass is for: it compares the work plan, the reports, and the actual code, and flags the dependents that need to re-run. The two-tier model trades a little rework risk for a lot of parallelism, and it keeps the rework detectable instead of silent.

But passes shouldn't be overloaded to mean "fully settled." A task can pass its implementation checks while still needing review follow-up. A milestone can look complete while reconciliation finds drift.

If you're building this yourself, start with passes; settlement can stay a convention until review and reconciliation exist.

The Minimum Viable Verification Layer

If we were starting over, or advising a team building its first harness, this is the place we'd tell them to start. What follows is a sketch of the shape, not our implementation, and it's smaller than it sounds.

def preflight(work_plan):
    validate_json(work_plan)
    validate_schema(work_plan)
    validate_structure(work_plan)

def run(work_plan):
    preflight(work_plan)
    task = select_one_eligible_task(work_plan)
    reload_context(task)   # global context, guide files, task source files
    execute(task)

def execute(task):
    if task_is_ambiguous(task):
        reason = ambiguity_reason(task)
        payload = build_blocked_payload(task, reason)
        write_report(task, Status.BLOCKED, payload)
        return

    implement(task)

    results = []
    for check in task.required_checks:
        result = run_check(check.command)
        passed = result.exit_code == check.expected_exit_code
        results.append(CheckResult(check, passed))

    if all(r.passed for r in results):
        commit_changes(task)
        payload = build_completed_payload(task, results)
        write_report(task, Status.COMPLETED, payload)
    else:
        payload = build_failed_payload(task, results)
        write_report(task, Status.FAILED, payload)

Once those boundaries exist, the rest of the harness has something stable to build on.

What Comes Next

Verification is necessary, not sufficient. It can't prove the agent made the right architectural choice, catch a guide that was missing, or know whether accepted code will survive a week of production hardening. Some engineering decisions stay judgment calls, and that's fine. The mistake is letting judgment disappear after the run. That's why the verification layer feeds the measurement layer.

The verification layer says whether this task crossed the mechanical boundary. The measurement layer asks whether the harness is improving. That is where the next post picks up: once the harness records what passed, failed, blocked, and later reached review, it can ask a harder question. Were the agent's decisions right?

Verification is the first place an agentic SDLC becomes an engineering system instead of an agent session. The model can reason about a task. The harness has to decide whether the result is allowed to change state.