A coding agent finishes its task and writes its last message:

Format, lint, unit tests, build: all green.

That sentence is the most important thing the agent says, and the one I can least check. I did not watch the run. I do not know which commit it ran on, whether the agent amended that commit afterwards, whether a fix was sitting uncommitted in the tree, or whether pnpm test in its shell is the same program as pnpm test in mine.

That last one is not hypothetical. In one of my passes, every executor’s shell inherited an environment variable that turned off pnpm’s version switch, so a bare pnpm resolved to the version baked into the image instead of the one the project pins. Same command, different program. That time, it failed loudly. The dangerous version of the same bug is the one that passes.

When one agent works while you watch, you can live with that. When several work in parallel and you read their reports the next morning, you cannot.

“Tests passed” is a claim about work you did not watch.

So I stopped accepting the sentence and started accepting a file: a receipt that says exactly what ran, where, and on what, written by code the agent does not control and checked by code that expects a specific answer.

This is the first of three articles on the harness I built to let coding agents work unattended on my repositories. The harness itself is private; the code below is written for this article and stands on its own.

A claim needs an address

A result is only checkable if it names what it is about. For a validation run, that means:

  • the repository, the branch and the commit that was validated;
  • the base commit, so you know which change was validated, not only which state;
  • the phase: a baseline before any work, a run inside an agent’s worktree, a run after each integration, a final run;
  • the exact commands with their arguments, and a version for the set;
  • a hash of the effective configuration;
  • whether the working tree was clean before and after.

And for each command: the exit code, the duration, and the SHA-256 of its log.

src/receipts/receipt.ts
export type Phase = "baseline" | "executor" | "incremental" | "final";

export type Command = { id: string; command: string; args: string[] };

export type CommandResult = Command & {
  exitCode: number;
  durationMs: number;
  logPath: string; // relative to the run directory
  logSha256: string;
};

export type Receipt = {
  schemaVersion: 1;
  commandSetVersion: string;
  commandSetHash: string;
  configHash: string;
  runId: string;
  phase: Phase;
  repositoryPath: string;
  branch: string;
  commit: string;
  baseCommit: string | null;
  cleanBefore: boolean;
  cleanAfter: boolean;
  startedAt: string;
  finishedAt: string;
  commands: CommandResult[];
  status: "passed" | "failed";
};

Three fields carry more weight than they look.

cleanBefore and cleanAfter. A green run on a dirty tree proves nothing about the commit; it proves something about the commit plus whatever the agent had not committed yet. Requiring a clean tree after the run also catches a check that rewrites files: a formatter that “passes” by fixing things.

baseCommit. The commit says which state passed. The base says which change was validated. Without it, a baseline receipt and a receipt taken after a change landed on the same SHA cannot be told apart.

commandSetHash. The hash of the commands as they were defined when the run happened. It is what lets a verifier notice that the rules changed afterwards.

The status is a field like the others. A failed run writes a receipt too.

Write it where the work runs

The agent never writes a receipt. It asks a guard to run the plan, and the guard writes the result.

That is the whole trick, and it is a boundary of responsibility, not of cryptography: the agent decides what to do; deterministic code certifies what was done.

The guard checks two things before running anything. HEAD must be the commit the caller expects, and the tree must be clean. If either fails, the guard does not throw. It runs nothing and writes a failed receipt, because the attempt is evidence: “an agent asked for validation on a commit that was not HEAD” is something you want in the record.

Logs are written next to the receipt and hashed as they are written. Git, the clock and the process runner are injected, so the guard is tested without a repository.

src/receipts/run-guard.ts
import { createHash } from "node:crypto";
import type { Command, CommandResult, Phase, Receipt } from "./receipt";

export type GuardDeps = {
  head(): string;
  branch(): string;
  isClean(): boolean;
  run(command: Command): { exitCode: number; output: string; durationMs: number };
  writeLog(name: string, content: string): string; // returns the relative path
  now(): Date;
  newId(): string;
};

const sha256 = (value: string) => createHash("sha256").update(value).digest("hex");
export const hashCommands = (commands: Command[]) => sha256(JSON.stringify(commands));

export function runGuard(
  input: { phase: Phase; expectedSha: string; baseCommit: string | null; commands: Command[] },
  context: { commandSetVersion: string; configHash: string; repositoryPath: string },
  deps: GuardDeps,
): Receipt {
  const startedAt = deps.now();
  const runId = deps.newId();
  const commit = deps.head();
  const cleanBefore = deps.isClean();
  const results: CommandResult[] = [];

  if (commit === input.expectedSha && cleanBefore) {
    for (const command of input.commands) {
      const { exitCode, output, durationMs } = deps.run(command);
      const logPath = deps.writeLog(`${runId}-${command.id}.log`, output);
      results.push({ ...command, exitCode, durationMs, logPath, logSha256: sha256(output) });
      if (exitCode !== 0) break; // fail fast: the failing prefix is the evidence
    }
  }

  const cleanAfter = deps.isClean();
  const passed =
    commit === input.expectedSha &&
    cleanBefore &&
    cleanAfter &&
    results.length === input.commands.length &&
    results.every((result) => result.exitCode === 0);

  return {
    schemaVersion: 1,
    commandSetVersion: context.commandSetVersion,
    commandSetHash: hashCommands(input.commands),
    configHash: context.configHash,
    runId,
    phase: input.phase,
    repositoryPath: context.repositoryPath,
    branch: deps.branch(),
    commit,
    baseCommit: input.baseCommit,
    cleanBefore,
    cleanAfter,
    startedAt: startedAt.toISOString(),
    finishedAt: deps.now().toISOString(),
    commands: results,
    status: passed ? "passed" : "failed",
  };
}

The loop stops at the first failure. A ten-minute test suite after a failed type-check tells you nothing new, and the prefix that did run is exactly the evidence you need.

Verify against an expectation, not a parser

A receipt that parses is not evidence. A receipt that matches what you expected is.

I learned that one the hard way. The first version of my receipts recorded the hash of each log and the hash of the configuration, and nothing ever read them back. During an audit of the harness, a receipt with forged values for both, and a log path pointing outside the run directory, validated untouched. The hashes were decoration.

The verifier starts from the other side. It does not ask whether the receipt is well formed. It rebuilds what it expects (the phase, the commit, the base, the branch, the command set from the current configuration) and compares field by field. Then it reads every log back and hashes it again.

It returns a list of reasons, not a boolean. When a closure is refused, this is what you read:

text
receipt commit does not match 4f1c2e9a
receipt command set does not match the current one

A boolean tells you to go and investigate. A reason tells you what to fix.

src/receipts/verify-receipt.ts
import { createHash } from "node:crypto";
import { lstatSync, readFileSync } from "node:fs";
import { join, relative, resolve } from "node:path";
import { hashCommands } from "./run-guard";
import type { Command, Phase, Receipt } from "./receipt";

export type Expectation = {
  phase: Phase;
  commit: string;
  baseCommit: string | null;
  branch: string;
  repositoryPath: string;
  commands: Command[]; // rebuilt from the current configuration
  commandSetVersion: string;
  configHash: string;
  runDir: string;
};

export function verifyReceipt(receipt: Receipt, expected: Expectation): string[] {
  const errors: string[] = [];
  const check = (ok: boolean, reason: string) => ok || errors.push(reason);

  check(receipt.status === "passed", "receipt status is not passed");
  check(receipt.cleanBefore && receipt.cleanAfter, "the tree was not clean around the run");
  check(receipt.phase === expected.phase, `receipt phase is not ${expected.phase}`);
  check(receipt.commit === expected.commit, `receipt commit does not match ${expected.commit}`);
  check(receipt.baseCommit === expected.baseCommit, "receipt base commit does not match");
  check(receipt.branch === expected.branch, `receipt branch is not ${expected.branch}`);
  check(receipt.repositoryPath === expected.repositoryPath, "receipt is from another repository");
  check(receipt.configHash === expected.configHash, "the configuration changed since the run");
  check(
    receipt.commandSetVersion === expected.commandSetVersion &&
      receipt.commandSetHash === hashCommands(expected.commands),
    "receipt command set does not match the current one",
  );

  const base = resolve(expected.runDir);
  for (const result of receipt.commands) {
    const path = resolve(base, result.logPath);
    const inside = relative(base, path);
    if (inside.startsWith("..") || inside === "") {
      errors.push(`log of ${result.id} escapes the run directory`);
      continue;
    }
    let cursor = base;
    const linked = inside.split("/").some((part) => {
      cursor = join(cursor, part);
      return lstatSync(cursor, { throwIfNoEntry: false })?.isSymbolicLink() ?? true;
    });
    if (linked) {
      errors.push(`log of ${result.id} is missing or reached through a link`);
      continue;
    }
    const actual = createHash("sha256").update(readFileSync(path, "utf8")).digest("hex");
    check(actual === result.logSha256, `log of ${result.id} does not match its recorded hash`);
  }

  return errors;
}

The log checks are the ones people skip. A log path is resolved against the run directory and refused if it escapes it, and every component of the path is checked for symbolic links. Otherwise a receipt can point at a log somewhere else, a passing log from another run for instance, and its hash would match perfectly.

The hash binds what you call, not what it does

Here is the limit of the command-set hash, and the one rule nothing enforces for you.

commandSetHash covers pnpm lint. It does not cover what the lint script in package.json runs. Change the script, keep the name, and every receipt made before the change still verifies. The pnpm incident from the opening is the same problem one level down: the hash names a command, and it cannot know which program answers to that name.

That is what commandSetVersion is for. Bump it whenever the catalog changes, or whenever what one of its commands means changes. The bump expires every receipt made under the old meaning, and the next run has to produce fresh evidence.

On this blog’s repository, the catalog started with format, lint and build. When I added the unit tests, nothing the agents had delivered until then had been checked against them, so the version moved with the catalog:

.agents/pass.json
{
  "validation": {
    "commandSetVersion": "2026-10-01.1",
    "catalog": [
      { "id": "format", "command": "pnpm", "args": ["format"] },
      { "id": "lint", "command": "pnpm", "args": ["lint"] },
      { "id": "unit", "command": "pnpm", "args": ["test:unit"] },
      { "id": "build", "command": "pnpm", "args": ["build"] }
    ]
  }
}

The stronger option is to hash what the commands resolve to: the scripts in package.json, the lockfile, the toolchain versions. That binds meaning too. I do not do it by default, because it expires every receipt on any unrelated dependency bump, and a check that fires on every change ends up ignored. A version you bump on purpose is cheaper, and enough, as long as you treat a catalog edit without a bump as a bug.

Changing what the checks mean without bumping the version is the one edit that quietly weakens the whole model.

Failed receipts are evidence too

A failed run keeps its receipt, with the failing prefix: every command up to and including the one that failed, each with its log.

Its verification differs from a passing one, but it is not weaker. The results must be a prefix of the plan, every command but the last must have exited 0, and the last must not. The identity fields and the log hashes are checked exactly as for a passing receipt.

That last part is another audit finding. My failed-receipt path once skipped the log and configuration checks that passing receipts got, so the record of past failures could be forged freely. Evidence you check less carefully is evidence someone will eventually rely on.

src/receipts/verify-failed-receipt.ts
import type { Command, CommandResult } from "./receipt";

export const isFailingPrefix = (actual: CommandResult[], plan: Command[]) =>
  actual.length > 0 &&
  actual.length <= plan.length &&
  actual.every(
    (result, index) =>
      result.id === plan[index]?.id &&
      (index < actual.length - 1 ? result.exitCode === 0 : result.exitCode !== 0),
  );

Why keep failures at all? When an integration breaks the branch and someone fixes it, the record names the failed receipt next to the passing one. The history says what went wrong and how it was corrected, not a cleaned-up story in which nothing ever failed. That is what you want the morning after.

What a receipt does not protect against

Tamper-evident is not tamper-proof, and pretending otherwise would be worse than the limitation.

In my setup, the guard, the coordinating agent and the executors run in the same container, as the same user. Anything that can write a receipt can forge one, and any signing key one process could use, the others can read.

So receipts defend against mistakes, drift and corner-cutting: a suite that was skipped, a catalog that changed under a running pass, a stale baseline, a run on the wrong commit. In my experience, that is most of what goes wrong.

They do not stop an agent that deliberately forges its own evidence. That defense lives one layer up, where the privilege boundary is: signed commits, protected branches, and a human reviewing the merge. An agent able to forge a receipt is also able to write code that passes every check, and review is the control for both.

If you need receipts that certify themselves, you need real privilege separation: the guard running as another user or in another container, with a key the agents cannot read. That is an architecture change, not a key file.

A receipt worth trusting

If agents, or any automation, report green and you merge on it, check your setup against this list:

  • It names the commit, the base commit, the branch and the repository it validated.
  • It names the phase, so a baseline cannot stand in for a final run.
  • It hashes the exact commands and carries a version you bump when their meaning changes.
  • It records a clean tree before and after.
  • It keeps every log and the hash of each, read back at verification.
  • It is checked against an expectation, and a refusal says why.
  • A failed run writes one too, and you keep it.

Let the agent decide what to do. Never let it certify that it was done: the certificate is written and checked by code that takes no part in the work.

Mohammed Hamdoune is a Staff Software Engineer specializing in software architecture and AI-assisted delivery. He writes about real-world engineering decisions at sidlynx.me and founded LRJI, a software architecture consulting firm at lrji.co.