We Gave Our Agent a Spec Language

TL;DR. Every coding agent has a plan. Almost none of them write it down in a form anything can check, so the plan lives in the model's head, mutates silently under context pressure, and the first time you learn it changed is when the output is wrong. We made our agent write the plan first, as a spec: typed steps, dependencies, and a postcondition on every step. A server holds the spec and grades each step, and the agent does not get to mark its own work done. The obvious objection is that an agent writing its own spec is grading its own homework. It is not. The reason is separation in time, and that turned out to be the most interesting design lesson in the whole system. The spec is a message from the agent's best self to its worst self.

In the last essay I told the origin story: building a product across more than twenty repositories with agents, watching the quality bar sag everywhere I was not standing, and deciding the bar had to live somewhere that was not me. This one is about the first place we put it. Before our agent writes a line of code, it writes a spec, in a small YAML language built for exactly this, and a server holds it to that spec for the rest of the session.

The Most Dangerous Artifact Is the One You Cannot See

Watch a coding agent work on anything non-trivial and you can feel the plan in there. It reads ten files, orients itself, and starts making changes in a sequence that clearly follows some internal structure. The plan exists. It just exists in the one place nothing can inspect it: the model's working context.

That placement fails in a specific, repeatable way. Context is under constant pressure in a long session. New file contents, tool output, error messages, all of it competes with the plan for the model's attention, and the plan quietly loses. Step four gets reinterpreted in light of a test failure. A constraint established early stops being load-bearing. The agent is not lying to you at any point. It is improvising against a memory of its own intent, and the memory is degrading, and there is no signal, because there is no artifact to diff against. You learn the plan changed when the output is wrong, which is the most expensive possible time to learn it.

Humans solved this so long ago we forgot it was a solution. Design docs, acceptance criteria, test-first development: all of them are the same move. Externalize intent before execution, in a form that can disagree with what you actually did. We just never made agents do it in a form that could disagree. A plan the agent describes in prose is theater. Prose cannot fail. What we wanted was a plan that could fail.

What the Agent Actually Writes

So the rule in our repositories became: for any non-trivial task, the agent writes a spec before it touches anything. The spec is a small YAML document. It declares the steps, what each step consumes and produces, which steps depend on which, and, on every step that matters, a postcondition: an expression evaluated against the step's actual output that either passes or fails. Tests pass. The file list is non-empty. The output parses against the contract. Whatever "this step genuinely happened" means for that step, written as a check.

Then the interesting part: the agent hands the spec to a server and the server takes over the bookkeeping. The agent asks for the next step, does the work, and reports back. The server evaluates the postcondition against what actually happened, and only then does the step count as done. If the check fails, the step is not done, whatever the agent's narration says, and the specific violation goes back to the agent as the retry context. Not "try again," but "your output failed this exact check for this exact reason." At the end of the run the server emits an audit trace, step by step, attempts, failures, what finally passed, and that trace goes into the commit.

One detail I did not expect to matter as much as it does: the spec never appears in the conversation. The agent narrates its progress in plain English, like any session, while the contract runs underneath. When a postcondition fails, the agent fixes the step and retries without ceremony. From the outside you see a normal working session that happens to never skip ahead, never declare done early, and always leaves receipts. The discipline is invisible until you read the trace.

The Self-Grading Objection

The objection everyone raises, and the one I raised myself: the agent writes the spec, so the agent controls the test it will be graded against. Surely it just writes itself an easy spec.

The answer turned out to be about time, not trust. The agent that writes the spec and the agent that executes step five are, for practical purposes, different agents. The spec-writer sits at the start of the session with a clean context, full attention on the task's actual shape, and no sunk cost. It has no incentive to cheat, because it is not yet the one who has to do the work. The executor, twenty minutes and forty tool calls later, is the one under pressure: context crowded, a test failing for the second time, every incentive pointing toward the shortcut. That is the agent that declares victory early, and that is precisely the agent that no longer gets a vote, because the check it must pass was written by its earlier, better self and is enforced by a server it cannot sweet-talk.

The spec is a message from the agent's best self to its worst self. Once I saw it that way, I stopped worrying about self-grading and started worrying about spec quality, which is the real limit and I will get to it.

This also explains why the enforcement has to live outside the model. The postcondition is only worth something if the entity evaluating it is not subject to context pressure. A model checking its own work inside the same degrading context is one system failing together. A server evaluating an expression is boring, and boring is the entire point.

The Check That Could Never Fail

Now the story that taught us the limits, because a system like this earns trust through its failures, and this one handed us two at once.

The first failure was loud. One of our pipeline spec files went missing entirely, swept away in an unrelated thirty-file commit, and for six weeks the command that depended on it crashed on contact with a clear error. The restore was a one-line git operation. Embarrassing, but honest: when a spec is simply gone, everything downstream refuses to run, and refusal is a kind of integrity.

The second failure was the quiet one, and the review that came with the restore is what surfaced it. The restored spec had a scaffolding step whose job was to create a feature's folder structure and files, and the step carried a validation that had been green for as long as the file existed. The validation was pointed at the wrong thing. It checked a criterion against an adjacent document that could never actually answer it, so the check passed vacuously, on every run, forever. Nothing had shipped broken through it yet, as far as we know. But a run that created zero files would have sailed straight through, and nothing would have said a word. Green did not mean "the step worked." Green meant "this question was never askable."

The fix was two text edits to the spec file: delete the check that could never fail, add one that could, a real postcondition on the step's structured output, the number of files created must be greater than zero. Now a scaffold run that creates nothing fails at the step where it happened, loudly, instead of silently until a user hits it.

Two lessons came out of that session, and the journal recorded both. The first is narrow and technical: a check has to be answerable from the thing it examines. If the truth you need lives in the step's output, check the output, not some adjacent artifact that feels related. Vacuous checks are worse than no checks, because they emit green, and green switches off human attention.

The second lesson is the one I would put on a wall. The dangerous defect was not the missing file, which announced itself instantly and loudly. It was the dormant check, which would have blessed a broken run with a straight face. And its fix was not code. It was two lines in a spec. When the process itself is a declared, versioned artifact, process bugs become data bugs, and data bugs are the cheapest bugs in the world to fix and review. The diff was legible to anyone in thirty seconds. That is what you buy when the plan stops living in anyone's head, human or model.

What It Does Not Solve

The pattern from the origin story holds here too, so let me be as hard on this as I was on everything else.

A postcondition only checks what you aimed it at. The check that could never fail is not just a war story, it is the standing risk. The spec language makes intent checkable, but it does not make intent correct. Writing a good postcondition is the same skill as writing a good test, and an agent under instruction to produce a spec can produce a weak one, checks that pass too easily, steps sliced so coarsely that "done" barely constrains anything. We handle that the same way you handle test quality in a human team, by reviewing the spec itself before execution starts, at the lifecycle layer above this one. The step-checker checks steps. Something else has to check the checker, and that layered arrangement, gates above postconditions, is the next essay.

And there is a real ceremony cost. For a one-file fix, writing a spec is overhead with no payoff, so trivial work skips the machinery entirely. The dial matters: the spec layer earns its cost exactly where the origin story said the bar sags, on multi-step work, across boundaries, in sessions too long for anyone's attention, mine or the model's, to hold the plan intact.

What I no longer accept is the alternative. An agent whose plan I cannot read, whose steps nothing checks, and whose "done" I have to take on vibes is an agent I have to watch, and watching does not scale. The spec language did not make our agent smarter. It made the agent's claims falsifiable, one step at a time, and falsifiable turns out to be the property everything else is built on.