How We Verify Agent Work

We wrote about the whole agent stack a fortnight ago: the idea panel, the brain-and-builder split, the review gates, the phone. This is the part that has changed most since, and the part that turns out to matter most.

Everything here answers one question. An agent says it is done. How do you know?

The premise: "done" is a claim

Our builders are competent. They are also, structurally, unreliable narrators. Two failures made that concrete.

One builder reported implementing a feature it had never written. The diff contained no trace of the thing it described. Another reported the test suite green. The runner had crashed before collecting a single test, printed nothing that looked like failure, and the builder read the absence of the word "failed" as success.

Neither is dishonesty. Both are the same mistake: treating output that nobody checked as evidence. That mistake is available to any agent, and it is available to us reading the agent's summary.

So the rule the rest of this follows from: the thing that judges the work is never the thing that produced it, and judgment reads artifacts rather than descriptions.

The brain re-runs the tests

For months our review gate could not run tests. It works in a read-only sandbox, which kills the test runner before it starts, so "the suite passes" reached us as an assertion from the builder and travelled no further.

Now the brain re-runs them itself on every returned build, before the gates and before believing any report. The check is small and it fails closed on the exact shapes that hide a bad build:

• No summary line is a failure, never a pass. The first version of this check reported success on a crashed runner, because the word "failed" never appeared in its output. Absence of failure is not evidence of success, and that is a bug we shipped before we caught it.
• Zero tests passed is a failure. A suite that runs nothing certifies nothing.
• A net deletion of test lines is a failure. That is how a red suite becomes green. Any test edit at all is surfaced for reading, because changing behaviour and moving its guard in the same commit is legitimate and must be seen.

It also detects which runner the project uses, after the first version was hardcoded to one and silently did nothing on a repository that used another.

A gate is validated by attacking it

The stack article said a gate nobody has attacked is decoration. Since then we built one and immediately proved the point on ourselves.

A security scan found an API key in a URL query string. The key was going to the service that owns it, so authentication was fine, but a URL travels through proxies, error messages and shell history, and a long-lived secret in one gets copied to places nobody audits. We fixed it and then built a check so it could not come back: something that reads a diff and blocks when a credential reaches a prompt, a log line or a URL.

Then we attacked the new check with the exact bug it was built for.

It missed it. The check understood credentials reaching prompts and logs, and knew nothing about URLs. Had we shipped it on the strength of its passing unit tests, we would have added a gate that gave us confidence about the one thing it could not see.

That was round one. Round two, tightened, produced fourteen false positives on real history, all from test files that deliberately set fake keys and from comments that merely mentioned one. A gate that cries wolf is worse than no gate, because people learn to wave it through. Round three replaced word-matching with something closer to a real flow check: the credential must be assigned to a variable, and the suspicious line must actually reference that variable. Measured against our own history afterwards: nothing over the last twenty-five commits, nine over five hundred, of which one was a genuine finding.

Three rounds to a gate we trust, and the only reason we know it works is that we spent two of them watching it fail.

Tests that have never failed are not known to work

The credential fix needed a regression test, so a future change could not quietly undo it. Writing the test is the easy half. The half people skip is proving the test can fail.

We reverted the fix in the source, ran the test, and watched it go red. Then restored the fix and watched it go green. Thirty seconds, and now the test is known to be load-bearing rather than assumed to be.

This matters more with agents than without. An agent asked to add a test will produce something that looks like a test, passes, and asserts nothing that would ever break. It satisfies the request perfectly. Only trying to break it tells you which kind you got.

When the instrument lies

The worst failure of the last fortnight was not code that broke. It was a gauge that read wrong.

Our research tool pulls from several sources. One of them, Reddit, blocks the datacenter our server sits in, so one of its access routes always returns a permission error. The tool classified that error as an authentication failure, and authentication failures are deliberately sticky: the reasoning is that a login problem needs a human to fix it, so it should not be downgraded just because some results arrived.

The result was a banner that read "Reddit blocked, authentication failed" on runs that had returned fifteen real, relevant posts from three communities. A session read the banner, concluded the evidence was unreliable, discarded it, and reported thinner findings than it actually had. Nothing was broken. The measurement was.

The first fix was in the obvious place and did nothing, because the error object computes its own classification at the moment it is created and the pipeline prefers that pre-computed answer. The second fix went to the single point every path passes through, which is also the only place that knows which source it is talking about. A blanket block from a datacenter address is now classified as unreachable, which is honest, and unreachable is not sticky, so a run that returns results describes itself as partial rather than failed.

The lesson generalises past this bug. Diagnostic output is code, and it can be wrong in the direction that costs you most: a false alarm trains people to ignore the alarm, and a lying gauge makes competent agents throw away good work. Any signal that steers a decision deserves the same scrutiny as the decision.

Rules that had to become mechanisms

The stack article described a mechanical sweep that exists because a written rule kept being violated by sessions that had the rule loaded. That pattern kept repeating, and each repeat cost the same way.

Every session ends with a status line saying done, waiting, or blocked, so it is obvious which state we are in without asking. Written down for weeks. Then a session wrote "done" twice while the builder it had dispatched was still mid-run, and we went looking for finished work that did not exist. The rule already said a running builder means waiting. Nothing checked. Now something does: on every attempt to end a turn, a hook reads the register of dispatched builders, asks whether any is still working, and refuses a "done" while one is. The rule was advice for weeks; it took ten minutes to make it a gate.

The knowledge base is updated when work ships, not at the end of a session. Also written down for months, also unenforced, also drifting. Now the deploy script fails when a change touches product code and no knowledge page changed, a hook nudges a session that merged something and wrote nothing, and a background job drafts a page from the diff for anything that still slips through. The drafted page is explicitly marked as a backstop, not a record: an automatic summary is better than silence and worse than a person who understood the change.

The general shape: if you find yourself writing the same rule down a third time, the rule is not the problem. Sessions do not forget because they are careless; they forget because remembering was never checked.

Auditing the tools themselves

An agent's skills are instructions that load into every session and tell it what to do. Ours had grown to over a hundred. We had never checked a single one for whether it was worth its cost, and we had never checked any of them for safety at all.

The audit was uncomfortable. Seventy-nine of them fired automatically on matching topics. Thirty-two of those had never once been used, while their trigger text was read on every turn of every session. Not one had ever been measured against a version of the task without it. The listing had grown to within a whisker of its budget, past which it silently truncates and routing degrades with no error.

Then we ran a security scanner over them. It reported one critical finding, eighty high and a hundred and fourteen medium, and the honest headline is that most of that was noise. It flagged our own anti-injection guidance as prompt injection, because that guidance quotes the phrase it is warning about. It flagged a skill reading our own settings file as snooping.

Three findings were real. Two skills shipped compiled bytecode, which discovery routines skip, so bad bytecode there could sit unread beside clean-looking source. Five invisible characters were embedded in skill text, one inside a URL where it would silently break the string for anyone who copied it. And one script authenticated by putting a key in a URL, which is the bug that started this article.

The number was not the finding. The triage was. Any scanner that returns two hundred results has told you nothing until someone has read them, and a team that treats the count as the answer will either panic or, far more likely, stop looking.

Where this sits in the stack

Verification is not a step at the end. It runs at four points:

Before code, the acceptance criteria are written first as a ledger of checks with expected outputs, so "done" has a definition that predates the work and cannot be quietly renegotiated once it is hard.

During the build, the builder works in an isolated checkout and can prove nothing to itself that matters.

On return, the brain re-runs the tests, then the cheap multi-model jury reads the diff, then an agentic reviewer from a different lineage reads the repository around it. Whoever built it does not review it.

At ship, the deploy script gates on the things that are mechanically checkable: the knowledge base was updated, no credential reaches a log or a URL, the build actually deployed, and the production page contains what it should.

None of those layers trusts the one before it. That redundancy looks wasteful right up until the day one of them is wrong, which, on the evidence of the last fortnight, is roughly every third week.

What we would tell you to steal

• Re-run the tests yourself. A builder's "green" is a claim, and the cheapest lie to catch.
• Fail closed on silence. No summary, zero tests, deleted test lines: treat all three as failure. Absence of an error is not evidence.
• Attack every gate with the bug it was built for. Ours missed on the first attempt. Yours will too.
• Make each new test fail once, deliberately. A test that has never been red is decoration.
• Treat a noisy gate as broken. Fourteen false positives is not a gate, it is a thing people click through.
• Audit your diagnostics. A lying gauge costs more than a missing one, because it is believed.
• When you write a rule down for the third time, build the mechanism instead.

The honest print: everything above comes from one founder's stack over about a fortnight, and several of these mechanisms are days old. The gates described here have caught real defects, including one live credential leak across nine call sites. They have also, in the same period, produced a false alarm on a finished build, blocked clean work over a template phrase, and briefly reported a healthy deploy as a failure because we deleted a directory while the script was still standing in it. Both halves are the report.