Externalized proof: confidence is cheap

I talked in the past about the pieces you need to build a reliable software factory: memory, process, proof, coordination, execution. In this one I’ll talk about the most essential piece and probably one of the most misunderstood ones, because it is the layer most workflows underbuild and the one I keep coming back to when something ships that shouldn’t have.

Confidence is the cheapest thing an agent produces.

It costs nothing. “Fixed.” “Tested.” “Reviewed.” “Ready to ship.” The same fluent tone whether the agent ran the suite, ran one test, ran nothing, or ran something and quietly stepped around the failures. That is not the model lying to you, it is doing exactly what it was built to do: produce a confident-sounding summary. The problem is on our end. I used to treat that summary as the trust boundary, and as much as it looks like it should be, it never was.

Confidence and proof are not interchangeable

When an agent says “I tested this and it passes,” it has produced a claim. A claim is not evidence, and the two drift apart in a dozen quiet ways while the summary reads the same to most of us.

Did it run the tests, or describe running them? If one failed, did it fix it or delete it? Did it run the right ones, against this branch or a stale checkout? Did it read the output, or pattern-match “no errors” off a wall of text? Did a flaky test get a quiet retry until it went green? Did it skip the failing one as “pre-existing,” or confuse “the build compiled” with “the behavior is correct”?

I cannot answer any of those from a sentence an LLM produces. I can answer most of them from a command log.

That gap, between the sentence and the log, is this whole layer. Externalized proof. The discipline is moving trust out of the non-deterministic path and into the deterministic one.

The failure mode is narrated proof

The worst version of this is not the agent that says “I’m not sure.” It is the agent that confidently narrates evidence it never produced.

I have watched an agent write “ran the migration dry-run, no destructive changes detected” in a summary when the dry-run command appears nowhere in the transcript. Not malice. Like a parrot filling in the air with sounds it’s heard before, it is filling in the shape of a competent answer, and “I ran the dry-run” is what a competent engineer would say there. The narration is a prediction of what proof would look like, served at the same temperature as proof itself, and the model is equally fluent reporting or imagining. The prose gives me no tell.

So the proof can’t live in the prose. Prose is where the agent is strongest and least accountable, exactly the wrong place to keep the thing I most need to trust the output. The evidence has to be the artifact itself: the command that ran, the exit code, the diff, the CI run I can click into, the screenshot. The thing that exists whether or not anyone narrated it well.

What makes a trail trustworthy instead of theater

It is easy to produce something that looks like proof and isn’t. A pasted block of green checkmarks. A “tests pass” with no command above it. A screenshot of the happy path that conveniently avoids the case that broke. Evidence theater is worse than no evidence, because now I have to un-believe something before I can review it honestly.

A trail I actually trust tends to share a few properties, none of them about volume. I want to see the command, not a description of it, so I can run it myself. I want output from a real run, not the model’s memory of what runs look like. I want a walkthrough that explains why these lines moved here instead of restating what they now say. And I want it honest about its own edges and weaknesses, which are the properties most models quietly drop.

The honesty is the crux of the problem. A trail that says “ran unit tests, did not run integration tests, did not check the staging deploy” is more useful to me than one that just says “tested.” The second sounds better and tells me less. Stated plainly, missing evidence becomes a kind of proof in its own right: it points at exactly where my judgment still has to go. A log that admits its gaps is doing the work; a summary that papers over them is putting on a performance.

The discipline is “enough”

The trap on the other side is treating every change like it needs a museum exhibit.

If a one-line copy fix demands a screenshot, a diff walkthrough, a CI run, and a written rationale, two things happen. The proof becomes ceremony, and everyone skips needless ceremony. And the agent learns that proof is a box to fill, so it fills it with whatever passes the check, which lands me right back in theater, just a more expensive one.

Proof should be proportional to the risk the change carries. A CSS tweak might need a before-and-after screenshot, or nothing if the diff is self-evident enough. A schema migration needs dry-run logs and a rollback note, because the cost of being wrong is somebody’s data. The verification should match the blast radius. Only enough for it to be clear without overwhelming.

What I’ve been chewing on since is that “enough” isn’t a property of the change, it’s a property of the reviewer: what would let a skeptical reader believe this before it moves, and nothing more. Too little and I have shipped confidence. Too much and I have built a ritual nobody sustains, which decays into nobody reading any of it, which is shipping confidence again by a slower road. I have erred in both directions in the same week. The only rule I trust is asking, per change, what a reasonable skeptic would actually need to see.

The harness makes it cheaper, not automatic

This is where Beislið lives for me. The point of the harness is to make capturing evidence the path of least resistance, so the command log, the verification step, and the diff walkthrough get produced as the work happens rather than reconstructed after the fact. The process post is the longer argument for why that has to be structural, not a polite request in a prompt.

But I want to be honest about the limit. A harness can make proof easy. It cannot make proof right. It runs the tests and captures the output, but it can’t decide which artifact actually matters here, or when “enough” means stopping to ask a human. That so-called “taste” is the residue that stays with me after everything else moves out of the chat.

It also has to feed the layers next door. A “review passed” gate that reads a confident summary instead of a real artifact is a rubber stamp with a progress bar. The same goes for coordination: “may I proceed to the next stage?” is really “has the external system accepted the evidence this stage requires?” If the proof is theater, the whole ledger is theater that happens to have timestamps.

Where do we end up?

In the software factory post I made the arithmetic point: I can’t read every line of every change forever, so I have to move up a layer. What I want to add is what “up a layer” does to the work. It isn’t a demotion and it isn’t less attention, it’s the same attention spent somewhere where you have more leverage. I’ve long stopped watching the agent type and have instead started interrogating the trail it leaves. The unit of review is no longer the line of code, it’s the evidence around it: did it test the thing that actually changed, is the missing-evidence list honest or suspiciously short, does the proof match the risk or is it diligence cosplay? Building the trail that lets me ask those well is most of the job now, and it survives the next model in a way reading diffs never could.

There’s a rhyme with the memory layer here. Memento is only useful when memory carries its receipts: source, date, certainty and the vault’s retrieval is worthless if it surfaces a stale fact as a current one. Proof has almost the same shape: worth nothing without enough provenance to be trusted without re-deriving it. Memory without epistemology is faster hallucination; proof without provenance is faster confidence.

The model writes, and it will tell me it’s done. Our job should be to make “done” mean something the system can verify, not something the agent merely tells. Confidence it gives away for free. The trail is the part that costs, and in the long run it turns out to be the only part worth paying for.