Our AI Agent Guardrail Signed a False PASS. Here’s the Fix.

A green check is only useful if you know what it checked.

During an adversarial review of our own local CLI, we found a case where CodeTruss v0.1.1 could sign a PASS receipt even though the captured change had never passed every configured check in that state.

This is the failure mode, the exact artifacts we tested, the fix implemented in an unpublished v0.2.12 candidate, and the result after that fix shipped publicly in v0.2.14.

The false PASS was caused by shared mutable state

CodeTruss can run several repository checks before a pull request. In v0.1.1, those commands ran in order inside the developer's mutable checkout.

That made the result order-dependent. If command one formatted, generated, migrated, deleted, or otherwise repaired a file, command two inspected command one's mutation instead of the agent result named by the receipt.

Our minimal fixture had one committed empty file and an untracked broken.flag. The allowed scope included the flag. We configured two checks:

1. rm broken.flag
2. test ! -e broken.flag

The first check removes the problem. The second confirms that it is gone.

The exact review invocation was:

codetruss review \
  --task "Keep broken.flag absent" \
  --allow "broken.flag" \
  --verify "rm broken.flag" \
  --verify "test ! -e broken.flag"

We ran it against the exact v0.1.1 tarball and the public v0.2.14 release archive on July 14, 2026. Their SHA-256 digests are 0996c295c4dfae04d886aab4690284add2f287441ae74046eabbfd10e08652e2 and 7fd85c939a3224c0678f6541b52ba4dd7e40b68a8dfaffd5bd6c8e0c12425da1, respectively.

The v0.1.1 result was false evidence. The flag existed in the state CodeTruss set out to review. Check one deleted it from the real checkout, check two observed that deletion, and the receipt recorded two successful checks.

We have no evidence that this flaw harmed a customer. Ordinary local receipts are intentionally invisible to CodeTruss, though, so an absence of synced or reported incidents cannot prove that no local user encountered it.

v0.2.14 gives every check the same starting state

The fix implemented in the unpublished v0.2.12 candidate treats evidence as immutable Git states instead of a live directory. It shipped publicly in v0.2.14:

1. Capture exact baseline and final Git-visible trees, including non-ignored untracked files.
2. Keep synthetic snapshot objects in a disposable private Git object store rather than leaving source blobs in the user's repository database.
3. Run scope classification, diff capture, and deterministic analyzers against those frozen trees.
4. Materialize a fresh copy of the exact final tree for every verification command.
5. Discard changes inside each disposable Git-tree materialization before starting the next command. Reused ignored dependencies, external symlink targets, absolute paths, and other host state are outside that guarantee.
6. Sign the baseline tree ID, final tree ID, command results, evidence coverage, and verdict into the receipt.

In the v0.2.14 run, the first check still deleted its copy of broken.flag and exited 0. The second check started from a new copy of the same final tree, saw the flag, and exited 1. The source checkout remained unchanged. The receipt returned FAILED.

The receipt recorded baseline tree 483afdad828a460eb2c23591968c9ff82c24b8b7 and final tree 77880070f5003e3a800739725c03782232191543. Those content-addressed IDs identify the two disposable evidence trees used by this fixture. They are not a remote attestation, and cleanup intentionally removes the underlying synthetic objects from the source repository after the completed command.

The maintained end-to-end regression also checks the opposite direction: command one deliberately corrupts its snapshot, while command two must still see the original final tree. That guards against both false failures and false passes caused by cross-command state.

The public release archive, the CodeTruss production download route, and the checked-in artifact all matched the v0.2.14 digest above. GitHub's attestation verifier also accepted the release archive's repository provenance and CycloneDX SBOM attestations. That verifies the artifact chain we tested; it does not prove that the program is defect-free.

Why verification isolation matters in real repositories

Many normal checks mutate files:

• formatters rewrite source;
• code generators create clients and schemas;
• package managers update lockfiles;
• migration tools update generated state;
• builds emit assets;
• test setup creates or removes fixtures.

Those mutations can be legitimate. They just cannot silently prepare the answer for the next check.

The useful question is not “did this sequence eventually become green?” It is “did each check pass from the exact final agent state?” Fresh materialization gives that question one stable meaning.

What the receipt proves—and what it does not

The signed receipt binds the captured baseline and final tree IDs, policy fingerprint, changed-file inventory, evidence coverage, analyzer result, verification exits, and verdict. When the signing key is trusted or pinned, re-running codetruss verify detects changes to the signed JSON, Markdown receipt, or captured patch.

A signature does not establish signer identity from an untrusted self-contained public key or prove trusted execution. It does not make a flaky or incorrectly written check correct, and it does not guarantee that every analyzer conclusion is true. CodeTruss receipts are inspectable evidence, not remote attestation.

Verification commands are also trusted local shell commands, not an operating-system sandbox. A command that deliberately uses an absolute path or other host access can still modify the machine. Repository-configured commands require explicit user-local approval before automatic execution.

Ignored files remain outside Git-visible evidence unless a project check covers them. Binary files are inventoried as changed assets but their contents are not generally analyzed, and a binary change does not by itself block PASS. Truncated or otherwise incomplete diff evidence is called out in the receipt and does block PASS.

Completed commands clean up their disposable evidence store in a finally path. Hook evidence can persist while an agent turn is still active or retryable, and a process or machine crash can leave local state for later cleanup. CodeTruss does not claim crash-proof deletion.

Local by default, explicit sync only

Deterministic scope, snapshot, analyzer, and verification work stays on the developer's machine. CodeTruss does not upload source, patches, or receipts during run, review, verify, or hook execution. Optional --llm review contacts the developer's selected provider, and developer-supplied agent or verification commands may use the network themselves.

sync is a separate explicit action. sync --dry-run shows the outgoing document first. The sync envelope omits the patch and local evidence filenames and redacts the absolute repository path, agent arguments, verification commands and output, unrelated paths, and duplicate whole-repository finding bodies. It still contains receipt-scoped facts such as the task, changed paths, relevant findings, verdict, policy hash, and evidence hashes, so developers should inspect the preview before uploading it.

Check the current CLI—or opt in to test it locally

Install the current free CLI and inspect its checksums, or review the v0.2.14 release and attestations.

We are also recruiting 10 independent developers for a consent-based, 14-day design-partner cohort. Participation can remain local-only: no repository access and no receipt sync are required.

To opt in, email zack@codetruss.com with the subject CodeTruss design partner opt-in and include your coding agent, language, and approximate repository size. Say whether you want local-only participation or are open to reviewing a redacted receipt before optional sync.

Participation does not grant permission to publish a name, quote, repository, or result. Each is a separate opt-in, and cohort participants are not added to a general marketing sequence.