All posts

August 7, 2026 · 12 min read

We Merged 18 Pull Requests in One Night. Our Own Gate Blocked Us First.


On the night of August 6, 2026, eighteen pull requests merged into CodeTruss main. Pull requests 11 through 28. The first landed at 6:09 PM Central and the last at 11:12 PM, so about five hours end to end.

In that window we shipped five new detection rules, a repricing of the whole product onto seats, a rebuild of the marketing site and the dashboard, five CLI releases from 0.2.28 through 0.2.32, a public benchmark, and a fix for the schema drift that had broken sign-in forty minutes earlier.

Every one of those commits went through the same pre-commit hook, which runs the tool we sell against the change we were about to make. Not a linter we wrote for ourselves. The shipping CLI, on the same policy file a customer gets, blocking on the same rules.

Here is what that actually looked like, including the parts where it stopped us.

What the gate is

One file, .git/hooks/pre-commit, and it does one thing:

codetruss review --staged --task "pre-commit"

Exit 2 blocks the commit. The repository policy in .codetruss.yml names two verification commands, pnpm lint and pnpm test, and pnpm test is 1,474 tests: 1,237 in the app suite across 137 files, plus 237 in the CLI suite across 19 files. Both commands run on a fresh materialization of the exact staged tree, not in the working directory, so one command cannot quietly prepare the answer for the next.

Then it writes a receipt. Four files per run: the signed JSON, a readable Markdown rendering, the captured patch, and the signature. There are 37 of them in the main checkout right now, 148 files, and 20 of those receipt sets are from that night. Agents working in their own Git worktrees write receipts into those worktrees, so another five live there. None of them were uploaded anywhere. Local receipts are invisible to us by design.

The gate blocked us, so we fixed us

At 1:44:16 PM, on the commit that became the first of those eighteen pull requests, the hook returned FAILED. Receipt 20260806T184416315Z-b7aa6c. The blocking line:

1 high/critical security or dependency finding(s) affect changed files
Table: Severity, Analyzer, Location, Finding
SeverityAnalyzerLocationFinding
HIGHsecretstests/analyzers.test.tsPossible Database URL with credentials committed

That commit was shipping a hardened secrets analyzer. The new analyzer found a credential-shaped string in a test fixture written to exercise the new analyzer. Both verification commands had already passed, pnpm lint exit 0 and pnpm test exit 0, so nothing was broken. The gate was doing its job on a file that only looked dangerous.

There were two ways out. Add an exception for test files, or change the fixture. We changed the fixture:

-file('.env', 'DATABASE_URL=postgres://<user>:<password>@db.prod:5432/app', { kind: 'config' }),
+// Credential-free on purpose: this case only needs a runtime .env to
+// exist, and a credential-shaped fixture trips the repo's own scanner.
+file('.env', 'DATABASE_URL=postgres://localhost:5432/app', { kind: 'config' }),

The test never needed credentials. It only needed a runtime .env to exist. The removed line is redacted above, and the last section of this post explains why it had to be.

Two minutes and nineteen seconds later, receipt 20260806T184635257Z-15b97b, same 101 changed files, HIGH finding gone, commit through. Diff the two captured patches against each other and that fixture is the only difference in the whole tree. The evidence went from 796,336 bytes to 796,490, so the entire fix is 154 bytes, and no rule got softened to make a green light appear.

That is the whole discipline in one paragraph. When the gate flags you, the tempting move is to widen the gate. Every time you do that, the gate is worth a little less to the next person who trusts it.

Every fresh worktree starts untrusted

Verification commands are trusted local shell commands, not a sandbox. So the CLI will not run repository-configured commands until a human has inspected them once. Approval is a SHA-256 over the canonical repository path and the exact ordered command list, which means a new Git worktree is a new path, which means a new hash, which means untrusted.

Our agents each work in their own worktree. So the ceremony ran over and over that night. Inspect pnpm lint and pnpm test, then run codetruss verify-policy trust, then commit.

One agent hit it while cutting the 0.2.30 release and made a different call, then wrote this in its own pull request:

The pre-commit hook blocked this commit (verification commands are not trusted (3170a791a03f)), so it was made with --no-verify after running lint and the CLI suite by hand. Worth a codetruss verify-policy trust if you want the hook active on this machine.

That is the right behavior. It ran the checks, it declined to add a persistent trust entry it was not authorized to add, and it said out loud that this commit has no signed receipt. A bypass that announces itself is a fact you can act on. A bypass that hides is the thing the receipt exists to prevent.

The worktree this post was written in started untrusted too, and it did not get a pass for being ours:

$ codetruss verify-policy status
untrusted 7b17005f555d2308b5b67c568d0832d50dc22de40787d8e4aa52da0d1a8ede29
- pnpm lint
- pnpm test

Same ceremony. Read what pnpm lint and pnpm test expand to in package.json, confirm .codetruss.yml is byte-identical to the one on main so nothing was slipped into the policy inside the worktree, then approve that exact path and that exact ordered pair of commands. Not "trust CodeTruss." Trust these two strings, here.

Using it is how we find its bugs

The gate erased a person's prompt. Baseline capture assumed every agent turn carries a submitted prompt. Harness machine events do not: background-task notifications, hook feedback, resumed sessions. Two failures came in that night. One turn reached the Stop hook with no baseline and went unreviewed. The other was worse: a human typed a prompt, the hook could not take a snapshot, and it blocked. Their text was gone and they were told to try again.

CLI 0.2.31 fixed both at the root. A promptless turn is a legitimate turn shape, because an exact baseline is a snapshot of the working tree and the tree exists whether or not anyone typed anything. And the rule that came out of it is worth stating plainly: instrumentation never blocks a person. Capture failures emit a note and let the prompt through. Stop remains the enforcement point and still fails closed on a turn with no provable baseline, so an agent still cannot finish unreviewed. Blocking is for verdicts, not for bookkeeping.

A comment could hide a live credential. This one came out of a cold practice round, one of us installing the published tarball into a scratch repository and behaving like a first-time user with no memory of how any of it works. A four-line file whose first line reads // AUTO-GENERATED FILE - DO NOT EDIT, with a live Stripe key on line four, produced a signed PASS. The same file without that comment FAILED.

Generated-file classification exists so that machine-written output does not produce spurious "oversized file" findings. It was silently disabling every analyzer, secret scanning included. One comment, whole scanner bypassed. The fix keeps the excluded text available to the secrets pass only, so line counts and quality analyzers still skip generated files and a credential stays a credential whatever wrote the line. It shipped in CLI 0.2.33.

That paragraph was written while this was still an open bug sitting on a branch, and it stays here now that it is fixed, because disclosing a bug only after it is safely closed is not the deal. The gate found this, and the gate is the product.

Failing loudly, applied to our own pipeline

Two of those merges shipped schema changes. Install tracking went in at 8:50 PM, the seat repricing at 10:02 PM. Production served the new code against the old database both times, because the Vercel build ran next build and never ran migrations.

Five minutes after each merge, production started throwing. The download counter first, on a DownloadEvent table that had never been created. Then, at 10:08 PM, signing in:

Invalid prisma.membership.findMany() invocation:
The column Subscription.receiptSyncGrandfathered does not exist in the current database.

P2022, on /auth/continue. Two migrations, two surfaces, one root cause. The same two migrations were also unapplied on the developer's local database, where they had already broken a tenancy isolation test, which is how the shape got recognized fast.

The fix was not to run the migration. Running the migration fixes tonight. The fix was to make the class impossible: the production build now begins with a migration step that applies pending migrations, exits non-zero when they fail, and only runs when VERCEL_ENV is production. Previews, CI, and local builds do nothing.

And then it failed. Three production builds in a row, starting with the very deploy that introduced it:

Table: Time, Result, Reason
TimeResultReason
10:49:44 PMERRORP1001 could not reach the database host on port 5432
10:50:11 PMERRORsame
10:59:50 PMERRORP1000 authentication failed for the migration user
11:02:41 PMREADY[deploy-migrations] applying pending migrations via DIRECT_URL (port 5432)

Every failure printed the same line before it exited:

[deploy-migrations] "pnpm db:deploy" exited with code 1.
Failing the build so old-schema code is not served.

Thirteen minutes of a red pipeline, three deploys refused, until the direct connection string was actually correct. That is not the gate malfunctioning. A gate that only goes green is decoration. The whole reason to build one is the thirteen minutes where it will not let you past, and the alternative to those thirteen minutes was another silent skew that surfaces later as a login page that does not work.

Why we think any of this is credible

The benchmark we published that same night came out of the same protocol. Five of nine planted bug classes caught at the exact file and line. Zero false positives across eight repositories and 177,703 lines. Every single fire opened at its line and argued both ways by hand. Every target that returned zero also got a canary fixture pushed through the identical harness, because a clean sheet from an engine that quietly gave up is the worst artifact in this business.

And before release we took a severity down. The mass-assignment rule fired at HIGH on a helper that passes data: any straight into an update. Every factual claim in the finding was true. The severity was not, because the only caller is an allowlisted literal built from a verified Stripe webhook, so nothing in the evidence showed a request body reaching the write. So the rule was split. HIGH now requires actual request-flow evidence at the site, and the open-record contract reports separately at MEDIUM. Severity is a claim about evidence, and it does not get to run ahead of the analysis that produced it.

What the gate still cannot see

A local receipt is not a security review, and it says so in its own text. On the night described here, no injection or taint analysis ran locally, which means SQL injection, command injection, path traversal, SSRF, open redirect, XSS, and insecure deserialization were never checked. No cross-file symbol graph is built, so architecture and dead-code conclusions cover only what a single-file pass can see. No model reads the diff under an agent hook, ever. No health score is calculated, because a number derived from the passes that did run would overstate what ran.

The sprint this post describes ended with one of those classes closed: CLI 0.2.35 runs the shared SAST engine and its source-to-sink taint tracking locally over JavaScript, TypeScript and TSX, so SQL injection is checked on your machine now, while command injection, path traversal, SSRF, open redirect, XSS, deserialization, every other language, and the symbol graph stay hosted.

A PASS means the checks that ran found nothing new. It is not a statement that the change is secure, and we would rather say that in the artifact than in a footnote.

Four of the nine benchmark classes were still misses that night, and each one needed dataflow we had not built: IDOR, missing authorization, a read-modify-write race, and a pagination off-by-one. The race is the one that moved. A detector for it merged at 10:50 PM that same night, swept across 1,917 files and 181,585 lines, and returned exactly one hit, which is the true positive. The benchmark page kept publishing five of nine that week, because the page gets updated when someone re-runs the whole corpus and not a minute before.

The corpus has since been re-run, after everything described here. The page now publishes six of nine with three misses, the race having moved into the caught half. It moved because it was closed, not because the bar moved.

This post has a receipt, and the first one was a FAILED

Same hook, same 1,474 tests, same signature. Receipt 20260807T042724068Z-9d5709, one changed file, and it would not let the commit through:

Table: Severity, Analyzer, Location, Finding
SeverityAnalyzerLocationFinding
HIGHsecretssrc/lib/blog.tsPossible Database URL with credentials committed
MEDIUMsizesrc/lib/blog.tsOversized file

The HIGH is the diff near the top of this post. Quoting a credential-shaped fixture puts a credential-shaped string in a file, so the rule that blocked the original blocked the article about the original. Same two options as before, same choice: redact the quote, leave the rule alone. Rewriting the removed line as <user>:<password> cleared it, because a value that announces itself as a placeholder is not a leak. The next receipt came back REVIEW_REQUIRED with the HIGH gone and only the MEDIUM left, and the MEDIUM is this file, which was already flagged as oversized before I added two thousand words to it. Also fair.

A gate that catches its own author is not being clever. It is applying one rule to one shape without caring who wrote it, and that indifference is the only property that makes it worth anything. You do not have to take any of this on faith, and that is the entire point.

Related CodeTruss guides

We Caught 6 of 9 Bugs. We Are Publishing the 3 We Missed.

CodeTruss built a corpus of nine bugs AI agents actually write, detected six at the exact line, and swept the new rules across eight repositories for zero false positives. Here is the whole result, including the misses and the rule we had to split before shipping.

Auditing an AI-Built SaaS: The LastSaaS Release Checklist Field Note

An independent public-source scan of jonradoff/lastsaas, a Go SaaS foundation built with Claude Code, shows how repository evidence becomes a fork release checklist — and why every automated security finding was rejected on manual review.

What the Official Next.js SaaS Starter Leaves for Your Release

An independent public-source scan of nextjs/saas-starter shows the release work a deliberately minimal template assigns to every fork: tests, CI, webhook configuration, seed hygiene, and tracking upstream security fixes.

Release Checklist for an AI-Ready SaaS Template: Open SaaS Field Note

A redacted public-source field note on wasp-lang/open-saas shows how CodeTruss turns auth, billing, jobs, email, file upload, CI, and Playwright evidence into a release checklist.

What a Release-Handoff Scan Found in an Open-Source AI Chatbot Template

An independent review of a pinned public repository shows how CodeTruss turns architecture and test signals into a release checklist—and why analyzer output still needs human judgment.

Our AI Agent Guardrail Signed a False PASS. Here’s the Fix.

CodeTruss v0.1.1 signed PASS for a change that had not passed every check. We reproduced the flaw and retested the immutable-snapshot fix in v0.2.14.

Put the same gate on your own agents

The CLI is free and local-first. It writes the same signed receipt on your machine, and no source, diff, or receipt leaves it.