Three minutes after I opened the pull request, a bot told me the endpoint I was rewriting could be knocked over by a small JSON body. I had written the bot.
The code I was changing takes text containing {{placeholders}} and replaces each one with a real value. Then it does the whole thing again, because a value it just substituted can itself contain another placeholder, and it keeps going until a pass replaces nothing. So give it a placeholder whose value contains that same placeholder twice. Every pass replaces one and produces two, the string doubles each time, and the loop never reaches a pass that changes nothing. Nothing bounded that, and the endpoint takes anonymous requests, so any stranger on the internet could send a small body and pin the worker. Billion laughs, no XML required.
The loop was not something I wrote, and the bot said so itself: "The core loop DoS predates this PR... but this PR rewrites exactly this loop and adds a termination-safety comment, so it's the natural place to close the gap." It was reporting somebody else's old bug and it was honest about that, which is the part that made me read the rest of the comment. A human reviewing my diff would not have looked at that loop at all, because the loop was not the point of the change. Thirty seconds later the code reviewer found the same loop from its own angle, self-referential data hanging the worker until the host times out. I spent the next 24 minutes on a pass cap, a per-field length budget, and a test for each shape of the problem.
That was 07:24 on a Friday. We run a monolith and thirty-something microservices, review stopped being something we could keep up with by hand, and these agents are what I built to close the gap, so the interesting part is not that a bot found a bug, it is where it was allowed to look. It started with a commit rule and a script on my own laptop.
Why review got hard
A single change usually means touching several of those services at once, the volume going out every day has gone up a lot because generating a PR is cheap now, and reviewing all of it landed on the same people as before. Three costs, stacked.
The diffs got big. AI makes it easy to produce a large, coherent, fully tested change, so a PR now arrives as around a thousand lines across multiple files, and reading that carefully is genuinely hard work. You cannot skim it, and you cannot honestly approve it without holding most of it in your head at once.
Then the context switching. The PRs come from different people working on different things, a new feature here, a maintenance change there, a bug fix somewhere else, each in its own repository, so you load the mental model of that service, what it owns, what calls it, what its config looks like, read the whole diff carefully, and then the next PR is a different person in a different repository and you drop all of that and load a different one. My reviews got worse the more I switched, and I could feel it happening.
And the thing that actually breaks production is often not in the diff at all. A perfectly correct change in one repository can break three services that consume it, and nothing in the pull request shows you that. You only find it if you happen to remember which services call this thing, on a day when you happen to have the energy to go and look.
None of this is an AI problem. AI made the volume and the size worse, but a human-written PR breaks the consumer in exactly the same way.
The cheapest fix was not a reviewer at all
Before automating any reviewing, I changed how the code arrives.
We had an AI software-engineer agent by then, and the rule I gave it was simple: do not commit everything at once, commit in related chunks. So a change arrives as several logical commits, each one doing one thing, instead of one thousand-line blob with a single message on top. Whoever reads it can walk it commit by commit, and the change explains itself in the order it was built.
One line in an agent file, and every review after it got easier, before a single review agent existed. Nothing else in this post had that ratio.
A tool that only helps me does not move the bottleneck
The review side started as small as it gets. I wrote a code-reviewer agent and a security-reviewer agent, kept them on my own machine, and added a hook so that whenever somebody assigned me a PR I handed it to Claude Code and the agents ran locally against it. No CI, no rollout plan. The reviews were good enough that I stopped opening big diffs cold: findings first, then the code, and the second read was a different kind of reading, because I knew what I was looking for.
It was also miserable to run. A terminal per PR, opened by hand. It went through my own Claude account, so a big review would hit a context limit or start crawling against a rate limit. The CLI crashed partway through often enough that I learned to check. And more than once I came back to a review that had died because the laptop went to sleep. None of that is an argument about architecture. It is just what it costs to run infrastructure on a machine that is also a laptop.
Which fixed my afternoon and nothing else. The same cost was landing on everyone, PRs were sitting waiting for whoever had enough context to read them properly, and review was what set how fast any of us could ship, so a fix that ran on one laptop left that cap exactly where it was. So I moved the agents into a repo the whole team could pull from. They used them, and they liked them, and I want to be careful with that sentence: it is a feeling the people around me reported, not a number I can show you.
Sharing them is also what forced the quality bar up, because a reviewer that only annoys me is my problem, and a reviewer that annoys the whole team gets muted and then quietly wastes everyone's CI minutes forever. So the code reviewer got rebuilt around Rapid Software Testing principles, question the assumptions, look for what is missing, prioritize by risk, and every finding got a mandatory Failure Scenario field: concrete inputs, and the wrong outcome they produce. If the agent cannot say what actually goes wrong, the finding gets downgraded or dropped. That one rule killed almost all the noise, and it survives verbatim into what we run today.
Good reviewers, still blind to the blast radius
By then the agents had solved the first two costs, and solved them well. A thousand-line diff got read carefully every single time, at the same quality on the last PR of the day as on the first, and nobody had to hold an entire service in their head to say something useful about it.
The third cost they could not touch at all. They reviewed the change and nothing the change touches, so the question that actually takes production down, does this break the services that consume it, was still nobody's job. They also only ran when somebody remembered to run them. An agent sitting on a laptop had no idea the other repositories existed, and it could tell you the code was correct without telling you anything about what it would do to the rest of the system.
So I tried to give it the other repositories. I cloned every one I needed into a single parent directory and ran the agents from inside it, and it worked, the cross-checks were suddenly possible. Then the clones went stale. I would have needed a rule making the agent pull the default branch of every repo before every review just to trust what it was reading, and I still had to keep that directory honest by hand, and missing one new repository means the review is confidently wrong. A directory on my disk was pretending to be the organization.
The access was the whole feature
Moving the agents into GitHub Actions was supposed to solve the "when somebody remembers" half. It solved the stale-mirror half too, because in Actions the checkout is fresh by definition and the organization is the source of truth instead of my disk, so there is nothing to pull and no new repository to remember to clone. The cross-repository review I had wanted turned out not to be a feature I had to build, it was a side effect of putting the agent where the code already lives.
What I did have to build was the map. One central file in the shared repo lists which projects relate to which, as symmetric groups. The reviewer resolves the group for the repo under review and gets that list handed to it, read-only, and the repo under review cannot grant itself extra reach.
There is no index and no clever retrieval. The reviewer decides for itself what to go looking for, and it does it the way you would: it takes the thing it just saw change — an endpoint route, a property name, a DTO, a method — and runs gh search code for that term across the sibling repositories, then pulls the specific files the hits point at and reads them. That is the whole mechanism, and its weakness is exactly the one you would expect. It finds the consumer when the consumer names the thing recognisably. A caller that reaches the endpoint through a generated client, or a field that gets mapped through two layers of renaming on the way in, is invisible to a text search, and the reviewer will tell you the change looks safe because as far as it could see, it was.
Nothing is cloned and nothing is uploaded wholesale. The reviewer queries the API for exactly the files and lines it needs, when it needs them, so a sibling repository gets read a few hits at a time rather than shipped as a body of code. That is a smaller surface, not a different one — the code still goes to a model, and a CI job holding an org-scoped token reading other teams' repositories on every push is not the same thing as an engineer choosing to point an assistant at their own. It was an easier conversation here because everyone already works this way, but "easier" is not "asked."
So now: change something in the backend, and the reviewer goes and looks at every repository connected to it, the consuming services, the gateway layer, and tells you whether the change is backward compatible or whether it is about to break somebody. That is the whole point of the thing, and it runs today on the monolith and on the 30-plus microservices around it.
The obvious objection is that this is what contract tests are for, and it is. We have some. We add more when we touch something, and coverage grows slowly, and it is nowhere near all of a monolith that size — so on any given morning most of the surface between our services is asserted by nothing at all. A reviewer that reads the consumer's code does not fix that. It covers the part that is not covered yet, and it is worse than a contract test everywhere a contract test exists: slower, non-deterministic, and it can be argued with. The version of this I actually want is the one where the reviewer's findings turn into tests, so the same breakage gets caught next time by something cheaper and more certain than a model.
The number I do have is turnaround. Opening a PR and getting usable review feedback on it used to take 30 to 60 minutes depending on the size of the thing, sometimes a lot more when everybody was heads-down. Now it is under 5 minutes, of which about two are the reviewers themselves running in parallel and the rest is queueing and checkout. That is wall clock from push to feedback, not engineer hours saved.
The plumbing is deliberately boring: a 25-line workflow in each repo calls a reusable workflow in the shared repo, which loads the agent file, resolves the related repositories and runs the model. I have put a sanitized version of it up at github.com/MojtabaTajik/PR-Review-Agents, about 520 lines: the composite action, the reusable workflow, both agent files, the relationship-map schema and an example caller workflow. The plumbing is not the part worth having. The agent files and the rules inside them are.
- Single source of truth. No repo carries a copy of an agent. Changing a reviewer across the whole fleet is one PR to one file, and adding the next repository costs a 25-line workflow file.
- The two reviewers run in parallel. They are separate jobs on the same push, so the wait is about two minutes for both of them, not two minutes each. That number is the difference between a tool and a tax, and if it ever doubles people will start merging before the review lands.
- It is not a required status check. Nothing in the pipeline physically stops a merge. The rule is a team agreement instead: critical and high findings block the merge, everything below that is the author's judgment. Enforced socially, not mechanically, which works right up until the day it doesn't.
- One review per push, with the reviewed commit linked in the title. Push again and you get another review, pinned to the head commit it actually saw, so on a PR you push to five or six times you can tell which state of the branch each verdict applies to, and watch findings escalate and clear instead of arguing about a moving target.
- Terse when clean. A clean verdict is two lines: "Looks good — nothing to flag." No "here is what I checked" essays. Silence about non-issues is the signal.
- Everything the reviewer reads is untrusted input. It reads diffs and other people's repositories, and both can talk back. A repo can append its own context file, but that file is handed to the agent as untrusted data that can add emphasis and cannot relax the severity rules. The cross-repo token is fine-grained, contents-read only, and the repo under review cannot widen its own reach. I built a security reviewer, so it gets treated like one.
- The reviewer must name the repositories it consulted. Every cross-repo claim ends with that list. Otherwise you cannot tell an informed finding from a confident guess.
The proof: one feature, three repos, nineteen reviews
Back to that Friday. One ticket, one behavioral change, three coordinated PRs: the engine itself, a gateway policy, and the MCP tools that document the new authoring rules.
After I pushed the DoS fix, the code reviewer read the new commit and came back with the finding I could never have gotten locally: the live consumer app calls this endpoint without setting the mock-data flag, so my new strict default would break document creation in production. Footer of the comment, the consumer repo, named. It had read the consumer's actual code instead of guessing.
I went to production telemetry to settle it, 90 days, around 61k invocations, zero mock-fallback traces. Then I stopped trusting my own zero, because our ingestion filters out Information-level traces entirely, and that is exactly the level those fallbacks were logged at. The evidence that would have answered the question was below the ingestion floor. So the PR now logs fallbacks at Warning, and the rollout became measurable instead of hopeful.
An hour later the code reviewer, cross-checking three sibling repos, found the sharpest thing of the day: the admin app's document preview flow proxies its body straight through to the same endpoint, and preview is exactly the case where real data does not exist yet, so strict validation would have broken every preview. That finding created the gateway PR, injecting the flag at the gateway for the two preview operations only, instead of touching the admin app at all.
Then the best moment of the exchange. The reviewers reviewed the gateway PR in its own repo, verified the policy, confirmed by cross-repo read that the current engine tolerates the extra property, and then flagged the one thing they could not verify from that side: if the engine reads that flag anywhere other than a top-level body field with exactly this name, the injection silently does nothing and previews start returning 400 with no error visible at the gateway. I verified the contract by hand, then added two tests to the engine PR that deserialize the exact JSON the gateway policy produces. A future rename now fails a test instead of quietly turning every preview into a 400.
The third PR was documentation only, and all four of its reviews were the one-liner. A bot that always finds something teaches you to ignore it, a bot that can say nothing to flag makes its warnings worth reading.
At 09:35 both reviewers cleared the merge commit, and after all of that the entire verdict was: "Safe · Ready for QA — Looks good — nothing to flag."
Final count for the three PRs: 19 automated review rounds, 7 substantive replies from me, 9 of the 11 commits driven by review findings, 13 findings I acted on or pushed back on, three of which were wrong, one MEDIUM security finding, tests 199 to 208 green, about two and a quarter hours end to end. One feature, one morning: this is an anecdote, not a benchmark.
No other engineer looked at any of it before it reached our QA team, and that part deserves saying out loud rather than counting as a win: the only human at the merge button was the person who had written both the code and the reviewers. For a behavior-neutral change sitting behind a setting that is a call I am comfortable making. As a standing policy it would not be.
The round I won
At 09:02 the code reviewer flagged that my new length cap would fire even on the supposedly behavior-neutral deploy, and suggested gating it to log-only. The finding was right. The fix was wrong, and I rejected it in the thread: log-and-continue returns half-expanded content, and a silently corrupt document is worse than a clear 400, and removing the bound reopens exactly the DoS the security reviewer flagged at 07:24. Instead I raised the default budget 10x, to 1M characters, far beyond any legitimate value while still being a bound.
What it still gets wrong
- There are no evals. A one-line edit to an agent file ships to every repository at once, and the only symptom of a bad edit is fewer findings, which looks exactly like good news. I tested the product code carefully and did not test the thing making the judgments, which is a bit embarrassing to write down. I am building them now, with other models judging the reviewers' output.
- Three of the thirteen findings were wrong, and the loop made them itself. The agent flags something, I fix it and leave a comment in the code explaining the fix, the next round of fixes makes that comment stale, and a later review flags the stale comment. Each of those findings was correct about what it saw. The work was manufactured by the review loop, and the more rounds a PR takes the more of it there is. No fix yet, and I do not think more rules is the answer.
- I do not know what it misses. Everything in this post is a finding the reviewers caught. I cannot show you the ones they cleared that broke later, because the engine has not been running long enough for me to have that list, and a review system with no known false negatives is not the same thing as one with none. Same reason I would not trust that zero in the telemetry: nobody is measuring at the level where a miss would show up.
- The map is maintained by hand. That central relationship file is a dependency graph a human writes. A new service that nobody adds to it is invisible to every reviewer in the fleet, and the review that misses it still reads exactly as confident as the ones that got it right. Which is the same failure I escaped when I stopped cloning repositories onto my laptop, moved up one layer. I do not have a fix for it.
- It costs real money, and I have not added it up. Around €1 to €2 per PR on average, depending on how much cross-repo checking it does and how big the PR is. I have not totalled that across the fleet, so I cannot tell you the monthly bill. Guards exist, max turns, concurrency cancel, docs and bot PRs skipped, per-repo model override, and Sonnet as the fleet-wide lever if spend gets out of hand.
- Cross-repo reads are stale. One round cross-checked the gateway repo, hit the default branch instead of the sibling PR I had opened minutes earlier, and re-raised a finding I had already addressed. Open PRs in other repositories are invisible to it. I am working on that now: permission to list open pull requests in the related repositories, and rules telling the code reviewer to check them, so it surfaces the sibling PR that already fixes the breakage instead of reporting it to me twice.
- The reviewers review code, not intent. There is a gate that blocks PRs without a linked requirement, but a change that faithfully implements the wrong requirement passes every check we have.
- It repeats itself. The security reviewer re-affirmed the same hardening in several clean verdicts. Harmless, slightly annoying.
The division of labor is the part I would keep, and the 09:02 round is what it looks like. The agents set the bar. They do not own the merge button.
The reviewer itself was never the goal. Review was the thing capping how fast we could ship, so review is what I went after, and when the constraint moves the fix will look like something else entirely.
None of which is the thing I actually remember from that Friday. A cross-repo contract, pinned by a test, because a bot asked a question it could not answer itself.