What a bot vote is allowed to decide

At KnowBe4 I worked on a CI pipeline where an AI code-review job doesn't just leave comments, it can cast an actual approval vote on a merge request through the platform's approvals API. Before that shipped, the question we had to answer in writing was narrower than "should an AI review code." It was: what is this bot specifically allowed to decide on its own, and what must always stay outside its reach. Two architecture decision records carry that reasoning, and the second one only makes sense once you understand what the first one refused to claim.

The bot's vote is not a lock

The first record starts from an uncomfortable fact: a merge request pipeline runs the source branch's own CI configuration. That means the branch under review can, in principle, rewrite the rules that review it. We documented three concrete ways this could go wrong. A branch could repoint the target-branch variable at itself, so the review loads a policy the author wrote and stamps it with a real, correctly-computed hash. A branch could manipulate the diff base so the pipeline sees zero changed files, tripping the shortcut that auto-approves anything with no visible diff. Or a branch could simply redefine or delete the review job outright, since a consumer repo's own pipeline configuration always wins over a shared, centrally maintained one.

The detail that mattered most in that record is about the hash we stamp onto every verdict. That hash proves the verdict is tied to a specific commit. It does not prove that a trusted, unmodified process is what produced the verdict, because anyone who can edit the pipeline that generates the hash can also edit the pipeline that checks it. A SHA embedded in an artifact you control the generation of is provenance data, not authentication. Treating it as proof of anything stronger than "this is the commit we're talking about" was the mistake we explicitly decided not to make.

So the first decision was to ship the bot's approval as a documented convenience, never described as tamper-resistant, with conservative defaults: it only fires on a clean pass with no shortcuts taken, it requires a human to click the button rather than running automatically, and if its eligibility to vote disappears after it already voted, there's no mechanism to withdraw that vote. We also named the actual fix for full tamper-resistance: a pipeline execution policy that runs the review itself inside a protected, non-editable configuration, so the process producing the verdict can't rewrite its own rules. That's a different tier of platform capability, scoped out on purpose rather than pretended into the convenience version.

What problem the bot vote is actually solving

That caution didn't make the bot's vote pointless. It made us precise about what the vote was actually for. The job that generates the verdict runs a real, bounded AI review against the diff: a deterministic scan first, at zero model cost, then a judgment pass and an independent validation pass that recomputes the merge-blocking decision from the underlying findings rather than trusting whatever the model claims about itself. Only a clean, unambiguous pass produces an approval-eligible verdict. Anything short of that, including a review that got fast-tracked through a shortcut path, is explicitly excluded from voting eligibility.

That verdict is what authorizes the vote, full stop. The bot never approves unconditionally, and never approves based on anything other than that recomputed result. What it buys is consistency and speed on the parts of review that don't need judgment: the standard checks a reviewer runs every time regardless of who wrote the code. Freeing a human from re-verifying those on every merge request means they spend attention on the calls that actually require it, architectural tradeoffs and business logic correctness, the kind of thing no verdict schema can capture.

Removing the human-first requirement, deliberately

The second record revisits something the first one got conservative about. In the initial design, the bot's vote could only ever supplement a human approval that had already landed, never stand in for it. In practice, that meant a human always had to approve first, and the bot's vote added nothing that wasn't already there. We changed that: a clean verdict now lets the bot's vote count on its own, including satisfying a code-ownership approval requirement, with no prior human approval as a precondition.

That change came with a genuine correctness fix. The eligibility check has to read a field describing which approvers count toward a given rule, and that field can show up four different ways: missing, malformed, empty, or populated with real entries. Under the old design, an ambiguous read defaulted to "require a human too," which was safe by construction. Once the same field decides whether to grant an approval outright, that default has to flip: any ambiguous read has to refuse to vote. Getting that polarity backwards would have meant the bot approving merge requests in exactly the cases where nobody could say for certain it was eligible to. We also caught a related bug in how a "hidden approval groups" flag was checked: treating anything other than the literal value true as false was quietly approving several cases that should have blocked.

We could make this a breaking behavioral change without a migration plan because nothing was relying on the old behavior yet. Nobody had this component turned on in a real pipeline at the time we made the change, so there was no installed base to break.

The freedom that won't be there next time

Nobody had this component turned on in a real pipeline when the eligibility default flipped, so there was no installed base to break and no migration plan to write. The next time this boundary needs to move, on this pipeline or the next one built the same way, that won't be true anymore. Whoever writes that record will be arguing not just for the new rule, but for changing behavior underneath approvals that already depend on the old one.