47 ADRs on one plugin in six weeks, what amend-in-place bought us

I built a Claude Code plugin at KnowBe4 that reviews e-learning course packages for quality issues before a human specialist signs off on them. Between the first tagged version and the one shipping six weeks later, the project accumulated 47 architecture decision records. That's more than one decision written down for every day of the six weeks it took to build. The number itself isn't the interesting part. What's interesting is the convention that made 46 of them worth having: when a later decision proved an earlier one wrong, the original record got a dated amendment appended to it, in place, with the original claim left standing right above the correction. Nothing got deleted. Nothing got quietly rewritten to look like the team had known better all along.

The founding rule

The very first record in the log, ADR 0000, states the convention before it states anything else: keep decisions in one file each, amend in place with dated amendments, never rewrite. The rationale given is blunt: a delivery arc that runs across many sessions produces decisions that "look wrong without their reasoning attached." A team member reading a record six weeks later needs to see not just what was decided, but what was known at the time and how that changed. Two alternatives were considered and rejected outright: a ticket tracker, because acceptance criteria aren't reasoning, and a single running decisions document, because a shared file produces merge conflicts and gives you no way to mark one entry as superseded by another without losing the thread between them.

A reversal in the record, not a rewrite of it

The clearest example of the convention doing real work is the record covering which model tier runs the plugin's visual quality check. The original decision, mandated by the project's acceptance criteria and costed against published list pricing, was Haiku 4.5 for a first pass over every frame, escalating to Sonnet 5 only on a finding or low confidence. The reasoning was sound for the evidence available at the time: most frames are clean, and triage with the cheaper model captures most of the value at a fraction of the cost.

Thirty-eight days later, a second measurement replaced the first. The shared rubric fed into every triage call ran 1,495 tokens, but Haiku 4.5's minimum cacheable prefix is 4,096 tokens, so the rubric could never actually cache on that model. Two thirds of the input on every single frame was being rebilled at full price, forever, with no way to fix it short of changing tiers. Measured on the same package, switching triage to Sonnet 5 with adaptive thinking cut total input tokens by 25 percent and surfaced four visual findings where Haiku had surfaced one, for roughly 2.2 times the cost per run. The amendment states its own conclusion plainly: "Both tiers are now the same model, so the two-tier COST model in this ADR no longer describes the pipeline." The original decision is still there, readable, with the numbers that justified it at the time. The correction sits right below it, dated, with the numbers that overturned it.

A second record shows the same discipline at a smaller scale. An ADR proposing to source spellcheck text from a module's i18n JSON files was written, built, and measured inside a single merge request: a 27-minute spellcheck run that got killed at the 150-second timeout with no output, six of nine findings on a clean module coming from a developer's data file rather than course content, and a false positive from a sibling English file. The decision was withdrawn in the same record it was proposed in. The stated lesson: "A thin, honest check surface beats a wide one that floods."

Why the pace made this necessary

Six weeks at more than one decision a day is not a pace at which anyone remembers the reasoning behind decision 12 by the time they're writing decision 40. A separate record in the log, covering the move from running the pipeline locally to running it as a hosted service, makes the stakes of that forgetting explicit. The move looks, from a distance, like reversing an earlier decision to prove the pipeline out locally first. The record calls that reading out and rejects it: the amendment states plainly that the move is "not a reversal of the evidence," because the local proof-of-concept answered a correctness question (can this pipeline detect real defects) that the hosting decision never touched. Without the dated history sitting in the same file, a reader six weeks removed has no way to tell a genuine reversal from a decision that was never in tension with the one before it.

What this bought

None of the reversed decisions in this log read as errors, because none of them were treated as errors. Each one is a claim that the evidence available at decision time supported, later replaced by a claim that newer evidence supported better, with both claims and the date between them on the record. That is the actual product of the amend-in-place convention: not a plugin that got every call right the first time, but a 47-entry paper trail where every claim about how the system behaves carries the measurement it stands on and the correction it survived, if any. A log that only ever recorded the final answer would look tidier. It would also tell you nothing about which of tomorrow's confident claims is due for the same treatment.