One number, three reasons, four moves
For the meeting where somebody proposes adding reviewers.
Producing a candidate answer got cheap. Judging whether it is any good did not, because judging needs knowing the domain well enough to recognize a wrong answer — and that was always the expensive part. So the amount produced grows away from the amount anybody looked at, and the remainder merges into what everything after it is built on.
Computing the ratio
step 03 — The ratio that tells you how wide your gap is
- Count what your team produced in a period. Whatever your unit is, applied consistently.
- Count how much of that somebody actually looked at — somebody with enough context to have caught a problem. Not review steps traversed, not checks that ran, not the absence of complaints.
- Divide. Write it down with the date.
- Take the strict reading of examined. An uncomfortable number you can act on beats a comfortable one that is counting review steps.
What counts as examined
| does it count? | why |
|---|
| somebody read it who could have caught a problem | this is the definition. Note the could have — not did |
| it passed through a review step | a step is a place, not an event |
| an automated check ran on it | useful, and not judgment. A check tests what somebody thought of in advance |
| nobody complained | unexamined work that happens to be fine looks identical to examined work |
The comfortable reading of examined gives you a number that means nothing. It is the only difficult part of the calculation and the whole value is in getting it right.
Three reasons more reviewers will not close it
- the headcount would have to grow with production. the capacity you need is your current capacity times your ratio, and it keeps pace with output
- the checkers have to know the domain. which is the scarcest thing you have, and they are currently doing the work you would be protecting
- nothing would tell you whether it worked. every number on your panel would get worse; what improved is the quantity with no gauge on it
The third survives any disagreement about the other two. Add checkers and every figure on your panel gets worse while the thing that improved is unmeasured — a visible cost, an invisible benefit, and no way to tell whether you bought anything. Which is why this remedy gets tried, abandoned after two quarters, and remembered as not having worked.
And watch for the fourth option. Faced with the cost of real checkers, organizations quietly assign people who do not know the domain. The examined count rises and the amount genuinely examined does not. The ratio improves on paper.
The four moves that do close it
| the move | what it trades away | where it works best |
|---|
| narrower tasks — ask for less at once | some capability, and some convenience | anywhere judging the output currently means reading all of it |
| outputs checkable without understanding them | expressiveness — a structured answer says less | anything where the result can be tested rather than read. Usually the largest gain available |
| failures that surface themselves | engineering effort, up front | anywhere a wrong answer currently looks like a right one |
| producing less on purpose | throughput, visibly. The least popular row here | where the marginal work was not worth judging, and so is not worth having |
What makes these different is not that they are cheap. It is that they scale: each reduces the judgment needed per unit of work, so the benefit grows with production instead of having to keep pace with it.
What only a director can do
- Name judgment in somebody's objectives, with hours — not as a quality initiative, as a share of one or two people's time, defended when it costs output.
- Promote somebody for it once, visibly. One promotion where the stated reason is judgment establishes more than any amount of saying it matters.
- Ask for the ratio in review more than once. A number nobody asks about twice is an event, not a measurement.
The honest alternative is also available. Decline all of it on the grounds that the failures are survivable and the money is better spent. That is a real position and much better than an aspiration nobody resources — and the ratio is what lets you revisit it deliberately rather than by surprise.
Every term the volume uses
- produced-to-examined
- How much work your team produced, divided by how much of it somebody actually looked at.
- examined
- Looked at by somebody with enough context to have caught a problem. Not the same as having passed through a review step.
- remainder
- Work that was produced and not examined. It does not go anywhere — it merges into what everything after it is built on.
- judgment
- Deciding whether a piece of work is good, which needs knowing the domain well enough to recognize a wrong answer.
- inspection
- Checking finished work at the end, rather than changing how the work is made.
- checkable
- Able to be judged without understanding the whole of it. The property that decides how much needs judgment at all.
- constraint
- The thing that limits output. Adding more of anything else does not help until it moves.
- rework
- Work that had to be revisited after somebody called it done.
- vigilance
- Staying alert for a rare problem across a long stretch of time.
Taken as already known, and so not defined: agent, model, review, throughput, headcount. That list is a claim about who is reading, and it is printed so it can be argued with.
Sources, and their state
None of these has been opened, and none of them is load-bearing. This volume's specification carried a kill condition: its part 3 was originally a claim about human attention that cannot be verified from here, so part 3 was rebuilt on arithmetic instead. The three reasons above rest on no source. Do not repeat a citation from this page until somebody has checked it.
Deming, 1982 unverified
Out of the Crisis, and the point about ceasing dependence on inspection
used here for: inspection at the end of a line sorts what arrives; it does not change what arrives
code review practice studies unverified
measurements of how review actually works at scale
used here for: review capacity is roughly linear in reviewers, and reviewers are not interchangeable
vigilance and automation-complacency research unverified
decades of work on people asked to supervise a usually-correct machine
used here for: the checkers themselves may degrade, which would make this worse