The Judgment Gap why the reporting improves while it happens › Unexamined work never announces itself 12 steps, no labs
04 — part 2, why the reporting improves while it happensreasoning

Unexamined work never announces itself

Work nobody looked at does not go anywhere or complain. It merges into what everything after it is built on.

The reason this can run for a year without anybody raising it is that unexamined work is not a pile somewhere. It went out. It is in production, or in a document somebody is relying on, or in a decision that has already been made on top of it.

work produced work examined the remainder it does not wait in a pile to be found and becomes what the next round is built on so the absence of a complaint is not evidence that anything was looked at.
It does not wait in a pile to be found. It becomes what the next round is built on, which is why the problem compounds and why there is no moment somebody could have caught it.
What moves: the reservoir has a dashed return path running back to the intake of the produced pipe, so its contents become the input to the next round.

So there is nothing to find. Work that nobody examined and that happened to be fine is indistinguishable, from the outside, from work that was examined. Work that nobody examined and that was not fine looks like the first kind until something built on top of it fails.

The absence of a complaint is not evidence

what people take as reassurancewhat it actually establishes
nothing has gone wrongthat nothing has gone wrong yet, at this level of the system
customers have not noticedthat the failures so far have been of a kind customers do not see
the tests passthat what somebody thought to test still behaves
the reviewers have not flagged anythingthat the share of work reaching a reviewer has not produced flags. Not what happened to the rest

Every row in that right-hand column is a true and much smaller statement than the left. None of them is about the remainder, and the remainder is the quantity that grew.

And there is no instrument you can buy for it. You cannot detect unexamined work by looking at outputs, because unexamined work that happens to be correct looks exactly like examined work. The only way to know the share is to count what was examined, which is the arithmetic in the previous step and the reason it comes before the evidence.

Every term this page uses

produced-to-examined
How much work your team produced, divided by how much of it somebody actually looked at.
examined
Looked at by somebody with enough context to have caught a problem. Not the same as having passed through a review step.
remainder
Work that was produced and not examined. It does not go anywhere — it merges into what everything after it is built on.
judgment
Deciding whether a piece of work is good, which needs knowing the domain well enough to recognize a wrong answer.
inspection
Checking finished work at the end, rather than changing how the work is made.
checkable
Able to be judged without understanding the whole of it. The property that decides how much needs judgment at all.
constraint
The thing that limits output. Adding more of anything else does not help until it moves.
rework
Work that had to be revisited after somebody called it done.
vigilance
Staying alert for a rare problem across a long stretch of time.

Taken as already known, and so not defined here: agent, model, review, throughput, headcount. That list is a claim about who is reading, and it is printed so it can be argued with.