Reading an Eval Report whether it supports the decision attached to it › Telling a real test from a case already made 12 steps, no labs
11 — part 4, whether it supports the decision attached to itsourced

Telling a real test from a case already made

If no outcome of the measurement would have led to a different ask, you were shown a case that was already made.

The fourth question, and the one that did not make the three despite costing nothing to ask. It is here rather than earlier because it is confrontational, and a question you will not actually ask is worse than useless — it displaces one you would.

what result would have changed the recommendation? a good result arrives, as expected a far worse result arrives instead the gate it opens both times the recommendation unchanged, both times if nothing that could have come out of the measurement would have changed the ask, the measurement was not what decided it.
A gate that opens is unremarkable. A gate that opens twice, on opposite inputs, is a decision that was never conditional on anything.
What moves: a good result travels to a gate, which opens, reaching an unchanged recommendation. The cycle repeats with a far worse result and the gate opens in exactly the same way.

The question has an exact form, and the exactness is what makes it answerable: what result would have led you to recommend against this? Not whether they are confident. What outcome, specifically, would have produced a different sentence at the bottom of the page.

Three things can come back. A number, which is the good answer and ends the conversation. Silence, which is its own answer. Or a reframing — that the project was always going ahead and the measurement was to guide the rollout — which is sometimes entirely legitimate and is a different document than the one you were handed.

Why this is worth the friction

Because a measurement built to support a decision already made will be, at some point, a measurement that somebody is rated on. A measure that becomes a target ceases to measure what it measuredGoodhart, 1975unverified, and once that happens the number stops being an instrument and becomes a target. The more an indicator is used to decide, the more it is worth gamingCampbell, 1979unverified.

How to ask it without an accusation. Put it in the future and in the plural: what would we want to see before we stopped this? Same question, no implication that anybody has done anything wrong, and it produces the threshold you actually wanted. Asking it before the measurement rather than after is better still, and is entirely within your gift.

Every term this page uses

denominator
The count a rate was worked out over. Ninety-four percent of what, and of how many.
baseline
What the same measurement says about doing it the old way, or the trivial way, or not at all.
held-out
Cases kept away from whoever built the thing, so passing them means something.
contamination
The thing being tested has already seen the test cases, so the score measures memory rather than ability.
grader
Whoever or whatever looked at each answer and decided it was right or wrong.
agreement
How often two graders looking at the same answers said the same thing.
abstention
A grader allowed to return a third verdict: I cannot tell.
stale
Measured on a version that no longer exists.
exhibit
A report made up for this volume, for you to judge. Its numbers are not claims about the world.

Taken as already known, and so not defined here: eval, test, model, agent, metric. That list is a claim about who is reading, and it is printed so it can be argued with.