Measuring a Team That Ships With Agents Introduction 12 steps, no labs

Which of your numbers stopped meaning what they meant

Four metrics examined, four different ways they broke, what replaces each one — and the one judgment no number left can support.

Your metrics did not get worse. They started measuring something else.

Every number you report went up. None of them is computed wrongly. And most of them were never measurements of anything you cared about directly — they were things that were easy to count and that used to move when the thing you cared about moved.

That connection is what broke, and a broken connection does not make any gauge read zero. This volume names which of your numbers are affected, in which of four ways, and what to put in their place.

two guesses in a row, and the middle one came apart effort what it cost you the thing you count what you report value what you wanted snapped before all three move together, and the middle one is a fair stand-in after the outer two still move. They are simply no longer connected. nothing on any face reads zero. Nothing flashes. Both ends keep climbing at their own rates, and the chain in the middle is gone.
The spine of the volume: counting something easy in place of something you care about is a chain of two guesses, and the middle of it came apart.
What moves: three gauges rise together, then the link between the second and third becomes a broken dashed line while both outer gauges carry on rising.

Four metrics, four different failures

the metrichow it broke
counting outputthe stand-in stopped standing in for effort or for value
cycle timethe stretch being timed moved, and the queue moved outside it
individual recordswho did the work stopped being a decidable question
coverage and its relativesit was only ever meaningful because it was expensive

The four were chosen because each breaks differently, so the set generalizes. By the third step you should be able to apply them to a number this volume never mentions.

Every condemned number here comes with a replacement, or an explicit admission that there is not one. There is exactly one of the latter. It is about judging individual people, and it is the most important page in the volume.

What you can do afterward

Mark every line on your own dashboard as readable, readable-with-a-caveat, or retired. Compute the one number most teams do not, from records you already keep. Walk into a calibration meeting knowing which number cannot help you. And retire a metric upward without it sounding like an excuse.

The four failures, the replacements and the retirement script are on one page you can print.

Every term this page uses

proxy
Something easy to count that stands in for something you actually care about and cannot count directly.
denominator
The count a rate was worked out over. How many, not what share.
attribution
Deciding which person or which change produced a result.
cycle time
How long one piece of work takes from being started to being shipped.
rework
Work that had to be revisited after somebody called it done.
outcome measure
Something measured about the result rather than about the effort spent.
gameable
Cheap to satisfy without doing the thing the measure was meant to reward.
lag
The delay between something happening and the number moving in response.
target
A measure somebody is expected to move, rather than one you only watch.

Taken as already known, and so not defined here: agent, metric, dashboard, pull request, coverage. That list is a claim about who is reading, and it is printed so it can be argued with.