Canary & Shadow Deployment

Find out on 5%, not on everyone

A canary sends a small share of real traffic to the new version and compares it against the old on live metrics: errors, latency, eval score, cost. If it looks worse, you have learned that from five percent of users rather than all of them.

Shadow deployment goes further in one direction and less far in another: the new version gets the same traffic but its answers are discarded. Zero user risk, and no signal about what users would have done with them.

Both matter more for models than for ordinary code, because a model change has no diff you can read. A new version, a new prompt or a new retrieval setting can be better on your eval set and worse in production, and this is how you find out cheaply.