Token Cost Engineering Concepts › What you are billed for 12 of 12 steps written
01 — part 1, conceptsconcept

What you are billed for

Token cost engineering is the practice of reducing what a workload costs to run without losing the answers you needed from it.

The bill has exactly two inputs

Every charge is a token count multiplied by a price. That is the whole of it, and the split between the two halves is the entire reason this subject has a method:

you controlyou do not control
how many tokens go in, on each requestwhat a token costs
how many come backhow prices change next quarter
how many requests are made at allwhich tier a vendor retires
A token is the unit a model is billed in — roughly a short word or a piece of one. This tutorial counts tokens with a crude rule that matches no vendor's, because every figure here is a ratio and the rule cancels.

Why this tutorial reasons in counts and not in dollars

Prices age. Token counts do not. Re-price everything in this tutorial and not one token count moves — only the dollar figures do, and they move together, so every comparison survives.

That is not a stylistic choice. It is the reason the method transfers to a reader whose prices differ from these, and it is checkable: the labs publish the conversion as a step you redo rather than as a result you inherit.

a request one call in tokens you send out tokens that come back x a price this quarter next quarter x a price this quarter next quarter one dollar total the only figure that moved the counters do not move when the prices do which is why every figure in this volume is a token count or a ratio
The counters climb, meet their prices, and resolve to one total. Then the prices change — and only the total moves. Everything this volume measures lives on the left of that multiplication, which is why the figures survive a price list you have not seen yet.
Nothing in this scene is measured. It is the shape of the bill; step 4 fills it in.

The two ways a cost reduction can fail

#how it failsworst casehow you would find out
1it saves nothing. The move was real, your traffic had nothing for it to act onthe engineering time is gonethe bill does not move
2it costs an answer. The bill fell and something you needed stopped workingyou ship a wrong answeryou might not

Most advice about this subject addresses the first and is silent about the second. The whole of Part 2 is an ordering that keeps the two apart.

The second failure does not appear on the invoice. The bill went down, which is what you were measuring, and something you needed stopped working, which you were not.