How to Calculate AI API Cost by Tokens

One request: input tokens × input rate plus output tokens × output rate, divided by one million. Output rates are usually multiples of input, so a chatty bot with short questions and long answers costs notably more than a reserved one at the same dialog count.

History is the hidden multiplier: every request carries the conversation tail, so the tenth message costs multiples of the first. Measure median and upper percentile of full input on real dialogs, not one test question.

Caching cuts repeated input cost but only on confirmed cached usage with a stable prefix. Image models bill per generation, video per second — keep them as separate budget lines, never mixed into token math.

Retries after errors are full new requests: retry transient errors with backoff, never 401/402/403. Unbounded max_tokens plus a loop equals an open bill — cap every call.

Take three load scenarios — current, 3× and 10×. Multiply each by your median input/output from history and the catalog rates. If even 10× fits the budget with margin, the architecture can stay.

Reconcile monthly totals against request history only: decompose the most expensive calls into input, output and cache. An average-check estimate without context and retries always undershoots.