How to Forecast What an AI Feature Will Cost to Run

Why the published price is the wrong starting point

Model pricing is quoted per million tokens, which is a unit designed to sound small and to be hard to translate into a monthly bill. The number that matters is cost per unit of work your product does — per support ticket handled, per document summarised, per user per month — and getting from one to the other is where forecasts go wrong. Teams that skip this step usually discover the real figure in the second month, after launch, when usage has stopped being a demo and the invoice reflects it. The work of forecasting is not arithmetic on the price list; it is being honest about how much text your feature actually moves.

Find the unit that drives your bill

Start by naming the billable event in your own product's terms and work outward from there. For a support assistant it is one conversation. For a document tool it is one file processed. For a coding feature it might be one completion request. Then measure, for a representative sample of real cases, how many model calls that single event triggers. This is the number people guess wrong most often: a feature that looks like one call is frequently three or four, because there is a classification step, a retrieval step, the main generation, and sometimes a second pass to check or reformat the output. A forecast built on one call per event will be wrong by a multiple, not by a percentage.

Size the tokens from real traffic

Once you know the calls per event, get the token counts for each of them from actual data rather than from the prompt you tested with. Input and output are usually priced differently, often by a factor of several, so count them separately. Take a sample of genuine production or beta traffic and look at the distribution rather than the average, because AI workloads have long tails and the expensive cases are rarely the typical ones. A median conversation and a ninety-fifth-percentile conversation can differ by an order of magnitude, and if a meaningful share of your traffic sits in that tail, the mean is the only number that predicts the bill.

Count the multipliers everyone forgets

Several things inflate the real total above the clean estimate. Retries: failed or malformed responses get retried, and every retry is billed. Conversation history: a chat feature resends the accumulated context with each turn, so a ten-turn conversation costs far more than ten times a single turn. System prompts and few-shot examples ride along with every single call, which makes a long system prompt a fixed tax on your entire volume. Retrieval-augmented features attach retrieved documents to the input, often dwarfing the user's own text. And evaluation, testing, and internal use all generate billable traffic that never appears in a customer-facing usage estimate.

Know which levers actually move the number

When a forecast comes back too high, the levers differ enormously in effort and in what they cost you elsewhere. Routing simple cases to a smaller, cheaper model is usually the largest win available and the one that most often survives evaluation, since much of the volume in a typical feature is easy work sent to a frontier model out of convenience. Prompt caching helps most where a long, stable prefix repeats across calls, which is exactly the system-prompt-plus-retrieval shape. Batch processing trades latency for a lower rate on work that is not interactive. Trimming the system prompt and capping conversation history are unglamorous and often reduce the bill more than anything clever. Test each against quality rather than assuming it is free.

Decide whether committed capacity fits yet

Providers sell reserved or provisioned capacity at a lower effective rate in exchange for paying whether you use it or not. The arithmetic is simple enough to do before a sales call: divide the committed cost for the period by your realistic volume for that period and compare it against the on-demand rate. It wins above a break-even volume and loses below it, and spiky traffic is exactly the profile that suits on-demand better, because you pay for the idle hours too. Check the commitment terms as carefully as the rate — whether it is cancellable, whether it is tied to a model version that may be deprecated during the term, and whether it is locked to one region.

Build something you can re-run every month

The output of this exercise should be a small model with your assumptions visible — events per month, calls per event, input and output tokens per call, retry rate, price per million — not a single number in a slide. Assumptions age fast: prices fall, your prompts get longer, usage patterns shift as the feature finds its audience. Re-run it monthly against actual invoices and note where reality diverged, because the gap teaches you more about your own system than the forecast did. Instrument the feature to record tokens per event from the start, so that after launch you are reading a measurement rather than defending an estimate.