Provisioned Throughput

Provisioned throughput is a purchasing model for hosted inference in which you reserve a fixed amount of model capacity for a period of time and pay for that reservation, rather than paying per token on shared on-demand capacity. Providers sell it under names like provisioned throughput units, reserved capacity, or dedicated capacity, and the unit is some measure of tokens per second guaranteed to you whether or not you use it. Two things are being bought, and they are worth separating. The first is predictable latency and freedom from the shared-pool rate limits that make on-demand traffic bursty under load — capacity nobody else can take. The second is price certainty, and sometimes a lower effective rate per token, but only above a break-even volume. Below that volume, reserved capacity is strictly more expensive than on-demand, because you pay for the idle hours too. The break-even arithmetic is straightforward and worth doing before a sales call rather than during one: divide the committed cost for the period by your realistic token volume for that same period and compare the result against the published on-demand rate. Teams get this wrong by planning against peak throughput when their traffic is spiky, which is exactly the profile that suits on-demand better. The commitment terms are where the risk sits. Reservations are typically monthly or annual, frequently non-cancellable, usually tied to a specific model version — which collides directly with model deprecation, so ask what happens to your commitment when that version retires — and often region-locked. A hybrid is common in production: reserved capacity sized to the steady baseline, with overflow routed to on-demand, which caps the idle waste while keeping headroom for spikes.