Test-Time Compute

Test-time compute (also called inference-time compute) is the idea that a model can produce better answers by spending more computation at the moment you ask a question, rather than only by being bigger or trained longer. Instead of writing the first answer that comes to mind, the model generates a longer chain of thought, samples several candidate answers and votes among them, or runs a search over possible solutions. This is the engine behind reasoning models. The striking finding is that a smaller model given a generous test-time-compute budget can match a much larger one on hard tasks — you buy capability with inference cost instead of model size. For SaaS builders, it is a lever: on a hard request you can raise the thinking budget for better quality, and on easy requests keep it minimal to save money and latency. The tradeoff is direct — more test-time compute means more tokens billed and slower responses — so tie it to task difficulty rather than applying it everywhere.