[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"glossary-time-to-first-token::en":3,"gloss-cluster-time-to-first-token::en":23,"gloss-next-time-to-first-token::en":9},{"slug":4,"category":5,"name":6,"definition":7,"meta_desc":8,"faq":9,"schema_markup":9,"related":10},"time-to-first-token","mlops","Time to First Token (TTFT)","Time to first token is the delay between sending a request to an LLM and receiving the first token of its response. It is dominated by the prefill phase — the model must process the entire input prompt before it can generate anything — so TTFT grows with prompt length: a 50-token question answers near-instantly while a 100K-token context can take many seconds before the first word appears. It is measured separately from inter-token latency (the pace of tokens after the first), and the two shape user experience differently: TTFT determines how long the interface feels frozen, while inter-token speed determines how fluid streaming feels. Practical levers for reducing TTFT include prompt caching (skip re-processing a shared prefix), trimming retrieved context, routing short queries to smaller models, and streaming so users see output the moment it exists. Perceived speed often matters more than total completion time. Two clarifications keep TTFT from being misread. It is not the same thing as total response latency: a model can start fast and still take a long time to finish a long answer, and a model can start slowly yet complete quickly once it does — so a product decision made on total latency alone will often optimize the wrong half of the experience. And it is not purely a function of model size. Infrastructure choices — batching strategy, hardware generation, whether a KV cache from a previous turn can be reused — move TTFT substantially and independently of which model you picked, which is why the same model can feel sluggish on one provider and immediate on another. The prefill relationship is the practical lever: because the whole prompt must be processed before the first token appears, TTFT grows with input length, and reusing cached prefix state is the single most effective way to avoid paying that prefill cost twice. That connection is why prompt and context caching exists as a priced product feature at all rather than as an invisible optimization. For conversational and voice interfaces the metric is close to decisive — a delay before any output appears reads as unresponsiveness in a way that slow-but-started output does not — so it belongs in your latency dashboards as a separate line from end-to-end time, measured at a high percentile rather than a mean, since it is the slow tail of requests that shapes how the feature is perceived.","Time to first token (TTFT) is how long an LLM takes to emit its first output token — the metric that decides whether your AI feature feels frozen.",null,[11,14,17,20],{"slug":12,"name":13},"inference","Inference",{"slug":15,"name":16},"latency","Latency",{"slug":18,"name":19},"prompt-caching","Prompt Caching",{"slug":21,"name":22},"streaming-generation","Streaming Generation",[24,28,32,35,38,42,45,48,51,54,57,60],{"slug":25,"category":5,"name":26,"updated_at":27},"annotation-guidelines","Annotation Guidelines","2026-08-24T03:30:02+00:00",{"slug":29,"category":5,"name":30,"updated_at":31},"baseline-model","Baseline Model","2026-08-24T02:46:38+00:00",{"slug":33,"category":5,"name":34,"updated_at":31},"batch-inference","Batch Inference",{"slug":36,"category":5,"name":37,"updated_at":31},"canary-prompt","Canary Prompt",{"slug":39,"category":5,"name":40,"updated_at":41},"champion-challenger","Champion-Challenger (A\u002FB Model Testing)","2026-08-24T02:46:37+00:00",{"slug":43,"category":5,"name":44,"updated_at":31},"class-imbalance","Class Imbalance",{"slug":46,"category":5,"name":47,"updated_at":31},"continuous-batching","Continuous Batching",{"slug":49,"category":5,"name":50,"updated_at":31},"cross-validation","Cross-Validation",{"slug":52,"category":5,"name":53,"updated_at":31},"data-labeling","Data Labeling",{"slug":55,"category":5,"name":56,"updated_at":41},"drift-detection","Drift Detection",{"slug":58,"category":5,"name":59,"updated_at":41},"eval-harness","Eval Harness",{"slug":61,"category":5,"name":62,"updated_at":41},"experiment-tracking","Experiment Tracking"]