Request Deduplication

Request deduplication is the practice of recognising that an incoming request is a repeat of one already in flight or already completed, and returning the existing result instead of doing the work again. It addresses a category of waste that is invisible in most metrics: the system appears busy and healthy, and a meaningful share of what it is doing has been done before. The duplicates come from predictable places. Clients retry on timeout without knowing whether the original succeeded. Users click a button twice when a response is slow. Queues with at-least-once delivery redeliver messages after a consumer restart. Load balancers and gateways retry idempotent-looking requests. And several users can ask the same question at the same moment, which is common in support tools and internal assistants. The standard mechanisms differ by kind. An idempotency key supplied by the client lets the server recognise a retry of a specific operation and return the original outcome rather than performing it twice — essential for anything that charges money or creates a record. In-flight coalescing collapses concurrent identical requests into one execution whose result is fanned out to all waiters, which is what prevents a cache miss from becoming a stampede. And a result cache handles repeats separated in time. For AI features deduplication is unusually valuable because each execution is expensive and slow. Two users asking the same question a second apart, a retried generation after a client timeout, an agent that calls the same tool twice in one loop — each duplicate costs a full inference. Exact-match deduplication on the request payload is cheap and safe. Semantic deduplication, where near-identical questions share an answer, saves more but introduces the risk of returning a response to a question that was not quite the one asked, so it needs a conservative similarity threshold and a clear scope. The correctness rules are what keep it safe. Only deduplicate operations whose repetition is genuinely redundant; a request that should produce a new record every time must not be collapsed. Scope the key by tenant and by user where results are permission-dependent, because a deduplicated response that crosses a tenant boundary is a data leak. Keep the window short enough that stale answers do not persist past their usefulness, and make sure a failed execution is not cached as though it had succeeded. Finally, measure it. A deduplication rate is a diagnostic as much as a saving: a sudden rise usually means a client is retrying too aggressively or a timeout is set below the real latency of the operation, and fixing that upstream is better than absorbing the duplicates downstream.