Cost Observability

Cost observability is the ability to explain a bill rather than merely receive one. It means spend is instrumented the way latency and errors are: broken down by service, feature, customer and version, available soon enough after the event to connect it to a change someone made, and queryable rather than only reportable. The test is simple — when the number moves, can you say why within an hour. Traditional cost reporting fails that test because it is organised around the provider's structure rather than yours. An invoice knows accounts, instance types and token counts; it does not know that the increase came from a retrieval change that doubled the context sent with every request, or from a single customer whose usage pattern shifted. Bridging that gap requires the application itself to emit cost data alongside its other telemetry. For AI workloads the instrumentation is specific and worth naming. Every model call should record input and output token counts, the model and version used, the latency, the feature it served, the tenant it belonged to, the workflow step, and whether it was a retry. With those fields, cost becomes a dimension you can slice like any other, and questions such as which feature drives spend, which tenant is unprofitable, and what a prompt change cost become queries instead of investigations. Cost per unit of work is the metric that makes the data useful. Absolute spend is dominated by usage growth, which is usually good news, so it hides regressions: a prompt edit that adds context, a retrieval setting that returns more chunks, a router that quietly sends more traffic to an expensive model. Watched as cost per conversation, per document or per resolved ticket, each of those shows up immediately and as an engineering problem rather than a finance one. Latency matters here too, because the two are often traded against each other. A change that improves quality by sending more context increases both cost and time to answer, and a change that reduces cost by using a smaller model may increase escalations to a human. Cost observability that sits beside quality and latency lets those trade-offs be made deliberately; cost data on its own tends to produce savings that show up as complaints somewhere else. The practical starting point is small. Log the fields, put cost on the same dashboard as latency and error rate, express it per unit of work, and alert on the ratio rather than the total. That is usually enough to find the first few surprises, and the surprises are what fund the rest of the work.