[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"guide-how-to-cut-the-running-cost-of-an-ai-feature::en":3,"guide-related-how-to-cut-the-running-cost-of-an-ai-feature::en":17},{"slug":4,"title":5,"excerpt":6,"body":7,"meta_title":5,"meta_description":8,"keywords":9,"category":15,"published_at":16,"updated_at":16},"how-to-cut-the-running-cost-of-an-ai-feature","How to Cut the Running Cost of an AI Feature","The bill for an AI feature is mostly decided by choices made once and never revisited: prompt length, retrieval size, model selection and retries. Working through them in the right order recovers most of the cost without touching quality.","\u003Ch2>Measure per unit of work before you change anything\u003C\u002Fh2>\n\u003Cp>Absolute spend rises with adoption, which is good news, so it hides efficiency problems. The number to watch is cost per unit of work — per conversation, per document, per resolved ticket — because that isolates how expensive each interaction is from how many of them there are.\u003C\u002Fp>\n\u003Cp>Getting it requires instrumenting the application, not reading the invoice. Every model call should record input and output token counts, the model and version, the feature it served, the tenant and the workflow step. Without those fields, cost is a single line nobody can break down, and every optimisation is a guess.\u003C\u002Fp>\n\u003Ch2>Find where the tokens actually are\u003C\u002Fh2>\n\u003Cp>Before optimising, look at the composition of a typical request. In most systems the largest input component is retrieved context, followed by the system prompt and then conversation history. Output is smaller in volume but usually more expensive per token, and reasoning-heavy models can spend a great deal producing a short visible answer.\u003C\u002Fp>\n\u003Cp>Look at the distribution too, not just the average. A small share of requests with enormous contexts or long agent loops often accounts for a disproportionate part of the bill, and fixing those pathological cases is cheaper and safer than trimming everything.\u003C\u002Fp>\n\u003Ch2>Send less: the retrieval and prompt pass\u003C\u002Fh2>\n\u003Cp>Returning fewer, better-ranked passages usually beats returning many, and it frequently improves accuracy as well, because models attend imperfectly across long inputs and marginal documents dilute the ones that matter. Reranking a small candidate set is often the cheapest quality improvement available.\u003C\u002Fp>\n\u003Cp>Then audit the system prompt line by line — every word is paid for on every call — and stop replaying conversation history verbatim once it grows, summarising older turns while keeping recent ones intact. Few-shot examples earn their tokens only until the pattern is established.\u003C\u002Fp>\n\u003Ch2>Choose smaller where the task allows\u003C\u002Fh2>\n\u003Cp>Not every request needs the largest model. Routing simple, well-specified work to a smaller model and reserving the expensive one for cases that need it is usually the single biggest lever available, and a cascade that escalates only when a cheap answer fails a check keeps quality while paying premium rates for a minority of traffic.\u003C\u002Fp>\n\u003Cp>This only works if you can tell when the cheap answer was inadequate. Structured tasks give you that cheaply — does it parse, does it match the schema, does the citation exist — while open-ended generation is much harder to route safely.\u003C\u002Fp>\n\u003Ch2>Repeat less: caching and deduplication\u003C\u002Fh2>\n\u003Cp>Where a provider supports caching a stable prompt prefix, ordering the prompt so the unchanging part comes first converts a repeated cost into a much smaller one. Beyond that, collapse concurrent identical requests into one execution, cache answers to questions that genuinely recur, and scope any cache by tenant and user so a shared answer never crosses a permission boundary.\u003C\u002Fp>\n\u003Ch2>Stop the accidental spend\u003C\u002Fh2>\n\u003Cp>Retries after a timeout set below the real latency of the operation quietly double cost. Agent loops with no step limit can run until something stops them. Failed generations that are retried without a backoff turn a provider incident into a large invoice. Each of these is a bug rather than an optimisation opportunity, and each is usually worth more than a week of prompt trimming.\u003C\u002Fp>\n\u003Ch2>Protect quality while you do it\u003C\u002Fh2>\n\u003Cp>Run every change against a fixed evaluation set before it ships, and watch the quality metric alongside cost in production. Cost reduction pursued alone will eventually find the trim that saves money and breaks the feature — a shorter prompt that drops an instruction, a smaller model that fails on the awkward cases, a cache that serves a stale answer.\u003C\u002Fp>\n\u003Cp>Finally, set the guardrails that stop the next surprise: spend caps, per-tenant limits, budget alerts on cost per unit rather than on the total, and a kill switch that falls back to something the product can survive.\u003C\u002Fp>","Most AI cost reduction comes from sending less, choosing smaller and repeating less. Here is the order to work through, and how to avoid cutting quality by accident.",[10,11,12,13,14],"ai cost optimisation","inference cost","prompt caching","model routing","token efficiency","pricing","2026-08-31T03:30:02+00:00",[18,23,27,31,36,41],{"slug":19,"title":20,"excerpt":21,"updated_at":22},"ai-tool-pricing-models-seat-vs-usage-vs-credits","AI Tool Pricing Models: Seat-Based vs Usage-Based vs Credits","The three common ways AI tools charge — per seat, per usage, and by credits — and how to reason about which one will actually be cheaper for the way your team works.","2026-08-05T14:32:26+00:00",{"slug":24,"title":25,"excerpt":26,"updated_at":22},"how-ai-image-generators-differ-diffusion-vs-the-rest","How AI Image Generators Differ: Diffusion vs the Rest, in Plain Terms","A non-technical explanation of how AI image generators work, why the diffusion approach became dominant, and what practical differences to expect between tools.",{"slug":28,"title":29,"excerpt":30,"updated_at":22},"how-to-automate-your-workflow-without-code","How to Automate Your Workflow Without Code","A practical sequence for building automations that survive: picking the right process, mapping it before touching a tool, and handling the failure cases that break most first attempts.",{"slug":32,"title":33,"excerpt":34,"updated_at":35},"how-to-build-a-chatbot-without-coding","How to Build a Chatbot Without Coding","A practical route to a working chatbot using no-code tools: deciding scope, connecting your own content, handling the questions it cannot answer, and knowing what it will cost.","2026-08-05T14:32:27+00:00",{"slug":37,"title":38,"excerpt":39,"updated_at":40},"how-to-build-an-evaluation-set-that-does-not-go-stale","How to Build an Evaluation Set That Does Not Go Stale","Most evaluation sets are assembled once, from whatever was easy to collect, and then quietly stop representing the work. How to build one from failures you already have, keep it fed from production, protect it from the system it measures, and know when it has expired.","2026-09-09T03:30:02+00:00",{"slug":42,"title":43,"excerpt":44,"updated_at":45},"how-to-change-a-prompt-without-breaking-production","How to Change a Prompt Without Breaking Production","Prompts get edited in a text box and shipped in seconds, which is why they break things quietly: no compiler, no stack trace, no obvious moment of failure. Give them the release discipline code gets.","2026-08-24T03:30:02+00:00"]