AI cost optimisation: how to cut your AI API bills
Practical ways to reduce the cost of AI features without hurting quality, from model choice and prompt design to caching, batching and monitoring.
What tokens are, how context windows limit what an AI model can consider at once, and how both affect cost, speed and the quality of answers.
Tokens are the small chunks of text that AI models read and write; in English, a token averages roughly three-quarters of a word. The context window is the maximum number of tokens a model can consider at once, including your instructions, documents, conversation history and its answer. Tokens determine cost and speed, and context limits shape how you design AI features.
If you use AI through an API or build AI features, you'll quickly meet two terms: tokens and context windows. They sound technical, but they explain a lot about cost, speed and why AI sometimes "forgets" things.
Language models don't read words the way people do. Text is split into tokens: common words may be a single token, while longer or rarer words are split into pieces. Punctuation and spaces count too. As a rough rule of thumb for English, 100 tokens is around 75 words, though this varies by language and content. Code, numbers and some languages use more tokens per word.
Every request has:
AI providers usually price API usage per million tokens, with separate rates for input and output. Output tokens often cost more than input tokens. Larger, more capable models cost more per token than smaller ones. So the cost of a feature depends on:
Our guide to AI cost optimisation covers practical ways to bring these down.
Models generate answers token by token. Longer outputs take longer. Very long inputs also take time to process. For interactive features like chat, streaming the answer as it's generated makes the wait feel shorter.
The context window is the maximum number of tokens the model can handle in a single request, including the answer. Think of it as the model's working memory for one task. Context windows have grown enormously; many current models can handle hundreds of thousands of tokens, enough for long reports or large parts of a codebase. Limits vary by model and provider.
If a conversation or document exceeds the context window, something has to give. Applications typically:
That's why a chatbot may "forget" something from much earlier in a long conversation.
Large context windows are useful, but they come with trade-offs:
For most business applications, retrieving the right few passages beats stuffing everything into the context. Our guide to embeddings and vector databases explains how retrieval finds those passages.
Say a support assistant sends 1,500 tokens of instructions, 2,000 tokens of retrieved help articles and 500 tokens of conversation with each question, and writes a 300-token answer. That's 4,300 tokens per reply. At 10,000 replies a month, that's 43 million tokens. Trimming instructions, retrieving fewer but better passages and using a smaller model for simple questions can cut that figure substantially without hurting quality.
Providers offer tokenizer tools and APIs to count tokens before sending a request. Building token counting into your application helps you predict costs, stay within limits and spot unexpectedly large requests.
Multimodal models also convert images and audio into tokens, so a high-resolution image can use many tokens. Our guide to multimodal AI covers how these models work.
Our AI solutions team designs AI features that balance quality, speed and cost, including retrieval, caching and model selection. Our web app developers build token tracking and limits into your software, and our AI chatbot builds manage long conversations sensibly.
Roughly 750 English words, although it varies with vocabulary, formatting and language.
Not necessarily. It lets the model see more, but focused, relevant context usually produces better and cheaper results.
Long conversations can exceed the context window, so older messages are trimmed or summarised.
This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.
Tell us about your project and get a free, fixed quote.
Get a free quotePractical ways to reduce the cost of AI features without hurting quality, from model choice and prompt design to caching, batching and monitoring.
How large language models like the ones behind ChatGPT, Claude and Gemini actually work, what they're good at, where they fall short and how businesses use them.
How to write prompts that get consistent, useful results from AI models, with reusable patterns, examples and the habits that separate good prompts from guesswork.