AI tokens and context windows explained simply
What tokens are, how context windows limit what an AI model can consider at once, and how both affect cost, speed and the quality of answers.
Practical ways to reduce the cost of AI features without hurting quality, from model choice and prompt design to caching, batching and monitoring.
AI API costs are driven by the number of tokens you send and receive, the model you use and how often you call it. The biggest savings usually come from using smaller models for simple tasks, sending only relevant context, caching repeated content and answers, batching non-urgent work, limiting output length and monitoring costs per feature so surprises are caught early.
AI features can start cheap and become expensive quietly. A prototype that costs a few pounds a day can cost thousands a month once real users arrive. The good news is that most AI costs can be reduced substantially with sensible engineering, often without users noticing any difference.
Most AI APIs charge per token, with separate rates for input and output, and different rates for different models. Our guide to tokens and context windows explains the basics. Your total cost is roughly:
requests × (input tokens × input price + output tokens × output price)
Every part of that formula can be reduced.
Large, top-tier models are excellent but often overkill. Classification, extraction, routing and short answers can often be handled by smaller models at a fraction of the cost. Our guide to small language models explains when they're enough.
A common pattern is routing: a small, cheap model handles routine requests, and only complex or uncertain ones go to a larger model.
Set maximum output tokens and ask for concise formats. Output tokens often cost more than input tokens, so verbose answers are expensive.
Many providers offer discounted batch processing for jobs that don't need an instant answer, such as overnight document processing, bulk product descriptions or report generation.
Errors, timeouts and agent loops can multiply costs. Set sensible retry limits, step limits for agents and alerts for unusual spikes.
At high, steady volumes, running an open-weight model on your own infrastructure can be cheaper than API fees. Factor in hardware, engineering and maintenance. Our comparison of open-source and proprietary AI covers the trade-offs.
Tag every AI request with the feature, customer or team that triggered it. Then you can see which features cost most, spot abuse, set budgets and price your product correctly. Set alerts for daily spend thresholds.
Every optimisation should be checked against your evaluation set. Our guide to AI evaluation and testing explains how. Saving 40% isn't a win if accuracy drops below an acceptable level.
A support tool sends a 3,000-token instruction block with every request. Moving that block into a cached prompt, trimming retrieved content from five passages to three and routing simple questions to a smaller model can reduce costs dramatically, while answers stay just as accurate in testing. Small changes like these compound at scale.
If you sell an AI-powered product, understand your cost per user and per action before setting prices. Usage-based limits or tiers protect you from heavy users driving costs above revenue.
Our AI solutions team reviews AI usage and cuts costs through model routing, caching and better retrieval, with quality checks at every step. Our web app developers add usage tracking and limits, and our AI chatbot builds are designed to stay efficient at scale.
Usually because of large prompts, long outputs, an expensive model used for simple tasks, or unexpected volume from retries or heavy users.
For simple tasks, often not noticeably. Test with your own examples to find the cheapest model that meets your standard.
A provider feature that reduces the cost of resending the same input, such as long instructions, across many requests.
This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.
Tell us about your project and get a free, fixed quote.
Get a free quoteWhat tokens are, how context windows limit what an AI model can consider at once, and how both affect cost, speed and the quality of answers.
What small language models are, where they match bigger models, where they don't, and how businesses use them for speed, cost and privacy.
How large language models like the ones behind ChatGPT, Claude and Gemini actually work, what they're good at, where they fall short and how businesses use them.