AI tokens and context windows explained simply
What tokens are, how context windows limit what an AI model can consider at once, and how both affect cost, speed and the quality of answers.
How large language models like the ones behind ChatGPT, Claude and Gemini actually work, what they're good at, where they fall short and how businesses use them.
A large language model (LLM) is an AI system trained on huge amounts of text to predict the next word in a sequence. That simple goal, at enormous scale, gives it the ability to write, summarise, translate, answer questions and follow instructions. LLMs don't look facts up by default and can be confidently wrong, so businesses pair them with their own data, clear instructions and human review.
Large language models are the engines behind most of the AI tools people now use at work. You don't need to understand the maths to use them well, but knowing roughly how they work helps you decide where to trust them and where to be careful.
At heart, an LLM is a prediction machine. Given some text, it predicts what comes next, one small piece at a time. Those pieces are called tokens, which are fragments of words. Our guide to tokens and context windows explains why they matter for cost and capability.
Predicting the next word sounds trivial. But to do it well across billions of examples, the model has to absorb grammar, facts, writing styles, reasoning patterns and the relationships between ideas. That's where its apparent understanding comes from.
Pre-training. The model reads a vast collection of text and learns to predict missing or next words. This stage uses huge computing resources and produces a general model with broad knowledge.
Fine-tuning and alignment. The general model is then trained further on examples of helpful answers and on human feedback about which responses are better. This turns a text predictor into an assistant that follows instructions, refuses harmful requests and responds in a useful format.
Updates. Providers release new versions regularly, improving reasoning, speed, safety and the ability to work with images, audio and tools.
Modern LLMs use an architecture called the transformer, introduced in 2017. Its key feature, attention, lets the model weigh which earlier words matter most when predicting the next one. Models are described by their number of parameters, the internal values adjusted during training. More parameters can mean more capability, but training data quality and technique matter just as much.
They can be wrong with confidence. LLMs generate plausible text, not verified facts. When they lack information, they may fill gaps with invented details. Our guide to AI hallucinations explains why and how to reduce it.
Their knowledge has a cut-off. A model only knows what was in its training data, unless it's connected to search or your own documents.
They don't remember by default. Each conversation is limited by the context window. Products add memory features on top.
They can reflect biases present in their training data.
They need clear instructions. Vague prompts give vague results. See our prompt engineering guide.
Several companies build leading LLMs, including OpenAI (GPT models behind ChatGPT), Anthropic (Claude), Google (Gemini) and Meta (Llama). There are also many open-weight models that businesses can run themselves. Our comparison of open-source and proprietary AI explains the trade-offs. Capabilities and pricing change quickly, so compare current versions for your specific task.
Bigger isn't always better. Smaller, faster models often handle simple tasks like classification and short answers well at a fraction of the cost. Larger models are better for complex reasoning, long documents and nuanced writing. Many applications use more than one model, routing each task to the most suitable option. Our guide to small language models covers the lighter end.
Our AI solutions team helps businesses choose models and build reliable applications on them. We build AI chatbots grounded in company content and add LLM features to web apps, with testing and safeguards designed in from the start.
ChatGPT is a product built on OpenAI's large language models, with extra features such as conversation history, tools and safety systems.
They model patterns in language extremely well, which often looks like understanding. They don't verify facts on their own, so outputs need checking.
Training a large model from scratch is very expensive. Most businesses use existing models and add their own data through retrieval or fine-tuning.
This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.
Tell us about your project and get a free, fixed quote.
Get a free quoteWhat tokens are, how context windows limit what an AI model can consider at once, and how both affect cost, speed and the quality of answers.
What AI hallucinations are, why language models produce confident errors, and the practical techniques businesses use to keep answers accurate.
How to write prompts that get consistent, useful results from AI models, with reusable patterns, examples and the habits that separate good prompts from guesswork.