What is RAG? Retrieval-augmented generation in plain English

How retrieval-augmented generation lets AI answer questions using your own documents, why it reduces made-up answers, and where businesses use it.

By the UK Technology Hub teamPublished 4 minute read

The short answer

Retrieval-augmented generation, or RAG, is a way of making AI answer questions using your own information. When someone asks a question, the system first searches your documents for the most relevant passages, then gives those passages to the AI model to write an answer. This keeps answers grounded in your content, makes them easier to check and avoids retraining the model.

AI language models learn from huge amounts of public text, but they don't know your prices, policies, products or internal procedures. And when they don't know something, they may still produce a confident-sounding answer. RAG is the most common way businesses solve both problems.

The library analogy

Imagine asking a very well-read assistant a question about your company. Without RAG, they answer from memory, which is broad but may be out of date or wrong about your business. With RAG, they first go to your company library, find the relevant pages, read them, and then answer based on what they found, pointing you to the source.

How RAG works, step by step

1. Prepare the content. Your documents, such as web pages, FAQs, manuals, policies and help articles, are collected and cleaned up.

2. Split into chunks. Long documents are divided into smaller passages, so the system can find precise information rather than whole documents.

3. Create embeddings. Each chunk is converted into a list of numbers, called an embedding, that represents its meaning. Passages with similar meanings have similar numbers.

4. Store in a search index. The embeddings are stored in a vector database or search index, often combined with traditional keyword search.

5. Retrieve. When a user asks a question, it's converted into an embedding too, and the system finds the chunks with the closest meaning.

6. Generate. The relevant chunks are given to the AI model with instructions such as "Answer using only this information and cite the source. If the answer isn't there, say so."

7. Respond. The user gets a natural answer, ideally with links to the original documents.

Why businesses use RAG

  • Accuracy: answers are based on your approved content, reducing invented answers.
  • Freshness: update a document and the next answer reflects it. No retraining needed.
  • Transparency: answers can cite sources, so users and staff can check them.
  • Control: you decide which content the AI can draw on, and can restrict content by user permissions.
  • Cost: far cheaper and faster than training or fine-tuning a custom model.

RAG or fine-tuning?

Fine-tuning adjusts a model's behaviour by training it on examples. It's useful for teaching a particular style, format or specialised task. RAG is better for giving a model knowledge that changes, such as product information or policies. Many business projects only need RAG; some combine both.

Where RAG is used

  • Customer chatbots that answer from your website, FAQs and policies. See our comparison of ChatGPT and custom chatbots.
  • Internal knowledge assistants that help staff find procedures, HR policies or technical documentation.
  • Customer portals where clients ask questions about their own account documents.
  • Sales support, drawing on past proposals and product specifications.
  • Compliance and quality, finding the right clause or procedure quickly.

What makes RAG work well

RAG isn't magic. Quality depends on:

  1. Good content. If documents are out of date, contradictory or vague, answers will be too. Content clean-up is often the most valuable part of a project.
  2. Sensible chunking. Passages need enough context to make sense on their own.
  3. Strong retrieval. Combining meaning-based and keyword search often finds better results.
  4. Clear instructions telling the model to stay within the retrieved content.
  5. Testing with real questions, including tricky and out-of-scope ones.
  6. Monitoring answers after launch, and improving content where gaps appear.

Security and permissions

If your documents include confidential or personal information, the RAG system must respect who is allowed to see what. A customer should never receive a passage from another customer's documents. Permissions need to be applied at the retrieval step, not left to the AI. Data protection rules apply too; see our guide to AI and UK GDPR.

RAG and AI agents

RAG gives AI knowledge. AI agents go further, letting AI take actions such as booking appointments or updating records. Many agents use RAG to find the information they need before acting. Our guide to AI agents explains the difference.

How we can help

Our AI solutions team builds RAG systems on your content, from customer-facing AI chatbots to internal knowledge assistants and AI features inside web apps. We start by reviewing your content, because better content means better answers.

Frequently asked questions

Does RAG stop AI from making things up?

It greatly reduces it by grounding answers in your content, but it doesn't eliminate it entirely. Good instructions, testing and monitoring are still needed.

Do I need to train my own AI model for RAG?

No. RAG uses existing models and supplies your content at the time of each question.

What kind of documents can RAG use?

Most text-based content, such as web pages, PDFs, Word documents, help articles and spreadsheets. Scanned documents need text extraction first.

This guide is general information, not legal, tax or financial advice, and details such as prices, features and rules can change. Product and company names mentioned are trademarks of their respective owners and are used only to identify those products; UK Technology Hub is not affiliated with or endorsed by them.

Related guides

All AI & automation guides