What is RAG (Retrieval-Augmented Generation)?

Large language models are great at generating fluent text, but they only know what they saw during training. Ask them about something recent, or something specific to your company's internal documents, and they'll either say "I don't know" or — worse — make something up. RAG is a simple, practical fix for that.

In one line: RAG lets a model "look things up" before answering, instead of relying only on what it memorized during training.

The problem RAG solves

An LLM's knowledge is frozen at training time and limited to what was in its training data. Two common issues follow:

Outdated or missing information: the model has no idea about anything that happened after training, or about private data it never saw.
Hallucination: instead of admitting it doesn't know, the model can generate a confident-sounding but wrong answer.

How RAG works

RAG stands for Retrieval-Augmented Generation. The idea is to give the model relevant, up-to-date information right before it answers, instead of relying only on what it memorized.

   User question
        │
        ▼
  ┌────────────┐
  │  Retrieve  │  search a knowledge base for the most relevant text
  └────────────┘
        │
        ▼
  ┌────────────┐
  │  Augment   │  insert that text into the prompt, next to the question
  └────────────┘
        │
        ▼
  ┌────────────┐
  │  Generate  │  the model answers using both the question and the context
  └────────────┘
        │
        ▼
       Answer

Plain LLM vs RAG

Plain LLM: knowledge frozen at training time, no access to private data, can hallucinate confidently, no traceability.
LLM + RAG: knowledge stays current via the knowledge base, private/internal data is searchable, answers are grounded in retrieved sources, and you can point to the exact document an answer came from.

A simple example

Imagine asking a chatbot about your company's vacation policy. A plain LLM has never seen your internal HR documents, so it can't answer correctly. With RAG, the system first searches your HR documents for the relevant policy text, adds it to the prompt, and only then asks the model to answer — using that real text as context.

RAG doesn't make a model smarter on its own — it makes the model's answers more grounded, current, and trustworthy by handing it the right information at the right time.

Conclusion

RAG is one of the most common patterns for building practical, reliable AI applications today: retrieve the relevant facts, augment the prompt with them, and let the model generate an answer grounded in real sources instead of guesswork.