What is RAG (Retrieval-Augmented Generation)?

The problem it solves

An LLM only knows what was in its training data. It cannot know your company handbook, this week's news, or yesterday's medical research — and asking it anyway invites hallucination.

The recipe

Retrieve. Your question is matched against a document collection and the most relevant chunks are pulled out — usually via embeddings and vector search.
Augment. Those chunks are pasted into the model's context alongside your question, like an open-book exam.
Generate. The model writes its answer grounded in the retrieved text, ideally citing the passages it used.

That is the whole trick: no retraining, and the model's knowledge updates the moment your documents do.

Where you already meet RAG

Customer-support bots that answer from the help center, Claude or ChatGPT searching the web before answering, and "chat with your PDF" tools all use RAG. It is the standard architecture for anything that needs your facts rather than the internet's averages.

Frequently asked questions

Why is RAG better than just asking the model?

A plain model can only use what it memorized in training. RAG feeds it the exact documents you care about at question time, so answers are grounded in sources you can check instead of vibes.

Does RAG completely stop hallucinations?

No, but it reduces them a lot. The model can still misread or overstate what the retrieved documents say, so good RAG systems cite the exact passages they used.