Grounding
AI Agents
Grounding means basing an AI model’s answers on specific, checkable sources, like your documents or search results, not just its training data.
AI Agents
What Is Retrieval-Augmented Generation (RAG)?
Retrieval-augmented generation (RAG) is a technique that retrieves relevant information from a source you choose, such as company documents, a knowledge base, a database, or the web, and gives it to an AI model along with the question. The model can then base its answer on that material instead of relying only on what it learned during training.
Also called retrieval augmented generation
Example
A medical spa adds a chat assistant to its website. Instead of letting the assistant answer from general knowledge, the team connects it to the spa’s own treatment guides, aftercare instructions, and booking and cancellation policies.
A client asks, “How long should I stay out of the sun after a chemical peel?” The system searches those documents, pulls the chemical peel aftercare section, and gives it to the model along with the question. The assistant answers from the spa’s guidance and names the source document. When the spa revises a policy, staff update the document, the system re-indexes it, and the answers reflect the change, with no retraining. The spa also sets up the assistant to hand off questions that need medical judgment, such as whether a treatment is safe alongside a specific medication, to a licensed provider.
How RAG works, step by step
Designs vary, but most RAG systems follow four steps:
- Prepare and index. Source content is split into smaller passages, often called chunks. Each chunk is commonly converted into an embedding, a list of numbers that represents its meaning, and stored in a searchable index such as a vector database.
- Retrieve. When someone asks a question, the system searches the index for the passages most relevant to it. Many systems match by meaning, and some combine that with traditional keyword search.
- Augment. The top passages are inserted into the prompt along with the question and instructions, such as “answer only from the material provided.”
- Generate. The model writes a response based on that material. Well-built systems also show citations that point back to the source passages, so people can check the answer.
In a typical setup, retrieval, augmentation, and generation happen each time a question is asked. Indexing happens up front and needs to be refreshed whenever the source content changes.
The term comes from a 2020 research paper by Patrick Lewis and colleagues at Facebook AI Research and university collaborators. Today it covers a broad family of designs, most of which add retrieved text to the prompt of a model that is otherwise unchanged.
Why it matters for your business
On its own, a general-purpose AI model draws only on what it learned during training. That knowledge has a cutoff date and does not include your internal documents. RAG fills the gap at the moment a question is asked, without changing the model itself.
- Current information. When a policy or price changes, you update the source document and re-index it, and the next answer can reflect it.
- Private information. The model can answer from your handbooks, contracts, product specs, or support history, none of which it saw in training.
- Grounded, checkable answers. RAG is the most common way to achieve grounding: the model answers from source text it was just given and can cite the passages it used, so a reader can check them. That can reduce hallucinations, though it does not eliminate them.
Common uses include internal help desks, customer support assistants, sales tools that answer from product documentation, and research assistants that summarize a document library. Features in AI assistants that search the web or your uploaded files before answering often work on the same principle.
Limitations to know
RAG is only as good as what it retrieves and the content it retrieves from.
- Retrieval misses. If the search step pulls the wrong passages, or a key detail is split across chunks, the model may give an incomplete or wrong answer that still sounds confident.
- Stale index. If an old policy and a new one are both indexed, or a changed document hasn’t been re-indexed yet, the system may retrieve the outdated version.
- Limited space. Only so much retrieved text fits into the model’s context window (the amount of text it can take in at once), so questions that need many documents at once can be harder to answer well.
- Permissions and privacy. If the index holds sensitive files, the system needs access controls so people only get answers drawn from documents they are allowed to see.
Testing with real questions, including ones your sources cannot answer, is a practical way to find these gaps before customers do.
When to use RAG
RAG is a good fit when answers depend on specific facts that live in your own content, change over time, or need to be traced to a source. It is often the first approach teams try for questions over documents, because it does not require training a model.
It is a weaker fit for questions that need the whole collection at once, such as “What themes come up across all of last year’s customer feedback?” Retrieval brings back a handful of the most relevant passages, not every document, so answers to broad summary questions can miss a lot. And if your reference material is short enough to paste into a prompt in full, you may not need a retrieval system at all.
RAG also depends on having source material worth retrieving. A well-organized knowledge base with clear, current, non-duplicated articles matters as much as the technology choice.
If you are weighing RAG against fine-tuning, which trains the model itself further, the comparison below sets out how the two differ.
RAG vs. fine-tuning
| RAG | Fine-tuning | |
|---|---|---|
| What changes | The prompt: relevant source text is added at question time | The model itself: it is trained further on examples |
| Best for | Answering from specific, private, or changing information | A consistent style, format, tone, or specialized task |
| Keeping information current | Update the source documents and re-index them | Needs another round of training, and is a less reliable way to add new facts |
| Sources and citations | Can point to the source passages it was given | No built-in way to show where an answer came from |
| Effort to set up | Content preparation, indexing, and retrieval testing | Curated training examples, training runs, and evaluation |
| Typical example | A support assistant that answers from your help articles | A model trained to write replies in your house format and tone |
RAG changes what the model can see when it answers; fine-tuning changes how the model behaves, and the two can be used together.
Further reading
- AI ImplementationUseful if you want help connecting the systems where your content lives and deploying AI inside the processes your team already runs.
From definition to deployment
See an AI agent doing real work.
SN3 AI Employees are role-trained AI systems deployed inside the tools your team already uses, with guardrails and support after launch.