Retrieval-augmented generation (RAG)
Quick definition
Retrieval-augmented generation (RAG) is a technique in which an AI system first retrieves relevant passages from a trusted source, such as a knowledge base or a product catalog, then gives them to a language model so that its answer rests on those passages rather than on its memory alone.
On this page
Why RAG exists
A language model learns from a fixed snapshot of text. It does not know your return policy, your current prices or the product you added this morning, and when asked anyway it may produce a plausible but wrong answer. Retraining a model every time your content changes would be slow and expensive.
RAG solves this by keeping knowledge outside the model. Your content stays in a searchable store that you can update at any time; the model is only asked to read the relevant extracts and phrase an answer. The approach was described by Lewis and colleagues at Facebook AI Research in 2020.
How it works, step by step
First, the content is prepared: documents, web pages or product records are split into passages, and each passage is turned into an embedding — a list of numbers that captures its meaning. These embeddings are stored in a vector index, often alongside a classic keyword index.
When a question arrives, it is converted the same way and the closest passages are retrieved, sometimes combined with keyword search or filters such as language or category. The best passages are inserted into the model's prompt with an instruction to answer from them. Many systems also return the sources, so the reader can check where the answer came from.
Limits to know about
RAG is only as good as what it retrieves. If the right passage is missing, out of date or badly split, the model answers from the wrong extract or falls back on its memory. Structured questions — "the cheapest jacket in stock in size M" — are poorly served by similarity search alone and need filters on real data. And the model can still misread a passage, so answers on sensitive topics should be checked or escalated.
That is why production systems combine RAG with structured search, freshness rules that re-sync content when it changes, and clear instructions to say when the sources do not contain the answer.
RAG in customer service
For a support or e-commerce assistant, RAG is what lets the agent answer from your FAQ, your policies and your catalog in the customer's language, and point to the page it used. In AI SmartTalk, the knowledge an assistant answers from — website pages, documents, FAQ entries and synced products — is indexed this way, and product questions also use filters on price, options and stock.
Frequently asked questions
Is RAG the same as fine-tuning?
No. Fine-tuning changes the model's weights by training it on examples; RAG leaves the model unchanged and hands it the right information at question time. RAG is usually the better choice for facts that change, such as prices, stock or policies.
Does RAG stop AI hallucinations?
It reduces them a lot, because the model answers from real passages, but it does not remove them. Good retrieval, an instruction to admit when the sources are silent, and a handoff to a person for sensitive cases are still needed.
What is a vector database?
A store that indexes embeddings so that the passages closest in meaning to a question can be found quickly. It is the usual retrieval layer of a RAG system, often paired with keyword search.
Can RAG work with a product catalog?
Yes, but products need more than similarity search: an assistant should filter on structured fields such as price, size or stock, then use RAG for descriptions and buying guides.