RAG — retrieval-augmented generation

Retrieval-Augmented Generation (RAG) is a technique where you give a language model relevant documents to read before it answers a question. Instead of relying on what the model memorised during training, you retrieve the right information from your own data and put it in the prompt.

RAG exists because LLMs hallucinate. They generate confident answers even when they do not know the real answer. By grounding the model in retrieved documents, you dramatically reduce hallucination and make the model's answers verifiable.

A RAG pipeline has three stages: indexing, retrieval and generation.

In the indexing stage, you take your documents (PDFs, web pages, databases), split them into chunks, convert each chunk into a numerical representation called an embedding, and store these embeddings in a vector database.

In the retrieval stage, when a user asks a question, you convert their question into an embedding, search the vector database for the most similar chunks, and retrieve the top results.

In the generation stage, you put the retrieved chunks into the prompt alongside the user's question, and the LLM generates an answer grounded in those specific documents.

RAG is not magic. Common failure modes include: chunks that are too large or too small (bad chunking splits an answer across two chunks, so neither gets retrieved). Poor embeddings that do not capture domain-specific meaning. Retrieved context that is relevant but not sufficient to answer the question. The LLM ignoring the retrieved context and hallucinating anyway.

Building a RAG system that works reliably requires careful chunking strategy, good embedding model selection, retrieval evaluation, and prompt engineering. This is why RAG engineering is a real specialisation, not a weekend project.

RAG is the most common way companies add AI to internal knowledge. Customer support bots that answer from the company's help docs. Legal tools that search case law. HR chatbots that answer policy questions. Internal search that understands natural language. Any time you want an LLM to answer from a specific corpus rather than its general training, RAG is usually the right approach.

In India, RAG is being adopted heavily in fintech (regulatory document search), edtech (course Q&A), healthcare (clinical guideline retrieval), and enterprise IT (internal knowledge management).

A common question: should I fine-tune the model on my data or use RAG? The answer is almost always RAG first. Fine-tuning changes the model's behaviour and is expensive, slow, and hard to update. RAG keeps the model unchanged and just gives it the right context — it is cheaper, faster to update (just re-index your documents), and easier to debug.

Fine-tuning makes sense when you need to change the model's style or teach it a completely new skill. For knowledge retrieval, RAG wins.