How Agentic RAG solves problem with current RAG limitations

--

In this volume 4 of coffee break concept, we will understand how AgenticRAG helps solve limitations of traditional RAG.

RAG Framework

The RAG (Retrieval Augmented Generation) framework operates in a specific sequence:

Document -> Chunks-> Vector DB -> Chunk Retrieval (Top K) -> LLM

However, this sequence encounters obstacles when dealing with certain types of queries.

Traditional RAG Pipeline

Problem 1: Summarization

Consider a query like “Summarize the document”.

  • The conventional RAG approach retrieves the top K chunks and summarizes them.
  • But wouldn’t it be more comprehensive if it retrieved all chunks of the document and summarized them?
Summarization problem with RAG

Problem 2: Comparing Documents

  • When tasked with comparing Document A and Document B, the basic RAG retrieves random chunks and attempts to compare these top K chunks.
  • This doesn’t paint an accurate picture as it doesn’t represent the full scope of the documents.
Problem 2: Comparing Documents

Problem 3: Structured Data Analysis

Consider a question like “When is the next leave?”.

  • The first step is to retrieve the region to which the employee belongs from a structured table.
  • Based on the region, the next leave for that region is extracted from the leave policy document.
  • This process isn’t as straight forward with the current RAG framework.
Problem 3: Structured Data Analysis

Problem 4: The Multi-part Question

Consider a question like “Identify common leave across all regions?”.

  • Imagine you have a leave policy document of a company present in 120 countries.
  • Since you are passing the top K contexts, the maximum number of regions that can be compared is limited to K, where K is the number of chunks passed to LLM.
Problem 4: The Multi-part Question

Agentic RAG

Agentic RAG can solve this 4 problems by replacing via custom agents.

  • Agents will interact with multiple systems.
  • RAG is now one part of this system which agents can use.
Agentic RAG
  • Agents uses LLMs to automate the reasoning and tool selection
  • RAG is just another tool which Agent may decides to use.

Routing Agent

  • Routing agents are simple agents which routes the queries.
  • An agent can route query in one or multiple tools.
  • Remember our question “Summarize the document” or a question if we want to combine “Summarization + Sematic search” can be solved using below example routing
Routing Agent

Query Planning Agent

  • Query planning agent breaks down the queries into sub-queries.
  • Each of the sub-queries can be executed against RAG pipeline.
Query Planning Agent

Tools For Agents

  • LLMs can have multiple tools like calling an API, infer parameters for API.
  • RAG is now a tool which LLM might use.
Tools For Agents

Summary

  • RAG has limitations when represented with complex questions.
  • Few of the use cases like summarization, comparison etc. can’t be solve with just RAG.
  • Agentic RAG can help overcome limitation of RAG.
  • Agentic RAG treats RAG as a tool which it can use for semantic search.
  • Agents equipped with routing, query planning and tools can out perform traditional RAG applications.

Your feedback as comments and claps encourages us to create better content for the community.

Can you give multiple claps? Yes you can

--

--

Mastering LLM (Large Language Model)

MasteringLLM is a AI first EdTech company making learning LLM simplified with its visual contents. Look out for our LLM Interview Prep & AgenticRAG courses.