Agentic RAG

Build Retrieval-Augmented Generation pipelines with reasoning and tool-aware agents.

Agentic RAG

Overview

Agentic RAG extends the classic RAG pattern by introducing an agent capable of reasoning, planning, and iteratively interacting with the vector store.

Instead of a single retrieve → generate pass, Agentic RAG allows the model to:

• decide what to retrieve next
• perform multiple retrieval steps
• refine its understanding iteratively
• optionally interact with external tools (MCP)

In RAGLight, Agentic RAG is designed to stay:

• explicit
• debuggable
• close to standard RAG semantics

---

How Agentic RAG differs from standard RAG

Standard RAG

Question
  ↓
Retrieve (once)
  ↓
Generate

• single retrieval step
• fixed context
• no planning

---

Agentic RAG

Question
  ↓
Agent
  ↳ Retrieve
  ↳ Think
  ↳ Retrieve again
  ↳ Think
  ↓
Final Answer

• iterative retrieval
• reasoning between steps
• adaptive use of context

The agent controls when and how retrieval happens.

---

Agentic RAG in RAGLight

RAGLight provides a dedicated high-level API:

• AgenticRAGPipeline

This pipeline builds on top of the same components as RAG:

• embeddings
• vector store
• LLM

But wraps them in an agent loop.

---

Option 1: AgenticRAGPipeline (simple API)

AgenticRAGPipeline is the recommended way to experiment with Agentic RAG.

Use it when you want:

• reasoning-aware retrieval
• minimal boilerplate
• fast experimentation

---

Basic example

from raglight.rag.simple_agentic_rag_api import AgenticRAGPipeline
from raglight.config.agentic_rag_config import AgenticRAGConfig
from raglight.config.vector_store_config import VectorStoreConfig
from raglight.config.settings import Settings
from raglight.models.data_source_model import FolderSource

Settings.setup_logging()

knowledge_base = [
    FolderSource(path="./data"),
]

vector_store_config = VectorStoreConfig(
    embedding_model=Settings.DEFAULT_EMBEDDINGS_MODEL,
    provider=Settings.HUGGINGFACE,
    database=Settings.CHROMA,
    persist_directory="./defaultDb",
    collection_name=Settings.DEFAULT_COLLECTION_NAME,
)

config = AgenticRAGConfig(
    provider=Settings.MISTRAL,
    model="mistral-large-2411",
    k=10,
    max_steps=4,
    system_prompt=Settings.DEFAULT_AGENT_PROMPT,
    knowledge_base=knowledge_base,
)

pipeline = AgenticRAGPipeline(config, vector_store_config)

pipeline.build()

response = pipeline.generate(
    "Explain how Agentic RAG differs from standard RAG"
)
print(response)

---

What happens during execution

At runtime, the agent:

1. receives the user question
2. decides whether retrieval is needed
3. queries the vector store
4. reasons over retrieved context
5. optionally repeats steps 2–4
6. produces a final answer

The loop stops when:

• the agent reaches max_steps
• or it decides it has enough context

---

Key configuration parameters

Agentic RAG introduces additional parameters compared to standard RAG.

max_steps

max_steps=4

Controls how many reasoning / retrieval iterations the agent can perform.

---

k

k=10

Defines how many documents are retrieved at each step.

---

system_prompt

The agent prompt defines:

• reasoning structure
• tool usage rules
• when retrieval should be invoked

RAGLight exposes default prompts via:

Settings.DEFAULT_AGENT_PROMPT

You are encouraged to inspect and customize them.

---

num_ctx

num_ctx=8192

Maximum context window size passed to the LLM. Increase this if your retrieved chunks are large or if you need longer reasoning chains.

---

verbosity_level

verbosity_level=2

Controls the verbosity of the agent’s internal logs. Set to 0 to silence agent traces, or 2 (default) for full step-by-step logging.

---

Option 2: Builder-based Agentic RAG

Under the hood, Agentic RAG is still built from the same primitives.

Using the Builder API allows:

• custom agent loops
• manual control over tools
• experimentation with reasoning strategies

(This approach is recommended only for advanced users.)

---

MCP integration (tools)

Agentic RAG can be extended with external tools via MCP servers.

This allows the agent to:

• execute code
• query databases
• fetch live data

Example configuration:

config = AgenticRAGConfig(
    provider=Settings.OPENAI,
    model="gpt-4o",
    k=10,
    mcp_config=[
        {"url": "http://127.0.0.1:8001/sse"}
    ],
)

Tools are invoked only when the agent decides they are relevant.

---

When to use Agentic RAG

Agentic RAG is useful when:

• a single retrieval pass is insufficient
• questions require exploration or refinement
• reasoning over large knowledge bases
• combining retrieval with tools

For simple Q&A, standard RAG is often faster and cheaper.

---

Summary

• Agentic RAG introduces an agent loop on top of RAG
• Retrieval becomes iterative and reasoning-driven
• RAGLight provides a simple AgenticRAGPipeline API
• Configuration stays explicit and debuggable
• Agentic RAG shines on complex, multi-step questions