Skip to main content

Getting started

This guide creates a local classic RAG pipeline with Ollama and Chroma, ingests a directory, and returns a structured response.

Prerequisites

  • Python 3.11 or newer
  • uv or pip
  • Ollama running locally
  • a directory containing .txt, .md, .html, source-code, or PDF files
Pull the default local models:

Install integrations

The ollama, chroma, and documents extras are independent. Installing the base package alone keeps the framework dependency-free.

Create the pipeline

Save this as local_rag.py:
Run it with your own directory:

What happens

  1. FolderSource discovers files recursively and skips generated directories and symlinks.
  2. The document-processor registry selects a processor from each extension.
  3. Processed chunks are indexed through add_documents() on Chroma.
  4. generate() retrieves the top matching chunks.
  5. The LLM receives the retrieved context and returns an answer.
  6. The response retains the sources and execution details.
Processing or indexing failures are isolated per file and reported by IngestionResult; one bad file does not discard successful files.

Persist the index

The example uses a temporary database. To reuse an index, replace the temporary directory with an application-owned path:

Next steps

RAG pipelines

Add reranking, configuration, streaming, and structured results.

Agentic RAG

Let a bounded agent choose retrieval and custom tools.

Providers

Use OpenAI, Gemini, Mistral, LM Studio, or Bedrock.

Ingestion

Configure recursion, ignored directories, and custom processors.