Document chunks. Directory ingestion resolves
the processor from the lowercase file extension, applies the source’s chunk settings, and adds the
result to the vector store.
Built-in mapping
Supported source extensions include Python, JavaScript, TypeScript, Java, C, C++, C#, Go, Rust,
Ruby, PHP, Scala, Swift, Solidity, Lua, Perl, and Haskell.
Install the processors with the
documents extra:
Processor contract
A processor receives a file path and chunk parameters. It returns a dictionary withchunks and
classes lists:
source path.
Register a custom extension
Extend the default document-processor registry when you need application-specific entries, then inject it into the pipeline:replace=True is explicit.
Multimodal PDF processing
VlmPDFProcessor extracts PDF text and sends embedded images to a vision-capable LLM for captions.
Override the built-in PDF factory because the processor needs a configured model: