FolderSource. Ingestion is explicit: it
runs when pipeline.ingest(...) is called, not during pipeline construction.
Source options
The default ignore list contains
.git, .venv, venv, node_modules, __pycache__, build,
and dist. Override it with a tuple of directory names:
Deterministic and resilient processing
Directory entries are processed in deterministic name order. Symbolic links and unsupported file types are skipped. A failure in one file is recorded without discarding files already indexed.IngestionResult exposes:
discovered_count,processed_count, andindexed_count;skipped_count, skipped paths, and reasons;failed_countand failures grouped by discovery, processing, or indexing stage;to_dict()for logs, APIs, and job records.