Component Protocols

This page is the authoritative contract for indx’s seven protocols: Stage, Parser, LLM, VLM, Embedder, Store, and OutputWriter. It also covers the rules for resolving a component by name.

indx uses structural typing. Every replaceable part of the pipeline is a Python typing.Protocol with a default implementation, and any object that satisfies the protocol’s shape can be dropped straight in. There is no base class to inherit from — to provide your own component, you only need to match the method signatures shown below.

See Bring your own components for worked examples and Authoring a plugin for packaging one for distribution.

Protocols vs. stages

The six stages (Walk → Parse → Chunk → Relate → Enrich → Embed+Pack) are units of pipeline orchestration. The six component protocols (Parser, LLM, VLM, Embedder, Store, OutputWriter) are the swappable engines that certain stages call into. The Stage protocol governs the stages; the rest govern the components. For the stage walkthrough, see the pipeline overview.

How components are bound

Every swappable component can be supplied two ways:

• By object — pass an instance in code, at construction or via DirectoryPipeline.use(...).
• By name — a short registry string (e.g. "docling", "qdrant"), optionally with a :model suffix (e.g. "ollama:qwen2.5"). Set it in code, on the CLI, or in indx.toml.

Both routes converge on the same typed protocol. Name resolution is covered in Resolution by name below and in Registry and defaults.

The Stage protocol

A Stage is a single unit of work in a DirectoryPipeline. The pipeline executes stages in registration order, passing one shared SpaceContext from each stage to the next.

from typing import Protocol, runtime_checkable

@runtime_checkable
class Stage(Protocol):
    """A single unit of work in a DirectoryPipeline.

    A stage MUST return the same SpaceContext instance it received, mutated in
    place. The pipeline executes stages in registration order, passing the
    context from each stage to the next.
    """
    name: str  # stable identifier, e.g. "walk", "parse", "chunk", "relate", "enrich", "embed-pack"

    def run(self, ctx: SpaceContext) -> SpaceContext: ...

Member	Contract
name	A stable string identifier ("walk", "parse", "chunk", "relate", "enrich", "embed-pack"). Stages are addressed by name for replace(name, ...) and drop(name).
run(ctx)	Does the stage’s work and returns the same SpaceContext instance, mutated in place. It must never return a new object and never None.

Return the same context

run must return the exact SpaceContext it was handed. Stages communicate only through the shared context, never via globals or side channels. Per-item failures are appended to ctx.errors rather than raised, unless fatal.

The built-in stages are not registered in any per-slot component registry — DirectoryPipeline wires them directly. Third-party stages are discovered through the indx.stages entry-point group.

See custom stages for the full authoring recipe and the pipeline reference for stage ordering.

The Parser protocol

Converts a single file into a ParsedDoc. Called once per file by stage 02 (Parse).

@runtime_checkable
class Parser(Protocol):
    """Converts a single file into a ParsedDoc. Default: docling."""
    name: str
    def parse(self, path: Path) -> ParsedDoc: ...

Member	Contract
name	A stable registry identifier for the adapter (for example "plaintext", "docling").
parse(path)	Takes a pathlib.Path to a source file and returns a fully populated ParsedDoc — normalized text, plus structural blocks, tables, and images that the Chunk stage uses to split with structure intact. Local parsers must not touch the network.

The parser reads the file directly from the given Path (for example path.read_text()) and emits a ParsedDoc whose Source records path, folder, and type.

• Default: docling → DoclingParser.
• Registry name(s): docling (plus any installed adapters, e.g. unstructured, llamaparse, markitdown, and the zero-dependency plaintext fallback that ships in core).

See Choosing a parser for the trade-offs between engines.

The LLM protocol

Text generation used by stage 05 (Enrich) to derive document type, topics, tags, and summaries.

@runtime_checkable
class LLM(Protocol):
    """Text generation for enrichment (type, topics, tags, summaries).
    Default: openai:gpt-5-mini."""
    name: str
    def complete(self, prompt: str, *, system: str | None = None,
                 max_tokens: int = 512, temperature: float = 0.0) -> str: ...

Parameter	Default	Meaning
prompt	— (required)	The user prompt to complete.
system	None	Optional system instruction.
max_tokens	512	Upper bound on generated tokens.
temperature	0.0	Sampling temperature; the default of 0.0 keeps enrichment deterministic.

complete(...) returns the generated text as a plain str.

• Default: openai:gpt-5-mini → OpenAILLM running the gpt-5-mini model; use ollama:qwen2.5 for local, no-egress enrichment.
• Registry name(s): ollama:<model>, none (and installed cloud adapters such as openai, anthropic, azure, vllm).

Reproducibility

The default temperature=0.0 is part of indx’s reproducibility story — same inputs, config, and model versions yield a byte-stable index.json. The resolved model name is recorded in the archive manifest and index.json.metadata for auditability. See Reproducibility.

Enrichment can be turned off entirely with --llm none, by dropping the enrich stage, or via [enrich] llm = "none" in indx.toml. See Enrichment with LLM and VLM.

The VLM protocol

Vision-language enrichment for images and figures, also used by stage 05 (Enrich). Disabled by default.

@runtime_checkable
class VLM(Protocol):
    """Vision-language enrichment for images/layout. Default: none (disabled)."""
    name: str
    def describe(self, image: bytes, *, prompt: str | None = None) -> str: ...

Parameter	Default	Meaning
image	— (required)	The raw image bytes to describe.
prompt	None	Optional instruction steering the description.

describe(...) returns a textual description as a str.

• Default: none → NullVLM, a no-op that ships in core and skips vision enrichment.
• Registry name(s): none, plus installed adapters such as qwen-vl, gpt4o, local (the local VLM ships via the indx[vlm-local] extra).

Because the default is none, image understanding is strictly opt-in. Enable it with --vlm <adapter> or [enrich] vlm = "<adapter>". See Enrichment with LLM and VLM.

The Embedder protocol

Turns chunk text into vectors. Driven by stage 06 (Embed+Pack), in batches.

@runtime_checkable
class Embedder(Protocol):
    """Turns text into vectors. Default: openai:text-embedding-3-small."""
    name: str
    dim: int
    def embed(self, texts: list[str]) -> list[list[float]]: ...

Member	Contract
name	A stable model/adapter identifier (for example "hash", "text-embedding-3-small"), recorded into the embeddings manifest.
dim	The vector dimensionality (an int). Pinned into the archive manifest so a loaded archive can validate query-time compatibility.
embed(texts)	Accepts a list[str] and returns a list[list[float]] — one vector per input, each of length dim, in the same order.

• Default: openai:text-embedding-3-small → OpenAIEmbedder, dimension 1536; use bge-m3 for the local profile.
• Registry name(s): hash, openai:<model>, bge-m3 (and installed adapters such as e5, cohere). hash is the zero-dependency offline fallback and is lexical rather than semantic.

The Embed+Pack stage groups chunk texts into batches (default 64) before each embed(...) call. See Choosing an embedder.

The VectorStore protocol (alias Store)

The vector-database adapter. Used by stage 06 to persist embedded chunks and to back space.search(...). The canonical class name is VectorStore; it is also re-exported under the public alias Store, so from indx.store import Store and from indx.store import VectorStore both resolve to the same protocol.

@runtime_checkable
class VectorStore(Protocol):  # exported as `Store`
    """Vector database adapter. Default: qdrant.
    Also: pgvector, chroma, lancedb, jsonl."""
    name: str
    def upsert(self, chunks: list[Chunk]) -> None: ...
    def search(self, vector: list[float], k: int = 5) -> list[SearchHit]: ...
    def delete(self, chunk_ids: list[str]) -> None: ...

Member	Contract
name	A stable registry identifier for the backend (for example "jsonl", "qdrant").
upsert(chunks)	Inserts or updates a list of Chunk objects (each carrying its own id, text, embedding, and source metadata). Returns None.
search(vector, k=5)	Returns the top-k matches as a list[SearchHit] in descending score order. Each SearchHit carries the matched chunk, its score, and any resolved neighbors.
delete(chunk_ids)	Removes the chunks with the given ids from the store. Returns None.

• Default: qdrant → QdrantStore.
• Registry name(s): qdrant, pgvector, chroma, lancedb, jsonl (the zero-dependency jsonl fallback ships in core and enables a fully offline run).

See Choosing a store and the .indx archive reference.

The Writer protocol (alias OutputWriter)

Serializes a finished KnowledgeSpace to disk. The final responsibility of stage 06. The canonical class name is Writer; it is also re-exported under the public alias OutputWriter, so from indx.output import OutputWriter and from indx.output import Writer both resolve to the same protocol.

@runtime_checkable
class Writer(Protocol):  # exported as `OutputWriter`
    """Serializes a KnowledgeSpace to a destination. Default: indx.
    Also: jsonl, langchain, llamaindex."""
    name: str
    def write(self, space: KnowledgeSpace, dest: Path, *, name: str = "handbook") -> None: ...

Member	Contract
name	A string naming the output format this writer emits (for example "indx", "jsonl").
write(space, dest, *, name="handbook")	Serializes the KnowledgeSpace to the dest directory (a pathlib.Path). The keyword-only name sets the archive/base file name. Returns None.

• Default: indx → IndxWriter, which seals the portable archive (the .indx is the file extension; the registry name is indx).
• Registry name(s): indx, jsonl, langchain, llamaindex (the jsonl writer ships in core).

See Output formats for what each writer emits.

Resolution by name

Each component sub-package maintains a registry mapping a short name to a class. A name may carry an optional :model suffix; the base name selects the adapter class and the suffix selects the model.

indx/llm/__init__.py

REGISTRY = {"ollama": OllamaLLM, "none": NullLLM}

def resolve(name: str) -> LLM:
    base, _, model = name.partition(":")
    cls = REGISTRY[base]               # raises RegistryError (CLI exit code 3) if the name is unknown
    return cls(model=model or None)

Slots, names, and defaults

Slot	Name strings	Class (default)
parser	docling	DoclingParser
llm	openai:<model>, ollama:<model>, none	OpenAILLM (gpt-5-mini)
vlm	none, <adapter>	NullVLM
embedder	openai:text-embedding-3-small, bge-m3 (plus adapters e5, cohere)	OpenAIEmbedder (dim 1536)
store	qdrant, pgvector, chroma, lancedb, jsonl	QdrantStore
output	indx, jsonl, langchain, llamaindex	IndxWriter

Local profile (opt-in)

The table above is the shipped zero-config default stack (cloud). The local profile — opted into with pip install "indx[local]" (alias indx[defaults]) or explicit flags/config — swaps llm to ollama:qwen2.5 and embedder to bge-m3 (dim 1024), keeping qdrant (or jsonl for a DB-free run). It is not the default; a bare indx <dir> --out <dir> uses the cloud stack and needs OPENAI_API_KEY.

Resolution order

For each unset slot the effective component is chosen in this priority order:

explicit object/string  >  indx.toml  >  documented default

An explicit instance or name passed in code (or to use()) wins; otherwise indx.toml is consulted; otherwise the documented default applies.

Unknown names fail fast

An unknown component name raises a fatal error before any stage runs — the pipeline never starts a partial build on a misconfigured slot. This surfaces as a configuration error on the CLI (exit code 3). See Errors and exit codes.

Third-party adapters register additional names through entry-point groups (indx.parsers, indx.llms, indx.vlms, indx.embedders, indx.stores, indx.writers, indx.stages). Once installed, an adapter is usable by name anywhere a built-in is. First-party builtins win on name collisions.

For the complete registry mechanics see Registry and defaults and Authoring a plugin.