Choosing a Vector Store

The Store slot holds your chunk embeddings and answers the nearest-neighbour lookups behind space.search(...) and indx query.

indx ships five backends behind one typed protocol, so picking a store is a one-line config change rather than a rewrite:

• The default qdrant gives you a real ANN index that runs embedded or as a server.
• The jsonl store needs no database at all and runs fully local with nothing installed.

For the embedding model that fills the store, see Choosing an Embedder. For running everything offline, see Local & Air-Gapped.

The decision at a glance

Store	Choose it when	Extra
qdrant (default)	General use. You want a real approximate-nearest-neighbour (ANN) index that works both embedded/on-disk and as a server with the same client code, plus rich payload filtering. Apache-2.0.	indx[qdrant]
pgvector	Your org already runs PostgreSQL and wants vectors beside relational data on one operational surface.	indx[pgvector]
chroma	Quick local prototyping with a simple, popular API.	indx[chroma]
lancedb	Large, file-based columnar vector data with zero server and good on-disk performance.	indx[lancedb]
jsonl (no DB)	Air-gapped, fully portable, zero-dependency indexes. Brute-force linear scan over small corpora; maximum portability inside the .indx archive.	none — ships in core

Why Qdrant is the default

qdrant is the default because it does not force a server. It runs in-process against a local on-disk path, then scales up to a hosted Qdrant later without code changes. The jsonl store goes one step further and needs no database at all. It is the offline fallback that ships in the core install.

How a Store fits the pipeline

A Store is consumed in stage 06 Embed+Pack (see the pipeline overview). The Embedder turns chunk text into vectors, those embedded chunks are upserted into the store, and the writer then materialises the embeddings into the archive’s embeddings/ layout. The resulting .indx stays portable regardless of which backend produced it.

The contract is the VectorStore protocol (normative names come from the protocols reference). The canonical class is VectorStore, also re-exported under the alias Store (from indx.store import Store):

from typing import Protocol, runtime_checkable

@runtime_checkable
class VectorStore(Protocol):  # exported as `Store`
    """Vector database adapter. Default: qdrant.
    Also: pgvector, chroma, lancedb, jsonl."""

    name: str

    def upsert(self, chunks: list[Chunk]) -> None: ...

    def search(self, vector: list[float], k: int = 5) -> list[SearchHit]: ...

    def delete(self, chunk_ids: list[str]) -> None: ...

• name is the backend’s registry identifier (e.g. "qdrant", "jsonl").
• upsert writes a batch of Chunk objects. Each chunk carries its own id, text, embedding, and source. indx batches chunks before calling it.
• search returns the top-k SearchHit results for a query vector, in descending score order. Each hit carries the matched chunk, its score, and resolved neighbors. This powers space.search(...) and indx query.
• delete removes chunks by id.

The slot is a structural Protocol. Any third-party class that implements these methods and exposes name works as a store — no subclassing, no fork of indx. See Custom Components and Authoring a Plugin.

Persistence: why your archive stays portable

A .indx archive opens on any machine, even one without your database. The pack step of stage 06 is what makes this work: when the writer seals the archive, it materialises the embeddings into the standard embeddings/ layout — regardless of which store backend ran the build.

handbook.indx                 (ZIP container)
├── manifest.json             # counts, embedder name + dim, store name, checksums
├── index.json                # the knowledge graph
├── chunks/                   # per-chunk JSON
└── embeddings/
    ├── manifest.json         # model, dim, count, backend
    └── vectors.f32           # contiguous little-endian float32 matrix (count × dim)

The vectors travel inside the archive as a float32 matrix, and the embedder name/dim are pinned in the manifest. A consumer that does KnowledgeSpace.load(...) can search the space without ever talking to Qdrant or Postgres.

The JSONL store stores vectors inline

With the jsonl store there is no external database. Vectors are kept inline in the archive and queried by brute-force linear scan. That is what makes a jsonl-built .indx fully self-contained. The trade-off is that linear scan suits only small corpora. For anything non-trivial, the default remains Qdrant.

Choosing a backend

Qdrant (default)

Qdrant works well across most workloads. It gives you a genuine ANN index with rich payload filtering, strong performance, and an Apache-2.0 license.

For indx, its key property is that the same client runs both embedded and against a remote server. The embedded mode is in-process against a local on-disk path, so you start local and move to a hosted deployment without touching your code.

pgvector

A good fit for teams already standardised on PostgreSQL. Keeping vectors in Postgres means one backup story, one access-control model, and the ability to JOIN vector results against your existing relational data. Choose it to consolidate operational surface area rather than run a second datastore.

Chroma

A simple, popular API that is good for quick local prototyping. Reach for Chroma when you want to spin something up fast and are not yet worried about scale or production operations.

LanceDB

A file-based, columnar vector store with no server and good on-disk performance. LanceDB suits a large vector set you want to keep as files without standing up a service. The files are easy to copy, version, and stage.

JSONL (no DB)

The zero-dependency floor. The jsonl store ships in the core install, requires no database, and stores vectors inline in the archive. Paired with the core hash embedder (--store jsonl --embedder hash), it produces a usable knowledge space with nothing else installed and no API key.

Note that jsonl only removes the database — the default OpenAI embedder still needs a key and network.

It searches by brute-force linear scan, which is fine for small spaces and well suited to air-gapped delivery. It is not an ANN index, so it does not scale to large corpora.

Selecting your store

You can pick a store three ways. Precedence is explicit code argument / use() → CLI flag → indx.toml → documented default.

On the CLI

# Default Qdrant (embedded/on-disk)
indx ./docs --out ./ai-ready

# Zero-dependency, fully portable
indx ./docs --out ./ai-ready --store jsonl

# pgvector
indx ./docs --out ./ai-ready --store pgvector

In indx.toml

The store slot lives in the [store] section. Backend-specific options go in a [store.<backend>] sub-table — those keys are passed verbatim to the adapter constructor and are otherwise opaque to the core.

[store]
backend = "qdrant"            # qdrant | pgvector | chroma | lancedb | jsonl

[store.qdrant]
url = "http://localhost:6333"   # adapter-specific; omit to run embedded/on-disk

[store]
backend = "pgvector"

[store.pgvector]
dsn = "postgresql://localhost:5432/indx"   # connection target; the password comes from env, not the file
# Override or supply the secret via env: INDX_STORE__PGVECTOR__DSN="postgresql://user:pass@host:5432/indx"

Secrets

Keep credentials such as DSNs and API keys out of the committed config file. indx layers environment-variable overrides on top of indx.toml, using INDX_* variables. Connection secrets belong in the environment, not in version control.

In Python

Pass a name string or a custom instance to the pipeline, or swap it later with use():

from indx import DirectoryPipeline

# By name
pipeline = DirectoryPipeline(embedder="bge-m3", store="lancedb")
space = pipeline.run("./docs", "./ai-ready")

# Swap by keyword later (accepts a name or an instance)
pipeline.use(store="jsonl")

Every backend is an extra

The core install (pip install indx) depends on no database client. Only the jsonl store ships in core. Each real database backend is an optional extra, which keeps the core light:

pip install "indx[qdrant]"      # QdrantStore   → qdrant-client
pip install "indx[pgvector]"    # PgVectorStore → psycopg + pgvector
pip install "indx[chroma]"      # ChromaStore   → chromadb
pip install "indx[lancedb]"     # LanceDBStore  → lancedb

If you select a backend whose extra is not installed, indx raises a single actionable error (MissingDependencyError) naming the exact pip install indx[...] to run. This happens only when that slot is actually selected, so unrelated runs are never affected.

The recommended local / air-gapped bundle is pip install "indx[local]" (Docling + Ollama + BGE-M3 + Qdrant). See the extras reference for the full matrix.

Things to keep in mind

• Dimension safety. The embedder’s name and dim (1024 for local bge-m3, 1536 for the default OpenAI embedder) are recorded in the archive manifest. A loaded archive can therefore detect a vector/dimension mismatch before querying. Changing the embedder means re-embedding.
• Batching is the lever. Upserts are batched per backend (Qdrant, pgvector, and Chroma batch at 256) to amortise round-trips. Tune batch size and concurrency per backend for throughput. See Performance.
• Portability regardless of backend. Whichever store you build with, the pack step materialises the same embeddings/ layout. A teammate can open the .indx with jsonl-style inline search even if they do not run your database. See the .indx archive reference.

See also

• Local & Air-Gapped — building and querying with zero network or external services.
• Inspect & Query — reading stats and running searches against a sealed archive.
• Protocols reference — the full Store contract and the other slot protocols.
• Registry & Defaults — how store names resolve to classes.
• Extras reference — the complete pip install indx[...] matrix.