Reproducibility & Determinism

A knowledge space is only trustworthy if you can rebuild it and get the same thing back. This page explains how indx makes that work: the same inputs, configuration, and component versions produce a byte-stable index.json, and the parts that can’t be made bit-reproducible are at least fully recorded for audit.

The sections below cover the guarantees indx makes, where they hold, and where they have edges.

The one-line promise

Re-running indx with identical inputs, config, and component versions produces a byte-identical index.json — there is no varying timestamp field to carve out.

Stable ids

Every Document and Chunk gets a zero-padded, stable string id, such as doc_0007 or chunk_0481. These ids aren’t random, and they aren’t assigned in the order work happens to finish. They follow a deterministic traversal order:

1. Folder lineage — folder ancestry, root to leaf.
2. Path — the file path within a folder.
3. In-document index — the 0-based Chunk.index position within its parent document.

The ordering key is derived entirely from the directory structure and document position. It never depends on wall-clock time, hash-map iteration order, or scheduler timing — so a rerun over unchanged input yields identical ids for the same content.

Downstream relations, neighbor links, and chunk_ids lists all point at these ids, which keeps the whole graph diff-friendly across builds.

policies/                        policies/
  data/retention.pdf  → doc_0007   data/retention.pdf  → doc_0007   (same id every run)
    chunk 0          → chunk_0480
    chunk 1          → chunk_0481
    chunk 2          → chunk_0482

Concurrency never affects ordering

indx parallelizes the expensive stages: Parse runs across files in a worker pool, Embed runs in batches, and Enrich issues bounded-concurrency LLM calls. Parallel execution means results arrive out of order.

To keep that from leaking into output, parallel results are re-sorted into the canonical deterministic order before ids are assigned. So the number of --jobs, the batch size, and which worker finishes first have no effect on the resulting ids or on the order of elements in index.json.

This is a deliberate design rule, not an accident of implementation: performance tuning must never trade away reproducibility. See Performance for the concurrency knobs themselves.

Note

Determinism applies to output ordering and ids, which the core computes. It does not require that two machines parse a PDF into byte-identical text when they run different versions of the parser. See “Pin your versions” below.

LLM enrichment determinism

Stage 05 (Enrich) uses an LLM to derive document type, topics, tags, and summaries. The LLM.complete(...) protocol defaults to temperature=0.0, which makes greedy decoding the default and removes sampling randomness.

That gets you as close to deterministic as the provider allows. But some providers aren’t bit-reproducible even at temperature 0 — batching, kernel nondeterminism, model-version rollovers, and floating-point reduction order can each shift a token.

indx doesn’t pretend otherwise. Instead, it makes enrichment auditable, recording the resolved component identity (the embedder/parser/store names plus the embedding model and dimension) in two places:

• index.json under metadata.
• The archive’s manifest.json inside the .indx container.

So even when the exact summary text can vary slightly between runs, you always know precisely which model and configuration produced it.

{
  "metadata": {
    "schema_version": "1",
    "indx_version": "0.4.2",
    "source_root": "/path/to/your/dir",
    "components": {
      "parser": "docling",
      "embedder": "openai",
      "store": "qdrant"
    },
    "embedding_model": "text-embedding-3-small",
    "embedding_dim": 1536
  }
}

Want full determinism? Drop enrichment

If you need an absolute byte-stable graph and don’t need LLM-derived metadata, remove enrichment with drop("enrich") (SDK) or --llm none (CLI). That drops the one stage whose output a provider can perturb; the remaining Walk → Parse → Chunk → Relate stages are deterministic on their own.

Embeddings: pinned and self-describing

Vectors are written to embeddings/vectors.f32 as a contiguous little-endian float32 matrix of shape count × dim. The embedder name and dim are pinned in both the embeddings sub-manifest and the archive manifest.json:

{
  "embedder": { "name": "text-embedding-3-small", "dim": 1536 },
  "store": "qdrant"
}

Pinning the model identity and dimensionality makes the archive self-describing. At load time, a consumer or space.search(...) can check that a query embedder matches the one that produced the stored vectors, refusing a dimension mismatch before it silently returns garbage.

As with LLMs, exact embedding values may differ across hardware or model versions, so the manifest records which model produced the vectors. If the default model changes, re-embed.

What is and isn’t byte-stable

Output	Reproducible?	Notes
Document / chunk ids	Yes	Deterministic traversal order; unaffected by concurrency.
Element ordering in index.json	Yes	Parallel results re-sorted before serialization.
index.json bytes	Yes	Same inputs + config + versions ⇒ byte-identical (serialized with sort_keys=True; no timestamp field).
Neighbor links, chunk_ids, relation targets	Yes	They reference the stable ids.
LLM enrichment text (topics, summary, tags)	Best-effort	temperature=0.0; some providers not bit-reproducible. Resolved model recorded.
Embedding vectors	Best-effort	float32 matrix; embedder name + dim pinned for validation.

Pin your versions

The guarantee is “same inputs, same config, and same component versions” — determinism is scoped to a fixed environment:

• Pin the indx version and the versions of any parser/LLM/embedder/store extras you use.
• Keep indx.toml under version control. The resolved snapshot is recorded in output, so you can diff it.
• On a --resume build, changing a component invalidates only the affected work: changing the parser invalidates Parse and everything downstream, while changing the embedder invalidates only Embed. This keeps reruns cheap without compromising correctness. See Configuration and Reproducibility-adjacent caching in Performance.

How indx tests this

Determinism is not a hope — it’s a guardrail enforced in CI with byte-stable golden-file tests. The suite runs the pipeline over a committed sample directory, using a fixed seed and offline, mocked LLM and embedder backends, then asserts that the produced index.json is byte-equal to a checked-in golden file:

def test_index_json_is_byte_stable(tmp_path):
    space = DirectoryPipeline(seed=0).run("tests/data/sample", tmp_path)
    produced = (tmp_path / "index.json").read_text()
    golden = Path("tests/golden/sample.index.json").read_text()
    assert produced == golden

Any unintended change to ids, ordering, or serialization fails the diff loudly. Golden files are regenerated deliberately, never blindly: when the serialized shape legitimately changes, a regeneration is gated on a schema_version bump.

See Testing for the full approach, including the mocked backends and deterministic seeding that make this stable offline.

Open question: the exact scope of determinism

Real LLM and embedding providers are not always bit-reproducible. So the precise scope of “deterministic” across live enrichment backends is still being pinned down. The committed guarantee is concretely about the core-produced graph: ids, ordering, structure, and serialization under fixed inputs and versions. Provider-driven enrichment is made auditable, with the model and config recorded, rather than promised bit-for-bit. When in doubt, run with --llm none for a fully reproducible graph, and treat enrichment text as best-effort.

Related reading

• The .indx archive — container layout, manifest.json, checksums, and sealing/loading.
• index.json reference — the serialized graph schema and metadata block.
• Testing — golden-file determinism tests and deterministic seeds.
• Local & air-gapped — the opt-in local profile that makes deterministic, no-egress, offline builds possible.