Core Objects

indx has a small data model: six objects that describe the source files, retrieved text, relationships, provenance, and in-flight pipeline state. A pipeline reads a directory and assembles them into a single portable result you can query.

This page is the conceptual tour. For every field, constraint, and serialized shape, see the data-models reference.

All core objects are Pydantic v2 models. Identifiers are stable, zero-padded strings such as doc_0007 and chunk_0481. They are assigned in a deterministic traversal order, so re-running over unchanged input yields identical ids.

The big picture

A run produces one KnowledgeSpace. Here is how the six objects fit together:

• KnowledgeSpace — the top-level result, holding everything below.
• Document — one per source file, forming a graph.
• Chunk — a flat list of retrievable units.
• Relation — the edges that wire documents and chunks together.
• Source — provenance, attached to every chunk and document.
• SpaceContext — the mutable carrier that holds the work in progress while the pipeline runs.

While the pipeline runs, the SpaceContext carries the work from one stage to the next. At the end, the context materializes into the KnowledgeSpace.

KnowledgeSpace  (top-level result → .indx archive)
├── documents:  Document[]   one enriched source file each
│     ├── Source             provenance (path, folder, type)
│     ├── chunk_ids ─────────┐  links into the chunk list
│     └── references / referenced_by : Relation[]
├── chunks:     Chunk[]  ◀───┘  retrievable units
│     ├── Source
│     ├── neighbors          adjacent chunk ids
│     ├── relations:  Relation[]
│     └── embedding          vector (stage 06) — in-memory; serialized to embeddings/vectors.f32, never inlined in index.json
└── relations:  Relation[]   graph-level edges

Where these come from

Each object is produced by a specific stage. Walk and Parse build provenance and ParsedDocs; Chunk creates Chunks; Relate adds Relations; Enrich fills in type, topics, tags, and summaries; Embed+Pack attaches vectors and seals the KnowledgeSpace. See the pipeline overview for the full flow.

KnowledgeSpace

The top-level result of processing a directory. It holds the document graph, the chunks, the embeddings, and run metadata. It serializes to a single .indx archive that re-loads without re-processing.

Most important attributes:

• root — absolute path of the walked directory or ZIP.
• documents — the Document graph (exposed as a callable accessor; see below).
• chunks — every Chunk in the space.
• relations — graph-level edges (a mirror of the per-object edges).
• metadata — tool version, resolved config snapshot, build time, and any non-fatal errors.

KnowledgeSpace also gives you a first-class API, so you rarely touch the raw fields:

from indx import KnowledgeSpace

space = KnowledgeSpace.load("./ai-ready/handbook.indx")  # filename is <name>.indx — default handbook, set with --name

space.stats                       # SpaceStats: counts, embed_dim, type histogram
space.documents(type="policy")    # filter the document graph by detected type
space.search("data retention", k=5)  # semantic search → list[SearchHit]
space.save("./copy.indx")         # seal back to a portable archive

• stats returns a SpaceStats with document/chunk/relation/embedding counts, embed_dim, and a per-type histogram.
• documents(type=...) returns the documents, optionally filtered by detected type.
• search(query, k=5) embeds the query and returns the top-k chunks as SearchHits, each with its resolved neighbor chunks and source.
• load(path) / save(path) open and seal .indx archives.

Same model, every surface

space.documents is both the data and a filter: calling it with no argument returns every Document; passing type= narrows the result. The same model surface backs indx inspect and indx query.

Document

One source file, enriched. A Document records everything a file-level parser throws away: folder lineage, document type, and cross-document references. It stores these alongside LLM-derived semantics.

Most important attributes:

• id — stable id, e.g. doc_0007.
• path / folder — original location relative to the walked root.
• lineage — the folder ancestry root→leaf (e.g. ["policies", "policies/data"]), so agents can filter and reason by location.
• type — detected/enriched document type (e.g. policy, guide), which drives type-aware enrichment.
• topics, tags, summary — semantic metadata added during Enrich.
• chunk_ids — the chunks produced from this document, in order.
• references / referenced_by — outgoing and incoming Relation edges resolved during Relate.

A Document does not embed its chunks; it links to them by id through chunk_ids. Resolve those against space.chunks to walk from a document to its retrievable content.

Chunk

The retrievable unit. This is the thing your RAG system or agent actually fetches. Every chunk remembers exactly where it sits, so retrieval never returns an orphaned fragment.

Most important attributes:

• id — stable id, e.g. chunk_0481.
• text — the retrievable text payload.
• source — the originating Source (path, folder, type).
• doc_id — id of the parent Document.
• index — 0-based position within that document.
• metadata — enriched topics, summary, and tags.
• neighbors — adjacent chunk ids (previous and next), for expanding context windows.
• relations — outgoing typed edges from this chunk.
• embedding — the vector, populated in Embed+Pack. It is an in-memory field, None until embedding runs, and never inlined into the chunk JSON. Vectors are serialized separately to the archive’s embeddings/vectors.f32 (see the .indx archive reference), so embedding is omitted from index.json.

Together, doc_id, index, neighbors, and relations turn a chunk from a flat bag of text into a connected node. An agent can climb back to the parent document, pull adjacent chunks for context, or follow a continues or references edge.

Relation

A typed, directed edge in the knowledge graph. A Relation can connect document to document, chunk to chunk, or chunk to document. This is how indx captures the structure that file-by-file parsing destroys.

Attributes:

• type — one of the five RelationType values.
• src — the source id or path: the edge’s origin (e.g. chunk_0481 or policies/onboarding.md).
• dst — the target id or path (e.g. legal/gdpr.md or chunk_0482).
• score — confidence/similarity score in [0, 1], defaulting to 1.0.

The five relation types:

RelationType	Meaning
sibling	Same folder / same logical group.
parent	Folder lineage / containment.
references	An outgoing citation, link, or mention.
continues	The next unit in a split sequence.
duplicate-of	Near or exact duplicate content.

Relations appear in three places: on a chunk’s relations, on a document’s references / referenced_by, and (as an optional mirror) on the space-level relations list. See Relate for how each type is derived.

Source

The lightweight provenance record attached to every chunk and parsed unit. It is what makes results traceable back to disk.

Attributes:

• path — original file path relative to the walked root.
• folder — containing folder, relative to root.
• type — detected/enriched document type (e.g. policy).

Source is small by design. It travels with chunks and ParsedDocs so that a SearchHit can expose hit.source.path without re-resolving the parent document.

SpaceContext

The shared, mutable carrier threaded through every pipeline stage. The objects above are the result; SpaceContext is the work in progress.

Each stage obeys the contract run(ctx: SpaceContext) -> SpaceContext, returning the same object it received, mutated in place. A stage reads what earlier stages produced, then appends to the collections relevant to its own phase.

What it carries:

• Inputs — root (the walked directory) and seed (the deterministic seed for any sampling/ordering). The output path, resolved config, and component instances live on the DirectoryPipeline, not the context.
• Accumulated work — space, a KnowledgeSpace the stages fill in place: Walk/Parse/Chunk/Relate/Enrich populate space.documents_, space.chunks, and space.relations; Parse also fills parsed, a doc_id → ParsedDoc map on the context. Embeddings are written onto each Chunk.embedding during Embed+Pack.
• Diagnostics — errors, a list of non-fatal StageErrorRecord items surfaced later under space.metadata["errors"].

When the pipeline finishes it returns ctx.space directly — the KnowledgeSpace the stages have been filling in place (there is no separate to_space() call).

Mutable, stage-filled models

Result and value models — Chunk, Relation, Document, ParsedDoc, and KnowledgeSpace — are mutable Pydantic models. Stages fill and refine them in place: Enrich sets Document.topics and summary; Embed sets Chunk.embedding.

A deterministic, diff-friendly index.json comes from stable serialization (sorted keys, fixed field ordering), not from immutability.

If you write a custom stage, mutate the context and the models it carries, then return it. Never construct a new one.

How they fit together: a quick walk

Conceptually, the models nest like this:

from indx import KnowledgeSpace, Document, Chunk, Relation, Source

space = KnowledgeSpace.load("./ai-ready/handbook.indx")

doc: Document = space.documents(type="policy")[0]
print(doc.path, doc.lineage, doc.topics)        # provenance + semantics

# Walk from a document to its chunks via chunk_ids.
chunks = [c for c in space.chunks if c.id in doc.chunk_ids]
first: Chunk = chunks[0]
print(first.text, first.neighbors)              # text + adjacent ids

# Every chunk carries a Source and typed Relations.
src: Source = first.source
for rel in first.relations:
    rel  # Relation(type="references", src="chunk_0481", dst="legal/gdpr.md")

From here, deepen your understanding with the pipeline and stages concept, the bring-your-own-stack model, or the complete data-models reference.