Data Models

This page documents every field of every core domain model. The models are Pydantic v2 types. They travel through the pipeline, serialize into index.json, and seal into a .indx archive. For a concept-level tour of how these objects fit together, see Core objects.

Conventions

A few rules apply across all models:

• Identifiers are stable strings generated from deterministic input such as relative path and position. In the current implementation they are short SHA-256-derived hashes, for example a3f4..., not sequential counters. Re-running over unchanged input yields identical ids.
• Vectors are list[float]. They are 32-bit floats in the on-disk matrix and plain Python floats in memory.
• Free-form metadata is typed dict[str, Any]. It is serialized with sorted keys for diff-friendly output.

Mutable, stage-filled models

Per the coding standards, the value/result models — Document, Chunk, Relation, and ParsedDoc — are standard mutable Pydantic v2 models that stages fill and refine in place. For example: Enrich populates a Document’s topics/summary, Embed sets each Chunk’s embedding, and a custom stage may rewrite chunk.text.

SpaceContext is the carrier that each stage receives and returns, threading these collections through the pipeline.

RelationType

Typed graph edges between documents and/or chunks. RelationType is a str enum, so its members serialize to their string values.

class RelationType(str, Enum):
    SIBLING       = "sibling"        # same folder / same logical group
    PARENT        = "parent"         # folder lineage / containment
    REFERENCES    = "references"     # outgoing citation, link, or mention
    CONTINUES     = "continues"      # next unit in a split sequence
    DUPLICATE_OF  = "duplicate-of"   # near/exact duplicate content

Value	Meaning
sibling	Same folder or same logical group.
parent	Folder lineage / containment.
references	Outgoing citation, link, or mention.
continues	Next unit in a split sequence.
duplicate-of	Near or exact duplicate content.

Tip

New RelationType members are additive within a major version. Consumers must ignore unknown values rather than fail.

Source

Provenance of a chunk or parsed unit — where the content came from in the walked tree.

Field	Type	Description
path	str	Original file path, relative to the walked root. Required.
folder	str	Containing folder, relative to root. Defaults to "".
type	str | None	Detected or enriched document type, e.g. "policy". Defaults to None until enrichment or type detection fills it.

class Source(BaseModel):
    path:   str
    folder: str = ""
    type:   str | None = None

Relation

A typed, directed edge in the knowledge graph. A Relation may connect chunk-to-document, document-to-document, or chunk-to-chunk ids.

The current implementation stores an explicit src and dst for every edge, so relations remain self-contained when serialized or moved between collections.

Field	Type	Default	Description
src	str	— (required)	Source id or path.
dst	str	— (required)	Target id or path.
type	RelationType	— (required)	The kind of edge.
score	float	1.0	Confidence / similarity score in [0, 1] where applicable.

class Relation(BaseModel):
    src:   str
    dst:   str
    type:  RelationType
    score: float = 1.0

ParsedDoc

The raw output of a Parser for a single source file. It is produced in stage 02 Parse, before chunking. It carries ordered structural blocks; text is exposed as a property that joins block text in order. Returned by Parser.parse(file) -> ParsedDoc.

Field	Type	Default	Description
source_path	str	— (required)	Source file path relative to the walked root.
parser	str	plaintext	Parser implementation that produced the record.
parser_version	str	0	Parser version string recorded for reproducibility.
blocks	list[Block]	[]	Ordered structural blocks: headings, paragraphs, table cells, captions, code, and similar units.
text (property)	str	derived	Normalized full-text rendering, joined from blocks in order.

class ParsedDoc(BaseModel):
    source_path: str
    parser: str = "plaintext"
    parser_version: str = "0"
    blocks: list[Block] = []

Chunk

A retrievable unit of content. A Chunk remembers its source document, its position within that document, the ids of its immediate neighbor chunks, and any typed relations.

The embedding vector is populated in stage 06 Embed+Pack. It is typically not inlined into index.json; it lives in embeddings/.

Field	Type	Default	Description
id	str	— (required)	Stable id generated from document id and position.
text	str	— (required)	The retrievable text payload.
doc_id	str	— (required)	Id of the parent Document.
position	int	— (required)	0-based position within the parent document.
prev_id	str | None	None	Previous chunk id, when one exists.
next_id	str | None	None	Next chunk id, when one exists.
source	Source | None	None	Originating document provenance.
index (property)	int	derived	Alias of position.
metadata	dict[str, Any]	{}	Enriched values: topics, summary, tags.
neighbors	list[str]	[]	Adjacent chunk ids (previous, next).
relations	list[Relation]	[]	Outgoing typed edges from this chunk.
embedding	list[float] | None	None	Vector, populated in stage 06. May be omitted from index.json.

class Chunk(BaseModel):
    id:        str
    doc_id:    str
    position:  int
    text:      str
    prev_id:   str | None = None
    next_id:   str | None = None
    embedding: list[float] | None = None
    source:    Source | None = None
    metadata:  dict[str, Any]        = {}
    relations: list[Relation]        = []

Document

One source file, enriched. A Document holds:

• Folder lineage and detected type.
• LLM-derived topics, tags, and summary from stage 05 Enrich.
• Resolved outgoing and incoming references from stage 04 Relate.

Field	Type	Default	Description
id	str	— (required)	Stable id generated from the relative path.
path	str	— (required)	Original path relative to root.
lineage	list[str]	[]	Folder ancestry, root→leaf.
size_bytes	int	0	Source file size in bytes.
doc_type	str | None	None	Detected / enriched document type.
type (property)	str | None	derived	Alias of doc_type.
folder (property)	str	derived	Containing folder, joined from lineage.
topics	list[str]	[]	Enrichment-derived topics.
tags	list[str]	[]	Enrichment-derived tags.
summary	str | None	None	Enrichment-derived summary.
chunk_ids	list[str]	[]	Chunks produced from this document, in order.
references	list[Relation]	[]	Outgoing references resolved in stage 04.
referenced_by	list[Relation]	[]	Incoming references (reverse edges).
metadata	dict[str, Any]	{}	Free-form additional metadata.

class Document(BaseModel):
    id:            str
    path:          str
    lineage:       list[str]     = []
    size_bytes:    int           = 0
    doc_type:      Optional[str] = None
    topics:        list[str]     = []
    tags:          list[str]     = []
    summary:       Optional[str] = None
    chunk_ids:     list[str]     = []
    references:    list[Relation] = []
    referenced_by: list[Relation] = []
    metadata:      dict[str, Any] = {}

SpaceStats

Aggregate counts surfaced via space.stats. The same shape appears under the stats key of index.json and is what indx inspect --json emits.

Field	Type	Default	Description
documents	int	— (required)	Number of documents.
chunks	int	— (required)	Number of chunks.
relations	int	— (required)	Number of relations.
embeddings	int	— (required)	Number of stored vectors.
embed_dim	int | None	None	Vector dimensionality, e.g. 1024 for bge-m3.
types	dict[str, int]	{}	Document count per detected type.
bytes_source	int	0	Total bytes of source material walked.

class SpaceStats(BaseModel):
    documents:    int
    chunks:       int
    relations:    int
    embeddings:   int
    embed_dim:    Optional[int]  = None
    types:        dict[str, int] = {}
    bytes_source: int            = 0

SearchHit

A single result from space.search(...). It exposes the matched chunk, its neighbor chunks (resolved into full Chunk objects for context windows), and a convenience source property.

Field	Type	Default	Description
chunk	Chunk	— (required)	The matched chunk.
score	float	— (required)	Similarity score; higher is better.
neighbors	list[Chunk]	[]	Resolved neighbor chunks of chunk.
source (property)	Source | None	derived	Provenance of the matched chunk — shorthand for hit.chunk.source.

class SearchHit(BaseModel):
    chunk:     Chunk
    score:     float
    neighbors: list[Chunk] = []

    @property
    def source(self) -> Source | None:
        return self.chunk.source

KnowledgeSpace

The top-level result of processing a directory. It holds the document graph, chunks, relations, and metadata, and serializes to a single portable .indx archive. Beyond its data fields, KnowledgeSpace provides first-class accessors for stats, document filtering, semantic search, and load/save.

Field	Type	Default	Description
manifest	Manifest	default manifest	Schema version, indx version, source root, selected components, embedding model, and embedding dimension.
documents	list[Document]	[]	The document graph. (Stored internally as documents_ with a property shim; the public callable is documents(type=...) below.)
chunks	list[Chunk]	[]	All chunks in the space.
relations	list[Relation]	[]	Graph-level edges (optional mirror of per-object edges).

documents is both data and method

documents is exposed as the serialized data key and as the filtering method space.documents(type=...). In the reference implementation the field is stored as documents_ with a property shim, so the public surface is the callable accessor.

Accessors

space.stats                       # -> SpaceStats
space.documents(type="policy")    # -> list[Document], optionally filtered
space.search("how long is data retained?", k=5)  # -> list[SearchHit]

KnowledgeSpace.load(archive)      # classmethod -> KnowledgeSpace
space.save(archive)               # -> None

Full signatures, behavior, and examples for these methods live in the SDK reference.

SpaceContext

The shared, mutable carrier threaded through every pipeline stage. Each stage receives this object and returns the same object, mutated — run(ctx: SpaceContext) -> SpaceContext. Earlier stages populate collections that later stages read; see Pipeline and stages for the full flow.

SpaceContext sets model_config = {"arbitrary_types_allowed": True} so custom stages can attach runtime objects without fighting Pydantic validation.

Inputs

Field	Type	Default	Description
root	Path	— (required)	Path being processed.
seed	int	0	Deterministic seed for ordering, sampling, and test fixtures.
space	KnowledgeSpace	empty space	The accumulated document graph, chunks, relations, and manifest.
parsed	dict[str, ParsedDoc]	{}	02 Parse — doc_id → ParsedDoc.
errors	list[StageErrorRecord]	[]	Non-fatal per-item failures. --strict promotes these to fatal pipeline errors.

Accumulated work

Most stage output is accumulated inside ctx.space:

Field	Type	Default	Description
Collection	Populated by	Notes
---	---	---
ctx.space.documents_	01 Walk onward	Source documents, then enrichment fields.
ctx.parsed	02 Parse	Parser output keyed by document id.
ctx.space.chunks	03 Chunk onward	Retrievable chunks with neighbor ids.
ctx.space.relations	04 Relate	Graph-level edges.
ctx.errors	any stage	Per-item skips and fatal diagnostics.

The pipeline returns ctx.space, which is the KnowledgeSpace that writers serialize and the .indx archive seals.

See also

• Core objects — conceptual overview of how these models relate.
• index.json schema — the on-disk serialized form of the graph.
• Component protocols — the typed interfaces bound into SpaceContext.
• SDK reference — full method signatures for the KnowledgeSpace accessors.