Make Your Monorepo Searchable by an Agent

The user story

As an OSS developer, I want my monorepo’s code, runbooks, and design docs in one relationship-aware space, so “how do I deploy the billing service?” surfaces the runbook and its neighbors instead of random matches.

Priya maintains a monorepo. Services, runbooks, ADRs, design docs, onboarding notes — deep folder structure, hundreds of files. Her first attempt at codebase search was a classic vector store: split every file, embed, query. It returned text, but the text was adrift. A runbook fragment with no folder context. A design doc chunk with no link to the service it describes. The model had no idea that services/billing/README.md and runbooks/billing/deploy.md were about the same thing.

The problem wasn’t retrieval — it was that her pipeline discarded the repo’s structure before retrieval ever ran. The folder tree is the knowledge graph. She just needed a tool that kept it.

What you’ll build

A relationship-aware knowledge space over a monorepo that:

• Preserves the folder tree as a relation graph — sibling, parent, references, and continues edges, not just chunks.
• Lets you filter by document type at query time — ask for runbooks only, or design docs only.
• Rebuilds incrementally with --resume so a pre-commit hook or local loop stays fast.
• Can run structure-only with --no-embed when you want the graph without paying for vectors.

By the end you’ll have a handbook.indx archive, a committed indx.toml, and a query surface your agents can call.

The build

1. Install indx.

   pip install "indx[cloud]"
   export OPENAI_API_KEY="sk-..."

   Fully offline?

   Swap to pip install "indx[local]" and build with --llm ollama:qwen2.5 --embedder bge-m3. Every flag and step below is identical — only the model names change.

2. Build over the repo directory.

   The directory is the input — no manifest, no glob. indx walks the tree, parses every file, chunks it, derives the relation graph from the folder structure and cross-file mentions, enriches each document with a detected type and summary, and embeds everything into one portable archive.

   indx ./repo --out ./ai-ready

   indx ./repo → ./ai-ready
     01 walk    318 files, 42 folders
     02 parse   314 ok, 4 skipped
     03 chunk   2740 chunks
     04 relate  891 relations
     05 enrich  314 documents (openai:gpt-5-mini)
     06 embed   2740 vectors → qdrant, sealed handbook.indx
   done: 2740 chunks, 314 docs, embed_dim=1536  (28.4s)

   That 04 relate 891 relations line is the part a flat splitter never produces. indx derives four relation types from the repo structure:

   • sibling — files in the same folder. services/billing/ and runbooks/billing/ are linked at the folder level.
   • parent — folder lineage. A deep config file inherits relations up to the root.
   • continues — adjacent chunks within the same file, so a long runbook reads as one continuous thread.
   • references — cross-file mentions. When deploy.md references config.yaml, indx traces that edge.

   These four edges are the signal a hand-rolled pipeline throws away at split time.

3. Commit an indx.toml so every contributor’s build is identical.

   Drop this file at the repo root. Every flag you’d pass on the CLI can live here, pinned to the project.

   [parser]
   engine = "docling"
   
   [enrich]
   llm     = "openai:gpt-5-mini"
   vlm     = "none"
   metadata = ["type", "topics", "tags", "summary"]
   
   [embed]
   model = "openai:text-embedding-3-small"
   
   [store]
   backend = "qdrant"
   
   [output]
   format = ".indx"

   Now indx ./repo --out ./ai-ready is the full command — the config is resolved automatically. CLI flags override the file when needed; see the Configuration reference for the full precedence order.

   Config precedence

   code/use() > CLI flag > INDX_* env var > indx.toml > built-in default. Override one key without editing the file by passing it as a flag.

4. Inspect the space before writing any agent code.

   indx inspect shows you the type histogram and a sample of the relation graph — the two things worth verifying before you wire retrieval.

   indx inspect ./ai-ready/handbook.indx

   archive:   handbook.indx
   documents: 314  chunks: 2740  vectors: 2740 (dim 1536)
   store:     qdrant
   
   document types
     runbook      47
     guide        38
     design-doc   31
     policy       18
     reference    91
     code         89
   
   relations (sample)
     sibling     services/billing/README.md  ↔  runbooks/billing/deploy.md
     parent      runbooks/billing/deploy.md  →  runbooks/
     references  runbooks/billing/deploy.md  →  services/billing/config.yaml
     continues   chunk_1042  →  chunk_1043

   That sample is the graph in action: deploy.md is a sibling of README.md, a child of runbooks/billing/, and it references the config file it describes. A flat splitter had none of that.

5. Query restricted to a document type.

   Once the space looks right, run a query. Use --type to restrict results to a single document type — exactly what you want when the question is deployment-specific.

   indx query ./ai-ready/handbook.indx "how do I deploy the billing service?" -k 3 --type runbook

   #1  score 0.91  runbooks/billing/deploy.md  (runbooks/billing · runbook)
       "To deploy billing, run `make deploy ENV=prod` from the service root…"
       neighbors: chunk_1042, chunk_1044
   
   #2  score 0.83  runbooks/billing/rollback.md  (runbooks/billing · runbook)
       "If the deploy fails, run `make rollback` — this reverts the last…"
       neighbors: chunk_1051, chunk_1053
   
   #3  score 0.79  runbooks/billing/smoke-test.md  (runbooks/billing · runbook)
       "After each deploy, hit /health and /metrics — both must return 200…"
       neighbors: chunk_1060, chunk_1062

   All three hits are runbooks in runbooks/billing/ — siblings of each other. Without the relation graph, a flat query for “deploy billing” can surface an unrelated CI config or a design doc that happens to mention billing.

6. Use --resume for fast incremental rebuilds.

   --resume reuses cached stage outputs for every file and config key that hasn’t changed. Only modified files re-run through the pipeline. Ideal in a pre-commit hook or a local edit loop.

   indx ./repo --out ./ai-ready --resume

   indx ./repo → ./ai-ready  (resume)
     01 walk    318 files, 42 folders  (cache hit: 311 unchanged)
     02 parse   7 ok, 0 skipped        (311 from cache)
     03 chunk   58 chunks              (2682 from cache)
     04 relate  12 relations updated
     05 enrich  7 documents (openai:gpt-5-mini)
     06 embed   58 vectors → qdrant, sealed handbook.indx
   done: 2740 chunks, 314 docs, embed_dim=1536  (3.1s)

   Graph-only mode

   Add --no-embed to produce a space with structure and relations but no vectors. Useful for cheap structural inspection or when you want to verify the relation graph before paying for an embed run.

   indx ./repo --out ./ai-ready --no-embed

Why the relation graph changes retrieval

Priya’s first pipeline retrieved text. The indx space retrieves grounded text — every chunk knows its source document, its folder, its detected type, and the chunks next to it.

That changes three things a flat splitter can’t do:

• Siblings surface together. When deploy.md scores highly, its folder-siblings (rollback.md, smoke-test.md) are connected by sibling edges — an agent can widen to them without a second query.
• Type filtering is structural, not heuristic. --type runbook isn’t a keyword filter; it’s filtering on the type field Enrich assigned each document. The same query over --type design-doc returns a completely different set.
• Neighbor chunks close the context gap. Each hit returns neighbor chunk ids. An agent that reads those gets the full runbook section, not an orphaned paragraph.

The Relate stage explains how each relation type is derived and how to tune the cross-reference detector.

You can also drive the same space from Python directly:

from indx import DirectoryPipeline

space = DirectoryPipeline(config="indx.toml").run("./repo", "./ai-ready")
hits  = space.search("how do I deploy the billing service?", k=3)

for hit in hits:
    print(hit.score, hit.source.path, hit.source.type)
    print(hit.chunk.text)
    print(hit.neighbors)

space.documents(type="runbook") returns all runbooks if you want to walk them programmatically.

Rebuild when the repo changes

The archive is a point-in-time snapshot. When files are added or updated, run indx ./repo --out ./ai-ready --resume — it re-derives only what changed. Wire this to a CI step or a pre-commit hook so the space stays current.

What you learned

• indx <dir> --out <dir> turns a folder tree into a relation-aware knowledge space — sibling, parent, continues, and references edges are derived automatically.
• A committed indx.toml pins the build config so every contributor and CI run produces the same archive.
• indx inspect shows the document-type histogram and a relations sample — verify the graph before writing any agent code.
• --type at query time filters by the Enrich-detected document type, so “how do I deploy?” can be scoped to runbooks only.
• --resume makes incremental rebuilds fast enough for a pre-commit hook or local loop.
• --no-embed produces a graph-only space for cheap structural inspection.

Where to go next

Ground an agent in your handbook The same knowledge space wired into an agent framework with one connect() call — cites sources, filters by type.

Relate stage How sibling, parent, continues, and references edges are derived — and how to tune the cross-reference detector.

Configuration reference Every indx.toml key, the full precedence order, and per-stage override patterns.