Index a Bank's Archive Without Touching the Internet
Devin runs ML infrastructure at a mid-size bank. The archive is ten years of document exports: loan agreements, compliance memos, audit reports, product guides. The data is sensitive and the perimeter is firm — no outbound traffic, ever. A standard indx install would egress chunk text to OpenAI on every build. That’s a non-starter.
The second constraint is harder: a silent skip is a compliance gap. If a file can’t be parsed and the pipeline quietly moves on, an auditor will ask why it’s missing — and Devin won’t have a good answer. Every failure must be loud.
What you’ll build
Section titled “What you’ll build”An offline knowledge space that:
- Never makes a network call — two paths depending on whether you can run Ollama locally or need the absolute zero-dependency core.
- Fails loudly on any parse or skip, so a compliance gap cannot hide behind a warning.
- Self-describes — the resolved config (model names, embedder, store) is recorded into the archive, so any space can be audited or reproduced months later without guesswork.
By the end you’ll have an exports.indx archive that lives entirely on disk, with a manifest.json that proves exactly how it was built.
The build
Section titled “The build”-
Choose your offline path.
indx offers two offline modes. Pick the one that matches your environment.
Path A — Local profile (
indx[local]): Requires Ollama running on the host, but gives you semantic embeddings and LLM enrichment with no cloud calls.Terminal window pip install "indx[local]"# Ollama must be running locally — no API key neededThis installs with
ollama:qwen2.5as the LLM andbge-m3as the embedder (dim 1024). Thedoclingparser is included. No byte leaves the host.Path B — Zero-dependency core (
indx+--offline): No Ollama, no model runtime. Ships withplaintextparser,hashembedder, andjsonlstore — everything needed to build a space with no installs beyond indx itself.Terminal window pip install indx# No extras, no optional deps -
Build the knowledge space — no database required.
The
--store jsonlflag inlines all vectors directly into the.indxarchive. There is no vector database to install, provision, or network to. The archive is a single self-contained file.Path A (local profile):
Terminal window indx ./exports --out ./ai-ready \--llm ollama:qwen2.5 \--embedder bge-m3 \--store jsonl \--strictPath B (zero-dependency core):
Terminal window indx ./exports --out ./ai-ready --offline --store jsonl --strictThe
--offlineflag fixes the pipeline to the offline-safe defaults:plaintextparser,noneLLM (no enrichment stage),hashembedder,jsonlstore. You can pass--store jsonlexplicitly for clarity, but with--offlineit’s already the default.Expected terminal output for Path A:
indx ./exports → ./ai-ready01 walk 2 841 files, 214 folders02 parse 2 841 ok, 0 skipped03 chunk 31 208 chunks04 relate 9 403 relations05 enrich 2 841 documents (ollama:qwen2.5)06 embed 31 208 vectors → jsonl, sealed exports.indxdone: 31 208 chunks, 2 841 docs, embed_dim=1024 (8m 14s)Expected terminal output for Path B (note:
05 enrichis omitted —--llm noneskips the enrichment stage entirely):indx ./exports → ./ai-ready01 walk 2 841 files, 214 folders02 parse 2 841 ok, 0 skipped03 chunk 31 208 chunks04 relate 9 403 relations06 embed 31 208 vectors → jsonl, sealed exports.indxdone: 31 208 chunks, 2 841 docs, embed_dim=<hash-dim> (2m 31s) -
Verify the archive is self-describing.
indx records the full resolved config — including model names, embedder, store, and parser — into
index.jsonundermetadata.configand into the archive’smanifest.json. You can read it back at any time.Terminal window indx inspect ./ai-ready/exports.indx --json{"documents": 2841,"chunks": 31208,"relations": 9403,"embeddings": 31208,"embed_dim": 1024,"types": { "contract": 812, "report": 1104, "memo": 491, "guide": 434 },"bytes_source": 4831200194}The
manifest.jsoninside the archive containsmetadata.configwith the exact model names used. Hand an auditor the.indxfile and they can reconstruct the build parameters without asking Devin anything. -
Confirm no network calls were made.
Both offline paths are structured to make this verifiable. With
--offline, indx refuses to load any component that could egress. With the local profile, every call goes tolocalhost(Ollama). A network monitor or a simple firewall rule on egress to0.0.0.0/0:443will stay silent for the entire run.If you want belt-and-suspenders assurance during the first run, build behind an egress-blocking firewall rule and watch for failures. With
--strictactive, any unexpected network dependency will surface as a fatal error rather than a quiet timeout. -
(Optional) Load the space from the SDK.
For downstream analysis scripts running on the same air-gapped host:
from indx import KnowledgeSpacespace = KnowledgeSpace.load("./ai-ready/exports.indx")The space loads from disk. No network call, no server to start.
Why this is safer than a hand-rolled pipeline
Section titled “Why this is safer than a hand-rolled pipeline”A typical ingestion script hides its dependencies — the embedder model name lives in a .env file, the store endpoint is an environment variable, and six months later nobody remembers which model version was in production.
indx makes the pipeline explicit and self-archiving:
- Privacy by construction. The default cloud stack egresses chunk text to OpenAI on every build. The local profile and offline core never make an outbound network call. There is no configuration knob to accidentally flip.
--strictas a compliance gate. A silent skip would mean a document is absent from the space with no record of why.--strictpromotes any per-item skip to a fatal error (exit code1), so a partial run cannot be mistaken for a complete one.- The archive proves itself.
metadata.configinmanifest.jsonrecords model names, parser, embedder, and store at build time. Some cloud LLMs aren’t bit-reproducible, but knowing which model was used is the first thing an auditor asks — and indx records it without any extra work. --store jsonleliminates the database. With vectors inlined into the.indxfile, there is no vector database to provision, secure, or network-fence. The compliance boundary is a single file.
What you learned
Section titled “What you learned”pip install "indx[local]"gives you semantic embeddings and enrichment withollama:qwen2.5+bge-m3— fully air-gapped, dim 1024.pip install indx+--offlinegives you the zero-dependency core:plaintext,none,hash,jsonl,.indx— no model runtime required.hashis deterministic and lexical, not semantic.--store jsonlinlines vectors into the.indxarchive — no database to install or serve.--strictturns any per-item skip into a fatal error (exit code1), closing the silent-skip compliance gap.- The resolved config is recorded into
index.jsonundermetadata.configand the archive’smanifest.json, making every space self-describing and auditable withindx inspect --json.