Skip to main content
While building domain-specific agents — legal, healthcare, sales, infrastructure — dependency on humans to steer your agents becomes the bottleneck on autonomy and accuracy. If you’re still using a markdown / file-system context layer, or pre-injected context, it’s only a matter of time before you hit these walls.

​
The four walls of file-based context

When you operate with 50+ markdown files in your context space — even neatly distributed by folder — the agent tends to wander between documents, pick up unrelated instructions, and drift off task.The obvious reaction is to build a navigational path for every use case and an Obsidian-style document structure with interlinks and references. But unrelated instructions sitting in the same document still pull the agent in a different direction. The structure doesn’t contain the blast radius.
When the task isn’t a repeated workflow but a slightly fuzzy decision-tree navigation that requires a chain of thought, the agent often can’t navigate past a certain point without steering.And once human steering is required, it caps the autonomy, speed, and value your agent delivers inside the workflow the human was trying to hand off.
Finding which reference the agent used to take an action gets harder as you collate more documents. When you can’t show the user the specific piece of context behind a proof-of-work, they can’t trust the agent’s responses.
When context is spread across many markdown files, updates become non-trivial: multiple documents hold related context and there’s no deterministic way to link them or update them accurately. Knowledge goes stale and self-contradicts.

​
The unlock: structured context management

If those problems sound familiar, the biggest unlock is changing how you think about context:
1

Structure over prose

Context should live in JSON / structured documents, not markdown. Agents navigate better when things look like code.
2

Discovery over injection

Your agent shouldn’t be fed context. It should be given meta-information about the available context and take on the responsibility of searching and discovering the right context itself.
3

Explicit correlations

Optional connections and correlations between data points should be a deliberate part of your context design, not an accident of which files sit in the same folder.

​
How we got here

We solved this at large enterprises for infrastructure debugging. Beyond the work on the overall harness, the context handling came down to three moves — each of which is now a primitive in Open Index:
  • Designed entity models for the use case. These became document schemas in an OpenSearch-backed index. → doc_types & schemas
  • Gave the agent a search tool with pre-injected navigational context instead of running RAG or prompt expansion ourselves and shipping the result into the prompt (think agentic search that spans every category of content). → search_brain & navigation_guidelines
  • Incremental update and deletion per doc_type instead of stuffing context into one document and driving updates from it. → storage: file vs index
It took multiple iterations to find the right form factor — so we open-sourced the standard so anyone building on similar lines doesn’t have to rediscover it.

Open Index on GitHub

The open-source context engine for autonomous agents.
This is not the whole harness — it’s one piece of it. Around this context layer, a production agent still needs work like soft gates to avoid early exits and critic sub-agents for in-session evals. See the Agent Guide for how the pieces fit together.

​
Keep reading

Core concepts

The four primitives — doc_type, doc_schema, entity, connector.

Quickstart

Build your first brain in a few minutes.

Reducing context poisoning

The structured-context playbook, in depth.

The agent harness

Where the context layer sits in a full autonomous-agent harness.