Everything is Context: Agentic File System Abstraction for Context Engineering

TL;DR

LLM agents live or die by what you put in their context window, but today that context is assembled by scattered, throwaway code: a RAG call here, a tool description there, a chat-history blob shoved in front of the prompt. Nothing is traceable, governed, or reusable across agents. This paper borrows the old Unix idea that “everything is a file” and applies it to context: memory stores, knowledge graphs, MCP tool servers, scratchpads, and human corrections are all mounted into one hierarchical namespace (/context/memory/, /context/history/, /context/tool/) with uniform list/read/write/search operations, metadata, and access control. On top of that file system they define a three-stage Context Engineering Pipeline — Constructor (select + compress what fits in the token window), Updater (stream it into the model and refresh it mid-reasoning), Evaluator (check the output, write verified facts back, escalate to a human when unsure). It’s implemented and open-sourced in the AIGNE framework, with two working demos: a chatbot with persistent SQLite memory, and an agent that talks to GitHub by treating GitHub’s MCP server as a mounted directory. There are no benchmarks — this is an architecture paper proposing a substrate, not a result paper proving it beats anything.

Problem & Motivation

The pain is concrete and you have probably felt it. You build an agent. To make it useful you need to feed it: the user’s history, relevant documents (RAG), tool definitions, a scratchpad for intermediate reasoning, maybe a long-term memory of facts about the user. Each of those is wired in differently — a vector DB client, a prompt template, a JSON tool schema, a session variable. The result is what the authors politely call “fragmented, transient artefacts.” In practice it means:

• No traceability. When the agent says something wrong, you can’t reconstruct which context elements produced it, because the assembled prompt was built on the fly and thrown away.
• No governance. Access control, retention, and provenance are afterthoughts. Agent A can accidentally see Agent B’s memory because isolation is bolted on per-feature, not architectural.
• Context rot and knowledge drift. Memory grows, fills with near-duplicates, goes stale, and silently degrades retrieval quality over time.
• No reuse. Swap your vector store for a knowledge graph and you rewrite the integration glue everywhere it touched.

Frameworks like LangChain, AutoGen, mem0, and Letta each solve pieces (memory, tool orchestration, retrieval) but, the authors argue, treat memory/retrieval/tools as independent components rather than a coherent, governed infrastructure. The deeper issue is architectural: a foundation model is a subsystem with a hard, bounded working memory (the token window). That constraint propagates up through your whole system — everything above it has to decide what to select, compress, and load. Without a principled substrate, every team reinvents that machinery badly.

If you can’t state the pain in one sentence: we have no standard, governed, auditable place to put the stuff agents reason over, so context handling is reinvented per-project and can’t be trusted in production.

What’s New (Core Contribution)

Four contributions, with “before → now” for each:

1. A file-system abstraction for context. Before: heterogeneous context sources (vector DBs, KGs, MCP tools, APIs, human notes) each had their own bespoke access pattern. Now: all of them are projected — via “programmable resolvers” (think GraphQL/OpenAPI-style declarative schema mappings) — into one namespace with uniform list/read/write/search. An agent reads a memory and calls a GitHub tool through the same interface, without knowing either backend’s physical format.

2. A Persistent Context Repository with an explicit lifecycle. Before: “memory” was a single fuzzy concept. Now: it’s split into three layers with distinct persistence semantics — History (immutable, global, append-only source of truth), Memory (agent/session-specific, mutable, indexed views derived from history), and Scratchpad (transient per-task workspace). Data flows History → Memory → (optionally back to) History, with every transition logged as a verifiable state transition.

3. A three-stage Context Engineering Pipeline tied to model constraints. Before: “context engineering” was a vibe — prompt-stuffing plus retrieval. Now: it’s a closed loop of Constructor → Updater → Evaluator, explicitly derived from three architectural constraints of GenAI models (bounded token window, statelessness, non-deterministic output). The Constructor even emits a context manifest — a JSON record of what was selected, excluded, and why — turning prompt assembly into a reproducible artifact.

4. Human-in-the-loop as a first-class architectural element. Before: human review was a wrapper around the system. Now: human corrections are stored as explicit context files (/context/human/) and triggered by the Evaluator when confidence is low — so tacit human knowledge becomes part of the auditable context base, not an external patch.

Honest read on novelty: the individual ideas (LLM-as-OS, memory hierarchies à la MemGPT, RAG, manifests) are not new. The genuine contribution is the unification — making “everything is a file” the single organizing principle so that governance, traceability, and composability fall out of the abstraction rather than being added per-feature. It’s an architecture/SE-principles paper, and should be judged as one.

How It Works (Technically)

There are two layers stacked on each other: the file system (storage + uniform interface) and the pipeline (the runtime loop that uses it). Let’s go through both, then trace one request end to end.

Layer 1 — The file system as context infrastructure

The file system applies five classic software-engineering principles to context. These aren’t decoration; each maps to a concrete mechanism:

• Abstraction. A uniform file interface hides whether a resource is a knowledge graph, a vector store, or a human note. Because it’s schema-driven, REST/OpenAPI endpoints, GraphQL types, and MCP tools get auto-projected into the namespace — no per-integration code. This matters for the token window too: you don’t hard-code 40 verbose tool descriptions into the prompt; the agent discovers tools by listing a directory.
• Modularity / Encapsulation. Each resource is a mounted component with metadata and a minimal operation set. Swap a relational DB for a vector store and nothing else changes. New sources mount dynamically, Unix-style.
• Separation of concerns. Non-executable files (config.yaml, results.csv) are data; executable files (analyser.py, simulate.sh) are tools. The agent knows to read one and invoke the other. Governance (access control, logging, metadata) is a separate layer from retrieval logic.
• Traceability / Verifiability. Every read/write — by agent or human — is logged as a transaction. You can reconstruct exactly what context existed at any point.
• Composability / Evolvability. A consistent namespace + interoperable metadata schema lets elements be combined without glue. A plugin architecture lets new backends (full-text indexers, vector DBs, KGs) mount without touching anything else.

One sharp idea worth flagging: files and directories can carry meta-defined actions — callable behaviours (summarise, validate, sync) attached to a node. A file isn’t just bytes; it’s an active node the agent can execute through the file interface. That’s the bridge between “data” and “tool” collapsing into one addressable space.

Layer 1.5 — The Persistent Context Repository (the memory lifecycle)

LLMs are stateless: end the session, lose everything. The repository fixes this with three components that differ precisely in persistence semantics:

The flow: a raw interaction is appended to History. Summarisation + embedding + indexing transform those raw records into Memory (optimized for retrieval). During reasoning, the agent jots intermediate work to a Scratchpad; when the task ends, useful scratch may be promoted into Memory or archived to History — closing the loop. Every transformation (history→memory, scratchpad→memory) is a logged, versioned state transition carrying creation context, ownership, and lineage.

The paper also gives a useful taxonomy of memory types along three axes — temporal (how long it lives), structural (token-level vs. fact-level vs. summary-level), representational (raw text vs. vector vs. triples vs. summary). Concretely: Episodic (session summaries), Fact (atomic facts as key-value/triples), Experiential (observation-action trajectories), Procedural (tool/function definitions), User (preferences/profile). Multiple types coexist under the namespace. This is the same conceptual ground MemGPT and mem0 cover, organized into one schema.

Layer 2 — The Context Engineering Pipeline

The pipeline exists because of three design constraints of the model layer. There’s no heavy math here — the one quantitative fact is that self-attention is O(n²) in sequence length n, so doubling your prompt roughly quadruples compute cost. Translation for building: long context isn’t just capped, it’s expensively capped, which is the economic reason to compress aggressively rather than dump everything.

The three constraints and what each forces:

1. Token window (e.g., 128K GPT-5, 200K Claude Sonnet 4.5) → you must select, compress, and incrementally stream.
2. Statelessness → you must keep an external repository; and because it grows with duplicates, you must deduplicate/consolidate.
3. Non-determinism (temperature-driven sampling means same prompt ≠ same output) → you must persist input/output pairs + provenance for audit and replay.

The pipeline is a closed loop of three components:

Context Constructor — selection + compression. On a new prompt, it queries file-system mount points (/context/memory/, /context/tool/), uses metadata (recency, provenance) to infer relevance, and trades off completeness vs. boundedness (cover everything relevant vs. respect the token budget and cost). It compresses via summarisation/embedding/clustering, aligns to the model’s prompt schema, and emits a context manifest — a JSON record of selected/excluded elements, their ordering, and estimated token contribution. The manifest is the reproducibility artifact: it’s why this prompt looked the way it did.

Context Updater — delivery + refresh. It moves constructed context into the bounded reasoning space and keeps three things in sync: the token window, the repository state, and the live dialogue. Three loading modes: (a) static snapshot — one-shot injection for a single task; (b) incremental streaming — load more fragments as reasoning unfolds; (c) adaptive refresh — replace stale/irrelevant fragments in response to model feedback or human input. In multi-agent settings it enforces isolation so one agent’s context can’t leak into another’s. Every load/replace is a logged metadata event (timestamp, source path, reasoning ID) for replay.

Context Evaluator — verify + write-back + escalate. After the model responds, it checks the output against its source context and provenance metadata to catch hallucinations/contradictions/drift (semantic comparison, factual-consistency checks, cross-referencing). Metrics (confidence scores, factual alignment, human-override rate) are recorded as metadata. Verified outputs become structured memory written back to the repository (with createdAt, sourceId, confidence, revisionId for audit/rollback). When confidence is low or contradictions appear, it triggers human review, and the human’s annotations are stored as explicit context files in /context/human/.

Architecture & data flow

flowchart TB
  subgraph FS["Agentic File System (uniform namespace)"]
    direction LR
    HIS["/context/history/<br/>immutable, global"]
    MEM["/context/memory/agentID<br/>indexed views"]
    PAD["/context/pad/taskID<br/>scratchpad"]
    TOOL["/context/tool/ + /modules/*<br/>MCP servers, APIs"]
    HUM["/context/human/<br/>annotations"]
  end

  Q[User prompt] --> CON
  subgraph PIPE["Context Engineering Pipeline"]
    CON["Constructor<br/>select + compress<br/>→ manifest.json"]
    UPD["Updater<br/>inject / stream / refresh"]
    EVAL["Evaluator<br/>verify · write-back · escalate"]
  end

  FS -- "list/read/search" --> CON
  CON --> UPD
  UPD --> LLM["LLM<br/>bounded token window"]
  LLM --> EVAL
  EVAL -- "verified facts (write)" --> MEM
  EVAL -- "raw trace (append)" --> HIS
  EVAL -- "low confidence" --> HUM
  HUM -- "human knowledge" --> FS
  PAD -- "promote on task end" --> MEM
  HIS -- "summarise + index" --> MEM

Schematic of one reasoning cycle: watch context flow from the mounted file system through Constructor → Updater → token window → Evaluator, then write verified facts back to memory (or escalate to a human). Click to advance the loop step by step.

Why the Constructor exists: a fixed token budget you have to pack. Drag the budget slider and watch which context items the Constructor keeps (by relevance × recency) vs. drops. The quadratic cost curve shows why "just use a bigger window" isn't free.

The algorithm, simplified

The “algorithm” is the closed loop. Here is the central idea as runnable-looking pseudocode — the part that makes this paper different is that everything is reached through one file interface and every write is logged with lineage.

# One reasoning cycle of the Context Engineering Pipeline.
# afs = the Agentic File System: list/read/write/search over a uniform namespace.

def reasoning_cycle(prompt, afs, agent_id, token_budget):
    # --- CONSTRUCTOR: select + compress what fits the window ---
    candidates = []
    for path in ("/context/memory/" + agent_id, "/context/history/", "/context/tool/"):
        for node in afs.list(path):
            meta = afs.stat(node)                       # recency, provenance, access scope
            if authorized(meta, agent_id):
                candidates.append((relevance(prompt, meta) * recency(meta), node, meta))

    candidates.sort(reverse=True)                       # most relevant/recent first
    selected, used = [], 0
    for score, node, meta in candidates:
        chunk = compress(afs.read(node))               # summarise / embed / cluster
        if used + tokens(chunk) > token_budget:
            continue                                    # boundedness beats completeness
        selected.append(chunk); used += tokens(chunk)
    manifest = {"selected": [n for _, n, _ in candidates], "used_tokens": used}  # reproducibility

    # --- UPDATER: deliver into the bounded window (here: static snapshot) ---
    context = align_to_prompt_schema(selected)
    afs.append("/context/history/", {"event": "inject", "manifest": manifest})

    # --- LLM call (stateless, non-deterministic) ---
    output = llm(prompt, context=context)

    # --- EVALUATOR: verify, write back, or escalate ---
    confidence = factual_consistency(output, selected)
    if confidence < THRESHOLD:
        afs.write("/context/human/", {"need_review": output, "ctx": manifest})  # human-in-loop
    else:
        afs.write(f"/context/memory/{agent_id}/fact/",                          # verified -> memory
                  {"value": extract_facts(output),
                   "createdAt": now(), "sourceId": manifest, "confidence": confidence})
    afs.append("/context/history/", {"output": output, "confidence": confidence})  # always trace
    return output

The whole thing fits in a screen because the file system hides the heterogeneity. afs.read over a vector store, a KG, or an MCP tool looks identical to the loop.

Built on Prior Work

Results & Evidence

Be clear-eyed here: there are no benchmarks, no metrics, no comparison against baselines. This is a design/architecture paper. The “evidence” is two exemplars (proof-of-feasibility demos) implemented in the open-source AIGNE framework:

• Exemplar 1 — memory-enabled chatbot. A few lines mount AFSHistory and UserProfileMemory backed by a SQLite file (file:./memory.sqlite3). Each dialogue round appends to memory and is auto-incorporated into later reasoning, giving stateful conversation with no manual state management. It demonstrates the History/Memory layers and the declarative mount API.
• Exemplar 2 — GitHub via MCP. The official GitHub MCP server (run as a Docker container) is mounted as an AFS module at /modules/github-mcp. The agent then calls afs_exec on /modules/github-mcp/search_repositories or /list_issues — interacting with GitHub as if browsing files. This demonstrates the “any MCP server becomes a mounted directory” claim.

What the evidence does establish: the abstraction is implementable, the API is ergonomic (mounting is genuinely a few lines), and MCP tools really do collapse into the file interface. What it does not establish: that this improves accuracy, reduces hallucination, scales to large memory bases, controls cost, or beats mem0/Letta/LangChain on any task. Claims like “verifiable,” “traceable,” and “reduces context rot” are architectural affordances the design enables, not measured outcomes. Treat the paper as a well-argued blueprint, not a validated result.

How You’d Use It

For an AI services company, this is less a product to adopt and more a reference architecture for how you build every client agent — and a story you can sell.

• Standardize your agent stack around a context namespace. Instead of bespoke memory + RAG + tool glue per client, every engagement uses the same /context/{history,memory,pad,tool,human} layout. Onboarding new engineers and new backends gets cheaper because the interface is fixed.
• Sell auditability as a feature. Regulated clients (healthcare, finance, legal) keep asking “why did the AI say that?” The context manifest + immutable history is a direct answer: you can replay any decision with the exact context that produced it. That’s a differentiator most agent shops can’t offer.
• Governance and multi-tenant isolation. The sandboxed-mount + access-control model is exactly what you need when one platform serves multiple clients or one client runs multiple agents that must not see each other’s data.
• MCP-as-mount is the practical quick win. If you already use MCP, the “mount any MCP server as a directory” pattern unifies tool access and gives you logging/governance over tool calls for free.
• Human-in-the-loop write-back as a managed service. The /context/human/ escalation path is the skeleton of a “human review queue” offering — low-confidence outputs route to a reviewer, corrections become first-class memory, the agent improves. That’s a recurring-revenue ops service, not a one-off build.

Realistic framing: AIGNE is a real open-source framework (TypeScript/Node, by ArcBlock) you could pilot. But you don’t need AIGNE — the valuable part is the pattern, which you can implement on whatever stack you already run.

Build Your Own (Minimal Recipe)

You can capture ~80% of the value without the full framework. Smallest useful version:

1. A virtual FS interface. Define list / read / write / search / stat over a backing store. For a weekend version: a directory tree on disk + SQLite for metadata. stat returns {created_at, source, owner, access_scope, type}.
2. Three roots with the right semantics. history/ = append-only log (never overwrite). memory/<agent_id>/ = mutable, indexed (add a vector index with FAISS/Chroma or just full-text via ripgrep/SQLite FTS). pad/<task_id>/ = scratch, deleted on task end.
3. Resolvers for external sources. Wrap each MCP server / API behind the same five methods so the agent never special-cases them. This is the load-bearing trick — keep the interface uniform.
4. The Constructor. Score candidates by relevance × recency, greedily pack until you hit the token budget, and emit the manifest JSON. Don’t skip the manifest; it’s the cheapest part and the biggest payoff for trust.
5. The Evaluator. A second LLM call: “Is this output supported by the provided context? Confidence 0-1.” Below a threshold, route to human/; above it, extract facts and write to memory/. Always append the raw trace to history/.

The two genuinely hard parts: (a) memory consolidation/deduplication — without it, memory rots into near-duplicates and retrieval precision craters; budget real effort for a “merge similar facts” job. (b) good relevance scoring under a budget — naive cosine similarity over-retrieves; you’ll iterate on hybrid (semantic + recency + provenance) ranking. Everything else is plumbing.

Libraries to reach for: Chroma/FAISS or SQLite-FTS for memory; ripgrep for content search (AIGNE uses it); the MCP Python/TS SDK for tool mounts; any LLM SDK for the Constructor compression and Evaluator checks.

How to Improve It

The paper is a blueprint, so the improvement surface is wide and testable:

1. Actually benchmark it. The glaring gap. Take a multi-turn agent task (e.g., a long customer-support dialogue or a coding agent), and measure: hallucination rate, task success, token cost, and latency with vs. without the manifest+evaluator loop. Until someone does this, “verifiable/reduces rot” is a hypothesis.
2. Learn the Constructor’s selection policy. Right now selection is heuristic (recency/provenance). Frame it as a bandit or RL problem: reward = downstream task success per token spent, and let the policy learn what to include. This is where RL would genuinely help — the action is “include/exclude context item,” the reward is bounded by the token budget, and the advantage signal is the success delta vs. a cheaper context.
3. Make consolidation a measured component. Add an explicit dedup/merge job and benchmark retrieval precision and storage over a long-running agent — the statelessness section names the problem but offers no mechanism.
4. Stronger evaluation than self-check. “Ask an LLM if the output is supported” is weak and itself non-deterministic. Add entailment models, citation-grounding checks, or ensemble agreement, and report calibration of the confidence scores.
5. Agentic navigation (their own future work) + caching. Let agents build their own indices over the namespace, and cache constructed contexts keyed by the manifest so identical sub-tasks skip re-construction — directly attacking the O(n²) cost the paper flags.

Glossary

• Context engineering — the discipline of capturing, structuring, selecting, compressing, and governing everything you feed an LLM, across its whole lifecycle (vs. prompt engineering, which is just wording one instruction).
• Token window — the fixed maximum number of tokens a model can attend to in one pass (e.g., 128K, 200K). A hard architectural ceiling.
• Self-attention O(n²) — compute and memory cost of attention grows with the square of input length; doubling the prompt ~quadruples cost. Why bigger context isn’t free.
• RAG (retrieval-augmented generation) — fetch relevant documents at query time and stuff them into the prompt so the model answers from them.
• MCP (Model Context Protocol) — Anthropic’s open standard for connecting agents to external tools/data via a server; here, mounted as a directory.
• LLM-as-OS — conceptual paradigm treating the LLM as a kernel that schedules context, memory, tools, and sub-agents like an operating system.
• Mount — Unix-style act of attaching an external resource into the file-system namespace so it’s accessed uniformly; here applied to memory stores, APIs, MCP servers.
• Resolver — a declarative mapping (GraphQL/OpenAPI-style) that projects an external backend’s structure into file-system nodes without changing the backend.
• Context manifest — JSON record the Constructor emits listing what context was selected/excluded, in what order, and its token cost — for reproducibility.
• Provenance / lineage — metadata tracing where a piece of context came from and how it was transformed, enabling audit, replay, and rollback.
• Scratchpad — transient per-task workspace where an agent writes intermediate reasoning; promoted to memory or archived on task end.
• Context rot — gradual degradation of an agent’s memory/context quality over time as it fills with stale or duplicate entries.
• AIGNE — the open-source (ArcBlock) GenAI agent framework where this architecture is implemented; AFS is its Agentic File System module.
• Human-in-the-loop — design where low-confidence model outputs are escalated to a person whose corrections become first-class, stored context.