Filesystem-Based Memory for LLM Agents

TL;DR

• Deployed agents (Claude’s memory tool, Claude Code’s memory folder, AGENTS.md/skills conventions) increasingly store long-term memory as a directory tree of markdown files that the agent itself reads, writes, and reorganizes with generic file operations — not a vector store, not a knowledge graph, just files.
• Nobody had tested the two assumptions this “filesystem default” rests on: that an agent can keep a growing file store organized as memories pile up, contradict each other, and go stale, and that organizing it is actually worth the trouble.
• The authors formalize the setup as three roles sharing one file store — a management agent that writes and reorganizes, a search agent that answers questions with citations, and (for task-based settings) an execution agent whose experience becomes new memories and whose retrieved memories become “skills” — and run this across long-conversation QA benchmarks and an embodied-task benchmark, varying store shape, stream size, tool set, and model strength.
• Headline result: organizing the store reliably buys search economy (roughly half the per-query cost once the raw material is large) but no single memory shape wins on answer quality everywhere, and a stronger management model just builds a more elaborate store — not a more useful one. The tool set you hand the agent (a shell vs. structured file-edit functions) reshapes the store almost as much as swapping the underlying model does.
• Within the horizons they tested, stores stay healthy (early memories survive, nothing decays by neglect) and get more useful as they grow — but staying well organized is a capability tax that only the strongest models keep paying as the store scales.

Problem & Motivation

Agents now work across horizons no context window can span — the same coding agent across months of a repo’s life, the same assistant across a user’s whole relationship with a product. Context windows don’t help here: they’re ephemeral, and models degrade on long contexts well before the window is even full (“lost in the middle”). So persistent memory outside the context window has become required infrastructure, not a nice-to-have.

Academic memory research has mostly responded by inventing bespoke representations: OS-style paged context (MemGPT), extracted fact stores (Mem0), temporal knowledge graphs (Zep), self-linking note networks (A-MEM), summary banks (MemoryBank), embedding-organized trees. Each ships with its own purpose-built interface, and each paper studies retrieval quality over its own structure.

Meanwhile, deployed practice quietly picked something else entirely: the filesystem. Coding agents already live in files, so files became the natural place to put extended memory too. Anthropic’s memory tool exposes memory as a directory behind six generic file operations. Claude Code keeps an indexed memory folder of agent-written notes. AGENTS.md and “skills” ship as plain markdown in a repo. None of these are exotic — they’re view, create, edit, delete, rename, grep. The filesystem earns its adoption honestly: it’s inspectable, it’s portable, and it’s operated with tools agents already know how to use. It’s also naturally hierarchical — folders are a taxonomy whose names are its labels, for free.

The problem: this default has never been tested. It rests on two assumptions nobody checked:

1. That an agent can keep a growing store organized — as memories accumulate, duplicate, contradict each other, and go stale, can the agent itself reconcile and reorganize them, or does the store just rot?
2. That organizing pays for itself — is a tidy filesystem actually better to search and more trustworthy than a flat dump, or is curation an expensive detour that a strong-enough search tool makes unnecessary?

Getting this wrong is not a hypothetical risk: prior work has shown that continuously rewriting a memory bank with an LLM can degrade it below having no memory at all. Industry’s current answer is to not trust the working agent’s incremental edits at all — OpenAI’s “dreaming” synthesizes a flat summary in the background, and Anthropic’s “Dreams” periodically rebuilds the whole store from past sessions from outside the agent, precisely because, in Anthropic’s own words, the working agent’s writes stay “local and incremental” and the store degrades between rebuilds. That’s treating the symptom. Whether the agent itself, as it works, can keep its own filesystem memory healthy — that question was still open. This paper is the first attempt to actually answer it.

What’s New

1. A three-role formalization that unifies declarative memory and skills in one store. Before: every memory system was a bespoke pipeline studied in isolation, and “facts” and “skills” were usually different systems entirely. Now: one minimal contract — a management agent that integrates and organizes, a search agent that answers with citations, and an optional execution agent whose trajectories become the chunks the management agent writes and whose queries the search agent serves — maps cleanly onto real deployed harnesses (in a coding agent, one model literally plays all three roles at once).

2. The “taxonomy contract”: five testable principles for what a well-organized file tree even means. Before: “the store is organized” was vibes. Now: sibling distinction, sibling relatedness, parent-child coverage, tree-wide proximity, and structural economy (defined below) give you metrics you can actually compute over a real store and score against.

3. An experimental design that separates model, scale, tool set, and store shape as four independent, controllable axes, run over both conversational memory (LoCoMo, PersonaMem, REALTALK) and procedural/skill memory (ALFWorld) — with a “growth study” that snapshots the store at every step of a 140-task chain, not just at the end. This is what lets them tell the difference between “the store looks different because the model is different” and “the store looks different because there’s more material,” which prior single-snapshot studies couldn’t separate.

4. An empirical characterization that turns “agents should organize their own memory” from an assumption products are shipped on into a conditional, falsifiable finding — with the finding, notably, being more cautionary than the industry narrative: no model they tested converts better organization into better answers, on its own.

How It Works (Technically)

The store. A memory store M is just a set of files in a folder tree. Each file is (path, one-line description, content) — e.g. /memories/people/alice.md, "Alice's diet and travel plans", and the markdown body. Folders carry no content of their own; they’re just shared path prefixes. Every file opens with YAML frontmatter (name, description, optional metadata.type: skill), because that’s what directory listings and search results show before an agent opens the file — the frontmatter is the searchable surface. Inside a file, markdown headings continue the same taxonomy below the file level. Facts carry inline source locators like [S6T5] (session 6, turn 5) so every claim can be traced back to where it came from, and repeated content is cross-referenced (see /memories/trips/...) rather than copy-pasted.

The taxonomy contract (what “organized” means). Both the conversational and skill management prompts encode the same five properties, which the paper adopts as its working definition of a well-organized tree:

• P1 — Sibling distinction: items under one folder can be told apart by name (or name+description) alone, without opening them.
• P2 — Sibling relatedness: things that share a parent actually belong together.
• P3 — Parent-child coverage: a folder’s contents fall entirely within what its name promises, and everything in that topic lives under it — so descending the tree narrows the search without losing the answer, and an exhausted subtree really is a dead end.
• P4 — Tree-wide proximity: more-related content sits closer together in the tree, wherever it lives.
• P5 — Structural economy: depth is added only where it helps routing to a fact; structure for structure’s sake is overhead.

These aren’t just prose — Section D.4 of the paper turns them into computable metrics (tree depth/fanout, a “distance mirrors relatedness” correlation, a “scope leakage” percentage for content that lexically belongs under a different folder than the one it’s filed in), so a real store’s organization quality can be scored, not just eyeballed.

Three roles, one contract each.

Management agent — integrates each new chunk into the store:

M_t = m(instruction, chunk_t, M_{t-1})      with M_0 = ∅

Plain English: it’s a function that takes the current store, one new piece of content, and an instruction, and returns the next version of the store. Feed it a stream of chunks one at a time and you get a trajectory of stores, M_1, M_2, ..., M_T. Nothing stops it from rewriting, splitting, merging, moving, or deleting anything already there — organization is part of the job, not a side effect.

Search agent — answers a question against a frozen store:

(answer, citations) = s(instruction, query, M)

The citations are file paths (optionally with line/section ranges) that must actually support the answer — this is graded separately from correctness, so a right answer with a fabricated citation still fails attribution. Search is read-only in intent: whatever store the search agent reads must be the exact store the answer gets graded against.

Execution agent (skill setting only) — attempts a task given a set of retrieved skill files, returns a trajectory and a success flag:

(trajectory, success) = e(task, retrieved_skills)

How the roles compose for skills (the interesting loop):

for each task τ_i in order:
    Γ_i        = search(retrieval_instr, τ_i, M_{i-1})     # fetch skills relevant to this task
    ξ_i, z_i   = execute(τ_i, Γ_i)                          # attempt it, using ONLY those skills
    M_i        = manage(curation_instr, render(ξ_i), M_{i-1})  # distill the attempt back into the store

Task i is attempted using a store built only from tasks 1..i-1 — it never sees the future. That’s what makes later-task performance a real measurement of what earlier experience actually transferred, not leakage.

Six memory variants (the “shape” axis being compared):

The “Reorganized store” row hides an important finding on its own: asked only to restructure, the model’s default behavior quietly drops detail while it rewrites (labeled Reorg. (condense)); adding one instruction — “keep every fact” — mostly fixes it (Reorg. (preserve)). That single-instruction gap is used throughout the paper as a clean measurement of the model’s own compression tendency versus an explicit override of it.

The tool harness. Agents never touch the store directly — every access goes through a tool set. The management agent writes through six file operations mirroring Anthropic’s deployed memory tool (view, create, str_replace, insert, delete, rename) plus regex search. The search agent, deliberately, gets a non-semantic, filesystem-native read set (view, regex search, table-of-contents, section-read) — no ranked embedding search — specifically so that it has to navigate whatever organization actually exists in the store, instead of a smart retriever papering over bad organization. A separate “harness study” swaps in BM25 ranked search or a raw bash shell to measure how much the tool choice alone moves things.

Architecture & data flow

flowchart LR
  EA[Execution agent<br/>attempts tasks / converses] -->|trajectory or dialogue chunk| MA
  subgraph Store["One memory filesystem"]
    FS[(/memories/<br/>folders · files · headings)]
  end
  MA[Management agent<br/>integrate · reorganize · maintain] -->|writes/edits| FS
  Q[Query or task] --> SA[Search agent<br/>navigate · answer · cite]
  FS -->|read-only| SA
  SA -->|answer + citations, or retrieved skill files| EA
  H[["Tool harness (interchangeable)<br/>file ops · shell · BM25 search"]] -.governs access.-> MA
  H -.governs access.-> SA

The same PersonaMem conversation, organized by three different management-agent models. Click a model to see where its hierarchy lives — in folders, in files, or inside markdown headings. Weaker and stronger models don't just organize better or worse; they put the tree in different places entirely.

The core loop, simplified

This is the management agent’s job in miniature — the one function that, called repeatedly over a stream, produces everything the paper studies:

# store: dict[path] -> {"description": str, "content": str}
# tools: view, create, str_replace, insert, delete, rename, grep — the ONLY way in

def manage(instruction, chunk, store, llm_with_tools):
    """One call = one step of Equation 1. The agent decides both
    WHAT to record and HOW to file it, in the same pass."""
    context = render_store_listing(store)          # paths + descriptions only, like `ls -la`
    plan = llm_with_tools(
        system=instruction,                          # encodes the 5-principle taxonomy contract
        user=f"New content to integrate:\n{chunk}\n\nCurrent store:\n{context}",
    )
    # plan is a sequence of tool calls the model chooses to make: e.g.
    #   view("/memories/people/alice.md")
    #   str_replace("/memories/people/alice.md", "vegan", "vegetarian since May 2026 [S6T5]")
    #   create("/memories/trips/2026-05-tokyo/itinerary.md", ...)
    for call in plan.tool_calls:
        store = apply(store, call)                   # each call mutates or reads the store
    return store                                      # this is M_t; loop again with the next chunk

The whole paper is a controlled experiment over what happens when you vary: what instruction says (condense vs. preserve), what chunk looks like (dialogue slice vs. task trajectory), what tools apply() exposes, and which model plays llm_with_tools.

Built on Prior Work

Results & Evidence

Setup in one line: gpt-5.4-mini (high reasoning effort) plays management and search agent in the main runs, with a nano→mini→full gpt-5.4 strength ladder in the model-strength studies; grading uses a single frozen judge-model prompt; conversational benchmarks are LoCoMo (158 questions), PersonaMem 32k/128k (32/42 questions), REALTALK (85 questions); the skill benchmark is ALFWorld (140 household tasks).

RQ1 — What do agents actually build when left to organize? All four filesystem variants organize by subject, but the shape is a signature of the model, not a response to scale. Holding the model fixed and moving from PersonaMem 32k to 128k (5x more material), the store doesn’t shard into more files — it consolidates: folders and files thin out while markdown sections inside those files quadruple (53 → 210 headings). Holding scale fixed and varying only the model tells the sharper story: the same 128k-token conversation becomes 122 files in 12 folders under a small model, 2 files under a mid model, and 105 files in 4 folders nested 7 levels deep under a strong model — an order-of-magnitude swing driven entirely by which model is doing the organizing. The clearest failure mode: asked to reorganize, models silently drop content unless explicitly told “keep every fact.”

RQ2 — Does shape change answer quality, and at what cost? No shape wins everywhere. The cheapest structured variant (an LLM just sorting existing session files into folders, without touching their content) is the single most consistent winner across benchmarks — beating the fully agent-curated store on two of four benchmarks. Where organization does pay unambiguously is search cost, and it scales with the size of the raw material: on the two PersonaMem tiers (the largest, densest stores), curated stores cut per-query search cost by half or more versus a flat verbatim dump; on the smaller conversational benchmarks, cost is near parity regardless of shape. On skills, the winner flips with who’s consuming the memory: a verbatim episode log wins under a strong execution agent (87.1% success) but a distilled, guidance-synthesizing store wins under a weak one (76.4% vs. the log’s 66.4%) — raw transcripts demand an execution agent strong enough to digest them; written-for-the-task guidance degrades gracefully instead.

RQ3 — Does organizing improve with model strength? This is the paper’s sharpest single finding. On conversation, the management agent’s strength buys organizational style, not answer quality — three models spanning an order of magnitude in build effort and store shape land within a 7-point, non-monotonic band on correctness (73.8% nano vs. 66.7% mid vs. 71.4% strong). Meanwhile the search agent’s strength is monotonically, strongly worth points (62% → 71% → 79% moving up the same three-model ladder, holding the store fixed). On skills, the opposite pattern appears: writing is a threshold, not a slope — the bottom two management models tie statistically (a 114-file sprawl vs. a 45-file store, same score), while the strongest model jumps 13 points by distilling 16 dense, genuinely transferable procedure files instead of just filing more of them.

RQ4 — Does a growing store stay useful and healthy? Within the horizons tested (140 chained tasks, one long conversation), yes on both counts. Task success rises across the whole chain at every model, steepest for the weakest execution agent — accumulated experience substitutes for execution-agent capability. Early memories are essentially never lost: files created in the first quarter of a chain survive to the end and are edited in place rather than replaced (stronger management agents show more in-place editing, not less). What does erode with scale is adherence to the taxonomy contract itself — only the strongest management agent holds it roughly flat as the store grows; everyone else’s organization quality slips. And the costs that scale are volume and effort, not price-per-use: a verbatim episode log’s “serve everything” retrieval cost climbs with the store size, while a curated store’s cost stays flat because the store itself stops growing (matures) partway through the chain and gets edited, not extended.

Store size and per-task retrieval cost across a 140-task chain (schematic, redrawn from Figures 4 and 7). The verbatim episode log grows without bound and its retrieval bill climbs with it; the curated store matures — stops growing — about a quarter of the way in, and its cost stays flat for the rest of the chain.

RQ5 — Does the tool set matter? Adding a tool (e.g., bolting ranked BM25 search onto the existing file tools) changes agent behavior but not outcomes — statistically a tie. Replacing the tool set entirely does move outcomes, and the direction flips by setting: on long dialogue, swapping in a raw bash shell makes the model shard memory into many more, smaller files, tying on answer quality; on skills, the same shell swap makes the model consolidate harder than any other tool set tried, and it wins outright (82.9% — the best score the mid-strength model produced under any configuration). The tool set is a real design lever over what your agent’s memory ends up looking like, independent of the model.

Caveats the authors are explicit about: benchmarks are small (32–158 questions per cell), a repeat-judging experiment found the judge itself moves ±1.3 points (about two questions) on a fixed cell, several “clean re-runs” were needed after prompt wording bugs were found mid-study, reasoning-model builds carry real run-to-run shape variance (one rebuild differing only in a corrected tool description moved a store from 2 files to 29), the model-strength grid is run on a single 42-question conversation, and every horizon tested tops out at one long conversation or 140 tasks — genuinely long-term (months-scale) accumulation is explicitly flagged as untested.

How You’d Use It

If you’re standing up agent memory for your own agent — a support agent that remembers a user across months, a coding agent that accumulates project knowledge, an ops agent that logs and reuses runbooks — this paper is a design checklist, not a library to import:

• Don’t assume curation pays. If your memory corpus is going to stay small (a few hundred KB), a verbatim dump plus a decent search tool is very likely to match or beat an agent-curated store, at zero build cost and comparable answer quality. Only invest in a management agent once the corpus is large enough that search cost is actually a line item.
• Match the store’s format to who reads it, not just to “best practice.” If the consumer is a strong model (Claude/GPT-tier), a raw log of what happened is fine and even wins. If the consumer is a cheaper/weaker model (a fast, cheap sub-agent, or an on-device model), distilled, task-specific guidance is worth the curation bill — it degrades gracefully where raw logs cause the weak model to take invalid actions at 2x the rate.
• The tool set is a cheap lever you control directly, independent of which model you’re paying for. Want a more consolidated, dense store on a task/skills use case? Point the agent at a shell instead of structured file-edit functions. Want fine-grained, browsable shards for a knowledge-base use case? Structured file tools tend that way.
• Bake in the “preserve every fact” instruction from day one if you ever let a management agent reorganize an existing store — the default behavior when a model rewrites content is silent, unannounced compression. This is a one-line prompt fix for a real, measured failure mode.
• Track the taxonomy contract, not just file counts. “My agent made 40 files” tells you nothing about quality. The five principles (sibling distinction/relatedness, parent-child coverage, tree-wide proximity, structural economy) are cheap to check with a periodic LLM-as-judge pass or the paper’s own metrics, and they’re what actually degrades as a weaker management model’s store grows.

Build Your Own (Minimal Recipe)

You can build a toy version of this whole apparatus in an afternoon — most of the machinery is prompting and file I/O, not modeling.

Components:

1. A store: a folder on disk, one markdown file per memory, each starting with ---\nname: ...\ndescription: ...\n---.
2. Seven tools for the management agent: view(path), create(path, content), str_replace(path, old, new), insert(path, line, text), delete(path), rename(old, new), grep(pattern). Anthropic’s own memory tool spec is the reference implementation to copy the schemas from.
3. A management system prompt encoding the five taxonomy principles verbatim, plus (critically) an explicit “never drop a recorded fact when you reorganize” instruction.
4. Four read-only tools for the search agent: view, grep, table_of_contents, section_read — resist the urge to give it embedding search on day one; forcing it to navigate the real structure is what makes organization matter and lets you evaluate whether your management prompt is actually producing something navigable.
5. A citation requirement in the search prompt: every answer must name the file/section it came from, and you grade that separately from correctness (an unsupported right answer is still a bug).

Build order: dump-and-search first (zero-cost baseline, gets you an eval harness) → add the management agent with the taxonomy prompt → add the “preserve every fact” guard → only then experiment with tool-set variants (shell vs. structured ops) and model strength.

The genuinely hard parts: (1) writing a management prompt that resists silent condensation without just becoming “never delete anything” (which creates its own bloat problem) — the paper’s own fix is one sentence, but tuning it for your domain will take iteration; (2) building a judge that separately scores correctness and attribution reliably enough to trust — the paper found their judge alone contributes about ±1.3 points of noise per cell, so budget for judge calibration on held-out examples before trusting any A/B result.

Reach for: any tool-calling LLM API for the management/search agents (the paper uses GPT-5.4 variants but nothing here is model-specific), a simple BM25 library (rank_bm25 in Python) if you want the chunk-retrieval baseline, and a long-conversation benchmark like LoCoMo if you want to eval against something public rather than building your own test set.

How to Improve It

1. Adaptive shape instead of one fixed strategy. The paper shows dump-and-search wins on small stores and curation wins on large ones — a real system could monitor store size/search-cost and switch strategies past a threshold, rather than committing to “always curate” or “never curate” up front.
2. Let the search agent’s toolset include semantic search, but keep a navigable-structure fallback. The paper deliberately withheld embedding search from the search agent to expose organization differences; a production system doesn’t need that constraint — the open question is whether hybrid (structure-aware + semantic) search erases the shape-sensitivity findings here, or whether the same patterns hold.
3. Test the months-scale horizon the paper explicitly flags as open. Everything here tops out at one conversation or 140 chained tasks. Whether taxonomy adherence keeps eroding, plateaus, or catastrophically collapses over a genuinely long-lived store (a year of a coding agent’s memory) is unmeasured — and is exactly the regime real deployed memory tools operate in.
4. Build a cheap automatic taxonomy-contract checker and feed it back into the management agent as a self-critique loop, rather than only measuring it after the fact. The paper’s five principles are already operationalized as metrics (Section D.4) — turning that into a live signal the management agent reads before finishing a write pass is a natural next step, and directly attacks the “only the strongest model holds organization” finding.
5. Separate the “what to write” and “how to file it” decisions explicitly, rather than one model doing both in one pass. RQ3’s cleanest result is that reading and writing decouple in capability — a system that routes content decisions to a strong model and filing decisions to a cheaper structural pass (closer to the “Foldered sessions” baseline, which punched above its weight) might beat either extreme on cost-adjusted quality.

Glossary

• LLM agent — a model that can call tools (read/write files, search, execute code) in a loop rather than just answering once.
• Tool harness — the specific set of tools/functions an agent is allowed to call; changing the harness changes what the agent can do without changing the model.
• Filesystem-based memory — long-term agent memory stored as a folder of files (usually markdown) rather than a database, vector store, or graph.
• Management agent — the role in this paper that writes new content into the store and keeps it organized.
• Search agent — the role that answers questions by reading the store and citing its sources.
• Execution agent — the role that actually does tasks; its experience feeds the management agent, and it consumes what the search agent retrieves.
• Taxonomy contract — the paper’s five-principle definition of what “well organized” means for a file tree (see How It Works).
• BM25 — a classic ranked keyword-search algorithm (the engine under most “search this document” features before embeddings became common); used here as the retrieval baseline.
• RAG (retrieval-augmented generation) — answering by first retrieving relevant chunks of text and feeding them to the model, instead of relying on the model’s own trained-in knowledge.
• Chunk — one unit of incoming content (a slice of dialogue, or a rendered task trajectory) that the management agent integrates in one step.
• Frontmatter — the ----delimited YAML block at the top of a markdown file (here: name + description) that search/listing tools see before opening the file.
• Judge model — a separate LLM used to grade answers against a gold reference, standing in for human evaluation at scale.
• Sign test — a simple statistical test (here: on paired per-question wins/losses between two variants) used to say whether an observed gap is likely real versus noise.
• LoCoMo / PersonaMem / REALTALK — three long-conversation question-answering benchmarks used to test conversational memory.
• ALFWorld — a text-based embodied household-task benchmark, used here to test procedural/skill memory.
• Reasoning effort — a model-configuration setting (used here for GPT-5.4-family models) controlling how much internal reasoning the model does before answering; higher effort costs more tokens.
• Guidance synthesis (GS) — a skill-setting variant where the search agent writes fresh, task-specific instructions from the store instead of just handing back raw stored files.