Meta^n: Recursive Self-Improvement through Emergent Depth

application reads the traces and code of everything below it and writes the next layer, so depth keeps growing without anything ever needing to be protected from self-edits.

TL;DR

Self-improving LLM agents today either only refine their answers (self-refine, Reflexion — no meta-level at all), bolt on one permanently fixed meta-level (evolutionary program search), or let the agent edit its own source code but must freeze part of that editing machinery to avoid corrupting itself — which caps how much real, working meta-depth any of them achieve at roughly 2.5 levels. Meta^n breaks the tradeoff by keeping the meta-operation Ω itself completely fixed — one prompt template, unchanged across every depth and every benchmark — and instead recursing on what Ω reads: each application inspects the execution traces of the whole stack below it plus the code that produced those traces, then writes the next layer as a small strategic pre-process and a library of callable helper functions. Because Ω never changes, it can’t destabilize anything; because its input strictly grows, layer 4 reasons from more information than layer 3 did. Depth isn’t set in advance, it’s set by convergence, and an evolutionary archive searches over many candidate layer chains at once. Across eight very different benchmarks and two model backbones, Meta^n beats prior self-improving agents on every single one, most dramatically on ARC-AGI-2 — a benchmark built to resist memorized skills — where Meta^n is the only system to score above zero. And distinct roles (generic helpers, task routing, correcting earlier mistakes) emerge at different depths on their own, even though no prompt ever tells a layer what role to play.

Problem & Motivation

Picture an agent that writes a script, the script crashes because it imports scipy and the sandbox doesn’t have it, and the task scores 0.0. The standard fix — self-refinement (Self-Refine, Reflexion) — retries variations of the same broken idea: same mechanism generates the script, diagnoses the failure, and repairs it, all in one flat loop. It never asks should I be solving this class of problem differently altogether? That question needs a level above the solver — a meta-level — and that’s where existing systems split into two camps, both with a ceiling:

• Evolutionary program search (FunSearch, AlphaEvolve, OpenEvolve) adds a meta-level search loop around the solver — mutate candidates, keep the winners. But the search loop itself is frozen forever. The solver improves; the thing improving it never does. Realized meta-depth = 1.
• Self-referential agents (Gödel Agent, Darwin Gödel Machine, HyperAgents) let the agent rewrite its own source code, which sounds unbounded. In practice every one of them has to hold some “driver” fixed — Gödel Agent’s action API, DGM’s archive maintenance and parent selection, HyperAgents’ outer evaluation loop — or the agent can corrupt its own scaffolding and the whole thing collapses. That frozen driver caps realized meta-depth (the paper’s term for “the highest level whose behavior actually changes during a run,” as opposed to what the architecture nominally allows) at roughly 2.5.

The dilemma in one sentence: recursing the improver buys you depth only by putting stability at risk, and every prior system resolves that tension by freezing something, which caps the very depth the recursion was supposed to deliver. Meta^n’s bet is that you don’t have to make that trade at all if you recurse on the improver’s input instead of the improver.

What’s New (Core Contribution)

1. A fixed meta-operation recursed on its own growing output, not on itself. Before: systems either freeze the meta-level after one application (evolutionary search) or let the agent edit its own improving machinery and must protect part of it (self-referential agents). Now: Ω — one LLM prompt template, same output schema and parser — never changes at all, at any depth, on any benchmark. What grows is Ω’s input: the trace log and the code stack of every layer built so far. Nothing needs to be frozen for safety because nothing that could destabilize the system is ever edited.
2. First demonstration that meta-depth beyond 2 produces structurally distinct layers, not redundant ones. Depth 2 emits generic, reusable primitives; depth 3 specializes and routes by task type (and starts causing regressions); depth 4+ corrects those regressions. This role progression emerges from accumulated context — no prompt assigns roles.
3. Evolutionary orchestration over entire layer chains, not just candidate answers. The archive holds many complete stacks (different Ω decisions at each depth), scores each on every task, and lets different tasks be won by different chains (archive-best) instead of committing to one chain end to end (best-single-chain). This decouples “how deep” from “how good.”
4. A consolidation guard that makes per-task improvement monotonic by construction. Because deeper layers can regress tasks that were already solved, an alternate orchestration mode targets one task at a time while freezing the archive’s best trace on everything else — a zero-regression guarantee, at the cost of some mean-score headroom.

How It Works (Technically)

Setup. A benchmark gives you N tasks and a scoring function eval(task, script) → [0,1]. Every solver run leaves a trace τ: the script that ran, its stdout/stderr, exit code, score, and any evaluator feedback. A base solver S1 is whatever you already have — one LLM call, or an 8-turn observe/act loop — Meta^n doesn’t care. The goal is a stack of solvers S2, S3, ..., Sn, each one wrapping the solver below it, that maximizes mean score across all tasks. Depth n is not chosen in advance; the stack keeps growing until it stops paying off.

Two phases per layer. Building a layer happens once, offline. Running it happens per task, online.

Build-step — Ω writes the layer. Ω takes four things: the previous depth’s traces across all tasks, the full stack of code every earlier layer wrote ([C2, ..., C_{d-1}]), the task descriptions, and the current depth number d. It returns one thing: Cd = (pre_process, library) — a small Python function that injects strategic context before the solver runs, plus zero or more reusable helper functions. Concretely, Ω’s response is a rationale (its reasoning, in text), a pre_process code block, and zero or more solver_lib:<name> blocks.

Run-step — the wrapper Md executes the layer. When a task comes in at depth d: (1) the outermost pre-process runs first and produces a context string; (2) that context threads inward through every pre-process below it, each one refining it; (3) the base solver sees the fully merged context and returns a script; (4) the union of every layer’s code library gets prepended to that script, with deeper layers overriding by name on collisions; (5) the result runs in a sandbox and produces this depth’s trace. Because Md never touches the inner solver’s internals — it only wraps — the whole stack is just nested function composition: Sd = Md ∘ M_{d-1} ∘ ... ∘ M2 ∘ S1. A layer can never corrupt the layer beneath it; the worst it can do is add a bad wrapper around a good one, which the next layer can then undo.

The subtle but important point: Ω is literally fixed. The same prompt template, output schema, and parser run at depth 2 and depth 6, on a combinatorial-optimization benchmark or a terminal-agent benchmark. The only thing that changes with depth is a formatter sitting above Ω that decides how much of the trace log to show it: at depth ≤ 2, Ω sees raw per-task traces; at depth ≥ 3, it sees a structured summary instead (a failure-pattern breakdown, an effectiveness analysis of the previous layer measured against depth d-2, and a few representative traces) because dumping dozens of raw traces stops being useful once there’s a code history to reason about instead. The paper is careful to call this formatter “a content adapter above Ω, not a separate driver” — it shapes what Ω sees, it never decides what Ω writes.

A concrete trace, start to finish (LawBench charge prediction, scored by F1). This is the paper’s own running example and it’s the clearest way to see why recursing on input beats a flat retry loop.

• S1 (base solver, one LLM call): 0.767.
• Depth-2 Ω looks at S1’s traces, notices a recurring label-formatting mismatch, and writes a fuzzy_match_label() helper function. S2 = M2(C2, S1): 0.807.
• Depth-3 Ω now sees S2’s traces and the code C2 that produced them. It writes a second helper, reconcile_labels(), but couples it to an over-prescriptive pre-process directive demanding an “Exhaustive Legal Analysis” of every plausible charge. The helper is sound; the directive isn’t. S3 = M3(C3, S2): 0.773 — a regression.
• Depth-4 Ω sees S3’s traces and the entire code stack [C2, C3]. Because it can read the code, not just the outcome, it can attribute the regression specifically to the directive rather than the helper. Its rationale says so explicitly (“roll back from Exhaustive Analysis to Structured Analysis as it caused a regression”), and C4 restores the lighter-touch directive while keeping reconcile_labels. S4 = M4(C4, S3): 0.833 — recovered, and net ahead of where it started.

A flat self-refinement loop, however many iterations you give it, cannot perform that depth-4 step — it only ever sees an accumulating log of what happened, never the code that made it happen, so it has no way to tell “the helper was fine, the directive wasn’t.” Ω at depth d≥3 always has strictly more to work with than Ω at depth d-1 did, whenever the previous layer changed anything observable.

Architecture & data flow

flowchart BT
  S1["Base Solver S1"] --> M2["Wrapper M2 + C2"]
  M2 --> M3["Wrapper M3 + C3"]
  M3 --> M4["Wrapper M4 + C4"]
  M4 --> Mn["... Meta-Layer n"]
  Omega{"Ω — one fixed prompt template"}
  Omega -.->|"reads S1 traces, writes C2"| M2
  Omega -.->|"reads S2 traces + C2, writes C3"| M3
  Omega -.->|"reads S3 traces + [C2,C3], writes C4"| M4

The recursive stack in 3D — drag to orbit. Each layer wraps the one below it; Ω (the sphere) is the same fixed operation applied at every level, but what it reads keeps growing.

The worked example, as a pipeline

flowchart TD
  S1["S1 base solver: 0.767"] --> O2{"Ω build-step"}
  O2 -->|"writes C2: fuzzy_match_label() helper"| S2["S2 = M2(C2,S1): 0.807"]
  S2 --> O3{"Ω build-step, reads traces + C2"}
  O3 -->|"writes C3: reconcile_labels() + over-strict directive"| S3["S3 = M3(C3,S2): 0.773 — regression"]
  S3 --> O4{"Ω build-step, reads traces + [C2,C3]"}
  O4 -->|"rolls back the directive, keeps reconcile_labels()"| S4["S4 = M4(C4,S3): 0.833 — recovered"]

What each depth is doing on average, from the paper's own rater-annotated data (596 Ω emissions, code-substrate benchmarks). Rollback is exactly zero at depth 2 — there's nothing to correct yet — and becomes the dominant behavior once there is.

The algorithm, simplified

This is the linear (single-chain) version of the recursion — the evolutionary orchestrator (used for all reported results) just runs many of these chains in parallel and keeps the best trace per task.

def grow_stack(tasks, base_solver, omega, epsilon=0.02, max_depth=10, patience=2):
    """Recurse Omega on its own growing output until it stops finding improvements."""
    stack = [base_solver]                 # stack[0] = S1
    code_history = []                     # [C2, C3, ...] every layer Omega has written so far
    traces = run_all(tasks, stack[-1])
    best_score, stale_rounds = mean_score(traces), 0

    for depth in range(2, max_depth + 1):
        if stale_rounds >= patience:
            break

        # Omega sees traces AND the code that produced them -- a flat self-refine
        # loop only ever sees the traces.
        pre_process, library = omega(traces, code_history, tasks, depth)
        if pre_process is None and not library:
            break                                    # Omega found nothing left to add

        Cd = (pre_process, library)
        Sd = wrap(Cd, stack[-1])                      # Md: composes, never mutates stack[-1]
        new_traces = run_all(tasks, Sd)
        new_score = mean_score(new_traces)

        stale_rounds = 0 if new_score - best_score > epsilon * score_range(tasks) else stale_rounds + 1
        stack.append(Sd)
        code_history.append(Cd)
        traces, best_score = new_traces, max(best_score, new_score)

    return stack, code_history                        # realized depth = len(stack) - 1


def wrap(Cd, inner_solver):
    """Md: compose Omega's new layer around the existing stack. Wrapping, never patching --
    inner_solver's internals are never touched."""
    pre_process, library = Cd
    def solve(task):
        ctx = pre_process(task)                       # this layer's strategic context
        script = inner_solver.solve(task, ctx)         # ctx threads inward through lower layers
        script = library.prepend_to(script)            # deeper layers override by name on collision
        return sandbox_execute(script, task)
    return Solver(solve=solve, libraries=inner_solver.libraries | library)

Built on Prior Work

Results & Evidence

Setup: two backbones (Gemma 4 31B-IT, GPT-5.2), eight benchmark families spanning Python solvers (CO-Bench’s 36 NP-hard problems, AlphaEvolve Math, 4-domain Symbolic Regression, AlgoTune kernel speedups, ARC-AGI-2’s 120 abstract-reasoning puzzles), a bash/Docker terminal-agent benchmark (TerminalBench 2.0, 89 tasks), and prompt-rewrite classification (Symptom2Disease, LawBench). Two prior self-improving systems as baselines: Gödel Agent (self-referential) and OpenEvolve (evolutionary search).

Headline pattern: Meta^n leads every benchmark family on at least one of its two reported estimators (archive-best, the mean of each task’s best score across the whole archive, vs. best-single-chain, committing to one chain end to end), under both backbones. The margin scales with how many distinct ways a benchmark can fail — CO-Bench’s 36 unrelated NP-hard problem shapes give the cross-task code library and per-depth specialization the most to work with, so that’s where the largest, most stable margins show up. ARC-AGI-2 is the categorical result: it was designed specifically to resist skill memorization, both baselines sit near zero, and even Meta^n’s own best single chain only reaches 0.123 — the win comes entirely from the meta-level stack (Ω abstracting transform primitives from traces, the archive composing them across tasks), not from better per-task search.

The ablation is the mechanistic finding worth remembering. Collapsing the stack to depth-1 (no recursion, same orchestration) drops CO-Bench archive-best from 0.845 to 0.714 — a +0.131 gain from recursion alone, and it reproduces on a second backbone (+0.080 on GPT-5.2 CO-Bench) and a second benchmark (+0.158 on GPT-5.2 AlphaEvolve Math). Decomposing where that gain lives: removing just the inter-layer context string (the plain-text ctx threaded between pre-processes) costs −0.094, roughly 72% of the whole recursion gain. Removing just the code-library injection costs −0.020, about 15%. The remainder (~13%) is the recursion machinery itself (beam width, retry, cross-candidate inspiration). In other words: most of what recursion buys is the free-text conditioning passed between layers, not the callable helper functions — a genuinely surprising result if you assumed the code library was the main event.

Caveats worth taking seriously:

• Same model at every layer, deliberately. The base solver and every Ω call use the identical model in every reported experiment, to cleanly isolate “depth helps” from “a stronger model at Ω helps.” That means the realistic deployment case — a strong model at Ω sitting over a cheap base solver — is explicitly untested and flagged by the authors as the natural next experiment.
• It doesn’t help everywhere. AlgoTune (kernel speedup) is the one benchmark where the agentic variant underperforms single-shot: the seed already extracts a pre-optimized kernel contract, and Ω’s extra context over-constrains rather than improves it (most of the loss traces to two FFT kernels specifically). On SWE-Bench, the seed is strong enough that Ω never activates at all — generation-0 wins.
• Cost is real. Agentic mode runs 4–10× the tokens of single-shot (up to 8 observe/act turns per task). A full CO-Bench agentic run is ~36M tokens and ~10 hours wall-clock; the 89-task TerminalBench run took ~28 hours, dominated by Docker startup.
• Rater agreement on the emergent-role analysis is uneven. Two independent LLM raters labeled 596 Ω emissions into 7 role categories; average agreement is κ=0.59, near-perfect on concrete categories (task routing, prompt engineering) but weak (κ=0.30–0.37) on the two most abstract ones. The headline patterns (rollback debuting at depth 3, the tactical-primitives-then-specialization shift) reproduce under either rater alone, but treat the fine-grained percentages as directional.

How You’d Use It

Your harness. This is a wrapper pattern, not a new base model — it slots onto whatever agent or solver you already run. Think of each Ω call as a meta-agent whose “memory” is the trace log and source code of the stack beneath it, and whose “output” is a wrapper agent — strategic pre-prompting plus a growing tool library — around that stack. It’s automatic prompt-engineering and tool-library synthesis, done recursively, orchestrated as evolutionary search over candidate wrapper chains. If you’ve already wired up a multi-agent system (message passing, roles, an orchestration loop), that machinery maps directly onto the archive/parent-sampling/children loop here — you’re not starting from zero.

Your automations and business processes. Where it fits: it wraps a solver you already have — one LLM call or an existing agentic loop — you don’t touch the base agent, you sit a recursive improver on top of it. It needs two things to be worth standing up: (1) a scorable eval harness per task (not always available — you may need to build this first, and it’s often the actual deliverable), and (2) enough task volume and diversity that cross-task patterns are worth mining — 30+ variations of a structurally similar problem (data-cleaning scripts across departments, terminal runbooks across services, classification prompts across product lines) is the sweet spot, per the paper’s own finding that the win scales with “how many distinct ways a benchmark can fail.”

Where it’s a poor fit. A single bespoke one-off task gives the recursion nothing to generalize a library across, and the depth-1 ablation shows recursion’s value shrinks exactly where the inner loop has little room left (AlgoTune, SWE-Bench in the paper’s own results) — don’t reach for this without real task diversity to mine.

Build Your Own (Minimal Recipe)

1. A base solver with a trace contract. Wrap whatever you already have (one LLM call or an agent loop) so it returns {script, stdout, stderr, exit_code, score, feedback} per task.
2. A sandboxed executor. Something that can run LLM-generated Python/bash safely: static analysis first (ast.parse, a blocklist for dangerous imports like os/subprocess/socket, a length cap), then a smoke test (exec in an isolated namespace to confirm a function actually defines a callable) before anything gets prepended to a real solve attempt.
3. The wrapper Md. A composition function, not a patch: run the outer pre-process, thread its context inward, call the inner solver, prepend the merged code library (later layers win on name collisions), execute in the sandbox. This is the piece that guarantees a layer can never corrupt what’s beneath it.
4. Ω itself. One fixed prompt: given recent traces (raw at shallow depth, a summarized digest once there’s a code history worth reasoning about), the accumulated code stack, the task descriptions, and the current depth, return a rationale, a pre-process, and zero or more named library functions.
5. A stopping rule. Keep applying Ω; stop when it returns nothing new, or the score gain over epsilon * score_range plateaus for P rounds, or you hit a max depth. That’s the whole linear version — a working toy in an afternoon if step 2 (sandboxing) is already solved.

The two genuinely hard parts: (a) getting the depth-aware summarization right — raw traces work fine at depth 2 but stop scaling once there’s a code history to reason about, so you need a structured digest (failure-pattern distribution, an effectiveness read on the previous layer) once depth ≥ 3, and it has to stay a dumb formatter above Ω, not a second decision-maker, or you’ve quietly reintroduced the “frozen driver” problem this whole design exists to avoid. (b) sandboxing generated code well enough that layers can compose indefinitely without one layer ever being able to corrupt another — this is exactly what buys you the freedom to never need a protected “driver.” The evolutionary archive (score-weighted parent sampling, K children per parent, per-task best tracking) is an upgrade you add once the linear version works, not a prerequisite.

How to Improve It

1. Put a stronger model at Ω over a cheaper base solver. The paper deliberately used the same model everywhere to isolate the depth effect; the realistic deployment case — cheap solver, expensive improver — is explicitly flagged as untested and is the highest-value next experiment.
2. Structure the conditioning string. The ablation shows the free-text context passed between layers carries ~72% of the recursion gain, far more than the code library. Replacing it with a small typed schema (failure category, confidence, which prior layer IDs it references) instead of prose is a direct, testable follow-up the paper itself suggests, and it would also make layer-to-layer interference detectable programmatically rather than only after a score drops.
3. Predict interference instead of discovering it after the fact. Right now a depth-4 layer detects and undoes a depth-3 regression after it has already happened (41% of chain/task pairs regress at depth 3 in the reported runs). Have Ω check a candidate directive against a held-out subset of tasks before committing it, catching the regression before it ships rather than one layer later.
4. Make the trace-summarization budget adaptive. The depth ≥ 3 structured summary is currently a fixed recipe applied everywhere; on low-diversity benchmarks (prompt-only tasks where every depth just keeps doing prompt engineering) it’s likely overkill. Scaling summary depth to measured task diversity could cut token spend without losing signal.
5. Push past depth 6 on purpose. Reported runs stop between depth 3 and 6 because Ω stops finding improvements, not because of a known hard ceiling — the authors explicitly don’t know yet whether the real limit is the base model’s reasoning capacity at high meta-levels or the context window filling with accumulated layer code. A controlled experiment that forces deeper stacks (disable early stopping, raise the depth cap) would identify which ceiling actually bites first.

Glossary

• Meta-operation (Ω) — the single fixed LLM-prompted procedure applied at every layer; reads the traces and code below it, writes the next layer’s pre-process plus a helper library.
• Meta-layer / depth d — one application of Ω; depth counts how many times the stack has been wrapped.
• Realized meta-depth — the highest level whose behavior actually changes during a run, as opposed to what the architecture theoretically permits — the paper’s yardstick for comparing systems.
• Driver — a component that controls modification in a self-improving system but is itself never modified; the thing prior systems must freeze to stay stable, and what Meta^n avoids needing.
• Execution trace (τ) — the record of one solver run on one task: script, stdout/stderr, exit code, score, evaluator feedback.
• Pre-process — a small function Ω writes that injects strategic context into a task before the solver runs it.
• Code library — the set of reusable helper functions Ω writes at a given depth that the solver may call.
• Wrapper (Md) — the mechanism that composes a new layer around the existing stack without ever touching the inner solver’s internals.
• Archive-best — mean, across tasks, of the best score any candidate chain in the evolutionary archive achieves on that task (a portfolio-style score, not one committed chain).
• Best-single-chain — the score of committing to one complete layer chain end to end.
• Consolidation guard — an orchestration mode where each candidate improves one focus task and inherits the archive’s frozen best on everything else, guaranteeing per-task scores never regress.
• ARC-AGI-2 — an abstract visual-reasoning benchmark built to resist memorized skills; used here as the stress test that separates “better search” from “genuinely new abstraction.”
• Evolutionary archive — a growing pool of candidate layer chains, searched via score-weighted parent sampling plus an exploration bonus for less-extended chains.