HyperAgents: Self-Referential Self-Improving Agents

TL;DR

Most “self-improving” AI systems have a fixed boss: a hand-written meta-mechanism that decides how the system gets modified. The system can only get as good as that fixed boss allows. This paper introduces hyperagents — programs that fold the task-solver and the self-modifier into one editable codebase, so the self-modification procedure is itself a thing that can be modified (they call this metacognitive self-modification). They build this on top of the Darwin Gödel Machine (an evolutionary archive of self-editing coding agents) to make DGM-Hyperagents (DGM-H), which works across coding, paper review, robotics reward design, and Olympiad math grading. The headline finding isn’t just that it improves at tasks (paper review 0.0 → 0.71, beating a hand-tuned baseline) — it’s that the learned habits of how to improve (persistent memory, performance tracking) transfer to brand-new domains it was never optimized on (imp@50 jumps from ~0 to 0.63 on math grading) and compound across runs. The catch: it’s expensive, runs inside heavy sandboxing, and the outer evolutionary loop is still hand-written.

Problem & Motivation

The concrete pain: if you want an AI system that gets better on its own, today you write a “meta agent” — the thing that inspects the base agent and decides what to change. That meta agent is hand-crafted and frozen. Your base agent can therefore only improve within the imagination of whoever wrote the meta agent. Hit the ceiling of the meta agent’s design and you’re stuck.

The obvious fix — add a meta-meta agent to improve the meta agent — just moves the wall up one floor. You’d need a meta-meta-meta agent, and so on forever. This is the infinite regress of meta-levels. The only way out is self-reference: a system that can analyze, modify, and evaluate any part of itself, including the part doing the modifying.

The Darwin Gödel Machine (DGM, 2025) got partway there for coding. It keeps an archive of coding agents; each agent edits its own code; good variants are kept as “stepping stones.” It works because of a lucky coincidence: both the task (coding) and the act of self-modification (editing code) are the same skill. Getting better at coding automatically makes you better at editing yourself. But that coincidence evaporates the moment you leave coding. If your task is grading math proofs or writing poetry, being a great proof-grader does nothing for your ability to refactor your own decision logic. So DGM’s self-improvement engine doesn’t generalize — its instruction-generation step is hand-written for coding and can’t be changed by the system. That’s the bottleneck this paper attacks.

What’s New (Core Contribution)

1. Hyperagents: task agent + meta agent fused into one editable program. Before: the meta-level (how you improve) sat outside the agent, fixed. Now: the meta agent is just more code inside the same repo the agent can edit — so the improvement procedure can rewrite the improvement procedure. This is the structural trick that kills the infinite regress.

2. Metacognitive self-modification. Before: self-improvement = “make the task-solver better.” Now: self-improvement = “make the task-solver better and make the thing that makes things better, better.” The system can invent new ways of analyzing its own failures.

3. Domain-general recursive self-improvement (DGM-H). Before: DGM only compounded inside coding because of the task/self-mod alignment coincidence. Now: because the self-mod mechanism is editable, the alignment assumption is dropped — it works on coding, paper review, robotics reward design, and math grading.

4. Empirical evidence that “how to improve” is a transferable, compounding skill. Before: nobody had shown meta-level learning transfers across domains. Now: hyperagents optimized on paper-review+robotics, when dropped into math grading, are dramatically better at generating improved agents than a fresh start (measured by their new improvement@k metric). The transferable artifacts are concrete: the system spontaneously wrote itself a PerformanceTracker class and a persistent memory store.

What’s genuinely new is #1/#2 (the editable-meta structure) and #4 (transfer of meta-skill). What’s inherited is the evolutionary archive machinery (#3 is DGM extended, not invented).

How It Works (Technically)

The system is two nested loops. Hold these two pictures in your head:

• Outer loop (the DGM evolutionary archive): a population of hyperagents, growing over time. You repeatedly pick a parent, let it modify itself into a child, evaluate the child, and add it to the archive. This is hand-written and frozen in the main experiments.
• Inner act (metacognitive self-modification): when a chosen parent “modifies itself,” it runs its own meta agent code, which emits a code diff that rewrites the hyperagent’s repo — and that diff can target the meta agent itself.

Let’s define the moving parts precisely, because the whole paper lives in the distinction:

• Agent = any computable Python program, optionally calling a frozen foundation model (FM) and tools. Not a neural net being fine-tuned — the FM weights never change. All “learning” happens by editing code, not gradients.
• Task agent = the agent instantiated to solve the actual task (review this paper, grade this proof, design this reward function).
• Meta agent = an agent whose only job is to look at the archive + past evaluations and produce a code diff that creates a new, hopefully-better hyperagent.
• Hyperagent = one repo containing both. Because it’s all Python (Turing-complete) and the agent can edit any file, in principle it can become any computable program.

One trace, start to finish

Take the paper review domain. The initial hyperagent is dead simple: the task agent just pipes the paper text into one FM call and returns whatever comes out. It scores 0.0 on test — it can’t even produce output in the required accept/reject format.

1. Select a parent from the archive (probabilistically — favor high performers that also produced compilable children; Appendix A.2).
2. Parent’s meta agent runs. It gets: the location of the agent’s repo, the location of past eval results, and how many iterations are left. It’s instructed to “modify any part of this codebase to do better in the future.” It makes a free-form decision about what to change. (Contrast: DGM’s meta step has a frozen hand-written prompt that tells it which part to edit.)
3. The meta agent emits a code diff → a new hyperagent (child). Early on, children add format parsing. Later children replace “act rigorous” prompt-tweaks with explicit multi-stage evaluation pipelines, checklists, decision thresholds.
4. Evaluate the child on a staged protocol (cheap 10-task screen first; only promising agents get the full 100-task eval — a cost-control trick).
5. Add to archive. Repeat for ~100 iterations.

Over the run, the meta agent itself also gets edited. The system autonomously wrote a PerformanceTracker (logs score per generation per domain, computes a moving-average improvement trend) and a persistent memory store that holds synthesized insights, not just numbers — e.g. “Gen55 has best accuracy but is too harsh; Gen64 has best balance; combine Gen55’s critical reasoning with Gen64’s balance” and later “Gen66 went backwards — Gen65 over-corrected, revert.” That memory is consulted on the next self-modification. This is the metacognition: the agent reasoning about improvement as an ongoing process, and writing that reasoning machinery into its own code.

Architecture & data flow

flowchart TB
  subgraph Archive["Outer Loop: DGM Archive (frozen, hand-written)"]
    SEL[Select parent<br/>prob. proportional to perf,<br/>inverse to #children]
    ADD[Add evaluated child<br/>back to archive]
  end

  subgraph HA["Parent Hyperagent (one editable repo)"]
    MA[Meta Agent<br/>reads repo + past evals<br/>+ persistent memory]
    TA[Task Agent<br/>solves the task]
  end

  SEL --> HA
  MA -->|emits code diff:<br/>can edit Task Agent<br/>AND edit Meta Agent| CHILD[New Hyperagent]
  CHILD --> EVAL[Staged Evaluation<br/>10-task screen -> full set]
  EVAL --> ADD
  ADD --> SEL
  TA -.->|self-written tools| MEM[(Persistent Memory<br/>+ PerformanceTracker)]
  MEM -.->|consulted next round| MA

  style MA fill:#ffe0b2
  style MEM fill:#c8e6c9

The orange box (meta agent) being inside the editable repo is the entire contribution. In plain DGM, the meta logic that decides “what to improve” lives outside, frozen.

Schematic of the evolutionary archive growing over iterations. Each node is a hyperagent; brighter = higher performance; edges are parent→child self-modifications. Watch how the population keeps diverse "stepping stones" instead of collapsing onto one line — that's why removing the archive (the "w/o open-ended exploration" ablation) kills progress.

Why two metrics matter

To measure not “is the agent good” but “is the agent good at getting good,” they define improvement@k (imp@k). Plain English: freeze a meta agent M, give it a starting task agent A, let it generate up to k descendants on some task, and record the test-performance gain from A to the best descendant.

imp@k = test_score(best of k generated agents)  −  test_score(starting agent A)

Crucially they hold the meta agent fixed during this measurement (using the “w/o self-improve” variants as the generator). That isolates the question: how good is this meta agent at producing improved task agents? If imp@50 is ~0, the meta agent can’t help. If a meta agent trained in domain X scores high imp@50 in unseen domain Y, then “how to improve” transferred.

The algorithm, simplified

# The DGM-H outer loop. The novelty isn't here (this is DGM's archive);
# it's that meta_agent lives INSIDE the editable hyperagent repo,
# so step (3) can rewrite step (3) itself.

def dgm_h(initial_hyperagent, task, n_iters):
    archive = [initial_hyperagent]            # stepping stones, never thrown away
    for _ in range(n_iters):
        # (1) favor strong parents that also breed compilable children
        parent = sample(archive,
                        weight=lambda h: h.perf / (1 + h.n_compiled_children))

        # (2) parent runs ITS OWN meta agent code. It freely decides what to edit:
        #     the task-solving logic, OR the meta agent itself (metacognition),
        #     OR write new tools like a PerformanceTracker / persistent memory.
        diff = parent.meta_agent(repo=parent.repo,
                                 past_evals=archive_eval_logs(archive),
                                 memory=parent.persistent_memory)   # self-written
        child = apply_diff(parent, diff)       # a new editable hyperagent repo

        # (3) cheap screen, then full eval (cost control)
        if not compiles(child):                # failed edits are common; just skip
            continue
        if staged_screen(child, task):
            child.perf = full_eval(child, task)
            archive.append(child)              # add back -> future stepping stone
    return best(archive)

The line that makes it a hyperagent rather than DGM: parent.meta_agent is a file in parent.repo, so diff can patch it. Everything else is bookkeeping.

Built on Prior Work

The honest lineage: structurally this is DGM with one box moved inside the editable boundary, plus a new metric to prove meta-skill transfers. That’s a smaller change than the framing implies — but it’s the right small change, and the transfer evidence is the real payoff.

Results & Evidence

Setup: 5 runs each, medians with 95% bootstrap CIs, Wilcoxon signed-rank for significance. Frozen FM + bash tool + file-edit tool as the starting hyperagent.

Task performance (the “it works” results):

• Coding (Polyglot): 0.084 → 0.267 on the full benchmark — comparable to the original DGM (0.142 → 0.307) despite not being hand-built for coding. This is the “we didn’t lose anything” check.
• Paper review: 0.0 → 0.710 on test. Beats the open-source static baseline (AI-Scientist-v2 reviewer, 0.630). Beats original DGM (0.0, expected — its coding-specific meta step is useless here) and beats DGM-custom (0.590, hand-tuned per domain) on median, though not significantly.
• Robotics reward design: 0.060 → 0.372 test, beating the default metric-optimizing reward (0.348). Qualitatively, DGM-H reliably finds jumping policies to maximize torso height while DGM-custom gets stuck standing tall.

Ablations (the “every part is necessary” results): Both fail badly, both significantly worse than DGM-H (p<0.05):

• w/o self-improve (frozen meta agent ≈ ADAS): ~0.0 paper review, 0.213 robotics.
• w/o open-ended exploration (no archive, child replaces parent): ~0.0 paper review, 0.116 robotics.

Meta-skill transfer (the actually-interesting result): On unseen math grading:

• Initial meta agent: imp@50 ≈ 0.0.
• Meta agent transferred from DGM-custom runs: imp@50 ≈ 0.0 (its gains were domain-specific hacks, not real meta-skill).
• Meta agent (hyperagent) transferred from DGM-H runs: imp@50 = 0.630 (p<0.05). The transferred persistent-memory + performance-tracking machinery let it self-improve fast in a domain it had never seen.

Compounding: Starting from ProofAutoGrader + a transferred DGM-H meta agent, 200 iterations pushed the strong baseline from 0.561 → 0.601 accuracy on full IMO-GradingBench.

What the evidence does NOT establish. This is preliminary and narrow. (1) Tiny samples — 5 runs, wide CIs (some 0.0–0.56), and the win over the hand-tuned DGM-custom is not statistically significant. (2) The “any computable task” claim is a Turing-completeness argument, not demonstrated breadth — four domains, all FM-shaped. (3) The grand “unbounded self-acceleration” framing rests on a 0.561→0.601 bump and one transfer experiment. (4) The outer loop (parent selection, evaluation) is frozen, so this is not yet the fully self-referential system the intro promises. (5) No cost-normalized comparison front-and-center — code-editing evolution over hundreds of iterations with FM calls is expensive. Read it as a strong proof-of-concept that meta-skill can transfer, not as a finished recipe for runaway self-improvement.

How You’d Use It

For an AI services company, the literal DGM-H is research-grade and costly — but three ideas inside it are immediately bankable:

1. Self-written evaluation and memory as a deliverable. The single most reusable artifact here is what the agent spontaneously built: a PerformanceTracker that logs score-per-version-per-domain with moving-average trend, and a persistent memory that stores insights (“Gen65 over-corrected, revert”) not just numbers. You can bolt this onto any client agent today without any of the evolutionary machinery. It turns a stateless prompt loop into something that reasons about its own track record across deployments — a strong, demonstrable upgrade.

2. “Improve the improver” framing for client agent factories. If you maintain a stable of client agents, DGM-H’s lesson is: don’t just optimize each agent’s prompt — invest in the procedure that generates new agent versions (the checklist-based eval pipeline, the memory of what’s been tried). That procedure transfers across clients. It’s the difference between selling 10 one-off prompt-tunings and selling an agent-improvement capability.

3. imp@k as a sales/QA metric. “How fast does this system get better with k iterations of automated optimization?” is a concrete, demoable number you can put in front of a client to justify an ongoing optimization contract, versus a one-time build.

Where it slots in: as the offline optimization layer behind a deployed agent. You run DGM-H-style search on a held-out task set, harvest the best task agent code (not the whole machine), and ship that. The structured multi-stage pipelines it discovers (checklists, explicit decision thresholds) are exactly the kind of brittle-prompt → robust-machinery upgrade clients pay for.

Build Your Own (Minimal Recipe)

You can capture ~80% of the value without the full archive. Build order:

1. A self-editing agent loop. An agent with two tools: bash and edit_file. Give it its own source directory and the instruction “modify this codebase to score better next time.” This is the DGM-H starting hyperagent — deliberately minimal.
2. An evaluator + staged screen. A function eval(agent, tasks) -> score. Add the cheap-screen trick: run on 5–10 tasks first, only do the full set if it clears a bar. (This is where most of your compute budget goes; the screen is what makes it affordable.)
3. A persistent memory file the agent reads at the start of each modification and writes insights to at the end. JSON of {insight, timestamp}. This one component delivers most of the practical lift.
4. The metacognitive twist (the hard part #1): put the “decide what to improve” logic in an editable file the agent is allowed and encouraged to rewrite, not in your harness’s frozen prompt. Without this it’s just ADAS.
5. The archive (hard part #2): keep every compilable variant, sample parents weighted by performance / (1 + compiled children). Skip this and you get the “w/o open-ended exploration” failure mode — it collapses.

Libraries/models: any strong frozen FM with reliable tool-use and long context (you’re feeding it whole repos + eval logs + memory). A sandbox is non-negotiable — docker with no network, CPU/memory/time limits. Budget for many failed/non-compiling children; that’s normal.

The two genuinely hard parts: (a) getting the meta agent to make useful structural edits instead of cosmetic prompt fiddling — the paper got there only via the archive providing stepping stones; (b) keeping eval cheap enough that hundreds of iterations are affordable.

How to Improve It

1. Co-evolve the tasks (their own #1 limitation). The task distribution is fixed, so the system can saturate it. Add an FM-driven task generator that proposes harder/novel tasks as the agent improves — a curriculum. This is where “open-ended” becomes real and is the most promising lever for breaking the performance ceiling.
2. Unfreeze the outer loop. Parent selection and the evaluation protocol are hand-written. Let hyperagents propose edits to those (their Appendix E.5 hints it’s feasible). Risk: stability/safety — an agent that edits its own evaluator can goodhart itself. Testable: does imp@k keep rising or does it learn to game the metric?
3. Cost-aware self-modification. Add a budget term to the reward so the meta agent learns to make cheap high-yield edits. Right now nothing penalizes burning iterations. This directly attacks the affordability wall and would make the whole thing deployable.
4. Cross-client / cross-domain memory bank. The transfer result says meta-skill generalizes. Maintain one shared persistent-memory store across many domains/clients and measure whether imp@k compounds super-linearly as the bank grows. This is the commercial moat version of the experiment.
5. Verifier hardening for hard-to-verify domains. Paper review and grading use AI judges that can be gamed (the paper notes overfitting risk, hence validation sets). Pair the metric with an adversarial checker, or ensemble judges, and re-measure — wide CIs (0.0–0.56) suggest the judge noise is currently large.

Glossary

• Self-referential agent — a program that can read, modify, and evaluate any part of itself, including the part that does the modifying.
• Meta agent — code whose only job is to produce edits that create a new, better agent. The “improver.”
• Task agent — code that actually solves the user-facing task (review, grade, design reward).
• Hyperagent — one editable repo holding both task agent and meta agent, so the improver can rewrite the improver.
• Metacognitive self-modification — improving not just the task-solver but the self-improvement procedure itself.
• Infinite regress of meta-levels — the trap where each fixed improver needs another fixed improver above it, forever; self-reference is the escape.
• Darwin Gödel Machine (DGM) — prior system: an evolutionary archive of self-editing coding agents. The base this paper extends.
• Archive / stepping stones — a growing population of past agent variants kept around so future search can branch from diverse starting points (quality-diversity idea).
• Open-ended exploration — search with no fixed goal that keeps producing new, increasingly capable artifacts; here, the population-based archive.
• improvement@k (imp@k) — metric: the test-performance gain a fixed meta agent achieves by generating up to k descendant task agents. Measures skill-at-improving, not skill-at-task.
• Frozen FM — a foundation model whose weights never change; all learning happens by editing code around it, not by training the model.
• Staged evaluation — cost trick: screen agents on a few tasks before paying for the full eval.
• DGM-custom — baseline where humans hand-tune DGM’s improvement instructions per domain; the bar DGM-H tries to match without human labor.
• Wilcoxon signed-rank test — a non-parametric significance test for paired samples, used here because run counts are small and not normal.