Recursive Harness Self-Improvement

TL;DR

• Big AI systems now improve in two places: the model (weights) and the harness (the scaffolding that tells the model how to act as an agent). This paper only touches the harness, and leaves the model frozen.
• The trick, called RHI (Recursive Harness Self-Improvement), writes the whole multi-agent workflow as a prompt (not code), then improves it by comparing the current version’s output to the immediately previous version’s output. One run, one comparison, per step. That is what makes it cheap.
• Across 30 synthetic “build me a full research repo” tasks, a low-reasoning agent given a harness improved 1–2 times beats the same model cranked to its maximum reasoning setting — and beats Anthropic’s built-in multi-agent mode (ultracode) too.
• The gains do not come from writing more tokens. They come from better context management: the improved harness tells agents exactly what to pass to each other, so the system reads and writes far less cached context. Cost drops 23–60%.
• The authors offer an information-theory explanation: RHI quietly pushes the workflow to carry more task-specific information in its communication channels while carrying less redundant overlap between parts.

Problem & Motivation

The concrete pain: provider-built harnesses are frozen generalists, and improving them by search is too expensive to do per task.

Here is the situation in plain terms. A modern coding agent is not just a model. It is a model wrapped in a harness: system prompts, sub-agent definitions, tool loops, memory rules, and the control flow that decides who does what and when. Anthropic ships one (Claude Code), OpenAI ships one (Codex), and they must work for everybody, so the vendor cannot re-tune them for your specific job. You can specialize them yourself — with skills, sub-agents, custom workflows — but nobody knows a good, cheap way to do that automatically.

The obvious automatic approach is population search: generate many candidate harnesses, run each on the task, score them, keep the winners, repeat. This is what most prior systems do (Meta-Harness, ADAS, AFlow, AlphaEvolve, GEPA, and friends). The problem is brutal cost. Every extra candidate is a fresh, full agent run plus a fresh evaluation. The paper cites Wang et al. (2026): once you honestly count that search cost, automatic harness evolution often fails to beat simple baselines like “just run the model a few more times.” So harness optimization is only worth doing if the search overhead is tiny.

That sets the bar: an optimizer that is computationally light, converges in a handful of iterations, and still meaningfully raises quality. Nobody had shown that was possible for open-ended, non-verifiable coding tasks. RHI is the attempt.

What’s New (Core Contribution)

1. Harness-as-prompt, not harness-as-code. Before: harness search meant mutating executable scaffolding code and re-running it. Now: the entire multi-agent workflow is written as a text specification that gets injected into the agent’s prompt — agent roles, instructions, what each agent must hand back (contracts), and the orchestration steps (hops). Editing text is far cheaper than editing and re-executing code, and it works against a black-box agent you cannot open up.

2. Trajectory-local self-comparison instead of population search. Before: score a candidate against a whole population of competitors (quadratic cost). Now: compare the current harness only against its immediate predecessor — one new run, one comparison per step. Cost per step drops from Θ(m²) to Θ(1). This is the single design choice that makes “a few cheap iterations” feasible.

3. A self-comparison history as the learning signal. Before: optimizers consumed scalar scores or execution logs. Now: RHI accumulates the running list of “did version i beat version i−1?” preferences and feeds that history to an LLM optimizer, which rewrites the harness. No gradients, no reward number — the history acts as a “momentum” signal in text space.

4. An information-theoretic account of why it works. Before: “multi-agent helps” was hand-wavy and often just meant “more compute.” Now: the paper measures the harness with embeddings and shows RHI raises the mutual information between the communication components (contracts, hops) and the task, while lowering the redundancy (total correlation) between components. That is a concrete, testable story: specialize the channels, de-duplicate the parts.

How It Works (Technically)

Trace one real task the whole way through (this is the paper’s own Appendix B example):

The task. “Implement a reproducible training recipe for an SE(3)-equivariant network that predicts protein side-chain χ angles from backbone context. Pull real data from the PDB, report angular error, profile speed/memory, run an ablation, and deliver a full repo: a conference-style research_report.md, five specific .png plots, metrics.json, ablation_results.json, and reproducible src/*.py.”

Harness H[0] (the starting point). A generic, domain-level “drug discovery ML team”: eleven agents — an orchestrator plus specialists for structural data, geometric ML, diffusion generative modeling, molecular dynamics, portfolio strategy, scientific communication, and so on. The workflow is one-pass: orchestrator decomposes, delegates once, integrates. Every agent’s contract is the same vague line: “return structured output to orchestrator.” Notice this team is mostly wrong for this task — you do not need diffusion or molecular-dynamics agents to predict χ angles.

One RHI step turns H[0] into H[1].

• The agent A runs H[0] on the task and produces output[0] (a repo).
• The agent runs the next harness and produces output[1]; an LLM judge compares output[1] vs output[0] on six criteria (deliverable coverage, numerical rigor, reproducibility, presentation, engineering quality, task alignment) and returns a preference: better / tie / worse.
• That preference is appended to the history.
• An LLM harness optimizer reads (current harness, history) and rewrites it.

Harness H[1] (what the rewrite produced). The team is now task-specific: it drops the diffusion / molecular-dynamics / portfolio agents and adds agent_structural_data_rcsb, agent_torsion_geometry, a model agent, an ablation agent, a profiling agent, a validator. The one-pass flow becomes a multi-round workflow (R0–R6): build an acceptance rubric, fan out design, reconcile everyone’s outputs into a typed InterfaceContract v1 (DatasetRecord, ModelIO, MetricsJSON, AblationJSON, ProfilingJSON, PlotManifest, ReportClaims), implement against those contracts, collect artifacts plus critiques, recall any agent whose contract failed, then gate. Each agent now has an explicit output_to_orchestrator_schema — that is the contract. And it hard-codes guards against known failure modes (“report text can contradict numeric metrics,” “memory profiling can report implausibly tiny deltas,” “ablation can contain only one model despite a variants requirement”).

That is the whole mechanism in one picture: vague generic team → specialized team with typed channels and a multi-round, self-checking workflow. Repeat once or twice more and it stabilizes.

Architecture & data flow

flowchart LR
  TASK[Task x] --> A
  H[Harness H_i<br/>roles · instructions<br/>contracts · hops] --> A[Coding agent A<br/>fixed model L]
  A --> OUT[Output repo_i]
  OUT --> EVAL[LLM judge L_eval<br/>pairwise, 6 criteria]
  PREV[Output repo_i-1<br/>cached] --> EVAL
  EVAL -->|preference| HIST[(Self-comparison<br/>history D)]
  HIST --> OPT[LLM optimizer<br/>L_harness]
  H --> OPT
  OPT -->|rewrite| H2[Harness H_i+1]
  H2 -.next iteration.-> A

The RHI loop, step by step

flowchart TD
  S0[Start: initial harness H_0<br/>run agent, cache output_0] --> RUN[Run agent A with H_i<br/>produce output_i]
  RUN --> CMP[Judge compares<br/>output_i vs output_i-1]
  CMP --> APP[Append preference to history D]
  APP --> RATE{improvement rate s_i<br/>below threshold epsilon?}
  RATE -->|yes: stop| DONE[Return best harness]
  RATE -->|no| UPD[Optimizer rewrites<br/>H_i+1 = L_harness of H_i and D]
  UPD --> RUN

Inside a single harness: why cost drops

The reason cost falls is the contract. A vague harness lets every agent read the entire shared history (dense, expensive). A task-specific contract tells each agent exactly which fields to pass, so downstream agents condition on a slim message instead of the whole transcript. The authors call this “a task-specific sparsity pattern over inter-agent communication,” directly analogous to sparse vs. dense attention.

sequenceDiagram
  participant O as Orchestrator
  participant D as Data agent
  participant M as Model agent
  participant E as Eval agent
  Note over O,E: H[0] — no contract: everyone re-reads the full history (dense, high cache cost)
  O->>D: full context
  O->>M: full context + D's raw dump
  O->>E: full context + everything
  Note over O,E: H[2] — typed contracts: each agent passes only its schema (sparse, low cache cost)
  O->>D: task slice
  D-->>O: DatasetRecord
  O->>M: DatasetRecord + ModelIO
  M-->>O: MetricsJSON
  O->>E: MetricsJSON + AblationJSON

Schematic: as RHI iterates, inter-agent communication goes from dense all-to-all (every agent reads everything) to a sparse, typed contract pattern. Fewer cells lit = less cached context re-read = lower cost. Illustrative, not the paper's raw matrix.

Now the math, demystified. There are only a few equations that matter.

1. The ideal objective (Eq. 1) — “win the most fights.” $$f_x(H) = \mathbb{E}{H’ \sim \mu,; y \sim A(H,x),; y’ \sim A(H’,x)}\big[\mathbb{1}{L{eval}(y, y’; x_{eval}) = y \succ y’}\big]$$ In English: take your harness H, run it to get output y; take a random competitor harness H' from some big pool μ, run it to get y'; ask the judge who wins. f_x(H) is just the fraction of those fights H wins. The best harness H* is the one with the highest win rate. Beautiful, and completely intractable — the pool of harnesses is enormous and every fight is a full agent run.

2. Population search (Eq. 2) — “sample a small league.” Prior work approximates by drawing a small set of m competitors and averaging wins over that set. Better, but still Θ(m²) comparisons, and every competitor is a fresh expensive run.

3. RHI’s move (Eq. 3) — “only fight your former self.” $$\tilde{f}^{(i)}x(H) = \mathbb{E}{y \sim A(H,x),; y^- \sim A(H^{(i-1)},x)}\big[\mathbb{1}{L_{eval}(y, y^-; x_{eval}) = y \succ y^-}\big]$$ Same formula, but the competitor pool μ collapses to a single opponent: the previous harness H^(i-1). That means one new run and one comparison per iteration (the predecessor’s output is cached). Cost per step: Θ(1). That is the whole efficiency trick.

Is fighting only your former self too weak? The paper’s answer: no, under a standard preference model. Assume there is some hidden “true quality” number u(H) for each harness, and the chance H beats H' is σ(u(H) − u(H')) for an increasing function σ with σ(0)=½ (this is the Bradley–Terry model — the same math behind Elo ratings). Then both the global objective and the local objective are increasing functions of the same hidden quality u. So if your new harness beats the old one more than half the time, its quality genuinely went up — you also climbed the global objective. Each comparison is a noisy local ascent step: winners are kept, losers are discarded.

4. The history (Eq. 4) — “remember every fight.” A single comparison is one noisy bit. So RHI accumulates them: $$D^{(i)}x = \big{ L{eval}(y^{(k)}x, y^{(k-1)}x; x{eval}) \big}{k=1}^{i}$$ Because the harness is text (discrete, no gradients), this history is not a gradient — the authors call it a “momentum-semantic signal”: an accumulating written record of what has been working, handed to the optimizer.

5. The update (Eq. 5) — “rewrite from history.” $$H^{(i+1)}x = L{harness}\big(H^{(i)}_x, D^{(i)}_x\big)$$ Crucial subtlety: the optimizer L_harness never sees the evaluation rubric x_eval directly. It only sees the preference history, which was produced under x_eval. So RHI optimizes an implicit objective, not an explicit reward. It aligns to the rubric second-hand, through the trail of comparisons.

6. The implicit objective (Eq. 6) — the information-theory hypothesis. The authors do not claim to know the optimizer’s true goal; they propose one that fits the observed behavior: $$J(g_i) = \underbrace{\sum_{hc \in C_{ext}} \frac{1}{K}\sum_k I\big(z^{hc}{Xk}; X\big)}{f_{ext}} ;-; \beta \underbrace{\text{TC}\big({z^{hc}{Xk}} \mid X\big)}{f_{int}}, \quad \beta > 0$$ Two ideas, both simple once unpacked:

• Mutual information I(A; B) = how much knowing A tells you about B. f_ext says: make the contracts and hops carry more task-specific information (C_ext = {contract, hop} are the components the optimizer prompt tells it to focus on). Measured, contracts and hops do rise in task MI across iterations (Table 2), while roles fall.
• Total correlation TC = the multi-variable version of mutual information; it measures how much a set of variables redundantly repeat each other. f_int is subtracted (β > 0), so RHI is pushed to reduce redundancy between roles/instructions/contracts/hops after conditioning on the task. Measured, that redundancy TC | task does fall monotonically (Table 3).

Put together: make each channel say more about the task, and make the channels stop repeating each other. The authors liken the redundancy term to classifier-free guidance in diffusion — a cooperative steering term that pushes components to specialize into distinct functions (roles = who, instructions = how, contracts = what to pass, hops = when).

The algorithm, simplified

# RHI: improve a prompt-represented harness by comparing each version to the last one.
# llm_agent(harness, task) -> repo         (a full black-box coding-agent run)
# judge(repo_a, repo_b, rubric) -> +1/0/-1 (pairwise preference; +1 = a is better)
# optimizer(harness, history) -> harness   (an LLM rewrites the workflow text)

def rhi(task, rubric, H0, max_iters=4, eps=0.5):
    H = H0
    prev_output = llm_agent(H, task)         # baseline run, cached as the opponent
    history = []                             # the "momentum" signal: past preferences
    for i in range(1, max_iters + 1):
        H = optimizer(H, history)            # rewrite roles/instructions/CONTRACTS/HOPS as text
        output = llm_agent(H, task)          # one new run  -> Θ(1) cost per step
        pref = judge(output, prev_output, rubric)   # one comparison vs former self
        history.append(pref)                 # optimizer never sees the rubric, only this trail
        improvement_rate = mean(p > 0 for p in history[-1:])  # did this step win?
        if improvement_rate < eps:           # a few iterations is enough; stop early
            break
        prev_output = output                 # your new self becomes next round's opponent
    return H

The contribution is entirely in three lines: the opponent is always your own previous output, the feedback is a pairwise preference (works even when there is no single correct answer), and the optimizer edits text, so a step is cheap.

Built on Prior Work

Results & Evidence

Setup. 30 synthetic ML-research tasks (10 each in quant finance, robotics, pharmacy), generated from real job postings. Each task demands a complete repo with standardized deliverables. Evaluation is pairwise LLM-as-judge on six criteria, using two judge families (gpt-5.5-max and opus-4.x-xhigh) and three seeds.

Headline results:

• sonnet-4.6-high + 2 RHI iters beats sonnet-4.6-max, winning 20 of 30 comparisons. Gains hold at iters 3–4.
• opus-4.7-high + 1 iter beats both opus-4.7-xhigh and -max.
• opus-4.8-high + 2 iters beats xhigh, ultracode, AND max — the strongest result. Notably it beats ultracode, Anthropic’s built-in dynamic multi-agent mode. A user-written prompt-level harness beat the provider’s system-level one.
• Cost: opus-4.8 +H[2] is 23% cheaper than max and 60% cheaper than ultracode; cache read/write down 32–64%.
• Not from longer outputs: output-token usage stays roughly flat across iterations (e.g. sonnet 1.71→1.86×) while quality rises. Gains track cache read/write, i.e. context management, not generation length.
• Mechanism evidence: contracts show the clearest task-specific clustering and stabilize fastest; task MI of contracts/hops rises while roles’ falls (Table 2); component redundancy TC | task falls monotonically (Table 3).

The opus-4.8 result, from the paper's Figure 7 numbers: pairwise wins over 30 tasks (bars) against normalized cache read/write cost (line). Step through the baselines and RHI iterations — RHI climbs in wins while cost falls below max and far below ultracode.

What the evidence does NOT establish (read this before you sell it):

• Synthetic tasks, LLM judge, no ground truth. “Quality” is one model’s pairwise opinion. They mitigate with two judge families + three seeds and a capped context, but there is no objective metric. Reward-hacking the judge is a live risk.
• RHI does not beat a genuinely stronger model. Section 6.1: RHI on sonnet-4.6 does not close the gap to opus-4.7. It is complementary to train-time scaling, not a substitute — it improves how a fixed model is used.
• The information-theory story is correlational. MI and total correlation are estimated from text embeddings with biased estimators (they permutation-debias). It is a plausible, testable hypothesis, explicitly “not a proof of the optimizer’s true objective.”
• Very few iterations tested (2–4) and one opus-4.7 case where output tokens rose with performance, so that model’s “not from more tokens” claim is inconclusive.
• Requires an agent that accepts a big harness spec in its prompt. The harness is prompt-injected text; this assumes a Claude-Code / Codex-style agent.

How You’d Use It

This maps almost one-to-one onto an AI services offering. You already run a MAS; RHI is a way to productize workflow tuning without fine-tuning, without infra, and without touching client models.

• The deliverable is a text artifact. For a client’s recurring open-ended job (“produce this class of research report / repo / analysis”), you run 2–4 self-comparison loops and hand back a task-specialized harness prompt. They then run a cheap, low-reasoning model that beats their expensive max-reasoning setting — at 40–60% lower cost. That cost delta is the whole sales pitch, and it is measurable on their own bill.
• Where it slots in: on top of Claude Code, Codex, or any agent that takes a system-prompt-level workflow spec. You are editing the orchestration text, not the model.
• The moat is a harness library. Because each iteration is Θ(1), producing a harness is cheap, so you can build a catalog of task-type harnesses (quant report, pharma repo, robotics eval, etc.) and reuse them — Figure 9e shows harnesses within a domain homogenize, hinting you can amortize one harness across a domain.
• Positioning against “just use the biggest model”: RHI’s own data says restructuring the workflow can dominate raising reasoning effort on an already-strong model, on cost-per-quality. That is a clean story for a client watching token spend.

Honest read for someone who sells this: the engine (run → compare → rewrite) is a weekend build. The hard, defensible part is the judge and the task-type harness templates — that is where your expertise and your moat actually live.

Build Your Own (Minimal Recipe)

Smallest version that captures ~80% of the value:

1. A harness schema (text). Agents as {role, instruction, contract} plus an ordered list of hops (workflow steps) plus auxiliary rules (acceptance gates, fallbacks, recall triggers). Keep it as JSON-in-a-prompt, like the paper.
2. A runner. run(harness, task) -> repo. Just inject the harness text into a Claude-Code / Codex-style agent and capture the output artifacts.
3. A pairwise judge. judge(repo_a, repo_b, rubric) -> +1/0/-1. Fix a rubric (coverage, rigor, reproducibility, presentation, engineering, alignment). Cap the input to ~30–40% of the judge’s context to avoid “context rot,” and truncate both repos identically.
4. An optimizer prompt. optimizer(harness, history) -> harness. Its system prompt must explicitly steer edits toward contracts and hops — that emphasis is what produced the paper’s gains (C_ext = {contract, hop}).
5. The loop. Run → compare to previous output → append preference → rewrite → stop when the improvement rate drops below a threshold. Cache the previous output so each step is one new run + one comparison.

Build order: schema → runner → judge → optimizer prompt → loop. The two genuinely hard parts: (a) a low-variance judge — position bias, truncation, and self-family bias will wreck you; use two judge models and multiple seeds. (b) getting the optimizer to make contracts typed and specific (MetricsJSON, PlotManifest) rather than just verbose prose. Reach for: any frontier model for judge + optimizer; a coding agent that takes a system-prompt workflow; and, only if you want to instrument the MI/redundancy story, text-embedding-3-large or all-mpnet-base-v2.

How to Improve It

1. Richer feedback than one bit. Roles and instructions barely move because a single pairwise preference is a weak signal. Return per-criterion preferences (which of the six axes lost) so the optimizer knows what to fix — the paper flags this as the obvious next step.
2. Keep a tiny best-so-far memory. Trajectory-local uses only H^(i-1), so one bad rewrite can erase progress. A 1–2 slot Pareto memory (GEPA-style) guarding the best harness would make ascent monotone at almost no extra cost.
3. Optimize Eq. 6 explicitly. Instead of hoping the optimizer implicitly maximizes f_ext − β·f_int, propose 2–3 candidate rewrites per step and select the one with the highest measured (task MI of contracts/hops) minus (component redundancy). Cheap, and directly tests the hypothesis.
4. Anchor the judge with verifiable checks. Where anything is objectively checkable (tests pass, required files exist, JSON parses), blend those hard signals into the preference to cut reward-hacking of the LLM judge.
5. Test transfer / amortization. Measure whether a harness learned on one task helps a sibling task in the same domain (Figure 9e hints yes). If it transfers, you tune once per domain, not once per task — a big cost win for a services business.
6. Close the second loop (the paper’s own future work). Harvest the higher-quality execution traces RHI produces and use them as post-training data for the next model. That is the “data flywheel” the intro promises but does not build.

Glossary

• Harness — the scaffolding around a model that turns it into an agent: system prompts, sub-agents, tool loops, memory, and control flow. Here, written as a text spec.
• Coding agent — an LLM-driven system that writes and runs code to produce a repo, given a task and a harness.
• Role / instruction / contract / hop — the four harness pieces. Role = an agent’s expertise; instruction = how it behaves; contract = the exact output it must hand back; hop = a step in the orchestrator↔sub-agent workflow.
• Orchestrator / sub-agent — the lead agent that decomposes and delegates, and the specialists it delegates to.
• Test-time scaling — getting more quality by spending more at inference (higher reasoning effort: high < xhigh < max; ultracode = a built-in multi-agent mode), without changing the model.
• Train-time scaling — getting more quality by using a bigger / better-trained model.
• Pairwise LLM-as-judge — an evaluator model that, given two outputs, says which is better (or tie), used when there is no single correct answer.
• Trajectory-local — comparing only against your immediate previous version, not a whole population.
• Bradley–Terry model — the assumption that the chance A beats B depends only on their hidden quality difference; the math behind Elo ratings.
• Mutual information I(A;B) — how much knowing one variable reduces uncertainty about another; measured in nats here.
• Total correlation (TC) — the multi-variable extension of mutual information; measures how much a set of variables redundantly share. RHI lowers it (given the task) so components stop repeating each other.
• KV cache / cache read-write — the stored attention context a model re-reads each step; the main cost driver in Claude-style agents, and what RHI shrinks.
• Nats — units of information using natural log (1 nat ≈ 1.44 bits).