EnvHarness - Awakening Static Worlds for Agent Learning

TL;DR

Agents learn by doing things in environments — a codebase, a browser, a spreadsheet, a simulated house. Those environments are hand-built, expensive, and frozen: they behave exactly the same on day one and on day one hundred, no matter how good the agent has gotten. The field’s usual fix is to generate more environments with an LLM, but that needs a new pipeline per domain and produces worlds whose grading you can’t trust.

EnvHarness takes the opposite route. It never touches the environment’s code. It slips a wrapper between the agent and the environment that intercepts the three standard calls (reset, step, observe) and reshapes what flows through them. Three wrapper types cover most of what you’d want: Stage changes where the episode starts, Contract rewrites which actions are allowed and what the agent sees, and Chain glues two environments into one long episode. Because the wrapper sits outside, the environment’s original human-written grader still scores the episode — so the training signal stays trustworthy.

The second half is EnvRigger, an LLM loop that does the wrapping for you. It watches the agent fail, writes a diagnosis in plain text, emits a candidate wrapper as Python source, re-runs the agent five times to check the wrapper made the task harder but still winnable, and revises until it lands. Across five benchmarks in four domains it beats both the original environments and purpose-built environment generators: up to +9.0 points on held-out ALFWorld tasks, +2.7 on SWE-bench Verified with 9.8% fewer steps, and +6.5 points under reinforcement learning. Most importantly, it keeps improving as you add environments where the alternatives flatten out.

Problem & Motivation

The pain in one sentence: a training environment that never changes stops teaching the moment your agent can beat it, and it never taught the specific thing your agent is bad at in the first place.

Unpack that. When you train or improve an LLM agent, the environment is doing three jobs at once: it presents a task, it reacts to actions, and it decides whether you won (the verifier). Building one is genuinely expensive — SWE-bench needed per-issue Docker containers and real test suites; WebArena needed cloned websites; ALFWorld needed a text-adventure engine with a physics-ish state model. That cost forces you to build it once and freeze it.

A frozen environment fails a learner two ways:

1. It’s blind to the learner. ALFWorld’s “put a clean mug on the desk” always starts with the mug sitting in plain sight. If your agent’s real weakness is searching closed containers, this task will never surface it — the agent walks over, grabs the mug, wins, and learns nothing. Every rollout you spend there is wasted compute.
2. It runs out of road. Once your agent solves the tasks, the environment has zero remaining signal. Success rate pins at 1.0 and the gradient (or the extractable lesson) goes to zero.

The field’s current answer is environment generation: use an LLM to invent new tasks, new websites, new repos. Two things break.

• It doesn’t transfer. A pipeline that synthesizes SWE-bench-style repo bugs (SWE-smith) has nothing in common with one that clones websites (VeriEnv) or one that simulates a house (GenEnv). Each is a new engineering project, because generating an environment means also generating its verifier — and verifiers are where the domain knowledge lives.
• You can’t trust the grader. If an LLM wrote both the world and the answer key, the answer key can be wrong. GenEnv goes furthest here and has the LLM simulate the transitions themselves — so a hallucinated state change becomes a hallucinated reward. Practitioners cope by over-generating and heavily filtering, which is expensive and still leaky.

EnvHarness’s framing: environment construction is a wrapping problem, not an authoring problem. You already own worlds with trustworthy graders. Reshape them.

What’s New (Core Contribution)

Three contributions, and it’s worth separating the genuinely new from the well-packaged.

1. The wrapper abstraction, applied to the environment side. (Genuinely new as a framing; mechanically it’s the decorator pattern.)

Before: “agent harness” is now a standard idea — you take a frozen LLM and bolt on tools, memory, and a loop to get a capable agent (Agent = Model + Harness). Everyone does this. Now: the paper points out that the other side of the agent-environment loop was never given the same treatment, and defines Customized Env = Static Env + EnvHarness. Same trick, mirrored. Nobody had named this, and naming it is what unlocks the domain-agnostic part.

The mechanical insight underneath: you can reshape a huge amount of an environment purely at the interface, without ever reading its source. If you can replay actions, filter actions, rewrite observations, and mask termination signals, you can move the starting state, constrain the action space, control partial observability, and extend the horizon. That’s most of what a curriculum designer wants.

2. Domain-agnosticism bought by refusing to touch the verifier. (This is the real load-bearing idea.)

Before: every environment-generation method reaches into environment internals, which is exactly why each one covers exactly one benchmark. Now: every EnvHarness component is written against one abstract type (ActionableEnv). The paper’s own reward axis note is telling: the reward function R is deliberately not exposed to the designer agent. Success stays the benchmark’s own verdict. That single restriction is what makes the whole thing safe — a wrapper can make a task harder, weirder, or longer, but it cannot make a wrong answer count as right. The cost of onboarding a new benchmark drops to writing one “Bridge” adapter; the designer agent, the loop, and every component work unchanged.

3. EnvRigger: black-box, policy-conditioned automation of the reshaping. (New composition of known parts.)

Before: curriculum learning either uses hand-designed difficulty schedules, or needs access to model internals / a learned difficulty predictor. Now: an LLM watches the policy’s own trajectories (nothing else — the policy is strictly a black box), writes a text diagnosis, emits Python source for a wrapper, and validates it by re-running the policy 5 times. Acceptance is empirical, not a guess: did the success rate move into the target band, and did failures shift for the intended reason?

Honest read on the overselling: the “co-evolution” language is stronger than the evidence. What actually happens is a repeatable loop where each round’s environments are conditioned on the skill-bank the policy currently carries. That is genuine, but it is co-evolution in a fairly loose sense — the policy in the skill-learning experiments isn’t being retrained, just re-equipped.

How It Works (Technically)

The environment as a 6-tuple, in plain English

The paper models an environment as E = (S, A, O, T, R, s₀). Read it as six knobs:

An EnvHarness component is a function w that takes an environment and returns a different environment:

E′ = w(E)

That’s the whole abstraction. E′ is still an environment — same interface, same method names — so you can feed it right back into w again and stack them. Crucially, w is defined over E alone. It knows nothing about the agent. (The choice of which w to apply is policy-dependent; the component itself is not.)

The three components, and what each equation actually does

Stage — moves the starting line.

E′ = w_stage,δ(E) = (S, A, O, T, R, s₀′)   where s₀′ = T(…T(T(s₀, a₁), a₂)…, a_k)

Plain English: “reset the environment normally, then quietly execute this list of actions δ = (a₁ … a_k) before the agent is allowed to look.” Everything else in the tuple is unchanged — same physics, same verifier, same goal.

What it computes operationally: it’s a replay. The wrapper calls inner.reset(), then calls inner.step() for each action in δ, then hands the resulting observation to the agent as if it were the initial one. This is why the nested-T notation looks intimidating and is trivial in code: T applied repeatedly is stepping through the environment.

Why this design is clever: the mutated start state is always reachable, because you got there by taking legal actions. No hand-editing of a state file, no risk of constructing an impossible world. And the saved form of a Stage is just the action list — a few strings.

Concrete: on the mug task, δ = ["take mug 1", "open drawer 1", "put mug 1 in drawer 1", "close drawer 1"]. The agent now boots into a room with no visible mug and must learn to search. The goal (“put a clean mug on the desk”) and the grader are untouched. Run it the other direction and you get scaffolding: pre-execute clean the mug and you’ve shortened a task the agent keeps timing out on.

Contract — rewrites the rules of engagement.

E′ = w_contract,r(E) = (S, A′, O′, T′, R, s₀)   where (A′, O′, T′) = (f_A(A), f_O(O), f_T(T))

Plain English: three optional hook functions, each defaulting to “do nothing”:

• f_A — action filter. Runs before the action reaches the environment. Can rewrite it or return Blocked. Example: delete high-level teleport navigation so the agent must walk the house step by step.
• f_T — transition rewriter. Runs after the environment responds. Can rewrite the response. Example: block clean mug unless the agent is holding the mug, and return an error message instead.
• f_O — observation filter. Rewrites what the agent sees. Example: truncate the room description to two sentences, forcing the agent to build a map over several turns instead of reading it off in one.

Note R and s₀ are unchanged in this equation — a Contract deliberately cannot touch the reward or the start state. Separation of concerns, and a safety rail.

What it computes operationally: three pure functions interposed on the step loop, reading only a data-only view of environment state (get_env_state() — plain dicts, no Docker handles, no browser sessions). That restriction is exactly what makes a Contract portable: the same hook code runs against an in-memory puzzle and a containerized Django repo, because it never touches the runtime underneath.

One design detail worth stealing: a blocked action leaves the environment untouched and returns the current re-observed state plus a reason. A rejection never strands the agent in a broken turn.

Chain — makes the episode longer.

E′ = w_chain,ℓ(E) = g(E, E_ext)   with A′ = A ∪ A_ext,   R′ = R_A ∧ R_B

Plain English: run environment A; when A’s task ends, splice in a handoff observation and continue in environment B under one shared step budget. The composite says “success” only if both sub-verifiers say success (that’s the ∧). Termination signals from the sub-environments are masked, so only the composite decides when the episode is over.

What it computes operationally: a per-step hook decides “stay in the current sub-environment or switch.” Serial concatenation is the default (switch when A terminates). The same hook expresses branching (route to a harder or easier env depending on whether A succeeded), mid-task switching (route on a specific action), or interleaving (alternate every step). B is reset lazily at the handoff, so if A fails early you never pay to boot the second container.

Why it teaches something: the mug task normally ends the instant the mug hits the desk. Chain it with “heat a potato and put it on the countertop” and the agent must carry its goal past the point where it used to stop. That directly attacks premature-termination behavior.

Composition. All three share the interface, so E′ = w_chain(w_contract(w_stage(E))) is one environment the agent can’t distinguish from a raw one. Order matters (w₁ ∘ w₂ ≠ w₂ ∘ w₁): a Contract underneath a Stage will filter the Stage’s replay actions; a Contract above it won’t.

The decorator stack in 3D — drag to orbit. An action (blue) descends through each wrapper layer to the frozen Bridge at the bottom; the observation (amber) rises back up, transformed on the way. Each layer only knows about the layer directly beneath it, which is why any layer can be added or removed without the rest noticing. Schematic, not the paper's data.

The class architecture (what you’d actually build)

flowchart LR
  POL[Policy π<br/>LLM agent] -->|"Action(tool, kwargs)"| CH[Chain / Link]
  CH --> CO[Contract / Rules<br/>f_A, f_T, f_O]
  CO --> ST[Stage / Setups<br/>replays δ on reset]
  ST --> BR[Bridge<br/>ALFWorld · SWE · WebArena · Sheets]
  BR --> ENV[("Frozen environment<br/>native T + human verifier R")]
  ENV -->|EnvResponse| BR
  BR --> ST
  ST --> CO
  CO --> CH
  CH -->|"Observation"| POL
  ENV -.->|"evaluate() → R<br/>never intercepted"| POL

Four levels:

• ActionableEnv — the one abstract contract. Gymnasium-style: reset(seed, options), step(action) → EnvResponse (a Pydantic wrapper over the classic (obs, reward, terminated, truncated, info) 5-tuple), evaluate() → EvaluationResult, observe(), get_env_state(), plus save_state() / from_state().
  • observe() being separate from reset() is not cosmetic: a Stage mutates the world after reset returns but before the policy acts, so the outer layer needs a way to re-read the world without paying for a second reset.
• Bridge — one per benchmark, the only layer that knows about the actual runtime. The paper ships seven across four runtime classes: in-memory (Toy24), text-adventure engine (ALFWorld/TextWorld), per-instance Docker containers (SWE-bench, OfficeQA, SpreadsheetBench — each step is a stateless docker exec), and Playwright browsers (WebArena, WebShop). Each Bridge publishes an env_state_schema() describing which fields hook code is allowed to read — and that schema is injected into the designer LLM’s prompt, closing the loop between what the environment exposes and what generated code can rely on.
• EnvHarness — the abstract decorator. Delegates every method to inner by default; a component overrides only the axes it affects.
• The three components — Setups (Stage), Rules (Contract), Link (Chain). Note for anyone reading the repo: the released code predates the paper’s vocabulary and uses those old class names.

Persistence is layered: each component serializes only itself, and a checkpoint is [environment, ordered list of components, innermost first], rebuilt outward. Heavy runtimes (containers, browsers) save only their reset arguments and accept that restore is valid at episode boundaries.

Two safety details you would not think of on your own, and both matter in production:

• Designer-emitted Rules code is stored as a source string, recompiled per episode in a namespace exposing only the abstract data types, and executed in a per-episode subprocess — so a bad generated hook crashes one episode, not the training run.
• The dense per-step reward hook is non-fatal by contract: exceptions are recorded, never episode-terminating.

EnvRigger: the automation loop

The formal object is a task-policy-conditioned map:

E′ = H(E, t ; π) = (w_k ∘ w_{k-1} ∘ … ∘ w₁)(E)

Plain English: “given a base environment E, a task t, and a specific policy π, produce a stack of wrappers that exposes π’s weaknesses on t.” The semicolon matters: E and t are the inputs; π is the conditioning — the loop never inspects π’s weights, only its behavior.

flowchart TD
  OBS["**Observe**<br/>run π on task t, K=5 rollouts<br/>collect successes AND failures"] --> DIA
  DIA["**Diagnose**<br/>LLM reads trajectories →<br/>textual root cause + direction<br/>(SR too high → harden;<br/>SR ~0 → scaffold)"] --> WRT
  WRT["**Write**<br/>emit candidate set:<br/>in_env_actions (Stage δ)<br/>+ _Rules class source (Contract)"] --> VAL
  VAL{"**Validate**<br/>wrap env, run K=5 FRESH rollouts<br/>SR in target band?<br/>failures shifted as intended?"}
  VAL -->|accept| ACC["Push component onto<br/>the active EnvHarness"]
  VAL -->|refine, ≤5 rounds| WRT
  VAL -->|reject: unsolvable<br/>or still trivial| DROP["Discard, task<br/>yields no component"]
  ACC --> USE["Collect trajectories in E′ →<br/>extract skill / compute RL reward"]
  USE -.->|next round: π now carries<br/>accumulated skills| OBS

Observe. Run the policy 5 times on the unmodified task. The prompt tells the designer to read the baseline for three specific things: (a) can the policy solve this at all — if baseline success rate is ~0, “make it harder” is nonsense, scaffold instead; (b) how much headroom does the solution leave — a 4-step solution can absorb far less perturbation than a 30-step one; (c) which parts of the environment does it actually use — perturbing a command the agent never calls is wasted effort. Successes matter as much as failures: they mark where the capability boundary sits.

Diagnose. The designer writes a plain-text root cause. The paper’s examples are the useful part — real diagnoses look like “repetitive action loops”, “fails to parse long observations”, “misreads tool constraints”, “submits a patch without running the failing test”, “guesses URLs instead of using site search”. These are behavioral, not statistical. And the direction flips on the numbers: perfect success rate means the environment is too forgiving and must be hardened.

Write. The designer emits a candidate with exactly two levers — this is the part to copy:

• in_env_actions — a list of tool calls the framework replays through env.step() before the policy starts. This is the whole s₀ mechanism. Note what it isn’t: you don’t write code to build a start state, you write a trajectory the environment walks for you.
• rules_code — Python source for a _Rules(Rules) subclass overriding up to three hooks. Standard library imports only.

Both compose freely, and multiple components can ship as one candidate — accepted or rejected as a whole.

Validate. Wrap, run 5 fresh rollouts, decide ACCEPT / REFINE / REJECT from aggregate statistics (success rate over K runs, failure distribution, timeout count) — the prompt explicitly forbids deciding from a single trace. The refinement rule is unusually good engineering: if the mutation moved the success rate toward the band, the perturbation TYPE is right — keep the working hooks verbatim and adjust only the magnitude. If it didn’t, throw it out and try a different type. Don’t discard code that cost you 5 rollouts to validate.

The failure mode the prompt guards hardest against: making the task unsolvable. The line is worth memorizing — “SR=0 from impossibility is exactly as useless as SR=1 from triviality.” Signals that you’ve overshot: most rollouts end in timeout, or SR=0 with failures pointing at the action axis. On those signals the next proposal must reverse or loosen the restriction; stacking more bans cannot climb back into the band.

Budget: K=5 baseline rollouts, K=5 validation rollouts per candidate, at most 5 write-validate rounds per instance. Designer backbone = the same model as the policy on every benchmark, so gains can’t be distillation from a stronger teacher.

The write-and-validate gate, animated. Candidates land at a measured success rate; the green band is the target zone. Too-easy candidates get hardened, unsolvable ones get loosened, in-band ones get accepted onto the stack. The right panel shows the paper's actual ALFWorld result: original tasks are bimodal (mostly always-solved or never-solved, 6% in band), and reshaping compresses them into the middle (80% in band, mean SR 0.74 → 0.48).

Two ways the reshaped environment becomes a better agent

The environments themselves aren’t the deliverable. Two consumption paths:

Skill-based learning (SL) — the main experiments. Run the policy in the reshaped environment, collect trajectories, distill a text skill from them (following ReasoningBank), add it to a skill bank, retrieve relevantly at test time. Nothing is fine-tuned; the model weights never move. A skill looks like this real example:

Verification-Driven Development Loop Description: Whenever a code change is made to fix a bug or implement a feature, especially where the test suite needs setup or configuration. Content: Before finalizing any change, run the relevant test suite to confirm the failure exists, then run it again after the patch to verify the fix, initializing the environment first when needed.

That skill was produced by a Contract that rejected patch submissions until tests had been run — the environment made the missing behavior mandatory, and the behavior showed up in the trajectories, and the distiller read it back out.

Reinforcement learning (RL) — the reshaped environment is the training environment. Here’s the RL vocabulary the paper assumes:

• Policy (π) — the agent’s decision function. For an LLM, “the model plus its prompt,” producing an action from an observation.
• Rollout / trajectory — one full episode: obs → action → obs → action → … → terminal, plus the verdict.
• Reward — the score. Here it’s almost always the verifier’s binary outcome at the end, plus a small penalty (coefficient 0.1) for unexecutable actions.
• GRPO (Group Relative Policy Optimization) — the training rule they use. Sample a group of G rollouts on the same task, score them all, and compute each one’s advantage as advantage_i = (r_i − mean(r)) / std(r) — literally “how much better than its siblings was this attempt.” Then push the model’s probabilities up on the tokens of above-average rollouts and down on below-average ones. No separate value network needed, which is why it’s popular for LLM agents.
• Why the difficulty band matters enormously here: if every rollout in a group succeeds, r_i − mean(r) = 0 for all of them. Same if all fail. The advantage vanishes and the gradient is zero. A task at 100% or 0% success rate teaches a GRPO learner nothing at all. This is the sharpest argument in the paper and it’s slightly under-stated: EnvHarness’s target band [0.4, 0.6] is exactly the region of maximum learning signal for group-relative RL. Moving in-band coverage from 6% to 80% means turning 74% of your task corpus from dead weight into live gradient.

Setup: Qwen3-8B-base, GRPO, 8×H100, vLLM rollouts, 50-step episode cap, temperature 0.4, 150 training steps.

The algorithm, simplified

# EnvRigger: one task's worth of the Observe -> Diagnose -> Write -> Validate loop.
# policy(env) -> Trajectory ; llm(prompt) -> str ; both are black boxes to us.

TARGET_BAND = (0.4, 0.6)      # sweet spot: hard enough to teach, easy enough to win
K = 5                          # rollouts per measurement -- never judge from one trace
MAX_REVISIONS = 5

def env_rigger(base_env, task, policy, harness=()):
    env = wrap(base_env, harness)                  # current env = frozen base + accepted components
    baseline = [policy(env, task) for _ in range(K)]
    sr = mean(t.success for t in baseline)

    # Direction is decided by the numbers, not by taste.
    if sr > TARGET_BAND[1]:   goal = "too forgiving -- inject a flaw-exposing obstacle"
    elif sr < TARGET_BAND[0]: goal = "too hard -- scaffold an early subgoal away"
    else:                     goal = "in band -- sharpen the specific failure mode"

    diagnosis = llm(f"""Root-cause these trajectories. Name the behavioral flaw
        (loops? unread observations? skipped verification?). {goal}
        {render(baseline)}""")

    feedback = ""
    for _ in range(MAX_REVISIONS):
        # Two levers only. delta is REPLAYED through step(); rules_src is Python source.
        delta, rules_src = llm_write_candidate(diagnosis, env.state_schema, feedback)
        candidate = wrap(env, [Stage(delta), Contract(compile_in_subprocess(rules_src))])

        fresh = [policy(candidate, task) for _ in range(K)]     # FRESH rollouts, not the old ones
        sr2 = mean(t.success for t in fresh)

        if TARGET_BAND[0] <= sr2 <= TARGET_BAND[1]:
            return harness + (Stage(delta), Contract(rules_src))     # ACCEPT: push onto the stack
        if sr2 == 0 and all(t.timed_out or t.blocked for t in fresh):
            feedback = "unsolvable -- REVERSE or loosen the restriction, do not add more bans"
        elif abs(sr2 - sr) < 0.05:
            feedback = "no movement -- wrong perturbation TYPE, start over with a different axis"
        else:
            feedback = f"right type, wrong magnitude ({sr2:.2f}); keep the hooks, tune the size"
    return harness                                  # budget exhausted: this task yields nothing

The reward R is nowhere in this function. That’s the point, not an omission.

Built on Prior Work

Results & Evidence

Skill-based learning, five benchmarks (mean of 3 runs). EnvHarness environments beat original environments everywhere:

The most interesting number isn’t EnvHarness’s — it’s Original Envs’. On SpreadsheetBench, skills extracted from unmodified environments (45.88) score below doing nothing at all (46.44). On SWE-bench they make trajectories longer (55.0 vs 53.6 steps). Practicing what you already do well produces redundant, sometimes actively harmful, skills. That’s a finding worth carrying into any skill-library product.

Environment scaling (SWE-bench, identical budget). This is the strongest result. At 300 environments: EnvHarness 54.79 (from a 47.67 base, +7.12 and still climbing), original environments 52.13, generated environments 50.37. Both baselines flatten; EnvHarness doesn’t, because each batch of 50 targets the policy as it currently is (skills accumulated so far).

Environment scaling on SWE-bench Verified, redrawn from the paper's Figure 5. Same environment budget, same extraction and retrieval protocol, same policy — the only difference is where the environments came from. Hover the endpoints. Curves between the paper's reported endpoints are interpolated.

Reinforcement learning (Qwen3-8B-base, GRPO):

Wins 3 of 4; ALFWorld OOD is a 0.8-point loss, within noise.

Chain, isolated (long-horizon): Stage/Contract only → SR 52.58, 49.61 steps. Chain only → SR 49.63 (slightly below the 49.88 baseline) but average steps collapse from 53.58 to 41.96. Combined → SR 54.30 at 43.12 steps, the best of both. Chain teaches efficiency and goal persistence, not raw task-solving; the two skill sets are complementary rather than redundant.

Cross-model: four policies from Gemini 3.1 Flash-Lite (30.7 no-skill) to Claude Sonnet 4.6 (67.2 no-skill). EnvHarness beats original-environment skills by 2.7–3.7 points on all four. The loop neither breaks on the weakest nor saturates on the strongest — what changes is the content of the diagnoses. Note though that any skills help the two weakest models most (+9.3 and +11.1 for EnvHarness vs under 5.5 for the two strongest), so absolute headroom shrinks as the base model improves.

Compute honesty (a good table to have published): on WebArena, EnvHarness total token spend is 137.3M vs VeriEnv’s 137.8M — essentially identical. Design tokens are 1.58M of that (~1%); rollouts dominate. On ALFWorld EnvHarness spends 228M vs GenEnv’s 64.2M, but GenEnv’s rollouts are LLM-simulated rather than executed. The extra cost is the cost of grounding.

What the evidence does NOT establish — read this before you build on it:

• The RL evidence is thin. One 8B model, two of the five benchmarks, one algorithm, 150 steps. The headline “+6.5” is a single in-distribution ALFWorld cell. Held-out RL generalization actually got slightly worse. Treat “EnvHarness improves RL” as promising, not demonstrated.
• Absolute numbers are modest and variance is large. SWE-bench improvement is +2.70 with standard deviations of 2.59 and 2.72 over three runs. The confidence intervals overlap. The scaling trend is the more convincing evidence than any single cell.
• Chain was excluded from the automated loop entirely. EnvRigger can’t observe the internal state of joined environments, so all Chain results come from randomly pairing environments by hand. The automated system is really Stage + Contract only.
• Verifier trust is inherited, not proven. “The original verifier still scores it” is true — but a Contract can still put the agent in a state the original verifier was never designed to judge. Nothing in the system detects that.
• No ablation of the validation loop. How much of the gain comes from diagnosis-driven wrapping versus any wrapping that lands in the difficulty band? A random-perturbation-plus-band-filter baseline is the obvious missing control, and its absence is the paper’s biggest methodological gap.
• Held-out ≠ transfer. The leave-one-out ALFWorld study is more honest: +3.1 average, but with one −8.7 regression on the heat task type. Reshaping is not uniformly beneficial.

How You’d Use It

Three places this maps directly onto agent work you’re already doing.

1. As a way to harden your own eval suite — the fastest thing to build.

If you run an agent in production, you likely have the same problem: your eval suite is a frozen set of golden tasks, the agent passes them, and it still breaks in the field. EnvHarness is a clean answer. Take your existing eval tasks — which already have trusted graders, because your own team wrote them — and generate a stress suite by wrapping: block the tool the agent leans on, truncate the context it over-reads, start the episode from a partially-completed or corrupted state, chain two tasks so it has to hold a goal across a handoff. You produce a hardened eval suite without authoring a single new task or grader, which is exactly where the cost and trust bottleneck sits in most eval work.

The Contract’s f_A axis alone is worth building on its own: “here are the twelve shortcuts your agent silently relies on, and here’s what happens when each one is unavailable.”

2. As the environment layer under a skill/memory system.

If you’re running skill banks or memory (ReasoningBank-style, or your own multi-agent memory), this paper’s negative result is the actionable part: skills extracted from environments the agent already passes make things worse. SpreadsheetBench went below the no-skill baseline. So gate your extraction on measured difficulty — only distill from episodes whose task sat in the interesting band. That’s a small change to an existing pipeline with real expected value.

3. In multi-agent orchestration — the cleanest structural fit.

An MAS is already a stack of message-passing layers with a standard interface. The ActionableEnv contract plus decorator stack is a design you can lift wholesale to build a rehearsal harness for a multi-agent system: wrap each agent’s environment view to inject partial observability, delay messages, block a coordination channel, or start the team mid-workflow with a subtask already botched. You get chaos-engineering-for-agents while preserving whatever completion check you already trust. The paper explicitly names multi-agent shared environments as future work — that’s an open lane.

Where the effort actually goes. The value here isn’t the wrapper code, which is a weekend. It’s (a) the Bridge you write for your own system, which becomes durable infrastructure once it exists, and (b) the diagnosis prompt tuned for your domain. Effort to stand up a first version against a system you already have with a working reset/step: roughly a week for the Bridge plus a week for the loop. The hard prerequisite is the honest gate below.

The honest gate — when to say no. EnvHarness requires a resettable environment. A Stage needs to put the world in a chosen state; a Chain needs to return it to a known state. Live production systems fail this: a sent email, a placed order, a charged card cannot be undone. If your agent acts on real accounts, you need a staging replica first, and that replica is the actual project.

Build Your Own (Minimal Recipe)

Smallest version that captures most of the value. Assume one environment you already have with a reset/step/is_done shape and a grader you trust.

Component 1 — the contract (half a day). One abstract class: reset(seed), step(action) -> (obs, reward, terminated, truncated, info), observe(), evaluate() -> bool, get_env_state() -> dict. That last one is the important one and the one you’ll be tempted to skip: it must return plain JSON-serializable data only — no DB connections, no browser handles, no container clients. Every rule of portability follows from it.

Component 2 — one Bridge (1–3 days, and this is where the real time goes). Adapt your actual system to that contract. Also expose env_state_schema(): a human-readable description of what’s in get_env_state(), because you’re going to paste it into the designer’s prompt.

Component 3 — the two wrappers (half a day). Skip Chain for v1; the paper did too.

class Wrapper(ActionableEnv):
    def __init__(self, inner): self.inner = inner
    def __getattr__(self, k):  return getattr(self.inner, k)   # delegate everything by default

class Stage(Wrapper):
    def __init__(self, inner, delta): super().__init__(inner); self.delta = delta
    def reset(self, seed=None):
        self.inner.reset(seed)
        for a in self.delta:                 # replay -- the mutated start is always REACHABLE
            self.inner.step(a)
        return self.inner.observe()

class Contract(Wrapper):
    def __init__(self, inner, hooks): super().__init__(inner); self.h = hooks
    def step(self, action):
        st = self.inner.get_env_state()
        action = self.h.filter_action(action, st)
        if action.blocked:                   # a block must NEVER strand the policy:
            return Response(obs=self.h.filter_observation(self.inner.observe(), st),
                            reward=0, terminated=False, info={"blocked": action.reason})
        r = self.inner.step(action)
        r = self.h.modify_transition(action, r, st)
        r.obs = self.h.filter_observation(r.obs, st)
        return r

Component 4 — the EnvRigger loop (2–3 days). The env_rigger pseudocode above is close to complete. What actually takes the time is the designer’s system prompt, so start from the paper’s — it’s reproduced in full in Appendix A and its three warnings (don’t make it unsolvable; read the baseline for headroom; keep working hooks when only the magnitude is wrong) are hard-won.

The two genuinely hard parts:

1. Reset fidelity. Everything rests on “5 fresh rollouts are comparable to 5 baseline rollouts.” If your reset is nondeterministic, or your container carries state across episodes, or your browser keeps cookies, your accept/reject decisions are noise. Test this first: reset twice with the same seed, run the same action list, assert the two get_env_state() dicts match. If they don’t, fix that before writing a single wrapper.
2. Sandboxing generated code. The designer emits Python source that runs inside your loop. Compile it in a restricted namespace, run it in a subprocess, and time-box it. The paper does exactly this and it’s the difference between one dead episode and a dead training run.

What to reach for: Gymnasium for the interface vocabulary; Pydantic for the response and state types (typed contracts are what let generated code fail loudly instead of silently); Docker for per-instance isolation if your environment is heavy; the same model for designer and policy (so gains can’t be teacher distillation you didn’t intend); vLLM if you go to RL, trl/verl for GRPO.

Sequencing that de-risks it: contract → Bridge → hand-written Stage and Contract, verified by reading the trajectories yourself → only then automate the designer. Building the LLM loop first means debugging generated code against an interface you haven’t validated.

How to Improve It

Five directions, ordered by ratio of payoff to effort.

1. Run the missing control: random perturbation + band filter. The paper never separates “the diagnosis was insightful” from “the wrapper landed in the difficulty band.” Generate wrappers at random from a small template library, keep only those whose validated success rate falls in [0.4, 0.6], and compare against full EnvRigger at equal rollout budget. If random-plus-band matches diagnosis-driven, you’ve found a 10× cheaper method (no trajectory-reading, no diagnosis tokens) and a genuinely important negative result. If it doesn’t, you’ve established the diagnosis is load-bearing — which the paper currently only asserts. Cheapest high-value experiment in this space.

2. Make the target band a function of the learner, not a constant. [0.4, 0.6] is fixed everywhere. For GRPO specifically, the quantity you actually want to maximize isn’t success rate near 0.5 — it’s the variance of the group reward, since that’s literally the numerator of the advantage. Target std(r) directly, or target the band that maximizes expected gradient magnitude for the group size you’re using. For a binary reward and group size G, that’s a computable optimum, not a hyperparameter. Should be a strict improvement for RL and costs nothing.

3. Add a state-validity check to close the verifier loophole. The safety claim — “the original verifier still scores it” — has a hole: a Contract can drive the environment into a state the verifier was never designed to judge, and nothing notices. Add a cheap invariant check as a fourth validation criterion: run the known-good solution trajectory (if you have one) or a strong reference policy through the wrapped environment, and reject any wrapper under which the reference solution stops verifying. This converts an assumption into a test, and it’s the change I’d make first for anything user-facing.

4. Reuse components across tasks instead of rediscovering them. EnvRigger currently pays 5+ rollouts per task per revision, from scratch, every time. But the same flaw recurs — “doesn’t run tests before submitting” is not a per-issue problem. Maintain a component bank keyed by diagnosis text, retrieve the nearest prior component, and validate it directly. Accepted-on-first-try means you skip the whole write-refine loop. Given design is only ~1% of tokens the direct savings are small, but the rollout savings from skipping revision rounds are not — and a component bank is also a reusable asset (“here are the 40 failure modes we’ve seen in coding agents, each with an executable probe”).

5. Two structural extensions the paper flags but doesn’t do.

• Bring Chain into the automated loop. It was excluded because EnvRigger can’t observe joined environments’ internal states. But it doesn’t need to — it can observe the handoff. Diagnose on the composite trajectory: where did the agent lose the goal? Then let the designer choose the pairing instead of pairing randomly, which is what produced Chain’s below-baseline standalone success rate. Semantic pairing (chain tasks that share objects, a repo, or a workflow) is the obvious first move.
• Multi-agent components. A component that places two policies in one shared environment, or that delays/drops messages between them, extends the whole abstraction to MAS rehearsal — and for anyone already running multi-agent systems, this is the version of the paper that’s actually worth building.

Glossary

• Agent harness — the software wrapper around a frozen LLM (execution loop, tool registry, context management) that turns it into an agent. Agent = Model + Harness.
• Policy (π) — the agent’s decision function: given what it sees, what it does. For an LLM agent, the model plus its prompt.
• Rollout / trajectory — one complete episode of an agent acting in an environment, plus the final verdict.
• Verifier — the code that decides whether an episode succeeded. Usually hand-written, usually the most expensive and most trusted part of a benchmark.
• Reward (R) — the numeric score from the verifier. In these benchmarks, almost always 1 for success and 0 for failure.
• Transition function (T) — the environment’s rules: given a state and an action, what state results.
• Black-box policy probing — diagnosing an agent’s weaknesses purely from its observable behavior (its trajectories), never from its weights or internal activations.
• Curriculum learning — ordering training tasks by difficulty so the learner always faces something it can just barely handle.
• UED (unsupervised environment design) — the RL research line that automatically generates training environments matched to the learner’s current ability; EnvHarness is UED done at the interface level for language agents.
• GRPO (Group Relative Policy Optimization) — an RL algorithm that samples a group of rollouts on the same task and reinforces the ones that beat the group average, using no separate value network.
• Advantage — how much better a given rollout was than the baseline expectation. In GRPO, (reward − group mean) / group std. If every rollout in a group succeeds or every one fails, the advantage is zero and nothing is learned.
• Target difficulty band — the success-rate window (here [0.4, 0.6]) where a task produces maximum learning signal: hard enough to fail sometimes, easy enough to succeed sometimes.
• Gymnasium interface — the standard RL environment API (reset(), step(action)), and the reason environment wrappers are a well-worn pattern.
• Decorator pattern — an object that implements the same interface as the object it wraps, delegating everything it doesn’t override. Lets you stack behaviors without anyone knowing how deep the stack goes.
• Bridge — in this paper, the per-benchmark adapter that makes a real runtime (Docker, browser, text engine) satisfy the abstract ActionableEnv contract. The only layer that knows what’s underneath.
• Skill (in ReasoningBank sense) — a short natural-language procedure distilled from trajectories, stored in a retrievable bank and injected into the agent’s prompt at inference. No weight updates involved.
• Partial observability — the agent can’t see the full state, only a filtered view. A Contract’s f_O hook manufactures it on demand.
• Held-out / OOD — evaluation tasks the training process never touched; OOD (out-of-distribution) additionally differ in kind, not just in instance.