SkillOpt: Executive Strategy for Self-Evolving Agent Skills

TL;DR

Agents today are steered by “skills”: plain-text files that tell a frozen model how to work in a domain (how to search, which tools to call, how to format answers). Today those files are hand-written, generated once, or rewritten by loose self-revision — none of which behaves like an optimizer, and none reliably beats its own starting point. SkillOpt makes skill-writing behave like gradient descent, but in text space: a separate “optimizer” model reads scored rollouts, proposes bounded add/delete/replace edits to one skill document, and an edit is accepted only if it strictly improves a held-out validation score. The result is the strongest no-weight-update adaptation method the authors tested: best-or-tied on all 52 (model, benchmark, harness) cells, +23.5 average points on GPT-5.5 in direct chat, and the learned skill files are tiny (300–2,000 tokens, 1–4 edits) and transfer across models, harnesses, and nearby tasks.

Problem & Motivation

If you run agents for clients, the thing you actually tune is rarely the model. You can’t retrain a closed frontier model, and fine-tuning open ones is slow and expensive. So the real adaptation layer is the procedure: the skill/system text that says “search the primary source first, verify the tool result, output the number with no label.” That text is where domain knowledge lives.

The pain: that text is written by hand or generated once and then frozen, or it is “self-improved” by asking a model to rewrite it after failures. All three are fragile.

• Hand-written / one-shot skills can’t learn. They help only when the author guessed right. They cannot see a rollout fail and fix themselves. In the paper, expert human skills (145–516 tokens) are beaten in every direct-chat model row.
• Loose self-revision drifts. Ask a model to “rewrite the skill to fix these failures” and it can erase a rule that was working, contradict itself, or overfit to the one failure it just saw. There is no step size and no check that the rewrite actually helped the real target model.
• Nobody was treating the skill as a trainable object. Prior systems mine trajectory lessons, grow skill libraries, or optimize prompts — but they leave the basic question open: if the skill is the thing we adapt, how should it be optimized? A “plausible” text diagnosis can still hurt the actual model, and without a gate you never find out until production.

The concrete “before” state: you have a skill file, you have a model, you have a domain, and you have no disciplined, reproducible way to make the file reliably better than where you started.

What’s New (Core Contribution)

The whole paper is one move: port the discipline of weight-space training onto a text document. Every optimizer concept gets a text-space twin.

• The skill document is the parameter. Before: skills were side artifacts of prompting, edited ad hoc. Now: the skill is the explicit external state being trained, with a start point S0 and a trajectory of versions S1, S2, ....
• A bounded “textual learning rate.” Before: rewrites were unbounded — one call could replace the whole file. Now: an edit budget L_t caps how many atomic edits apply per step, with constant/linear/cosine/autonomous schedules (bigger early, smaller later). This keeps each version close to the last, so the optimizer can actually learn what helped.
• A held-out validation gate on every edit. Before: self-revision was accepted unconditionally. Now: a candidate skill is run on a selection split with the real frozen model, and accepted only if the score strictly improves (ties rejected). This is the single most important safety mechanism — it turns “self-editing” into “propose-and-test.”
• A rejected-edit buffer (negative feedback for free). Before: a failed rewrite was just discarded. Now: rejected edits and the failure patterns behind them are stored for the rest of the epoch, so later optimizer calls avoid repeating them — negative feedback with zero extra deployment cost.
• An epoch-wise slow/meta update (a momentum term). Before: no long-horizon memory. Now: at each epoch boundary the optimizer compares the same tasks under last-epoch vs. this-epoch skill, writes durable guidance into a protected region of the skill file that fast edits can’t overwrite, and separately keeps an optimizer-side “meta skill” (teacher-only, never shipped).

What is genuinely new is not any single trick (reflection, prompt optimization, and skill evolution all predate this). It is the integration into a controlled, gated, reproducible training loop whose deployed output is a static file that adds zero inference-time model calls.

How It Works (Technically)

Set the vocabulary first. M is the frozen target model (the one doing the client’s work — never changes). O is the optimizer model (a frontier model used only during training — like the training machinery, thrown away at deploy). h is the harness (direct chat, Codex CLI, or Claude Code CLI). A skill s is a natural-language policy prepended to the agent before it runs.

Running one task produces a trajectory and a score:

$$(\tau(s),, r(s)) = h(M, x, s), \qquad r(s)\in[0,1]$$

Plain English: put skill s in front of frozen model M, run task x inside harness h, and you get back the full trace τ (messages, tool calls, outputs) plus a scalar score r between 0 (fail) and 1 (perfect). That score is the only feedback signal — it plays the role that a loss plays in normal training.

Data is split three ways, exactly like ML: train D_tr (generates experience), selection D_sel (the validation gate — decides accept/reject), test D_test (locked until the final number). The reported result is:

$$s^\star_{sel} = \arg\max_{s\in C(D_{tr})}; \frac{1}{|D_{sel}|}\sum_{x\in D_{sel}} r(s), \qquad \text{Test}(s^\star_{sel}) = \frac{1}{|D_{test}|}\sum_{x\in D_{test}} r(s^\star_{sel})$$

Plain English: among all candidate skills produced from training experience, pick the one with the best selection-split average score (that’s s*), then report its average on the untouched test split. Because selection ≠ test, the headline numbers measure generalization, not memorizing the gate.

Now the loop, step by step, tracing one real example (SpreadsheetBench on GPT-5.5).

1. Forward pass — rollout batch. Run a batch (default 40 tasks) from D_tr with the current skill. The harness records everything: tool calls, command outputs, verifier feedback, spreadsheet previews. This batch is the “evidence unit.” Small batches update fast but noisily; big batches expose recurring patterns. In our trace: the initial skill just says “use Python spreadsheet libraries and preserve unrelated content,” and many rollouts fail because the agent trusts a cell preview or writes an Excel formula the grader can’t evaluate.

2. Backward pass — minibatch reflection. Split the batch into failures and successes, then into reflection minibatches (default size 8). The optimizer O reads a minibatch, not a single trace — this is deliberate: one trace gives an anecdotal fix, a minibatch reveals the systematic error (“agent consistently relies on preview values”). Failure minibatches propose corrective rules; success minibatches propose rules that lock in what already works. Each reflection returns structured add / delete / replace edits. In our trace: the failure analyst proposes “inspect the actual workbook, not previews” and “compute and write evaluated static values even if the prompt mentions INDEX/MATCH or XLOOKUP.”

3. Merge. Proposals are merged hierarchically: consolidate failure edits, consolidate success edits, then a final failure-prioritized merge that deduplicates, resolves contradictions, and drops example-specific suggestions. Each surviving edit carries a support count (how many independent analyses proposed it).

4. Bounded update (the learning rate). Rank the merged pool by expected utility and clip to the top L_t edits (default 4, cosine-decaying to a floor of 2). This is the key difference from ad-hoc rewriting: unbounded rewrites can erase good rules or overfit; a bounded update preserves continuity while still adding new procedure. Apply the kept edits → a candidate skill s̃.

5. Validation gate + rejected-edit buffer. Run s̃ on D_sel with the same frozen M and harness. If its selection score is strictly greater than the current skill’s, it becomes the new current skill (and, if it also beats the running best, it is written to best_skill.md). Otherwise it is rejected, and the failed edits + the failure patterns they targeted go into an epoch-local buffer that later reflection calls receive — so the optimizer stops re-proposing what already failed.

6. Epoch-wise slow/meta update (momentum). After each epoch, re-run the same ~20 tasks under last-epoch’s skill and this-epoch’s skill, bucket the results into improvements / regressions / persistent failures / stable successes, and have O write a longitudinal guidance block into a protected <!-- SLOW_UPDATE_START ... END --> region that step-level edits cannot touch. This guidance also passes the validation gate. Separately, O updates an optimizer-only meta skill (“which kinds of edits help here, which backfire”) that is prepended to future optimizer prompts but never shipped with the deployed skill.

End of trace: after 4 epochs, best_skill.md is ~1,995 tokens built from just 4 accepted edits, and SpreadsheetBench test accuracy goes 41.8 → 80.7. The deployed file is a static ~2k-token markdown document; at inference it is just prepended to GPT-5.5. No optimizer calls, no weight changes.

Why the caps and gates matter together: if consecutive skill versions jumped too far or in random directions, the rejected-edit history and prior accepted edits would be meaningless — you couldn’t tell what helped. Bounded + gated updates keep every version close enough to the last that the optimizer’s memory is actually informative. That’s the whole thesis: stability is what makes text-space optimization behave like optimization instead of thrashing.

Architecture & data flow

flowchart LR
  subgraph Data["Dataset splits"]
    TR[(Train)]
    SEL[(Selection<br/>= validation gate)]
    TEST[(Test<br/>locked)]
  end
  TR -->|rollout batch| M
  SKILL[/Current skill S_t/] --> M[Frozen target model M<br/>+ harness h]
  M -->|scored trajectories| O[Optimizer model O]
  O -->|add/delete/replace<br/>proposals| MERGE[Merge + rank]
  MERGE -->|clip to top L_t edits| CAND[/Candidate skill S~/]
  CAND --> GATE{Selection score<br/>strictly up?}
  SEL --> GATE
  GATE -->|accept| BEST[/best_skill.md/]
  GATE -->|reject| BUF[Rejected-edit buffer]
  BUF -.negative feedback.-> O
  BEST -.next step.-> SKILL
  BEST -->|final eval only| TEST

Schematic of the validation gate as a ratchet. The optimizer keeps proposing candidate edits (dots). Most land below the current best and are rejected; only a strictly-improving candidate is accepted, so the deployed skill's score can only climb or hold — never drift down. Press to run more steps.

The training loop, in text space

flowchart TD
  S0[Initial skill S0] --> ROLL[Run rollout batch on train split]
  ROLL --> SPLIT[Split into failures / successes -> minibatches]
  SPLIT --> REFLECT[Optimizer proposes add/delete/replace edits]
  REFLECT --> RANK[Merge + rank + clip to L_t edits]
  RANK --> CAND[Candidate skill]
  CAND --> GATE{Selection score up?}
  GATE -->|yes| ACCEPT[Adopt candidate; maybe save best_skill.md]
  GATE -->|no| REJECT[Store failed edits in buffer]
  ACCEPT --> NEXT[Next step]
  REJECT --> NEXT
  NEXT --> ROLL
  NEXT -->|epoch boundary| SLOW[Slow/meta update into protected region]
  SLOW --> ROLL

Schematic of bounded vs. unbounded updates. The green path takes small, validation-gated steps from S0 and climbs steadily to high ground; the red path takes big unchecked rewrites, overshoots, and thrashes. Same start, very different destinations — this is why the edit budget (textual learning rate) matters. Drag to orbit.

The algorithm, simplified

# SkillOpt core loop. Stubs: run(), reflect(), evaluate() are model/harness calls.
# The idea IS the gate: an edit survives only if a held-out score strictly improves.

def skillopt(M, O, harness, D_tr, D_sel, D_test, s0, epochs=4, L=4):
    cur = best = s0
    cur_score = best_score = evaluate(M, harness, s0, D_sel)  # baseline on the gate
    seen = {hash(s0): cur_score}                              # cache: never re-score a skill
    buffer = []                                               # rejected edits + failure notes

    for epoch in range(epochs):
        buffer.clear()
        for batch in rollout_batches(D_tr):
            traces = [run(M, harness, x, cur) for x in batch] # forward pass: scored rollouts
            fails  = [t for t in traces if t.score < 1.0]     # separate by outcome...
            wins   = [t for t in traces if t.score == 1.0]    # ...minibatches expose PATTERNS

            edits  = O.reflect(fails, wins, cur, buffer)      # propose add/delete/replace
            edits  = O.merge_and_rank(edits)[:L]              # L = textual learning rate (clip)
            cand   = apply(cur, edits)                        # bounded candidate skill

            score  = seen.get(hash(cand)) or evaluate(M, harness, cand, D_sel)
            seen[hash(cand)] = score

            if score > cur_score:                             # THE GATE: strict improvement only
                cur, cur_score = cand, score
                if score > best_score:
                    best, best_score = cand, score            # -> best_skill.md
            else:
                buffer.append((edits, fails))                 # negative feedback, no deploy cost

        if epoch >= 1:                                        # momentum: cross-epoch consolidation
            guide = O.slow_update(prev_skill, cur)            # writes PROTECTED region only
            if improves(guide, D_sel): cur = apply_protected(cur, guide)

    return best, evaluate(M, harness, best, D_test)           # test touched exactly once

Built on Prior Work

SkillOpt sits at the junction of three lines and takes the “controlled training” idea further than each.

The honest read: none of the individual analogies (parameter, gradient, LR, validation, momentum) is invented here. The contribution is showing that wiring all of them together with a strict gate is what finally makes text-space skill training monotone and reproducible — and proving it at unusual breadth.

Results & Evidence

What was tested. 6 benchmarks spanning QA (SearchQA), spreadsheets (SpreadsheetBench), enterprise docs (OfficeQA), multimodal docs (DocVQA), math MCQ (LiveMathematicianBench), and embodied decision-making (ALFWorld); 7 target models (GPT-5.5 down to GPT-5.4-nano, plus Qwen3.5-4B and Qwen3.6-35B-A3B); 3 harnesses (direct chat, Codex CLI, Claude Code CLI). Baselines: no-skill, human skill, one-shot LLM skill, Trace2Skill, TextGrad, GEPA, EvoSkill.

Headline numbers.

• Best or tied on all 52 evaluated cells. Not “on average” — every single (model, benchmark, harness) cell.
• GPT-5.5 direct chat: +23.5 points average over no-skill (58.8 → 82.3). It also beats an oracle baseline that picks the best of six competing methods per cell by +5.4 points.
• Biggest gains on procedural tasks: SpreadsheetBench 41.8→80.7, OfficeQA 33.1→72.1, LiveMath 37.6→66.9. Near-ceiling factual tasks move least (SearchQA +9.6).
• Works in tool harnesses: +24.8 (Codex) and +19.1 (Claude Code) over no-skill on GPT-5.5; beats EvoSkill by +14.0 / +3.2.
• Small models benefit most in relative terms: GPT-5.4-nano roughly doubles on DocVQA and triples on ALFWorld — consistent with “a skill file supplies procedure that a small model doesn’t hold in its weights.”
• Cheap and compact: final skills are 379–1,995 tokens from 1–4 accepted edits (OfficeQA’s +39.0 came from a single edit). Training cost is 0.6M–46.4M tokens per test-point, paid once before deploy.
• Transfers: a Codex-trained spreadsheet skill moves to Claude Code for +59.7; a GPT-5.4 skill lifts every smaller GPT; an OlympiadBench skill gives positive gains on Omni-MATH. No transfer row falls below the target’s no-skill baseline.

Ablations that matter (GPT-5.5 as both target and optimizer). Gains are insensitive to exact rollout batch, minibatch, and LR schedule (most cells wobble inside ±1.5 pts), but sensitive to the design pillars: removing both slow+meta update drops SpreadsheetBench 77.5→55.0 (−22.5); removing the rejected-edit buffer costs 1.6/4.6/2.4 pts; unbounded rewriting (“without lr”) underperforms any bounded budget. Figure 3 shows the validation gate tends to pick checkpoints that generalize to test, not just fit the gate.

Caveats — read these before you sell it.

• Needs a reliable scorer. The entire loop rides on r(s) and a held-out split. It shines where you have automatic verifiers, exact-match, or executable checks. For subjective/open-ended work you’d need a trustworthy judge model or human eval, and the gate is only as good as that judge.
• Model names are near-future / possibly synthetic. “GPT-5.5”, “GPT-5.4-nano”, “Qwen3.6” and several cited papers carry 2026 arXiv ids. Treat the mechanism as the takeaway; treat the exact leaderboard as illustrative, and re-run on the models you actually deploy.
• Single skill, single domain. By design it trains one portable file, not a library. Highly heterogeneous domains needing many disjoint procedures may not fit one document.
• Training isn’t free. It’s cheaper than fine-tuning and amortizes over reuse, but a one-off task may not justify the rollout + optimizer spend.
• The oracle-gap framing flatters. “+5.4 over an oracle that picks the best of six methods per cell” is a strong claim, but the oracle is an artificial construct; the more honest number is “+5.4 over the best-achievable cherry-pick,” which is still good but not magic.

How You’d Use It

This maps cleanly onto an AI-services business, because the deliverable is a static, auditable, model-agnostic text file — not a fine-tune you have to host.

• Productize “skill tuning” as an offering. For any client task with a checkable outcome (extraction accuracy, spreadsheet correctness, ticket-field compliance, retrieval QA), you sell a one-time optimization run that yields a best_skill.md they drop into their existing agent. No model access needed beyond API calls; no weights to manage; the artifact is theirs to read and edit.
• It’s a natural fit for your MAS work. In a multi-agent system, each role already has a system prompt / skill. SkillOpt is a per-role training loop: freeze the role’s model, optimize its skill against role-level success, ship the file. The “optimizer model” is just another (offline) agent — this is orchestration you already know how to build.
• Harness-portability is the commercial moat. The same file format runs in direct chat, Codex, and Claude Code, and transfers between them (+59.7 in the paper). You can optimize once and deploy across a client’s chat product, their coding agent, and their batch pipeline.
• Auditability sells to enterprise. Every change is logged (edit_apply_report.json), the artifact is <2k tokens of plain English a domain expert can read in minutes, and there’s a protected region for durable policy. That’s a much easier compliance story than a fine-tuned checkpoint.
• Where it slots in: it replaces “our senior prompt engineer hand-tunes the skill and hopes” with “we run a gated optimization and can prove the shipped file beats the starting file on held-out data.” That proof is the value proposition.

Build Your Own (Minimal Recipe)

You can capture ~80% of the value in a weekend for one benchmark. The gate is the point; everything else can start crude.

Components (build order):

1. Harness adapter — one function run(model, task, skill) -> (trace, score). Prepend skill to the system prompt, call the model, score with the task’s checker. This is where most real work lives (wiring the scorer).
2. Three splits — deterministic train/selection/test from your task set (a fixed seed). Non-negotiable: selection ≠ test.
3. Rollout + bucket — run N train tasks, split into failures/successes.
4. Optimizer call — one prompt: “here are K failed traces and the current skill; propose at most L generalizable append/insert_after/replace/delete edits as JSON.” (The paper’s Appendix C prompts are copy-pasteable.) Apply the JSON edits to the markdown.
5. The gate — score the candidate on the selection split; keep it only if strictly better; save the best-ever to best_skill.md. This 5-line check is what separates SkillOpt from “ask a model to rewrite my prompt.”

Add next (the last 20%): the edit budget L with a cosine decay; a rejected-edit buffer you feed back into the optimizer prompt; the epoch-wise slow update writing a protected <!-- SLOW_UPDATE --> region.

Reach for: any capable model as the optimizer (the paper shows a stronger optimizer helps and costs nothing at deploy, but a target-matched optimizer still recovers 56–74% of the gain); JSON-mode / structured output for parseable edits; a skill-hash cache so you never re-score an identical candidate; difflib/simple string ops to apply patches.

The one or two genuinely hard parts: (a) a scorer that is cheap, deterministic, and actually correlates with the outcome you care about — garbage score, garbage skill; (b) making the optimizer propose general rules, not task-specific ones — the merge/rank step and the “must generalize, don’t hardcode” instruction are doing real work.

How to Improve It

Each of these is a testable extension you (or a client project) could actually run.

• Preference / rubric gates for open-ended tasks. The biggest limitation is the need for a scalar scorer. Swap the strict > gate for a pairwise-preference judge (or a small learned reward model) so SkillOpt applies to writing, support replies, and other subjective work. Test: does a preference-gated run still climb held-out human ratings?
• Skill libraries with a router. The paper trains one file per domain. Train many, add a lightweight classifier that picks the right skill per request, and share the optimizer/meta infrastructure. Test: does a routed library beat one monolithic skill on a heterogeneous task mix?
• Reuse the meta skill across domains. The optimizer-side “which edits help here” memory is currently thrown away per run. Persist and transfer it as a training accelerator so new domains converge in fewer epochs. Test: does a warm-started optimizer hit the same score with fewer rollouts?
• Self-distill the skill back into weights. For open models, use the optimized skill as a teacher to generate training data, then fine-tune so the procedure lives in weights (the paper flags this in its Outlook). Test: does distillation let you drop the skill file with no accuracy loss?
• Regularize against skill bloat / overfit. Skills grew up to ×53. Add an explicit length penalty or an L1-style “prefer deleting a stale rule over adding one” term to the ranking, and a periodic consolidation pass. Test: can you hold accuracy while shrinking the file, improving transfer?
• Curriculum + adaptive LR. The “autonomous” schedule is barely explored. Let the edit budget respond to the selection-score slope (big edits while improving fast, tiny edits near plateau) and order tasks easy→hard. Test: fewer epochs to the same test score?

Glossary

• Skill (document) — a plain-text/markdown policy prepended to a frozen agent that encodes procedures, tool policies, and output rules; here it’s the trainable object.
• Target model M — the frozen model doing the actual task; its weights never change.
• Optimizer model O — a (usually frontier) model used only during offline training to propose skill edits; not shipped at deploy.
• Harness h — the execution environment: direct chat, or a CLI agent like Codex / Claude Code with tools and files.
• Rollout — one run of the model on a task inside the harness, producing a trace and a score.
• Trajectory τ — the full record of a rollout: messages, tool calls, observations, outputs.
• Train / selection / test splits — experience source / validation gate / final-report set; selection is the “validation set” that decides accept-or-reject.
• Validation gate — the strict rule that a candidate skill is adopted only if its selection score is strictly higher than the current skill’s.
• Textual learning rate (edit budget L_t) — the max number of atomic edits allowed per step; the text-space analogue of a learning rate, with constant/linear/cosine/autonomous schedules.
• Atomic edit — one of append / insert_after / replace / delete applied to the skill markdown.
• Rejected-edit buffer — an epoch-local memory of edits that failed the gate (and the failures they targeted), fed back to the optimizer as negative feedback.
• Slow / meta update — an epoch-boundary consolidation: durable guidance written to a protected region of the skill (slow), plus optimizer-only “how to edit better here” memory that is never shipped (meta); together they act like momentum.
• Protected region — a <!-- SLOW_UPDATE_START ... END --> block that fast step-level edits cannot overwrite.
• Held-out — data not used for training the skill, so scores on it measure generalization rather than fit.
• Best-or-tied cell — for one (model, benchmark, harness) combination, SkillOpt equals or beats every measured baseline.