GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning

TL;DR

The standard way to adapt an LLM system to a task is reinforcement learning (RL) — run the system thousands of times, collapse each run into a single number (the reward), and use those numbers to estimate gradients that shift the model’s weights. That throws away almost everything: a whole rollout of reasoning, tool calls, and error messages becomes one scalar. GEPA’s bet is that the text of those rollouts is a far richer learning signal than the scalar, because modern LLMs can read and reflect on it. GEPA samples a few rollouts, has an LLM read the traces to diagnose what went wrong, rewrites the offending prompt, and tests it — while maintaining a Pareto frontier of candidates so it doesn’t get trapped optimizing one local winner. Across four tasks (multi-hop QA, instruction-following, fact verification, privacy-aware delegation) it beats GRPO by up to 19% using up to 35x fewer rollouts, and beats the prior best prompt optimizer (MIPROv2) by 10%+. It also works as inference-time search for code generation, pushing GPT-4o’s NPU-kernel utilization from 4% to 30%.

Problem & Motivation

Here is the concrete pain. You have a compound AI system — say a multi-hop retrieval pipeline with a query-writer module, a summarizer module, and an answerer module, each driven by an LLM prompt. You want it to perform better on your task. The popular answer is RL with verifiable rewards (RLVR), with GRPO as the workhorse algorithm. The catch: GRPO in practice needs tens of thousands to hundreds of thousands of rollouts to fit a new task. Each rollout may fire expensive tool calls, hit rate-limited APIs, or require GPU time you don’t have. And if your best model is a frontier API model, you often cannot fine-tune its weights at all.

Worse, RL is throwing away information by design. A rollout produces a rich natural-language trace — the module instructions, the chain-of-thought, the tool calls and their outputs, even the compiler error messages inside the reward function. RL collapses all of that into one scalar reward at the end, then back-propagates a gradient from it. That is an enormous compression. The signal-to-effort ratio is terrible: thousands of runs to extract a few bits of gradient direction.

The authors’ observation: LLMs are built to read language. If you keep the trace in natural language and let an LLM reflect on it — “the answer failed because the query in hop 2 never mentioned the second entity” — you get targeted, interpretable credit assignment for free, no gradients required. That’s a much higher-bandwidth learning medium than a scalar.

What’s New (Core Contribution)

GEPA’s novelty is the specific combination of three ideas into a working optimizer, plus one genuinely new operator:

1. Reflective prompt mutation as the learning step (before: scalar-reward gradients → now: LLM reads the trace and rewrites the prompt). Prior prompt optimizers like MIPROv2 use Bayesian search over instruction/demo candidates scored only by a number. GEPA instead feeds the full execution trace plus evaluation trace to an LLM and asks it to diagnose and propose a new instruction. The trace, not the score, drives the update.

2. Evaluation traces as diagnostic signal (before: metric returns a scalar → now: metric also returns feedback_text). GEPA wraps the eval metric µ into a feedback function µf that surfaces the natural-language byproducts the metric already computes — compiler errors, which constraints failed, which documents were still missing — and hands them to the reflection step. This is the cheapest, highest-leverage change in the paper.

3. Pareto-based candidate selection (before: greedily evolve the single best candidate → now: keep every candidate that’s best on at least one task instance, sample among them). Greedy “always mutate the current best” collapses into a local optimum: it finds one good strategy and burns the whole budget failing to improve it. GEPA instead tracks, per training instance, which candidates achieve the top score, and stochastically samples from that frontier — preserving diverse winning strategies.

4. System-Aware Merge / crossover (new operator). When two candidates have independently improved different modules of the system, GEPA can splice them — take module A’s best version from lineage 1 and module B’s from lineage 2 — to produce a single candidate combining complementary lessons.

The honest read: ideas 1–3 each have precedent (Reflexion-style self-critique, DSPy/MIPRO prompt optimization, MAP-Elites “illumination”). The contribution is welding them into one sample-efficient loop and showing it beats weight-space RL, which is a strong and somewhat surprising claim.

How It Works (Technically)

GEPA optimizes the parameters of a compound system Φ — concretely, the set of prompts (and optionally few-shot demos), written ⟨Π, Θ⟩. A “candidate” is one full assignment of prompts to all modules. GEPA never touches model weights; it only evolves text.

Inputs: the system Φ with its starting prompts, a training set Dtrain, the eval metric µ, a feedback function µf, and a rollout budget B. Dtrain is split into Dfeedback (used to derive learning signal) and Dpareto (a held-out validation set used only for scoring/selecting candidates).

The loop has three interacting parts.

1. The genetic loop (Section 3.1). Maintain a pool of candidates P, starting with just the base system. Each candidate records its parent (ancestry), so lessons accumulate down a genetic tree. While budget remains: select a candidate to evolve, mutate it (reflection) or merge two (crossover), test the child on a small minibatch, and — only if it improved over its parent on that minibatch — pay to evaluate it fully on Dpareto and add it to the pool. At the end, return the candidate with the best average Dpareto score. The minibatch gate is the sample-efficiency trick: cheap local test first, expensive full eval only for survivors.

2. Reflective prompt mutation (Section 3.2) — the actual “learning”. Pick a target module within the chosen candidate (round-robin, so every module eventually gets attention). Run the system on a minibatch from Dfeedback, recording each rollout’s trace and outcome. Extract that module’s inputs, outputs, and reasoning from the traces, plus the feedback_text from µf. Hand all of it to an LLM with a meta-prompt that says, roughly: here is the module’s job, here are runs that succeeded and failed and why — propose a better instruction. The LLM performs implicit credit assignment (attributing the system’s success/failure to this module’s prompt) and writes a new instruction. The child candidate is a copy of the parent with that one module’s prompt swapped. This is the replacement for the gradient step — and it’s why one update can produce a huge jump (Figure 5 shows a single mutation taking PUPA from 82% to 91%).

3. Pareto-based selection (Section 3.3) — escaping local optima. This is Algorithm 2 and the part worth slowing down on. Build a scores matrix S: rows are candidates, columns are the Dpareto task instances, entries are scores. For each task instance i, find the best score any candidate achieved, s*[i] = maxₖ S[k][i], and collect the set of candidates that hit it, P*[i]. Union those sets across all instances → C, the candidates that are best on at least one task. Then prune dominated candidates: if candidate X is best only on tasks that candidate Y is also best on (and Y wins more), drop X. Finally, sample a candidate with probability proportional to how many task instances it’s the champion of (f[Φ]). Net effect: you keep every distinct winning strategy alive and bias exploration toward the ones with the broadest wins — exploration and exploitation in one rule, no temperature to tune.

Let me demystify the two equations that matter (Algorithm 2, lines 4–5):

• s*[i] ← maxₖ S[P[k]][i] — “the best score any candidate has ever achieved on task instance i.” This defines the Pareto frontier per-task.
• P*[i] ← {P[k] : S[P[k]][i] = s*[i]} — “the set of candidates that tie for that best score on i.” These are the champions of task i.

There’s no calculus here. “Pareto frontier” sounds heavy but operationally it’s just: don’t throw away a candidate as long as it’s the reigning best on some problem, even if its overall average is mediocre. That candidate may hold the one insight that, when merged, unlocks the global best.

Trace one example through (HotpotQA): Base system’s hop-2 query-writer has the prompt “Given question and summary_1, produce query.” It keeps missing the second entity, so retrieval fails and the final answer is wrong → score 0 on those instances. GEPA runs a minibatch, the µf feedback reports “documents still missing: [X, Y],” the reflection LLM reads the trace and rewrites the prompt to “…identify gaps in hop-1’s documents, use explicit names/locations from the summary to surface new documents, avoid restating the answer.” That child scores higher on the minibatch → gets full-evaluated → enters the pool. Because it’s now the champion on the multi-entity instances (even if weaker elsewhere), Pareto selection keeps sampling it for further refinement. Eventually Merge splices this improved query-writer with a lineage that improved the answerer module.

Architecture & data flow

flowchart TD
  A[Base system Φ with starting prompts] --> B[Candidate pool P]
  B --> C{SelectCandidate<br/>Pareto-based}
  C --> D[SelectModule round-robin]
  D --> E[Run on minibatch from D_feedback]
  E --> F[Collect traces + feedback_text via μf]
  F --> G[Reflection LLM:<br/>diagnose + rewrite prompt]
  G --> H[Child candidate:<br/>one module's prompt swapped]
  H --> I{Improved on<br/>minibatch?}
  I -- no --> B
  I -- yes --> J[Full eval on D_pareto<br/>update scores matrix S]
  J --> K[Add child to pool,<br/>record parent ancestry]
  K --> L{Budget left?}
  L -- yes --> C
  L -- no --> M[Return best avg-score<br/>candidate on D_pareto]
  C -. alternate path .-> N[System-Aware Merge:<br/>splice best modules<br/>from two lineages]
  N --> I

Interactive: the scores matrix and Pareto frontier. Each row is a candidate prompt, each column a task instance; highlighted cells are per-task champions. Click "greedy" vs "pareto" to see why keeping all champions (not just the best average) finds a higher peak. Schematic, not the paper's exact numbers.

The algorithm, simplified

# GEPA core loop. llm(), run_system(), feedback_fn() are stubbed.
# A "candidate" is a dict {module_name: prompt_string}.

def gepa(base_system, D_feedback, D_pareto, mu_f, budget, b=3):
    pool    = [base_system]                       # candidates
    parents = [None]                              # ancestry, parallel to pool
    S = [eval_all(base_system, D_pareto, mu_f)]   # scores matrix: S[k][i]

    while budget > 0:
        k = select_candidate(pool, S)             # Pareto-based, see below
        module = round_robin_module(pool[k])      # ensure every module gets turns
        mb = sample(D_feedback, b)                # tiny minibatch

        # --- the "gradient" step is just reflection over text ---
        traces, scores = run_system(pool[k], mb, mu_f)   # mu_f returns feedback_text too
        before = mean(scores); budget -= b
        new_prompt = llm(meta_prompt(module, pool[k][module], traces))  # diagnose + rewrite
        child = {**pool[k], module: new_prompt}          # copy, swap one module

        _, after_scores = run_system(child, mb, mu_f)    # cheap local test
        budget -= b
        if mean(after_scores) > before:                  # only survivors get full eval
            S.append(eval_all(child, D_pareto, mu_f)); budget -= len(D_pareto)
            pool.append(child); parents.append(k)

    return pool[argmax(mean(row) for row in S)]          # best average on D_pareto

def select_candidate(pool, S):
    n_tasks = len(S[0])
    champions = set()                                    # candidate-of "best on >=1 task"
    win_count = [0] * len(pool)
    for i in range(n_tasks):
        best = max(S[k][i] for k in range(len(pool)))
        for k in range(len(pool)):
            if S[k][i] == best:
                champions.add(k); win_count[k] += 1      # f[Φ] = how many tasks it wins
    champs = list(champions)                             # (paper also prunes dominated ones)
    return sample_proportional(champs, [win_count[k] for k in champs])

Built on Prior Work

Results & Evidence

Setup: four tasks — HotpotQA (multi-hop QA), IFBench (instruction-following, with out-of-distribution test constraints), HoVer (multi-hop claim verification), PUPA (privacy-aware delegation). Two models: open Qwen3-8B and proprietary GPT-4.1-mini. Budget is matched across optimizers per benchmark (within ~10%) so differences come from the algorithm, not search budget.

Headline numbers:

• vs GRPO (24,000 rollouts, LoRA): GEPA wins by up to 19% (HotpotQA) and +10% on average, using up to 35x fewer rollouts. GEPA matches GRPO’s best validation score after as few as 6–402 rollouts on some tasks (up to 78x sample efficiency).
• vs MIPROv2: GEPA wins on every benchmark and both models, by up to 11.1%. Aggregate gain over baseline +14–16% vs MIPROv2’s +7% — more than double.
• Pareto vs greedy ablation: Pareto selection beats “always evolve the best” by up to 8.17%, +6.4% aggregate.
• Prompt length: GEPA prompts are up to 9.2x shorter than MIPROv2’s (which bloats from joint few-shot optimization) — cheaper inference, lower latency.
• Inference-time search (preliminary): on NPUEval kernels, GPT-4o’s Sequential10 jumps from 4.25% → 30.52% mean vector utilization (some kernels hit 70%), beating RAG (16%) and RAG+MIPROv2 (19%). On KernelBench CUDA, fast₁ goes from ~0% to >20%.

What the evidence does NOT establish — read this before selling it:

• Only four tasks, mostly from one prior suite (Tan et al. 2025). Strong, but not a broad battery; generalization to your domain is unproven.
• GRPO ran with LoRA, not full fine-tuning, and with hand-explored hyperparameters. The “RL is worse” claim is real for this RL setup, not a universal verdict.
• Validation rollouts dominate GEPA’s budget. Most counted rollouts go to scoring candidates on Dpareto, not learning. The authors note you could shrink the validation set — but the impressive “35x fewer” ratio depends on how you count.
• Merge is fragile. GEPA+Merge helped GPT-4.1-mini but degraded Qwen3-8B on 3 of 4 tasks with the same hyperparameters. Crossover timing/budget is unsolved.
• The kernel-optimization results are explicitly “preliminary” with a single model (GPT-4o) and small task counts.

How You’d Use It

This is unusually well-suited to an AI services business, because it optimizes prompts of an existing system without touching weights — which is exactly the constraint when your clients use frontier APIs.

• Client offering: “prompt optimization as a service.” Take a client’s existing compound system (RAG pipeline, agent, classifier chain), wrap their eval metric to emit feedback_text, run GEPA on a few hundred labeled examples, hand back optimized prompts + a measured lift. No GPUs, no fine-tuning contract, works on GPT-4.x/Claude/Gemini. This is a deliverable you can scope in days, not weeks.
• Replace flaky few-shot stuffing. If you’re currently pasting 5 long demos into prompts (and paying for those tokens every call), GEPA’s instruction-only prompts are up to 9x shorter and generalize better — direct margin improvement on token costs.
• Multi-agent systems (your ARC MAS background applies directly). GEPA optimizes every module’s prompt with per-module credit assignment via round-robin + µf. For a MAS where one agent’s instruction is silently sabotaging the whole pipeline, GEPA’s trace reflection is the credit-assignment mechanism you wished you had.
• Inference-time search for hard generation tasks. Put the whole task set in as both train and pareto, let GEPA “overfit” by iteratively proposing better solutions — and inject domain docs via µf (retrieve manual sections keyed on the compiler error). This is a strong pattern for code-gen, config-gen, or any task with a rich textual verifier.

Realistic effort: GEPA is open-sourced and DSPy-aligned. The hard part of standing it up is not the algorithm — it’s writing a good µf that surfaces actionable feedback text from your evaluator.

Build Your Own (Minimal Recipe)

You can capture ~80% of the value in a weekend without Merge or the full Pareto machinery.

1. Wrap your metric into a feedback function. This is the single most important piece. Instead of score = metric(output, gold), return (score, feedback_text) where the text explains why — “missing constraints: X, Y”, “compiler error: …”, “retrieved 2/4 gold docs”. Garbage feedback → garbage mutations.
2. Write the reflection meta-prompt. Template: “This module’s job is {role}. Here are runs with their traces and outcomes: {traces}. Here is what the evaluator said: {feedback}. Diagnose what the current instruction gets wrong and propose a better instruction. Output only the new instruction.”
3. Implement the genetic loop with a minibatch gate. Pool of candidates, round-robin module selection, mutate → test on a 3-example minibatch → only full-evaluate survivors. This gate alone delivers most of the sample efficiency.
4. Start with greedy selection, then add Pareto. Greedy (always evolve current best) is 20 lines and works okay. Add the per-task champion set + proportional sampling once greedy stalls in a local optimum (you’ll see the search tree go one-deep and stop, exactly as Figure 6a shows).
5. Skip Merge initially. It’s the least robust part and needs lineage divergence to pay off.

Reach for: DSPy (compound-system abstraction + a GEPA optimizer exists), any chat model for both the system and the reflection (the reflection model can be a stronger model than the system model). The genuinely hard parts: (a) µf quality, (b) deciding Dfeedback/Dpareto split sizes so you don’t blow the budget on validation.

How to Improve It

1. Adaptive / subsampled validation. The authors admit most rollouts go to scoring on the full Dpareto. Track candidates on a dynamically selected validation subset (bandit-style: spend more eval on candidates that look promising). This directly attacks the biggest cost.
2. Learned Merge timing. Merge helps only when lineages have diverged into complementary module improvements. Detect lineage divergence (e.g., which modules each lineage has mutated) and trigger Merge automatically instead of a fixed “max 5 times” — fixing the Qwen3 degradation.
3. Better module selection than round-robin. Round-robin spends equal budget on every module. Use the credit-assignment signal from µf to prioritize the module most implicated in failures — a softmax over per-module blame.
4. Hybridize with RL, don’t replace it. GEPA gets you a strong prompt cheaply; a short GRPO run could then fine-tune weights around that prompt. Reflection for the coarse high-bandwidth gains, gradients for the last mile.
5. Reflection-model ensembling / self-consistency. A single reflection call can mis-diagnose. Sample several reflections, propose multiple children, and let the Pareto front sort them out — trading a few cheap LLM calls for fewer wasted full-evaluations.
6. Cache and transfer lessons across tasks. In inference-time mode GEPA already reuses insights across tasks. Persist a library of reflectively-discovered “rules” and seed new optimization runs with them — a compounding asset for a services firm.

Glossary

• Compound AI system (Φ) — a pipeline of multiple LLM calls/modules (e.g., query-writer → summarizer → answerer) plus tools, optimized as one unit.
• Rollout — one full execution of the system on one task instance, producing a trace and a score. The unit of “cost” in the paper.
• GRPO (Group Relative Policy Optimization) — an RL algorithm that runs a group of rollouts, computes each one’s advantage relative to the group’s mean reward, and updates model weights toward higher-advantage outputs. Sample-hungry.
• Advantage — in RL, how much better a given outcome is than the baseline (here, the group average); the signal that tells the gradient which direction to move.
• RLVR (RL with Verifiable Rewards) — RL where the reward comes from an automatic checker (tests pass, answer correct) rather than a learned reward model.
• Policy gradient — updating a model’s weights in the direction that increases expected reward; what GEPA deliberately avoids.
• Reflective mutation — GEPA’s learning step: an LLM reads execution traces and rewrites a prompt, replacing the gradient update.
• Feedback function (µf) — the eval metric augmented to also return natural-language feedback_text (errors, missing items) for reflection.
• Pareto frontier (per-instance) — the set of candidates each of which is best on at least one task instance; nothing on the frontier is strictly dominated by another candidate across all tasks.
• Dominated candidate — one whose set of “tasks it wins” is a subset of another candidate’s; pruned from selection.
• Illumination / MAP-Elites — an evolutionary strategy that keeps a diverse archive of elites across a behavior space rather than a single global best.
• System-Aware Merge (crossover) — splicing two candidates by taking the best version of each module from different lineages.
• MIPROv2 — prior SOTA prompt optimizer using Bayesian search over instructions + few-shot demos, scored only by a scalar.
• LoRA — low-rank adapters; a parameter-efficient fine-tuning method (used in the GRPO baseline here).
• Minibatch gate — testing a mutated candidate on a tiny sample first, and only paying for full validation if it improves — the core sample-efficiency trick.