Towards a Science of Scaling Agent Systems

TL;DR

Everyone repeats “more agents is all you need,” but nobody had measured when a multi-agent system actually beats a single agent on real, interactive tasks. This paper runs 180 carefully matched experiments — 5 architectures × 9 models × 4 agentic benchmarks, all with identical tools, prompts, and token budgets — and finds the answer is wildly task-dependent: centralized coordination gives +81% on decomposable financial analysis, while every multi-agent variant degrades sequential planning by 39–70%. The core finding is that coordination is a tax, not a free lunch: under a fixed token budget, splitting work across agents fragments the per-agent budget and amplifies errors (independent agents amplify errors 17.2×; a central orchestrator contains it to 4.4×). They fit a 20-parameter regression on measurable coordination metrics that explains 51% of performance variance out-of-sample and correctly picks the best architecture 87% of the time. The practical takeaway: if your single agent already clears ~45% accuracy, or your task is sequential, more agents will probably make it worse.

Problem & Motivation

The pain is concrete and it costs money. You are building an agentic feature for a client. Do you ship one strong agent with tools, or an orchestrated team of specialists? The field’s folklore says “teams win on hard tasks,” backed by papers like More Agents Is All You Need and various “collaborative scaling law” claims. So you build the multi-agent version, it burns 15× the tokens (Anthropic’s own number), and on half your tasks it performs worse than the single agent you started with. There was no principled way to predict this in advance.

Two methodological failures kept the field stuck:

1. Confounded comparisons. Prior multi-agent vs. single-agent studies used different prompts, different tools, and different compute budgets for each architecture. So when a team “won,” you couldn’t tell whether it was the coordination or just that it got more tokens to think with. Causal attribution was impossible.
2. Wrong benchmarks. Most multi-agent wins were demonstrated on non-agentic tasks — single-shot things like HumanEval or GSM8K, where five agents voting just cancels random errors (an ensemble effect). On genuinely agentic tasks (sustained tool use, partial observability, adaptive strategy) the dynamics flip: errors cascade through execution chains instead of canceling, and agents drift onto divergent world states (only 34% trajectory overlap after 10 interactions). The encouraging benchmark numbers were measuring the wrong thing.

The deeper tension: a single agent maintains one unified memory stream — every reasoning step sees the full history (constant-time global context). A multi-agent system fragments that context; the global picture has to be lossily compressed into inter-agent messages. That compression is the “coordination tax,” and nobody had quantified when the tax is worth paying.

What’s New (Core Contribution)

Three things, each a real delta over the prior literature:

• A confound-controlled benchmark of agent architectures. Before: architectures compared with mismatched prompts/tools/budgets. Now: 180 configurations holding tools, prompt structure, and total token budget identical across all five architectures, so performance differences are causally attributable to coordination structure alone. (Fairness detail: multi-agent teams get fewer per-agent iterations to keep total budget matched against the single agent’s longer solo reasoning.)
• A predictive scaling law from measured coordination metrics, not architecture labels. Before: “use a hierarchical MAS for hard tasks” (categorical, heuristic). Now: a mixed-effects regression on continuous, trace-measurable quantities — efficiency, overhead, error amplification, redundancy, message density — that hits cross-validated R²=0.513 (and R²=0.89 leave-one-domain-out). Crucially it has no dataset-specific parameters, so it extrapolates to unseen task domains. The model with measured metrics (R²=0.513) beats one using only architecture labels (0.43) or only model intelligence (0.28).
• Three quantified mechanistic laws that replace folklore with numbers: (1) a tool-coordination trade-off (β=−0.330, the single strongest effect) — tool-heavy tasks suffer disproportionately from coordination overhead; (2) a capability ceiling (β=−0.408) — once a single agent clears ~45% accuracy, extra agents give negative returns; (3) topology-dependent error amplification — independent agents 17.2×, centralized 4.4×, because the orchestrator acts as a validation bottleneck.

What is not new: the agent/MAS taxonomy itself (Independent/Centralized/Decentralized/Hybrid) is standard, and the individual benchmarks are pre-existing. The novelty is the controlled measurement and the predictive equation built on top.

How It Works (Technically)

The formal setup, in plain terms

An agent system is a tuple S = (A, E, C, Ω): a set of agents A, a shared environment E, a communication topology C, and an orchestration policy Ω. When |A|=1 it’s a Single-Agent System (SAS); |A|>1 is a Multi-Agent System (MAS). Each agent is itself (Φ, A_actions, M, π) — a reasoning policy Φ (the LLM), an action space A_actions (tool calls), internal memory M, and a decision function π mapping observation history to the next action.

The per-step loop is just the standard agent loop written formally:

• α_{i,t} = π_i(h_{i,t}) — agent i picks an action from its history (the LLM reads the transcript and emits the next tool call).
• o_{i,t} = E(α_{i,t}) — the environment returns an observation.
• h_{i,t+1} = h_{i,t} ⊕ (α_{i,t}, o_{i,t}) — append the (action, observation) pair to history, truncating at MAX_TOKENS.

Nothing exotic — this is the ReAct loop in math notation. The interesting part is what C (topology) does to the budget and the errors.

The five architectures differ only in topology C and aggregation Ω:

• SAS — one loop, C=∅, complexity O(k) for k reasoning steps. Zero communication overhead, full context integration, but no decomposition or self-checking.
• Independent MAS — n agents, C=∅, outputs combined by majority vote/synthesis. O(nk). Maximal parallelism, zero error correction.
• Centralized MAS — orchestrator + n sub-agents, C = orchestrator↔agents. O(rnk) over r rounds. The orchestrator is a validation bottleneck: it reviews sub-agent outputs before aggregating.
• Decentralized MAS — all-to-all peer debate, C = every pair. O(dnk) over d debate rounds. Consensus through discussion.
• Hybrid MAS — orchestrator hierarchy plus limited peer edges. Most communication channels, highest overhead (up to 515%).

The coordination metrics (this is the load-bearing part)

The whole predictive model is built from five quantities you can measure directly from execution traces, no human labels needed:

• Coordination overhead O% = (T_MAS − T_SAS)/T_SAS × 100% — extra token/turn cost vs. the single agent. Ranges 58% (independent) to 515% (hybrid).
• Message density c — inter-agent messages per reasoning turn.
• Redundancy R — mean cosine similarity of agents’ output embeddings (how much they’re duplicating each other). 0.41–0.50 for MAS.
• Coordination efficiency E_c = S / (T/T_SAS) — success per unit of relative cost. SAS=0.466; MAS ranges 0.074 (hybrid) to 0.234 (independent). This is the key one: MAS is 2–6× less efficient per token.
• Error amplification A_e = E_MAS / E_SAS — relative failure probability. SAS=1.0, centralized=4.4, hybrid=5.1, decentralized=7.8, independent=17.2.

The scaling law, demystified

The model is one big linear regression (Eq. 1) with 20 standardized predictors: main effects (intelligence I, I², log tool count, single-agent baseline P_SA, the coordination metrics) plus nine hand-chosen interaction terms. “Mixed-effects” just means it also accounts for benchmark-level grouping so it doesn’t overfit one dataset. Don’t memorize the equation — memorize what its biggest coefficients say:

• E_c × T = −0.330 (the dominant effect). Efficiency loss compounds with tool count. Worked example from the paper: on a 16-tool task, the efficiency penalty is −2.46 for SAS’s high efficiency vs. only −0.39 for MAS’s low efficiency… wait — read it the operational way: because MAS efficiency is low and gets multiplied by many tools, tool-rich environments magnify the coordination tax. Below ~4 tools the effect nearly vanishes (|ΔP|<0.05), which is exactly why MAS can win on simple decomposable tasks.
• P_SA × log(1+n_a) = −0.408 (the capability ceiling / “baseline paradox”). The higher your single-agent baseline already is, the more adding agents hurts. Solving for the break-even: the decision boundary lands at raw single-agent accuracy ≈ 0.45. Above it, don’t add agents.
• O% × T = −0.141. Overhead cost scales super-linearly with task complexity. There’s a threshold O%_max(T) beyond which no coordination benefit can recover the overhead; for T=16 tools it’s ~150% overhead — which rules out every MAS except decentralized (whose parallelization compensates).
• A_e × T = −0.097. Errors propagate worse in tool-rich settings. For independent MAS with 16 tools: −0.097 × 17.2 × 16 ≈ −26.7 standardized units — catastrophic. This is why independent agents universally underperform: no inter-agent check, so every agent’s mistakes survive to the final answer.
• I² = +0.256 (accelerating returns to model quality). Smarter base models benefit disproportionately — top-quartile models (Intelligence Index > 60) beat the linear prediction by 23%. Practically: capability improvements compound, so a better base model often beats a more elaborate architecture.

The headline result of the law: it picks the optimal architecture for 87% of held-out configs (vs. 20% random, 54% capability-only).

A concrete trace through the logic

Take Finance-Agent (decomposable: split into revenue analysis, cost analysis, market comparison, then synthesize). Single-agent baseline ≈ 0.35 — below the 0.45 ceiling, so there’s room to gain. Tool count is moderate (T≈5), so the E_c × T and O% × T penalties stay small. Centralized adds a validating orchestrator (A_e only 4.4). Result: +80.9%. Now take PlanCraft (Minecraft planning: each action mutates inventory state the next action depends on — strictly sequential). Splitting it across agents means each one burns its shrunken token budget on state-tracking before it can even message a peer; the compression destroys reasoning quality. Result: −39% to −70% for every MAS variant. Same model, same tools — opposite outcome, fully explained by decomposability + the budget-fragmentation mechanism.

Architecture & data flow

flowchart TD
  T[Task] --> P{Measure task properties:<br/>tools T, baseline P_SA,<br/>decomposability}
  P --> M[Scaling-law equation<br/>20 standardized predictors]
  M --> D{Predicted best<br/>architecture?}
  D -->|"P_SA > 0.45 or sequential"| SAS[Single-Agent System<br/>O of k, A_e=1.0]
  D -->|"decomposable, mid tools"| CEN[Centralized MAS<br/>orchestrator validates<br/>A_e=4.4]
  D -->|"tool-heavy, parallel"| DEC[Decentralized MAS<br/>peer debate, A_e=7.8<br/>but parallel efficiency]
  SAS --> R[Run + measure<br/>E_c, O%, A_e, R, c]
  CEN --> R
  DEC --> R
  R -.->|feeds back as<br/>empirical metrics| M

Interactive: drag the sliders for single-agent baseline and tool count to watch the predicted multi-agent advantage flip from positive to negative. The 0.45 baseline ceiling and the tool-count tax are the two crossover lines. Schematic, built from the paper's reported coefficients.

Interactive: how an initial error rate propagates to the final answer under each topology, using the paper's amplification factors (independent 17.2×, decentralized 7.8×, hybrid 5.1×, centralized 4.4×, single 1.0×). The orchestrator "bottleneck" is why centralized stays flattest.

The algorithm, simplified

The paper’s real contribution is a selection rule, not a training loop. Here’s the decision procedure as runnable-looking Python:

# Empirical coordination metrics measured from traces (Table 5 of the paper)
METRICS = {
    "SAS":          dict(E_c=0.466, O=0.0,   A_e=1.0,  R=0.0),
    "independent":  dict(E_c=0.234, O=0.58,  A_e=17.2, R=0.45),
    "centralized":  dict(E_c=0.120, O=2.85,  A_e=4.4,  R=0.41),  # orchestrator = validation bottleneck
    "decentralized":dict(E_c=0.150, O=2.63,  A_e=7.8,  R=0.50),  # peer debate, highest redundancy
    "hybrid":       dict(E_c=0.074, O=5.15,  A_e=5.1,  R=0.47),
}
# Standardized coefficients from the fitted scaling law (Eq. 1 / Table 4)
B = dict(I=-0.180, I2=0.256, logT=0.535, P_SA=0.319,
         Ec_T=-0.330, O_T=-0.141, Ae_T=-0.097, R_na=0.041, PSA_na=-0.408)

def predict_perf(I, T, P_SA, n_a, m):
    # m = the metrics dict for one architecture. All terms standardized.
    return (B["I"]*I + B["I2"]*I**2 + B["logT"]*log1p(T) + B["P_SA"]*P_SA
            + B["Ec_T"]*m["E_c"]*T          # efficiency tax compounds with tools (biggest term)
            + B["O_T"]*m["O"]*T             # overhead scales super-linearly with complexity
            + B["Ae_T"]*m["A_e"]*T          # errors cascade worse in tool-rich envs
            + B["R_na"]*m["R"]*n_a          # redundancy: weak positive error-correction
            + B["PSA_na"]*P_SA*log1p(n_a))  # baseline paradox: high baseline -> agents hurt

def choose_architecture(I, T, P_SA, n_a=3):
    # Score every topology, pick the predicted winner. This IS the paper's deployment tool.
    scores = {name: predict_perf(I, T, P_SA, n_a, m) for name, m in METRICS.items()}
    return max(scores, key=scores.get), scores

That’s it. No fine-tuning, no RL — the “science” is a calibrated formula you evaluate before deciding how many agents to ship.

Built on Prior Work

Results & Evidence

What was tested: 180 configs = 5 architectures × 9 models (GPT-5 nano/mini/full, Gemini 2.0 Flash / 2.5 Flash / 2.5 Pro, Claude Sonnet 3.7/4.0/4.5) × 4 benchmarks (Finance-Agent, BrowseComp-Plus, PlanCraft, Workbench), Intelligence Index 34–66, matched token budgets.

Headline numbers:

• Finance-Agent (decomposable): Centralized +80.9%, Decentralized +74.5%, Hybrid +73.2%.
• PlanCraft (sequential): all MAS degrade — Independent −70.0%, Centralized −50.4%, Decentralized −41.4%, Hybrid −39.0%.
• BrowseComp-Plus (dynamic web): Decentralized +9.2%, Centralized ~flat (+0.2%).
• Workbench (tool-heavy, T=16): marginal, −11% to +6%; Decentralized best because parallelism offsets overhead.
• Overall mean MAS effect: −3.5% (95% CI [−18.6%, +25.7%], σ=45.2%) — i.e., on average MAS does nothing good, with enormous variance.
• Model fits: R²_train=0.589, R²_CV=0.513, leave-one-domain-out R²=0.89, architecture-selection accuracy 87%.
• Team-size cost: turn count grows T ∝ n^1.724 under fixed budget — super-linear, capping practical teams at 3–4 agents.
• Model-agnostic: cross-family slope difference Δ_max=0.023, CV<0.02 — the scaling behavior holds across OpenAI/Google/Anthropic.

What the evidence establishes: Coordination value is genuinely task-contingent and predictable from measurable properties; the “more agents” heuristic is wrong on agentic tasks; error amplification is real and topology-dependent.

What it does NOT establish (caveats — read these before quoting the numbers):

• R²=0.513 means the model leaves ~half the variance unexplained; “predicts performance” is half-true. The flashier R²=0.89 is leave-one-domain-out and rests on only four domains — a thin basis for a “universal equation.”
• Prompts were held identical but not optimized per model/architecture. LLMs are prompt-sensitive; architecture-specific tuning could shift the rankings.
• The exact per-architecture metric values (E_c, A_e, etc.) come from Table 5 averages; treat the equation’s point predictions as directional, not precise.
• Four text-based benchmarks only — no embodied, multimodal, long-horizon, or multi-user tasks. The crisp thresholds (0.45 baseline, ~150% overhead) may not transfer.
• Team sizes were small (≤4). Nothing here speaks to large swarms.

How You’d Use It

For an AI services company, this is unusually directly monetizable as architecture diligence:

• Pre-sales scoping / “agent architecture audit.” Before committing to a multi-agent build for a client, measure two cheap things: (1) single-agent baseline accuracy on a sample of their tasks, (2) tool count and whether the task decomposes. If baseline > ~45% or the task is sequential, you can confidently quote a single strong agent + tools — cheaper to build, cheaper to run (avoiding the 15× token blowup), and likely better. That’s a defensible recommendation backed by a paper, not vibes.
• Default to single-agent + tools. The strongest practical message: most “let’s make it multi-agent” instincts are wrong on real interactive workloads. Reach for orchestration only on decomposable, parallelizable, moderate-tool, low-baseline tasks (classic: research synthesis, multi-source financial/market analysis).
• When you do go multi-agent, pick Centralized for safety. The orchestrator-as-validator caps error amplification at 4.4× vs. 17.2× for independent fan-out. If you’ve ever shipped a parallel “spawn N agents and merge” pattern, this paper is telling you that’s the single worst topology for error propagation.
• Cost/latency story for clients. T ∝ n^1.724 and 58–515% overhead give you concrete numbers to set expectations: a 3-agent team isn’t 3× the cost — it’s worse, super-linearly, often with negative performance return.
• Model upgrades over architecture. The accelerating I² returns mean “swap to the better base model” frequently beats “add coordination complexity.” Good for keeping client systems simple and your maintenance burden low.

Build Your Own (Minimal Recipe)

You don’t need to reproduce 180 runs to get 80% of the value — you need a task profiler and a thin orchestrator harness.

1. Harness with swappable topology (1–2 days). Wrap any LLM in a ReAct loop. Make topology a parameter that routes messages: none (SAS), fan-out+vote (independent), orchestrator (centralized), peer-debate (decentralized). LangGraph or a 200-line hand-rolled loop both work. Keep tools, prompts, and a total token budget identical across modes — that matched budget is the one non-negotiable design choice.
2. Instrument the five metrics (the genuinely useful part). Log per run: total turns/tokens (→ overhead O%), success (you need a validator), failure rate per architecture (→ A_e), inter-agent message count/turn (→ density c), and cosine similarity of agent outputs (→ redundancy R). Compute E_c = success / (turns/turns_SAS).
3. Build the validator first — this is the hard part. Every metric depends on knowing success/failure. For deterministic tasks (Workbench-style pass/fail) easy; for fuzzy outputs you need an LLM-judge or rubric (the paper reports Cohen’s κ ≈ 0.87–0.91 for theirs, so it’s doable but real work).
4. Profile new tasks cheaply. Run SAS a handful of times → get P_SA. Count tools T. Eyeball decomposability. Drop into the choose_architecture function above.
5. Libraries/models to reach for: LangGraph or AutoGen for topology; any frontier model family (results are model-agnostic); sentence-transformers for redundancy embeddings; a small held-out task set per client for P_SA calibration.

The one or two hard things: (a) a trustworthy success validator, and (b) genuinely matching token budgets across architectures — most teams accidentally give the multi-agent version more compute and then “discover” it’s better.

How to Improve It

1. Per-architecture prompt optimization (their own limitation iv). Re-run with DSPy/automatic prompt tuning per topology. The fixed-prompt design is clean for causal inference but probably understates MAS — a tuned orchestrator prompt might move the 0.45 ceiling. Testable: does prompt optimization shift the decision boundary?
2. Adaptive / hybrid-per-task routing. Instead of one architecture per task type, use the scaling law as an online controller: start single-agent, measure live E_c and A_e, escalate to centralized only when error amplification trends up. Effectively a learned policy over the topology — a clean RL framing where reward = success per token.
3. Sparse communication & early exit (their limitation vi). The overhead O% × T term is the killer. Test whether sparse message routing (agents talk only when their redundancy R drops below a threshold) or early-exit on consensus recovers the parallelization benefit without the token blowup.
4. Distilled coordinator. The orchestrator’s value is validation, which may not need a frontier model. Distill a small, cheap “validator/router” model and test whether centralized’s 4.4× error containment survives — that would make MAS economically viable.
5. Break the four-domain ceiling. The “universal equation” claim is the weakest link. Add embodied, long-horizon, and multi-user benchmarks and check whether the coefficients (especially the 0.45 baseline threshold and I² term) are stable. If they drift, the law is domain-specific, which is itself a publishable finding.

Glossary

• Agentic task — a task where interactive, multi-step environment feedback beats any single-shot answer by more than threshold δ (the paper’s formal definition). Web browsing yes; GSM8K no.
• SAS / MAS — Single-Agent System (one reasoning locus) vs. Multi-Agent System (multiple LLM agents communicating).
• Topology (C) — the inter-agent communication pattern: Independent (none), Centralized (hub-and-spoke through an orchestrator), Decentralized (all-to-all peer debate), Hybrid (orchestrator + some peer links).
• Orchestrator (Ω) — the agent that decomposes tasks, routes work, and validates/aggregates sub-agent outputs; acts as an error “bottleneck.”
• Coordination overhead (O%) — extra tokens/turns a MAS spends vs. the single agent, as a percentage (58%–515% here).
• Coordination efficiency (E_c) — success per unit of relative cost; the paper’s most predictive single metric.
• Error amplification (A_e) — ratio of MAS failure rate to SAS failure rate; how much the topology multiplies mistakes (1.0 to 17.2×).
• Redundancy (R) — how similar agents’ outputs are (cosine similarity); too much = wasted duplication, a little = error correction.
• Baseline paradox / capability ceiling — once single-agent accuracy exceeds ~45%, adding agents yields negative returns.
• Mixed-effects model — a regression that adds group-level (here, benchmark-level) random effects so the fit isn’t dominated by one dataset.
• R²_CV — cross-validated R²: variance explained on held-out data, the honest version of fit quality (0.513 here).
• Intelligence Index — an external composite capability score (reasoning + coding + knowledge) used to rank the nine models, 34–66.
• Inverted-U scaling — performance rises then falls as coordination complexity increases; the central qualitative finding.