Unifying Language Agent Algorithms with a Graph-Based Orchestration Engine (AGORA)

TL;DR

Building LLM agents today is a mess of one-off code: every reasoning trick (CoT, ReAct, Tree-of-Thoughts, and so on) ships in its own repo with its own scaffolding, so you can never cleanly compare them. AGORA fixes the engineering problem first — a Directed Acyclic Graph (DAG) workflow engine where each agent algorithm is a reusable module (“operator”) that shares common plumbing (LLM calls, memory, tools) — and then uses that clean base to fix the science problem: it runs 10 agent algorithms across many LLMs on the same math and multimodal benchmarks with identical metrics. The headline finding is deflating in a useful way: the simplest method, Chain-of-Thought, often beats the fancy algorithms on accuracy while using a fraction of the tokens. Complexity mostly adds cost and error-accumulation, not intelligence. The most dramatic result is that adding one sentence to a ReAct prompt — “You can take as many steps as needed” — jumped accuracy from 34% to 65%.

Problem & Motivation

If you have ever tried to answer a client’s simple question — “should we use ReAct or Tree-of-Thoughts for this?” — you already know the pain this paper is about.

• Every agent algorithm is a snowflake. CoT, ReAct, ToT, RAP, PoT, and the rest were each published with bespoke code. They don’t share a memory layer, a tool interface, or an execution model. Reusing one means reading and re-plumbing someone else’s repo.
• You can’t compare them fairly. Because each lives in its own harness, differences in results might come from the algorithm or from incidental differences in prompting, parsing, token counting, or which model was used. There was no apples-to-apples bench.
• The engineering tax is real. Standing up any one of these for a new domain means re-writing orchestration, retry logic, logging, and evaluation glue every single time. That’s weeks of undifferentiated work before you learn anything.
• “More sophisticated” was assumed to mean “better.” The field kept publishing more elaborate search-and-planning schemes. Nobody had run them side-by-side under a cost budget to check whether the complexity actually pays off in production-like conditions.

The concrete pain in one sentence: there was no shared substrate on which to build agent algorithms once and evaluate them honestly, so teams paid a re-engineering tax and made architecture decisions on vibes instead of numbers.

What’s New (Core Contribution)

AGORA is not a new reasoning algorithm. Its novelty is a platform plus a study — and the study’s conclusion is the real payload.

• A graph-based orchestration engine as the common substrate (before → now). Before: each algorithm defined its own control flow in ad-hoc Python. Now: every algorithm is expressed as a DAG of tasks running on a shared engine (built on Netflix’s Conductor library), which gives you visual tracing, async/distributed execution, branching, and looping for free. Built on top of the authors’ earlier OmAgent framework.
• Agent algorithms as modular “operators” (before → now). Before: memory access, LLM inference, and tool use were re-implemented per algorithm. Now: these are shared services, and each of the 10 algorithms is a self-contained operator with clean input/output ports that snap into a workflow. This is what makes the comparison fair — everything below the algorithm is held constant.
• A standardized evaluation framework + public leaderboard (before → now). Before: results were reported on inconsistent setups. Now: four fixed metrics (accuracy, cost in USD, token usage, pass rate), three plug-and-play client interfaces (web chat, batch JSON runner, CLI), and a public “Open Agent Leaderboard” so results are reproducible.
• Two engineering upgrades to existing algorithms. ReAct-Pro: splits ReAct’s combined “think + act” into two separate model calls so the model focuses on one job at a time (inspired by Reflexion). General GoT: extends Graph-of-Thoughts beyond its original hard-coded tasks (like sorting) to arbitrary tasks.
• The empirical finding itself is a contribution: across models and benchmarks, simple CoT is a robust, cheap default, and complex algorithms frequently hurt — especially on small models and on top-tier models where there’s little headroom to add.

How It Works (Technically)

There are three layers to understand: the workflow engine (the DAG), the operators (the algorithms), and the clients (evaluation). Then we trace one problem end-to-end.

Layer 1 — The DAG workflow engine. A workflow is a Directed Acyclic Graph: nodes are tasks, edges are the order of execution, and “acyclic” means no node can loop back onto itself directly (loops are handled by explicit logical control-flow nodes, not by tangled edges). Two node types:

• Simple task — developer-defined custom logic (e.g., “call the LLM with this prompt”, “run this Python”).
• Logical task — built-in control flow: branching (if/else), looping (do-while), fork/join. This is how an “acyclic” graph still expresses iteration — the loop is a first-class node that repeats a subgraph until a condition holds.

Because it’s built on Conductor, the engine renders workflows visually, runs tasks asynchronously, and can distribute long-running jobs. For you, the practical payoff is that “what the agent did” becomes a traceable graph instead of a stack trace.

Layer 2 — Operators. An operator is a reusable node with defined input/output connections. The key move: memory, LLM inference, and tool use are shared services every operator can call, so an operator only has to encode the part that makes its algorithm distinctive. Examples of how the 10 algorithms map to graph shapes:

• CoT = one operator: prompt the model to “think step by step,” read the answer. A single node.
• SC-CoT (Self-Consistency) = fork into N CoT paths at temperature 1, then a join node that takes a majority vote. Config in the paper: 5 paths.
• ReAct / ReAct-Pro = a loop node wrapping think → act → observe, up to a max of 10 steps.
• DnC (Divide-and-Conquer) = a loop that alternates two operator roles: a divider that breaks a hard problem into sub-problems and a conqueror that solves the easy ones, until a stop condition.
• ToT / RAP = tree-search operators. ToT expands a tree of “thoughts” and explores with BFS/DFS (config: BFS, branch b=1, depth 6). RAP goes further and runs Monte Carlo Tree Search with four phases — selection (pick a promising branch), expansion (break the question into sub-questions), simulation (roll out and score a path), backpropagation (push the score back up the tree to steer future selection). The one-line difference from ToT: RAP backpropagates scores, so earlier decisions learn from later outcomes.
• V* = an LLM-guided visual search over a high-res image (see the algorithm below).

Layer 3 — Clients. After you wire a workflow, you attach a client: WebPageClient (live chat for qualitative study), ProgrammaticClient (reads a JSON test file, runs the batch, logs outputs, and summarizes scores — this is the evaluation workhorse), or DefaultClient (CLI for debugging). Same workflow, swappable front door.

The four metrics, demystified. Accuracy = fraction of predictions that exactly match ground truth. Cost = total US-dollar spend on API calls (only meaningful for hosted models; self-hosted models under 7B report no cost). Token usage = input + output tokens summed per sample, then averaged. Pass rate = fraction of predictions that are “valid” (not empty/null) — this catches algorithms that crash or produce unparseable output, which is exactly how the fancy visual-search method V* got a low score (72% pass rate means ~28% of its answers were unusable).

Trace one input → one output

Take a GSM8K word problem — “A robe takes 2 bolts of blue fiber and half that in white. How many bolts total?” — run through DnC on GPT-3.5:

1. ProgrammaticClient loads the JSON test case and hands the question to the DnC workflow.
2. The divider operator calls the shared LLM service: “break this into sub-problems.” → [“compute white = 2/2 = 1”, “sum 2 + 1”].
3. The loop logical-node routes each sub-problem to the conqueror operator, which calls the LLM to solve each. → white = 1, total = 3.
4. The loop’s stop condition (all sub-problems solved) fires; the workflow emits “3”.
5. The client compares “3” to ground truth (exact match → accuracy++), records the summed input/output tokens and dollar cost, and marks the prediction valid (pass rate++). Now swap the DnC operator for a single CoT operator and re-run: one LLM call, far fewer tokens, and — per the paper’s results — usually equal or better accuracy. That swap-and-rerun, holding everything else fixed, is the paper.

Architecture & data flow

flowchart LR
  subgraph Clients
    WC[WebPageClient]
    PC[ProgrammaticClient]
    DC[DefaultClient]
  end
  subgraph Engine[Graph Workflow Engine · DAG on Conductor]
    OP1[Operator: CoT]
    OP2[Operator: ReAct-Pro]
    OP3[Operator: DnC]
    OP4[Operator: V*]
  end
  subgraph Shared[Shared Services]
    LLM[LLM / VLM inference]
    MEM[(Memory)]
    TOOL[Tools]
  end
  WC --> Engine
  PC --> Engine
  DC --> Engine
  OP1 --> LLM
  OP2 --> LLM
  OP3 --> LLM
  OP4 --> LLM
  OP2 --> TOOL
  OP1 <--> MEM
  Engine --> METRICS[Metrics: accuracy · cost · tokens · pass rate]

Schematic 3D view of one agent algorithm as a DAG of operators sharing a common services layer. Drag to orbit. This is the abstraction that lets any algorithm be swapped in without rebuilding the plumbing beneath it.

The data-flow of the evaluation loop

flowchart TD
  J[JSON test file] --> R[ProgrammaticClient: next case]
  R --> W[Run workflow: operator graph]
  W --> P[Prediction]
  P --> V{Valid? not empty/null}
  V -->|no| LOG
  V -->|yes| M{Exact match ground truth?}
  M -->|yes| LOG[Log: acc, tokens, cost, pass]
  M -->|no| LOG
  LOG --> R
  LOG --> SUM[Summarize scores across all cases]

The algorithm, simplified

The one central idea is “an agent algorithm is a graph of operators over shared services, and the harness swaps the graph while holding everything else constant.” Here is that idea as toy Python:

# Shared services every operator can call (held constant across algorithms).
def llm(prompt, temperature=0): ...        # -> str  (also tallies tokens + cost)
memory = {}                                 # scratch state operators read/write

class Operator:                             # a reusable DAG node
    def run(self, x): raise NotImplementedError

class CoT(Operator):                        # simplest algorithm = ONE node
    def run(self, q):
        return llm(f"{q}\nLet's think step by step.")

class DnC(Operator):                        # complex algorithm = a loop of two roles
    def run(self, q, max_iter=6):
        subproblems = llm(f"Break into sub-problems:\n{q}").split("\n")  # divider
        answers = [llm(f"Solve:\n{s}") for s in subproblems]            # conqueror
        return llm(f"Combine answers {answers} for:\n{q}")

def evaluate(algorithm: Operator, test_file):     # the harness — same for every algo
    acc = tokens = valid = 0
    cases = load_json(test_file)
    for case in cases:
        pred = algorithm.run(case["question"])     # swap `algorithm`, keep all else fixed
        valid  += bool(pred and pred.strip())      # pass rate
        acc    += (extract(pred) == case["answer"])# exact-match accuracy
        tokens += token_meter.pop()                # cost/token bookkeeping
    return dict(accuracy=acc/len(cases), pass_rate=valid/len(cases), tokens=tokens)

The teaching point: CoT.run is one line; DnC.run fans out to many LLM calls. Every extra call is another chance to accumulate an error — which is exactly why the paper finds the one-liner wins.

Built on Prior Work

AGORA is a synthesis layer. It borrows the algorithms wholesale and contributes the common ground they run on plus the comparison.

Results & Evidence

What was tested. Math reasoning (GSM8K 8-shot, AQuA zero-shot, MATH-500 4-shot) across commercial models (GPT-3.5, GPT-4o, Doubao-lite) and open-source (Qwen2.5 72B/7B, Llama-3.3-70B, deepseek-r1-1.5B, and smaller). Multimodal reasoning on MME-RealWorld (2K–4K images) with V* and ZoomEye on open VLMs.

Headline numbers.

• Simple wins. CoT + Doubao-lite hit 89.31% on GSM8K for $0.0558 — beating more complex algorithms while using the fewest tokens. On the score-vs-cost plot, CoT sits in the ideal top-left corner (high accuracy, low cost).
• The one-sentence miracle. ReAct on AQuA (GPT-3.5) = 34.25%. Splitting think/act (ReAct-Pro) → 40.16%. Adding “You can take as many steps as needed” to the prompt → 64.57% — an ~90% relative jump from a single sentence. On GSM8K, ReAct 38.13% → ReAct-Pro 74.91%.
• Complexity backfires. PoT (writes code) and ToT (tree search) underperform simple methods on math — PoT because small models generate buggy code; ToT because “think + evaluate state” balloons token usage without easing the reasoning.
• Small models can punch up. deepseek-r1-1.5B beat InternLM2.5-7B. And in multimodal, Qwen2.5-VL-7B + ZoomEye (48.06) beat Qwen2.5-VL-72B plain IO (44.47) — a good agent workflow closed a 10x parameter gap.
• V* was fragile: lowest multimodal score (15.14) driven by a 72% pass rate — ~28% of its outputs were invalid.

Schematic score-vs-token scatter in the spirit of the paper's Figure 3. The ideal corner is top-left: high accuracy, low cost. CoT lands there; tree-search and code-writing methods drift right (pricier) without gaining height. Hover a point to read it. Illustrative positions, not the paper's exact data.

What the evidence does and does NOT establish.

• Does: under a fair, shared harness with a cost lens, algorithmic complexity is often a bad trade on math and multiple-choice tasks; prompt phrasing can dominate algorithm choice.
• Does NOT: these are mostly math and multiple-choice benchmarks with exact-match scoring. That format flatters CoT and penalizes tool-use/search methods whose value shows up on open-ended, long-horizon, or genuinely tool-dependent tasks (web browsing, coding agents, retrieval). The authors themselves flag tool-use and web interaction as future work. GoT, RAP, and DnC were excluded from the main cost comparison for high token use — so “simple wins” is partly a statement about which tasks were chosen. Exact-match also under-credits correct-but-differently-formatted answers.

How You’d Use It

For an AI services company, the value is split between the finding (use immediately) and the framework (adopt selectively).

• Ship the finding as a default policy. Start every client build with plain CoT (or CoT + self-consistency). Only escalate to ReAct/tool-use/search when you have evidence CoT is insufficient for that task. This is a defensible, money-saving default you can put in a proposal: “we benchmark, then use the cheapest approach that passes.”
• Sell “algorithm selection” as a service. The paper legitimizes a productizable offer: run a client’s real task through several agent strategies under a fixed cost budget, and hand them a score-vs-cost chart. Most clients have never seen their options quantified this way.
• Adopt the eval harness even if you skip the engine. The four-metric + JSON-batch + leaderboard pattern is the reusable gold. A standardized “prediction → valid? → match? → log tokens/cost” loop turns “which prompt is better?” from an argument into a number. Build this once; reuse on every engagement.
• Steal ReAct-Pro and the “as many steps as needed” line. Both are free upgrades to any agent loop you already run. Separating think/act into distinct calls and explicitly licensing the model to take more steps are one-line changes with outsized payoff.
• For multimodal / document-heavy clients: ZoomEye-style “let a small VLM zoom into the image” is a cost-effective pattern — a 7B model with the right search loop matched a 72B model. That’s a real margin play.

Build Your Own (Minimal Recipe)

You do not need Conductor to capture 80% of the value. You need the operator abstraction and the eval loop.

1. Define an Operator interface with one method, run(input) -> output, and a shared services object exposing llm(), memory, and tools. (Half a day.)
2. Implement 3 operators to start: CoT (one call), SC-CoT (N calls + majority vote), ReAct-Pro (think-call, act-call, observe, loop with a step cap). (One day.)
3. Wire an evaluation client that reads a JSON list of {question, answer}, runs an operator over each, and logs accuracy, pass rate, and token count. Add a per-call token/cost meter — this is the part people skip and regret. (One day.)
4. Add a config file so switching algorithm, model, and dataset is a one-line change, not a code edit. (Half a day.)
5. The one or two genuinely hard parts: (a) answer extraction/parsing — getting a clean final answer out of free-form model text is where pass rate lives or dies; budget real time here. (b) fair token/cost accounting across streaming, retries, and multi-call algorithms so comparisons aren’t lying.

Reach for: any agent library you already trust (LangGraph gives you the DAG for free), a small local model plus one hosted model for the cost axis, and a handful of public benchmark JSONs (GSM8K is tiny and ideal for a first run).

How to Improve It

The paper’s limitations are your roadmap — several are directly testable.

• Test on tool-dependent, open-ended tasks. The “simple wins” claim is under-tested outside math/MCQ. Re-run the same harness on web-navigation, coding, or retrieval tasks where ReAct-style tool use should actually earn its cost. If CoT still wins there, that’s a strong publishable/marketable result; if it loses, you’ve mapped where complexity pays.
• Build an adaptive router. The authors list this as future work: a lightweight classifier that reads a task and picks the cheapest algorithm likely to solve it, escalating only on failure. This is the productizable version of the paper’s advice and a clear moat.
• Fix answer extraction to raise pass rate. V* lost purely on validity, not reasoning. A robust structured-output layer (JSON mode, retries, a parser operator) would re-score several algorithms and separate “bad reasoning” from “bad plumbing.”
• Swap exact-match for a judge on open-ended tasks. Exact-match penalizes formatting. An LLM-judge or semantic-match metric would give search/tool methods a fairer shot and generalize the harness beyond MCQ.
• Add a latency/wall-clock axis. Cost and tokens are tracked, but a client also cares about response time. RAP/ToT’s many sequential calls are slow; quantifying that strengthens the “start simple” case and makes the chart more decision-useful.
• Cache and reuse across operators. Multi-call algorithms repeat sub-computations; a shared cache on the services layer would cut the token penalty that got DnC/RAP excluded — possibly reviving them as viable.

Glossary

• Agent algorithm — a reasoning strategy an LLM follows to solve a task (e.g., think step-by-step, or search a tree of options).
• DAG (Directed Acyclic Graph) — a flowchart of tasks with a direction and no direct loops; here, a workflow where nodes are steps and edges are order.
• Operator — AGORA’s reusable workflow node encapsulating one agent algorithm, sharing common LLM/memory/tool services.
• Conductor — Netflix’s open-source workflow-orchestration engine AGORA builds its DAG execution on.
• CoT (Chain-of-Thought) — prompt the model to show intermediate reasoning steps before answering.
• SC-CoT (Self-Consistency CoT) — run CoT several times and take the majority-vote answer.
• ReAct — an agent loop alternating reasoning (“thought”) and tool use (“action”), reading each observation before the next step.
• ReAct-Pro — AGORA’s tweak: put “think” and “act” in separate model calls so each is more focused.
• ToT (Tree-of-Thoughts) — explore a branching tree of candidate reasoning steps with BFS/DFS and pick the best path.
• RAP (Reasoning-as-Planning) — treat reasoning as planning and search it with Monte Carlo Tree Search (select → expand → simulate → backpropagate).
• MCTS (Monte Carlo Tree Search) — a search that rolls out sample paths, scores them, and pushes scores back up the tree to guide future choices.
• PoT (Program-of-Thought) — have the model write and run code to compute the answer instead of reasoning in words.
• DnC (Divide-and-Conquer) — alternate a “divider” that splits a hard problem and a “conqueror” that solves the pieces.
• GoT (Graph-of-Thought) — like ToT but thoughts form a graph, allowing aggregation and refinement, not just a tree.
• V* — an LLM-guided visual search that recursively crops a high-res image to find a target, storing hits in a Visual Working Memory.
• ZoomEye — a training-free method that treats an image as a tree and “zooms in” on relevant regions based on visual cues.
• VWM (Visual Working Memory) — scratch memory where a visual-search agent stores located targets before answering.
• Pass rate — fraction of predictions that are valid (non-empty, parseable); catches crashes and garbage output.
• Exact-match accuracy — scoring that only credits a prediction if it exactly equals the ground-truth answer.
• VLM — Vision-Language Model; an LLM that also takes images as input.