Agent Behavior Mining: Generative AI Agent Governance in Business Processes

TL;DR

Companies are handing real business processes (order-to-cash, claims, procurement) to LLM agents, but once an agent is “reasoning” instead of executing a fixed flowchart, the process owner loses the ability to see why anything happened. Existing AgentOps tools (LangSmith, LangFuse, Phoenix) log plenty, but as raw debug traces meant for the engineer who built the agent, not as structured process data a compliance or ops person can query. This paper’s fix, Agent Behavior Mining (ABM), is a deliberately unglamorous event schema: six new event types (prompt, agent_start, agent_finish, call_llm, execute_tool, transfer_to_agent) and a handful of ai:* attributes (tokens, cost, reasoning text, tool args, errors), all layered on top of the 20-year-old XES standard instead of replacing it. Because it stays inside XES, any off-the-shelf process-mining tool can immediately run discovery, conformance checking, performance analysis, and variant analysis on agent logs with zero new tooling. The authors built this into a real 4-agent coffee-shop order-to-cash system (371 traces), surfaced concrete governance findings (a step the agent silently skips, a 5x cost gap between agents, two legitimate-but-different ways of handling the same order), and then had 18 industry practitioners react to the dashboards. The headline result isn’t a benchmark number — it’s that 83% of practitioners said the dashboards meaningfully increased their trust-relevant transparency into agents they otherwise treat as black boxes, and that “process variants” (agents doing the same job two different ways), not cost or speed, was what practitioners cared about most.

Problem & Motivation

Classic BPM governance works because the process is deterministic: a BPMN diagram says exactly what step follows what, so a deviation is either a bug or an approved exception, and both are visible in the model. GenAI agents break this assumption on purpose — the whole value proposition of an LLM agent is that it can interpret ambiguous input and choose a path the designer didn’t hard-code. The paper calls this the autonomy-control paradox: the flexibility you want from the agent is the same thing that defeats the standardization BPM governance depends on.

The concrete pain shows up in the paper’s running example: a four-agent coffee-shop order-to-cash (O2C) system (Order Agent, Inventory Agent, Barista Agent, Customer Service Agent). A customer orders “large caramel latte, oat milk, refund if delayed.” The Order Agent has to interpret that instruction, and nothing in a normal deployment tells the process owner how it interpreted it — whether it correctly applied a real refund policy or invented one on the spot. Scale that up and you get two silent failure modes the paper names directly: an Inventory Agent that restocks unnecessarily (a financial leak nobody notices because restocking “looks like normal behavior” in aggregate) and a Customer Service Agent that can be talked into an undeserved refund by an adversarial prompt (a compliance failure that’s invisible without the reasoning trail behind the decision).

The paper is careful to point out that this isn’t an instrumentation gap — agent frameworks already produce verbose traces. The actual gap is structural: those traces are built for a developer debugging one run, not for an analyst who wants to ask “across 10,000 runs, which agent violates policy the most, and how much is that costing us?” That second question needs a process-mining question answered on process-mining data, and today’s AgentOps traces aren’t shaped like that.

What’s New (Core Contribution)

1. An event data model for GenAI agent behavior that stays inside the XES standard. Before: the two research efforts that tried to make process mining agent-aware — Agent System Event Data (ASED) and Object-Centric Event Data (OCED) — both did it by extending the XES meta-model itself (new entity types for agents, new object-linking structures). That means any tool that wants to use them needs new, non-standard algorithms. Now: ABM adds new event types and attributes (call_llm, execute_tool, ai:response_thought, token counts, cost) without touching the meta-model. A log written this way opens in any process-mining tool that already reads XES — no adapter, no custom parser. This is the paper’s real technical bet: narrower ambition (don’t change the meta-model), broader compatibility (works with everything already built for XES), and it’s explicitly designed to be complementary to ASED/OCED rather than a competitor to them.

2. A working instantiation on a real multi-agent system, not a synthetic example. Before: AgentOps platforms show you a waterfall of spans for one conversation. Process-mining papers on “agent systems” mostly work with abstract, simulated event logs. Now: the authors built an actual 4-agent LangGraph/ReAct-style O2C system, ran it for 371 cases (with deliberately induced failures — hallucinations, loop errors), converted the raw traces into ABM events, and ran real discovery/conformance/performance/variant analysis against them, reporting concrete findings (a skipped calculate_total step, a $9.82 total token bill, two coexisting “variants” for order prep).

3. An empirical read on whether this actually matters to practitioners — not just whether it’s technically feasible. Before: most agent-observability work stops at “look, you can build a dashboard.” No one had asked people who’d actually have to use this for governance whether it changes anything for them. Now: an 18-practitioner exploratory study (enterprise architects, process managers, security experts, consultants) using a TAM-based (Technology Acceptance Model) survey after hands-on exposure to the dashboards. This is a genuinely useful contribution but a modest one — it’s perception data from a workshop, not evidence of better governance outcomes in production (the paper says this itself; see Results below).

Be honest about what’s not new here: process discovery, conformance checking, variant analysis, and XES itself are all 15–20-year-old process-mining techniques, unmodified. The Devil’s Quadrangle (cost/time/quality/flexibility) used to derive requirements is a standard BPM redesign framework, not new. The genuine novelty is narrow and specific: the exact set of event types/attributes that make LLM-agent traces legible to that existing toolchain, plus the first data point on whether practitioners care.

How It Works (Technically)

The core idea in one sentence: wrap every atomic thing an agent does — get a prompt, call an LLM, call a tool, hand off to another agent, finish — in one of six standardized event types, tag each with cost/time/reasoning/error attributes, and group them by conversation into a case, so the resulting file is a normal XES event log that any process-mining tool already knows how to read.

XES in one paragraph, for context. XES (eXtensible Event Stream) is the standard file format process-mining tools speak. A log is a set of cases (think: one instance of the process — one order, one support ticket, one conversation). Each case is a trace: an ordered list of events. Every event carries standard attributes from three extensions: concept (what happened — a name), org (who/what did it — a resource), and time (when). Process-mining algorithms don’t care what generated the log — they just read cases, traces, and events, and compute statistics over the sequences. That “algorithm doesn’t care what generated the log” property is exactly what ABM exploits: if you can get agent behavior into this shape, decades of process-mining tooling becomes usable for free.

Requirements, derived two ways. The authors don’t just invent event types — they derive seven requirements (R1–R7) from two angles, then design the schema to satisfy all seven:

• Goal-oriented (from BPM’s Devil’s Quadrangle — the four levers you trade off when redesigning any process: cost, time, quality, flexibility):
  • R1 Cost — agents cost real money per token; the model must tie every step to its exact spend.
  • R2 Time — a tool call might take 10ms, an LLM reasoning call might take 10s; the model must capture that variance.
  • R3 Quality — since execution is probabilistic, you need to trace a bad outcome back to the reasoning step that caused it.
  • R4 Flexibility — agents legitimately take different paths for similar inputs; the model must make those different sequences visible as distinct, comparable “activities.”
• System-driven (from how agent frameworks and observability standards actually work):
  • R5 Behavior — capture atomic events (an LLM call vs. a tool call vs. a hand-off) separately, so a failure can be isolated to “the model reasoned wrong” vs. “the API call errored” vs. “the wrong agent got the ball.”
  • R6 Semantics — align attribute names with OpenTelemetry’s GenAI semantic conventions (the emerging industry-standard vocabulary for LLM telemetry), so the model doesn’t invent its own dialect.
  • R7 Interoperability — stay strictly inside the XES standard so no custom tooling is required downstream.

The event type hierarchy. All events inherit the standard XES concept/org/time attributes. On top of that, the model defines:

• prompt — a user turn. Carries ai_message (what the user said). One or more prompt events sharing a case_id make up a multi-turn conversation treated as a single case.
• An abstract AgentEvent type (never appears directly in a log — it just factors out attributes every agent-related event shares, like ai_agent_name), with five concrete subtypes:
  • agent_start / agent_finish — bracket how long an agent’s turn took (R2).
  • call_llm — one LLM invocation: which model (ai_model), input/output token counts (ai_input_tokens, ai_response_message_tokens), and duration. This is where cost (R1) and, when the model exposes it, reasoning content (ai:response_thought, R3) live.
  • execute_tool — one tool/function call: ai_tool_name, ai_tool_args (the actual payload, e.g. {"order": [...], "customer": "Max"}), and duration. This is what lets an analyst reconstruct what a decision actually did, not just that a decision happened.
  • transfer_to_agent — one agent handing work to another. This is the event type that turns a swarm of independent logs into one legible multi-agent process, since MAS delegation is otherwise invisible across separate agent logs.

Two attributes do quiet but important work: concept:name is the raw technical event type (call_llm), while concept:instance is a human-readable activity label ("[Order Agent] processes order"). Process-mining visualizations (like a directly-follows graph) group and draw nodes by these labels — so concept:instance is effectively “what shows up as a box in the diagram,” decoupled from the underlying technical event.

Tracing one real case end to end (this is the paper’s own worked example, Figure 3): a user says “Can I please have an espresso for Max.” That becomes a prompt event (ai_message: "Can I please have an espresso for Max"), opening a new case_id. An agent_start event fires with ai_agent_name: "Order Agent". The Order Agent then makes a call_llm event: ai_model: gpt-4.1-2025-04-14, ai_input_tokens: 372, ai_response_message_tokens: 27, duration: 691ms — this is the moment the agent “decides” what to do with the request, and it’s the first place cost and reasoning attach to a concrete step. The decision produces an execute_tool event: ai_tool_name: process_order, ai_tool_args: {"order": [...], "customer": "Max"}, duration: 3.7s — this is the effect of the reasoning, logged as structured data instead of buried in a text blob. From here the trace would continue through further call_llm/execute_tool/transfer_to_agent events until an agent_finish, and the whole sequence — because it’s a normal XES trace — can be dropped straight into a process-mining tool.

classDiagram
  class XESConcept {
    concept:name
    org:resource
    time:timestamp
  }
  class Prompt {
    ai_message
    case_id
  }
  class AgentEvent {
    <<abstract>>
    ai_agent_name
    concept:instance
  }
  class agent_start
  class agent_finish
  class call_llm {
    ai_model
    ai_input_tokens
    ai_response_message_tokens
    ai_response_thought
    duration
  }
  class execute_tool {
    ai_tool_name
    ai_tool_args
    duration
  }
  class transfer_to_agent {
    ai_source_agent
    ai_target_agent
  }
  XESConcept <|-- Prompt
  XESConcept <|-- AgentEvent
  AgentEvent <|-- agent_start
  AgentEvent <|-- agent_finish
  AgentEvent <|-- call_llm
  AgentEvent <|-- execute_tool
  AgentEvent <|-- transfer_to_agent

sequenceDiagram
  participant U as User
  participant L as XES Event Log
  participant O as Order Agent
  participant LLM as LLM (gpt-4.1)
  participant T as Tool: process_order
  U->>L: prompt: "espresso for Max"
  L->>O: agent_start (Order Agent)
  O->>LLM: call_llm (372 in / 27 out tok, 691ms)
  LLM-->>O: decision: place order
  O->>T: execute_tool: process_order({order, customer:"Max"})
  T-->>L: execute_tool event logged (3.7s)
  O->>L: agent_finish
  Note over L: Case closed — one XES trace,<br/>readable by any process-mining tool

Watch raw agent activity (prompt → LLM call → tool call → hand-off) stream in and accumulate into a directly-follows graph in real time — this is literally what "process discovery" does to an ABM event log: no one draws this diagram, it falls out of replaying enough traces.

What you get once the log exists. The paper doesn’t invent new process-mining algorithms — it applies the four standard ones to the ABM log and shows each answers one governance question:

• Process discovery builds a directly-follows graph (a “what happened, and how often” map) straight from the traces. On the 371-case O2C log this showed the Order Agent orchestrating ~1,200 downstream instances, delegating to Inventory Agent (191 times) or Customer Service (66 times), with Inventory Agent handing off to Barista Agent 180 times after checking stock 249 times. For a process owner this is the first time the “black box” resolves into an actual map, built from what agents really did rather than what a designer assumed they’d do.
• Conformance checking compares real traces against a reference sequence and flags two kinds of gaps: skips (the Order Agent omitting a calculate_total step it’s supposed to always perform) and insertions (the Barista Agent skipping estimate_prep_time and then, later, executing an unplanned remake_order_item). This is where “policy violation” becomes a countable, filterable thing instead of a hunch.
• Performance analysis aggregates the cost/time attributes: across the 371 cases, 4.29M total tokens, $9.82 total spend, 1m49s average end-to-end time, 2.3 agents touched per case on average — and, critically, per-agent attribution (Barista, Inventory, and Order agents each burned ~1M tokens; Customer Service only 604k). That per-agent breakdown is what turns “AI is expensive” into “this specific agent is expensive, go look at it.”

The four O2C agents sized by real token consumption from the paper's performance analysis (Barista/Inventory/Order ≈1M tokens each vs. Customer Service's 604k), connected by edges sized to the real delegation counts from process discovery. Drag to orbit — this is what "cost attribution" looks like once agent behavior is standardized event data instead of a black box.

The algorithm, simplified

There’s no single “ABM algorithm” in the paper — the contribution is the schema, and the process-mining techniques are off-the-shelf. The one piece of logic a builder actually has to write is the transformer: turning whatever your agent framework emits (spans, callbacks, OpenTelemetry traces) into ABM-shaped XES events. Here’s that core loop, simplified to the shape you’d actually implement:

# Turns one raw agent-framework trace into a list of ABM-shaped XES events.
# `raw_trace` is whatever your framework gives you (e.g. LangGraph callbacks,
# OpenTelemetry spans) — a list of steps with a type, payload, and timestamps.

def to_abm_events(raw_trace, case_id):
    events = []
    for step in raw_trace:
        base = {                      # every event gets the shared XES attributes
            "case_id": case_id,
            "time:timestamp": step.start_time,
            "duration": step.end_time - step.start_time,
        }
        if step.kind == "user_message":
            events.append({**base, "concept:name": "prompt",
                            "ai_message": step.text})
        elif step.kind == "agent_span_start":
            events.append({**base, "concept:name": "agent_start",
                            "ai_agent_name": step.agent_name,
                            "concept:instance": f"[{step.agent_name}] starts"})
        elif step.kind == "llm_call":
            events.append({**base, "concept:name": "call_llm",
                            "ai_model": step.model,
                            "ai_input_tokens": step.usage.input_tokens,
                            "ai_response_message_tokens": step.usage.output_tokens,
                            # only present if the model exposes chain-of-thought:
                            "ai_response_thought": getattr(step, "reasoning", None)})
        elif step.kind == "tool_call":
            events.append({**base, "concept:name": "execute_tool",
                            "ai_tool_name": step.tool_name,
                            "ai_tool_args": step.arguments})
        elif step.kind == "handoff":
            events.append({**base, "concept:name": "transfer_to_agent",
                            "ai_source_agent": step.source, "ai_target_agent": step.target})
        elif step.kind == "agent_span_end":
            events.append({**base, "concept:name": "agent_finish",
                            "ai_agent_name": step.agent_name})
    return events   # a valid XES trace once wrapped with case metadata

The interesting engineering isn’t the branching above — it’s case_id: getting every step of a multi-agent, possibly multi-turn interaction tagged with the same case id is what makes the log analyzable as one process instead of a pile of disconnected spans, and it’s also exactly where the paper admits its model is weakest (see Limitations below).

Built on Prior Work

Results & Evidence

Two separate bodies of evidence, and they should be weighted differently.

The process-mining demonstration (371 real execution cases from the O2C system) is solid proof of feasibility: the schema really does let standard discovery/conformance/performance/variant analysis run on real agent traces and surface real, specific findings (the skipped calculate_total, the 5x token-cost gap between Barista/Inventory/Order vs. Customer Service, the two legitimate prep-time variants). This part of the paper is a working demo, not a claim about generalization — it’s one hand-built scenario on one model.

The practitioner study (18 people: 4 enterprise architects, 1 product owner, 1 IT manager, 6 consultants, 2 process managers, 1 business analyst, 1 security expert, 1 data scientist, 1 developer) is where the interesting numbers live, and where the caveats matter most:

• 78% picked process variants as the most valuable insight type — ahead of agent performance (50%) and compliance (44%). The authors’ read: for probabilistic systems, controlling drift may matter more to practitioners than the classic BPM metrics of speed and cost.
• 83% reported “moderate” to “extreme” transparency improvement.
• 72% rated the insights “helpful” or “very helpful” for governance decisions.
• Adoption barriers clustered almost evenly across governance/compliance fit (25%), tool integration (25%), and lack of skills (20%) — i.e., people believed the insights but weren’t sure their organization could operationalize them yet.

The paper is unusually explicit about what this does not establish, and it’s worth taking them at their word rather than the framing in the abstract:

• It’s perceived usefulness, not measured outcomes. No task accuracy, no decision quality, no actual governance action was measured — participants explored dashboards under lab conditions and self-reported value (a TAM survey), which the authors flag can capture “potential value” that never materializes as adoption.
• Single model, single architecture. Every agent ran gpt-4.1; reasoning-trace availability, error rates, and variant counts are all plausibly model-dependent, and the study only covers sequential ReAct-style agents — a different orchestration pattern (parallel/async agents, planner-executor splits) might expose entirely different governance gaps.
• Small, non-representative sample. 18 practitioners recruited by purposive sampling for role diversity, not statistical power — the authors call this “analytical generalization,” i.e., useful for refining a theory of what matters, not for estimating how a real population of process owners would react.
• Synthetic traces. The 371 cases came from a 60-minute workshop simulation with deliberately induced failures, not production traffic — the mix and frequency of failure modes may not resemble a real deployment at all.

Net: this is honest, well-scoped exploratory work that establishes plausibility and a first signal of practitioner interest, not a validated governance tool with measured ROI.

How You’d Use It

If you’re running agents against real business processes — order-to-cash, support, procurement, anything with a policy behind it — ABM maps onto three concrete uses in your own stack.

• A drop-in observability layer for existing BPM/process-mining investments. Because the output is plain XES, any existing Celonis, Signavio, Disco, or PM4Py pipeline built for your non-AI processes can point the same tooling at your agent fleet with zero new licenses — a much smaller lift than standing up a whole new AI-observability platform, and it directly answers the adoption barrier practitioners raised most (tool integration, 25%).
• A recurring conformance audit built into your own ops. Instrument your agents, run the log through discovery/conformance/variant analysis on a schedule, and generate a findings report structured like the paper’s O2C example: skipped policy steps, cost outliers by agent, and a breakdown of “legitimate variant” vs. “unexplained drift.” This is a concrete addition to an existing monitoring workflow, not an open-ended audit project.
• A trust-building artifact before a multi-agent system goes to production. The paper’s own quote from a security expert (“seeing the stability of the process technically end-to-end”) is the exact question a security or compliance reviewer asks before approving an agent rollout — a conformance dashboard answers it far better than a slide deck about the system prompt.

The realistic framing: this is an instrumentation-and-reporting layer, not a control system. It tells you that an agent deviated and roughly why (if reasoning traces are available); it doesn’t stop the deviation from happening. The paper itself lists closed-loop intervention (auto-triggering a guardrail or prompt update on conformance drift) as future work, not something ABM does today.

Build Your Own (Minimal Recipe)

You can get real value from a scoped, few-day build:

1. Pick your six event types and lock the attribute names to the paper’s vocabulary (or OpenTelemetry GenAI semconv names) so you’re not inventing a private dialect: prompt, agent_start, agent_finish, call_llm, execute_tool, transfer_to_agent.
2. Wrap your agent framework’s callback/hook system (LangGraph callbacks, CrewAI’s step hooks, or your own MAS’s message bus) to emit one of these events per step, tagged with a case_id — this is the to_abm_events transformer above.
3. Write out a real XES file (or push events straight into PM4Py’s in-memory event log format) so you inherit an ecosystem of tools instead of building visualizations yourself.
4. Point an off-the-shelf process-mining tool at the log. PM4Py (open source, Python, scriptable) gets you discovery and conformance checking for free; Celonis or Disco if your org already has enterprise licenses.
5. Define one reference sequence for conformance checking (e.g., “Order Agent must always emit calculate_total before agent_finish”) — this is genuinely optional for discovery/performance/variant analysis, but it’s what makes conformance checking possible, and it’s usually where your organization’s actual policy lives.

The two genuinely hard parts, both of which the paper flags as open problems rather than solved:

• Case correlation. A “case” here means “everything that belongs to one conversation/intent.” That’s easy for a single-turn, single-user, synchronous interaction and gets hard fast for concurrent agents, async fan-out, or a user who changes topic mid-conversation — the paper’s own model assumes none of that happens, and says so.
• Reasoning-trace availability. ai:response_thought is only as good as what the underlying model exposes. Models without visible chain-of-thought (or with it stripped by API defaults) leave you with “the agent deviated” but not “why” — you’re then inferring cause from ai_tool_args and ai_error_message instead of reading the actual reasoning.

Reach for: PM4Py for the mining side, OpenTelemetry’s GenAI semantic conventions as your attribute vocabulary (so you’re aligned with where the ecosystem is heading), and whatever tracing hooks your agent framework already exposes — you are very likely not starting from zero on instrumentation, just reshaping what you already capture.

How to Improve It

The paper’s own limitations section is a genuinely good todo list — here’s it translated into testable builds:

1. Structure the reasoning trace instead of treating it as one opaque blob. ai:response_thought is currently a single text field per LLM call. Run it through a lightweight extraction step (even a second LLM call) to pull out discrete sub-decisions, and test whether that finer granularity catches conformance violations the coarse version misses — versus how much noise it adds from non-deterministic phrasing.
2. Build dynamic case-splitting for multi-turn sessions. Add a topic-shift detector so a single chat thread can spawn a new case when the user pivots to an unrelated request, instead of assuming (as the current model does) that everything in one session belongs to one case.
3. Handle concurrent/async agents. The model’s case_id correlation assumes a linear, mostly-sequential ReAct loop. Test it against a fan-out architecture (planner dispatches N parallel sub-agents) and see where the single-thread case notion breaks — this is explicitly flagged as unsolved.
4. Build the closed-loop half the paper only proposes. Right now ABM detects a conformance-violation rate; nothing acts on it. Wire a threshold-triggered action (flag for human review, auto-suggest a prompt patch, page a guardrail update) and measure whether it actually reduces recurrence — this is the highest-leverage “improvement” because it turns ABM from a dashboard into a control loop.
5. Re-run the study across model families. Everything here used gpt-4.1. Repeat the O2C scenario on a model with no exposed reasoning (quantify how much governance value is lost) and one with extended thinking (Claude, o-series) to see whether richer reasoning traces meaningfully change what conformance checking can catch.

Glossary

• XES (eXtensible Event Stream) — the standard file format for process-mining event logs: cases, made of ordered traces, made of timestamped events with attributes.
• Case / trace / event — a case is one instance of the process (one order, one conversation); its trace is the ordered sequence of events that happened within it.
• concept:name / concept:instance — the raw technical event type vs. a human-readable label used to group and draw activities in diagrams.
• ASED (Agent System Event Data) — a prior XES extension that adds agents as first-class entities (roles, org membership) to the meta-model itself.
• OCED (Object-Centric Event Data) — a prior XES extension for many-to-many relations between business objects (orders, shipments, items).
• Directly-follows graph (DFG) — a process map auto-built from event logs: an arrow from A to B means “B directly followed A at least once,” with counts.
• Process discovery — automatically building a process map (like a DFG) from raw event logs instead of a human drawing it.
• Conformance checking — comparing real traces against a reference/expected sequence to flag skips, insertions, or reorderings.
• Variant analysis — grouping traces by their exact event sequence to see how many distinct “ways” the process actually runs, and how often each occurs.
• Performance analysis (process mining sense) — aggregating time/cost attributes across cases to find bottlenecks or outliers.
• OpenTelemetry GenAI semantic conventions — an emerging industry-standard vocabulary of attribute names for LLM/agent telemetry (model, tokens, latency), so different tools/vendors describe the same thing the same way.
• Devil’s Quadrangle — a BPM framework naming the four things any process redesign trades off: cost, time, quality, flexibility.
• ReAct (Reason + Act) — the common agent loop pattern of interleaving LLM reasoning steps with tool-call actions.
• TAM (Technology Acceptance Model) — a standard survey framework for measuring whether people find a new technology useful and easy to use, used here to structure the practitioner feedback.
• AgentOps — the emerging category of observability tools (LangSmith, LangFuse, Phoenix) built specifically for monitoring LLM agents in production.