One Step at a Time: Trading LLM Autonomy for Process Predictability

Motivation

• Prompt-based agents must reconstruct each procedure from text, and can produce correct answers without running its prescribed verification tools.On the KYC-style domain, 31–49% of correct answers bypass the tools, including 48% for a frontier model.Source: §1 · pp. 1–2

Approach

• MCP delivery releases one SOP step at a time, and the server advances only after the model reports completion.The model’s autonomy is scoped to the current step.Source: §1 · p. 2
• Step-level delivery fixes the execution path in advance and records step inputs and outputs, enabling audit trails and step-localized diagnosis.Capable executors lose the ability to inspect future steps or self-correct against the full procedure; raw accuracy declines slightly, and branching domains can lose grounded accuracy.Source: §1 · p. 2

Contributions

• The evaluation covers 15,475 trials across 13 SOP-Bench domains, four open-weight executors, and prompt-based versus step-level delivery.The models range from frontier Kimi K2.5 to lightweight Ministral 3 8B.Source: §1 · p. 2
• 2.1–4.5% of prompt-delivery trials are correct without executing the SOP, while step-level delivery reduces this ungrounded rate to 0.2–0.3% for every model.The lightweight executor gains +6.5pp grounded accuracy, while frontier and large executors trade a small raw-accuracy decrement for process visibility.Source: §1 · p. 2

2.1 SOP-Bench

• SOP-Bench contains expert-authored business procedures, and its reported results show that newer models do not guarantee better performance or dominate across domains.These observations motivate the hypothesis that instructional specificity and its delivery mechanism affect adherence.Source: §2.1 · pp. 2–3

2.2 RFC 2119 Formalization

• RFC 2119 formalization raises DeepSeek V3.2’s raw TSR by +21.8pp and grounded TSR by +17.8pp across 12 domains.Keyword insertion alone was insufficient; effective conversion also specified tool-to-step mappings, decision thresholds, and edge-case handling.Source: §2.2 · p. 3

2.3 SOP-MCP: Step-Level Delivery via MCP

• SOP-MCP releases one instruction at a time and advances only after the model reports a step_output, preventing the model from seeing ahead.This scopes autonomy to the current step and prevents compensation for errors in the SOP itself.Source: §2.3 · p. 3

2.4 Trust and Delegation

• Prompt-based execution can yield dozens of distinct paths, whereas MCP enforces a defined path and provides step-level evidence for compliance review.The associated autonomy–control tension parallels checklist use in human organizations: consistency may come at the cost of catching checklist errors.Source: §2.4 · p. 3

2.5 Decomposition: Advisory vs. Enforced

• Unlike decomposition strategies where the model retains sequencing control, MCP makes the decomposition enforced by giving sequencing control to the server.The paper measures the consistency and auditability of this enforced variant.Source: §2.5 · p. 3

3.1 Experimental Design

• The experiment compares prompt and step-by-step MCP delivery of identical SOP content across 13 domains, with 15,475 trials and one trial per task and condition.The four open-weight models span a capability range; one domain was excluded because a pre-existing tool bug made faithful execution systematically incorrect.Source: §3.1 · p. 4

3.2 Bridge Study: Connecting SOP-Bench to Our Evaluation

• RFC 2119 formalization raises DeepSeek V3.2’s raw TSR by +21.8pp but grounded TSR by only +17.8pp; ungrounded answers rise from 0.3% to 5.1% of correct answers.The study uses the bridge comparison to check comparability with SOP-Bench and assess shortcut-derived accuracy.Source: §3.2 · pp. 4–5
• The plain_sop baseline is 61%, within SOP-Bench’s reported 57–100% per-domain range, while adding tool information to informal SOPs yields only +0.7pp raw TSR.The bridge study ran DeepSeek V3.2 across all 14 domains and four conditions, totaling 8,618 trials.Source: §3.2 · pp. 4–5

3.3 SOP-MCP Protocol

• The run_sop tool returns one step per call and requires step completion before advancing; the system prompt requires the model to call it first.The same RFC 2119 SOP is decomposed into discrete steps for MCP delivery.Source: §3.1 · p. 4, §3.3 · p. 5

3.4 SOP Quality Assurance

• Pilot trace review and linting found tool-name mismatches, schema conflicts, and incorrect branching across four domains, and all corrections preceded the primary evaluation.Pilot tasks were not a disjoint development split, and debugging used MCP traces without an equivalent pass for the prompt condition.Source: §3.4 · p. 5

3.5 Evaluation

• Grounded TSR counts correct answers only when process adherence reaches pa_coverage ≥0.5; raw TSR equals grounded plus ungrounded TSR.Adherence uses a task-specific expected-tool set and name-based, order-insensitive coverage, so it does not verify tool order, parameters, or use of outputs.Source: §3.5 · p. 5
• Path consistency is the share of adherent trials following the most common ordered tool-name sequence within each domain, model, and condition group.Higher dominant tool-sequence percentages indicate more reproducible behavior.Source: §3.5 · p. 5

4 Results

• Grounded TSR counts correct answers only when process adherence reaches pa_coverage ≥0.5; reported effects pool 12 domains, excluding video_annotation.The fixed adherence threshold is unreliable for video_annotation because of branching.Source: §4 · p. 5

4.1 Failure Modes Under Prompt-Based Delivery

• Under prompt delivery, 2.1–4.5% of trials are correct without executing prescribed tools, and the lightweight executor averages 20 distinct tool-call sequences per domain.On know_your_business, 31–49% of correct answers bypass the SOP, including 48% for the frontier executor.Source: §4.1 · p. 6

4.2 Effect on Ungrounded Answers

• ≈0.3% ungrounded rates under step-level delivery, with declines of Ministral −4.2pp, DeepSeek −2.9pp, Kimi −2.1pp, and GLM 5 −1.9pp.On know_your_business, ungrounded shares of correct answers fall from 31–49% to 0–8%; the near-elimination is partly structural because the server withholds the next step until the current one executes.Source: §4.2 · p. 6

4.3 Process Adherence and Path Consistency

• Adherence rises significantly for every model, up to +18.8pp, while consistency gains are significant only for DeepSeek V3.2 (+13.7pp) and GLM 5 (+7.0pp).On dangerous_goods, GLM’s paths collapse from 6 to 1, with a 100% dominant path.Source: §4.3 · p. 6, Table 3 · p. 6
• Ministral reaches 95% adherence versus 97–99% for capable executors, but its path consistency moves only from 53% to 58% and its distinct-path count rises under MCP.Trace variation is concentrated on branching domains rather than execution noise.Source: §4.3 · p. 6

4.4 Effect on Grounded Accuracy

• Ministral 3 8B gains +6.5pp grounded accuracy, while capable models show no grounded gain; raw TSR falls for GLM 5 (−4.7pp), Kimi (−3.8pp), and DeepSeek (−2.5pp).Ministral’s gain is adherence-driven (76%→95%), with accuracy among adherent trials roughly unchanged; capable models’ raw declines reflect shortcuts removed without replacement.Source: §4.4 · pp. 6–7, Table 4 · p. 7
• Grounded-accuracy gains concentrate where prompt-based adherence was poor, while losses concentrate in branching decision-tree domains.Traffic-spoofing detection averages +39.8pp and patient intake +13.3pp; video classification averages −21.6pp and referral-abuse detection v2 −14.0pp.Source: §4.4 · p. 7

4.5 Cost, Latency, and Errors

• Ministral is cheapest and fastest at $0.015 per grounded answer and 17 seconds per MCP trial, while step-level delivery costs 2.1–2.9× more and tool errors remain 1–3% of calls.The lightweight executor does not match the frontier executor’s grounded accuracy.Source: §4.5 · p. 7

5.1 The Autonomy–Predictability Trade-off

• All four models near-eliminate ungrounded answers, while capable executors consolidate paths and the lightweight executor gains grounded accuracy.Capable executors do not gain grounded accuracy, and branching domains needing look-ahead can lose it; Ministral 3 8B gains +6.5pp grounded accuracy as adherence rises from 76% to 95%.Source: §5.1 · p. 7
• At ≈3.5× lower cost per grounded answer, the lightweight executor remains below larger executors in grounded accuracy: 67.5% versus 79.0–83.8%.Most of its grounded gain is shortcut answers becoming process-backed; raw gain was +2.3pp, not significant, and it often does not execute the process reproducibly.Source: §5.1 · pp. 7–8

5.2 SOP Structure and Delivery Fit

• Step-level delivery brings sequential SOPs near-deterministic execution, but legitimate branching keeps path consistency at 21–37%.For sequential SOPs, dominant-path consistency reaches 97–100%; branching variation reflects different inputs triggering different decision paths, and should not be eliminated.Source: §5.2 · p. 8

5.1 The Autonomy–Predictability Trade-off

• Capable executors’ small raw-accuracy decrement accompanies the predictability trade, while branching domains requiring look-ahead can lose grounded accuracy outright.Step-scoped models cannot look ahead; most grounded accuracy gains track repairs to poor prompt-based adherence.Source: §5.2 · p. 8

5.4 Evidence a Domain Expert Can Check

• Fixed-shape step inputs and outputs make the audit trail consumable by another program, enabling an automated revision cycle.Each MCP step records what the model did, called, and concluded; the full-SOP prompt baseline evaluated here returns only a final answer.Source: §5.3 · p. 8, §5.4 · p. 8

5.3 Traceability Enables Targeted Optimization

• Reordering the customer_service decision tree reduced its pilot MCP penalty from −14pp to −4pp, while a separate conditional-execution issue remained.The optimizer reordered RESOLVED before ESCALATED within 60 seconds; the remaining gap traced to Step 9.Source: §5.3 · p. 8
• Step-level traces enable a trial–diagnose–revise–rerun cycle that prompt-based delivery would require manual conversation analysis across hundreds of trials to reproduce.Traceability also exposed a benchmark defect in the excluded warehouse_package_inspection domain: higher prompt accuracy came from circumventing the tools.Source: §5.3 · p. 8

• Step-level SOP delivery fixes the execution path in advance, raises adherence to 95–99%, documents every step, and cuts hallucinated process execution to ≈0.3%, at the price of a small raw-accuracy decrement for capable executors.Source: §6 · p. 8

Limitations

• Step-level delivery bundles information presentation, process enforcement, and behavioral meta-instructions, so their individual contributions are not disentangled.A sequential-chunk ablation without enforcement could isolate these effects.Source: Limitations · p. 9
• SOP-Bench uses mock tools with precomputed results, leaving real-world API failures such as timeouts, rate limits, and schema changes outside the evaluation.Source: Limitations · p. 9
• The adherence threshold excludes video_annotation, whose wide branching defeats the fixed coverage threshold.For some branching domains, expected tools are empirically derived rather than specified per task.Source: Limitations · p. 9
• Trust is measured indirectly through technical proxies; whether predictability gains change human delegation decisions remains an open question for user studies.Source: Limitations · p. 9
• Trace-guided SOP corrections may be tuned to the evaluation tasks because pilot defect discovery used the same task set, not a disjoint development split.Revised SOPs were frozen before the reported full-task-set reruns.Source: Limitations · p. 9
• Grounded-accuracy deltas are aggregation-sensitive: GLM 5 shifts from −2.9pp trial-pooled to +0.8pp domain-mean.Per-domain figures use 30–274 tasks without per-domain confidence intervals or multiple-comparison control.Source: Limitations · pp. 9–10
• The adherence measure checks tool-name coverage, not call order, parameters, or whether outputs were consumed; stricter completeness is not identifiable across the benchmark.Per-task required-tool annotations are missing, and the independent output-field reference is available for only two of twelve domains.Source: Limitations · pp. 9–10

A Per-Domain MCP Lift

• Average grounded MCP lifts concentrate in domains with poor prompt adherence, while losses cluster in branching decision-tree domains where step-scoped autonomy prevents look-ahead.Source: Table 5 · p. 11, A Per-Domain MCP Lift · p. 11

B Grounded TSR and Path Variance: Full Results

• Table 6 covers Grounded TSR for each domain-by-model-by-condition cell, with unique execution-path counts in parentheses.Source: B Grounded TSR and Path Variance: Full… · p. 11, Table 6 · p. 12
• Grounded TSR indicates the fraction of trials that answered correctly while following the SOP.Source: B Grounded TSR and Path Variance: Full… · p. 11
• Parenthesized path counts enumerate distinct tool-call sequences among adherent trials, enabling comparison of execution-path variation across cells.Source: B Grounded TSR and Path Variance: Full… · p. 11, Table 6 · p. 12

C Warehouse Package Inspection: When Higher Accuracy Is the Wrong Signal

• Under MCP, the model followed the SOP in every trial, yet 55% were marked incorrect because tool outputs and ground truth disagreed.Source: C Warehouse Package Inspection: When… · p. 12
• The validateBarcode mock uses PO-number parity, conflicting with ground truth in 62 of 150 cases (41%) and capping perfect tool-following accuracy at 58.7%.The observed MCP result was below that ceiling because of additional downstream tool-logic failures.Source: C Warehouse Package Inspection: When… · p. 12
• 57% prompt TSR exceeded MCP's 45%, but the prompt-based model bypassed tools by reading barcode_match directly from task input.This shortcut concealed the benchmark defect from prompt-based evaluation.Source: C Warehouse Package Inspection: When… · p. 12

D Failure Modes Under Prompt-Based Delivery

• Table 7 quantifies prompt-delivery failures in process execution and reinvention, with columns covering ungrounded answers and execution-path variation.Source: D Failure Modes Under Prompt-Based… · p. 12, Table 7 · p. 12

E Ungrounded Answers on know_your_business

• 31–49% of correct know_your_business answers bypassed prescribed tools under prompt delivery, versus 0–8% under step-level delivery.Table 8 reports 90 tasks per cell.Source: E Ungrounded Answers on… · p. 12, Table 8 · p. 13

F Definition of pa_coverage

• pa_coverage counts a trial as adherent when tool-name coverage reaches 0.5, with expected tools defined per task and branch where applicable.For some single-output branching domains, expected tools are estimated from tools used by the majority of correct prompt-condition executions of the same task.Source: F Definition of pa_coverage · p. 13
• Coverage ignores call order, duplicate calls, parameters, tool outputs, and extra tools, so it is a proxy for procedural compliance rather than proof of correct execution.An empty expected-tool set receives coverage 1.0 and is vacuously adherent.Source: F Definition of pa_coverage · p. 13

G Adherence Threshold Sensitivity

• Across tested thresholds, Grounded TSR effects remain directionally stable for Ministral, Kimi, and GLM 5, while DeepSeek stays near zero and never approaches significance.Ministral ranges from +6.3 to +8.2pp, Kimi from −1.7 to −2.2pp (not significant), and GLM 5 from −2.9 to −3.0pp.Source: G Adherence Threshold Sensitivity · p. 13, Table 9 · p. 13

H Grounded TSR Decomposition

• Acc|Adh declines under MCP for all models, reflecting a reclassification effect as previously non-adherent trials enter the adherent population.Those newly included trials are disproportionately hard cases the model previously abandoned or shortcut.Source: H Grounded TSR Decomposition · pp. 13–14
• Grounded TSR is the product of adherence rate and accuracy among adherent trials; Adh% and Acc|Adh denote those two components.The decomposition is trial-pooled across 12 domains.Source: H Grounded TSR Decomposition · p. 13, Table 10 · p. 14
• Grounded TSR rises only when adherence gains outweigh declining accuracy among adherent trials, stays flat when the changes roughly cancel, and dips slightly when adherence was already high.Ministral’s adherence rises by 18.8pp; Grounded TSR stays flat for DeepSeek and dips slightly for GLM 5 and Kimi.Source: H Grounded TSR Decomposition · pp. 13–14, Table 10 · p. 14

I Token, Cost, and Latency Economics

• 2.1–2.9× input tokens produce a 2.1–2.9× cost tax and 2.2–2.8× latency tax under step-level delivery.The accumulated conversation is re-delivered at each step without prompt caching; caching would substantially reduce the tax.Source: I Token, Cost, and Latency Economics · p. 14
• $0.010 per trial at 17s for Ministral under MCP compares with $0.044 per trial at 81s for Kimi, making Ministral 4.3× cheaper and 4.8× faster.Ministral is also approximately 3.5× cheaper per grounded answer.Source: I Token, Cost, and Latency Economics · p. 14
• Median output-token usage is 15–77% of the smaller 4,096-token cap, and each individual response is smaller than the trial-wide total.A trial spans multiple model responses, while the max_tokens cap applies to a single response.Source: Table 12 · p. 14