PaperlayerView exact revision on arXiv

Source-linked AI summary

Can MCP Clients Decide What to Do After Failure? A Result-Only Actionability Audit

Rishabh Mehan

arXiv:2609.00072v1cs.SE

TL;DR

A completed MCP failure result may expose that something went wrong without giving deterministic software enough information to choose a concrete recovery action. The paper measures this gap with a six-part actionability profile and finds that recovery in its sample often depends on prose or information outside the result.

  • Problem

    A failure marker can reveal that something went wrong without distinguishing whether software should repair an argument, authenticate, wait, switch tools, or stop.

    Source: §I · p. 1

  • Method

    The paper applies a separately coded six-part actionability profile to completed MCP failures and demonstrates a fail-closed control-plane prototype.

    Source: §I · p. 1, §III · p. 2, B. Descriptive analysis and… · p. 4

  • Results

    Completed results often expose failure and sometimes a broad response, but in the sample usually lack cause, target, executable repair, or replay conditions without prose or other state.

    Source: §IX · p. 6

  • Takeaways & Limitations

    MCP standardizes observability more strongly than recovery semantics for ordinary completed tool errors, with exceptions for typed specialized workflows.

    Source: §IX · p. 6

    The 21 failures exercise the construct but cannot estimate registry-wide prevalence, raw-wire behavior, deployed-agent performance, or end-to-end recovery.

    Source: §I · p. 1

Abstract

from arXiv · show

A client that receives isError:true knows that something went wrong. It may still have no machine-readable basis for deciding whether to fix an argument, authenticate, wait, choose another tool, or stop. This paper studies what deterministic software can learn from a completed MCP failure result alone; request arguments, schemas, discovery history, authen- tication state, transport metadata, host policy, and ap- plication state are outside that boundary. We introduce a six-part actionability profile and apply it with record- level evidence. In a small illustrative study of 21 safely induced failures from ten reachable sampled servers, typed fields expose failure in 18 cases and a broad policy in 8, yet expose no specific cause, target, executable repair, or replay constraint. Prose often carries more cause and target information, at the price of making semantic interpretation part of the recovery path. A lexical source audit finds the same text-centered pat- tern. Finally, a fail-closed prototype demonstrates how a separate experimental control plane could support deterministic branching. The result is deliberately nar- rower than an ecosystem survey or an agent benchmark: completed MCP results often make failure observable, sometimes make a broad response possible, and rarely make concrete recovery or safe replay self-contained in this sample.

I. Introduction

The paper asks how far deterministic software can safely act from a completed MCP failure result alone, since observing failure does not distinguish among possible next actions.

I. Introduction

  • A Boolean failure marker cannot distinguish whether to repair an argument, authenticate, wait, switch tools, or stop; repeating some failures can waste calls or cause damage.Source: §I · p. 1

A. MCP failure surfaces

  • MCP tool-execution failures can appear in successful JSON-RPC exchanges with isError:true, while protocol failures use typed errors and missing-input workflows have a specialized result.Source: §I · p. 1

I. Introduction

  • The study isolates completed failure results from request arguments, schemas, discovery, authentication, transport, host policy, and application state.Source: §I · p. 1

A. MCP failure surfaces

  • SDKs shape the observed error surface: TypeScript guidance puts recovery hints in text, while the study analyzes Python’s normalized client view rather than wire data.Source: A. MCP failure surfaces · p. 1

B. Machine-facing precedents

  • HTTP and gRPC separate machine-facing controls from prose, but their mechanisms do not solve application-specific recovery.Source: B. Machine-facing precedents · p. 2
  • Prior MCP proposals already describe structured recovery; this paper instead contributes an auditable actionability measure and a fail-closed conformance refinement.Source: B. Machine-facing precedents · p. 2

C. Agent failure benchmarks and policy systems

Related work studies faults and agent recovery, while this paper examines what a completed MCP failure record lets deterministic software infer.

C. Agent failure benchmarks and policy systems

III. A Non-Cumulative Actionability Profile

The six-part profile separates failure visibility from increasingly specific recovery information, coding each capability independently rather than as a ladder.

III. A Non-Cumulative Actionability Profile

  • The profile separately codes failure detection, coarse policy, cause, repair target, executable repair, and replay constraints for each observation.Source: §III · p. 2
  • Its coordinates are not cumulative: timing can be explicit without a corrected request, and a target can be identified without a precise cause.Source: §III · p. 2
  • The executable-repair threshold combines a procedure, replacement value, and authority to act, while safe-policy sets include stop/escalate in all 21 scenarios.Source: §III · pp. 2–3
  • Record coding contrasts opaque control fields with content-assisted interpretation; text can identify a target without supplying a valid replacement, while prose-dependent recovery inherits interpreter fallibility.Source: §III · pp. 2–3

IV. Illustrative Case Study

The illustrative case study combines a frozen specification and repository audit with low-load probes, analyzing normalized client observations rather than raw protocol frames.

A. Specification, sample, and static candidates

  • The study froze the specification and schema at the August 19, 2026 snapshot, drew 54 repositories from a 23,569-record registry frame, and documented sampling and replacements.The release can replay sampling from the derived frame but cannot reconstruct the omitted raw registry response.Source: A. Specification, sample, and static… · p. 3
  • A narrow lexical scanner adjudicated 16 production-classified isError matches, reducing them to 12 semantic construction paths across ten repositories.The scanner, candidate projection, and adjudication ledger make filtering inspectable, though repository acquisition is not fully reproducible.Source: A. Specification, sample, and static… · p. 3

B. Low-load probes and released observations

  • Low-load probes initialized ten of 15 advertised remote endpoints and yielded 21 verified failures, while omitting rate limits, upstream faults, timeouts, partial commits, and uncertain idempotency.The probes used nonexistent tool names, wrong required-field types, and a conservative subset of type-correct out-of-domain sentinels.Source: B. Low-load probes and released… · p. 3
  • The analyzed records are Python SDK 1.26.0 normalized views, not untouched HTTP bodies or JSON-RPC frames; identifiers and sensitive endpoint data remain private.The release also minimizes selected service-specific observations and keeps restricted source records private.Source: B. Low-load probes and released… · p. 3

C. Coding procedure

The six capabilities are coded independently, with borderline records illustrating why evidence for one capability does not imply another. The model exercise separates inference from answer-bearing instructions and measures policy labels rather than completed recovery.

C. Coding procedure

  • Each of six capabilities was coded independently in both views by the sole author; the AI-assisted pass was sensitivity evidence, not inter-rater agreement.An archival reliability claim would require two trained independent coders and adjudication.Source: C. Coding procedure · p. 3
  • S11 and S18 support failure detection without a unique policy, cause, or target, while S21 supports policy and cause but names no repair parameter.These borderline decisions are checkable in the row-level ledger.Source: C. Coding procedure · p. 3

A. Conditions and evidence limits

  • The four paired conditions range from normalized observations and cause-code-only messages to structured-action JSON and prose stating the cause and preferred policy.The code-only/normalized pair is the only comparison that tests recovery inference without revealing the preferred answer.Source: A. Conditions and evidence limits · p. 3
  • Structured and prose action conditions are descriptive rather than controlled format comparisons because they disclose different words as well as different formats.Their scores measure compliance with an answer-bearing instruction, not independent recovery reasoning.Source: A. Conditions and evidence limits · pp. 3–4
  • The study records 840 repeated judgments across two small models, four conditions, and 21 failures, but tests policy labels rather than actual recovery.No argument is repaired, credential acquired, alternative discovered, side effect checked, or task completed; the findings should not be generalized to frontier or deployed agents.Source: A. Conditions and evidence limits · p. 4
  • Exact agreement with the preferred policy is the main outcome; coverage counts valid non-stop policies, while immediate replay returns the same stored failure for at most three attempts.Always stopping is only a safety check because it belongs to every non-harm set.Source: A. Conditions and evidence limits · p. 4

B. Descriptive analysis and deterministic baseline

The analysis reports exact descriptive counts without pooling incomparable models or conditions. Its deterministic client validates an experimental recovery contract and stops safely when an input is malformed or unsupported.

B. Descriptive analysis and deterministic baseline

  • Exact counts and predeclared paired differences are reported without pooling models or conditions, and no population interval is used for these non-probability observations from ten servers.Wilson and ten-server cluster-bootstrap calculations remain sensitivity analyses in the artifact.Source: B. Descriptive analysis and… · p. 4
  • The deterministic client checks the completed-result shape and isError:true, validates experimental recovery fields, then uses a local cause registry to select host policy and replay class.Malformed or unsupported inputs stop safely; the prototype is an executable design baseline, not a production extension or model leaderboard.Source: B. Descriptive analysis and… · p. 4

A. Specification and static code

The specification and source audit find that ordinary completed failures rely on text rather than a general structured recovery contract. In the 21-record sample, interpreting content reveals more actionability than typed fields, but no record supplies a complete repair or safe-replay basis.

A. Specification and static code

  • An ordinary completed CallToolResult has resultType, isError, content, and optional structured content, but no standard failure code, repair target, policy, retry delay, or replay-safety field.Specialized InputRequiredResult/MRTR and typed protocol errors exist, but do not form a general taxonomy for completed failures.Source: A. Specification and static code · p. 4
  • All 12 adjudicated production error-construction paths combine isError with text; none provides a standard code or action, though one audited repository has a private structured contract.Seven paths use prose alone and five serialize JSON or mixed JSON inside a text block; the private contract demonstrates feasibility but also fragmentation.Source: A. Specification and static code · p. 4

B. Joint capability profiles

  • The 21 failures comprise ten nonexistent tools, seven invalid argument types, three invalid values, and one authentication failure; their normalized views include eight protocol errors, ten isError results, and three success-shaped errors.These are safely induced observations from the sampled servers.Source: B. Joint capability profiles · p. 4
  • 18 of 21 records expose failure through typed fields, and eight -32602 observations support only a broad request-repair policy, not a specific cause.The eight codes cover five unknown tools and three invalid arguments; once content is interpreted, all 21 failures are visible, 19 support a coarse policy, 19 identify a cause, and 18 name a target.Source: B. Joint capability profiles · p. 4, Table I · p. 4
  • No observation supplies a complete replacement request, explicit retry timing, unchanged-request safety, or side-effect state.The content-assisted view reaches profiles 111100 (18), 111000 (1), and 100000 (2), using D/P/C/T/E/R coordinate order.Source: B. Joint capability profiles · p. 4
  • The profile sets form chains in this sample, so the partial order's advantage is not exercised; E = R = 0 reflects missing standard fields in the control view and only this safely inducible mix in the content view.The content-view scope does not include transient, rate-limit, upstream, timeout, partial-effect, or idempotency cases.Source: B. Joint capability profiles · p. 4

C. Exploratory model-by-condition results

The exploratory model results show no gain from a symbolic cause code in the answer-free comparison, while supplied actions produce many more strict matches. A separate deterministic prototype passes its counterfactual controls as designed, without establishing interoperability or protocol security.

C. Exploratory model-by-condition results

  • 42 to 39 strict matches for Llama and 6 to 5 for Qwen followed addition of a symbolic cause code in the only answer-free comparison.With disclosed actions, structured JSON yields 95 and 39 strict matches, while prose yields 105 and 30; the lexical differences rule out a clean format comparison, and code-registry and explicit-policy baselines reach 21/21 by construction.Source: C. Exploratory model-by-condition… · p. 5
  • Qwen accounts for all terminal false-success judgments and most needless replay; these counts characterize the tested models and controller, not load amplification on a degraded service.Qwen makes 24, 24, 26, and 23 needless replays across the four conditions, producing 48, 48, 55, and 42 additional calls; Llama adds one, one, zero, and zero.Source: C. Exploratory model-by-condition… · p. 5

D. Executable conformance result

  • The deterministic prototype handles all 21 counterfactual controls as designed, including two synthetic transient cases for safe immediate replay and delayed retry.Seventeen malformed or unsafe inputs fail closed with an exact reason and stop/escalate; the suite demonstrates executability in one implementation, not interoperability or protocol security.Source: D. Executable conformance result · p. 5

VII. An Evidence-Motivated Two-Plane Candidate

The candidate separates validated host controls from explanatory content for people and models. Its prototype branches deterministically on controlled cases, but its placement in MCP and interoperability remain open.

VII. An Evidence-Motivated Two-Plane Candidate

  • The proposed design keeps validated recovery controls in a host control plane and explanations in content for models and people.It is presented as a modest design rule, building on earlier structured-recovery proposals.Source: §VII · p. 5
  • The illustrative envelope pairs a cause code and parameter target with a revise-arguments policy, unsafe same-request replay, and no committed side effects.Its text content separately explains that walletAddress must be a string.Source: §VII · p. 5
  • The host validates the result and experimental controls before applying local authorization, confirmations, retry budgets, backoff, or circuit breakers.Server assertions about policy, replay, and side effects are not automatically trusted; unknown codes or unavailable capabilities stop safely.Source: §VII · p. 5
  • The control's place in MCP remains open, and the 21-case prototype does not test independent interoperability, downgrade behavior, SDK validation, or adversarial assertions.A reserved member would need capability negotiation and independent versioning; the example namespace is non-official.Source: §VII · p. 5

VIII. Threats, Ethics, and Reproducibility

The study's sampling, coding, model tests, and simulator impose narrow interpretive boundaries; its ethical safeguards and reproducibility materials also have explicit limits.

VIII. Threats, Ethics, and Reproducibility

  • The sample is not a census: reachability and low-risk inducibility select failures, while transient, rate-limit, upstream, timeout, partial-commit, and idempotency cases are absent.The lexical audit covers one high-precision candidate class, and the data do not estimate prevalence or preserve raw-wire behavior.Source: §VIII · pp. 5–6
  • The rule-based profile was coded by one author, and the universally safe always-stop option makes 100% coverage uninformative as progress.Preferred policies remain judgments and may omit clarification, discovery, or user-controlled authorization.Source: §VIII · p. 6
  • Two small local models, one prompt, and 21 scenarios cannot characterize deployed agents, especially when treatments reveal the answer.The authors lacked frontier-model credentials and budget; the corpus and ten server clusters also offer little inferential reach.Source: §VIII · p. 6
  • The simulator records policy labels rather than repaired requests, downstream completion, or side effects, and replays only the same stored response.It includes no backoff, shared load, service recovery, or real transient outage, so its extra calls are not a degradation result.Source: §VIII · p. 6
  • Probes were low-volume and limited to initialization, nonexistent names, malformed arguments on selected read-only tools, and sentinel values.The authors did not obtain maintainer consent or seek an IRB determination; proposed follow-on safeguards include community notice and coordinated disclosure.Source: §VIII · p. 6
  • The package reproduces derived analysis, but absent raw model responses prevent third parties from re-auditing parser and invalid-output decisions.It includes sampling metadata, record-level rationales, scanner materials, analysis and conformance code, and hashes; raw registry responses remain unreconstructable.Source: §VIII · p. 6

IX. Discussion and Conclusion

The paper finds that completed MCP results more reliably expose failure than specify recovery, leaving concrete action dependent on prose or information outside the result. Its two-plane example offers a fail-closed conformance target, while model and interoperability conclusions remain limited.

IX. Discussion and Conclusion

  • Completed results often expose failure and sometimes a broad response, but usually lack cause, target, executable repair, or replay conditions without prose or other state.Relying on prose makes semantic interpretation, with its failure modes, part of the recovery path.Source: §IX · p. 6
  • Showing two small models a taxonomy name did not improve policy selection, while revealing the answer mainly tested whether they followed it.Needless retries and false-success judgments motivate separating failure belief from recovery policy, not estimating deployed-agent behavior or outage load.Source: §IX · p. 6
  • The two-plane example is a conformance target, with a stable cause code, compatible policy, optional target, and explicit replay and side-effect assertions.The client branches deterministically and fails closed on controlled cases, but independent interoperability and adversarial robustness remain unknown.Source: §IX · p. 6
  • Testing the conclusion further requires independent coding, raw-wire captures across SDKs, missing failure classes, end-to-end recovery experiments, and interoperability and adversarial trials.For ordinary completed tool errors, MCP standardizes observability more strongly than recovery semantics, with exceptions for typed specialized workflows.Source: §IX · p. 6
Loading 2609.00072v1…