PaperlayerView exact revision on arXiv

Source-linked AI summary

ProbeID: Verifying MCP Tool Identity for LLM Agents under Registry Drift

Guanxing Chen, Sining Zhang, Haohuai He, Qiujie Lv, Yu-An Huang, Zhi-An Huang

arXiv:2610.33633v1cs.CRcs.AI

TL;DR

Copyable descriptions and admission-time metadata binding do not establish which tool behavior an agent will receive. ProbeID compares schema-valid probe responses with a trusted admission reference and abstains on ambiguity. It reaches 0.956 strict-choice hit@1 on cached cases, reduces tainted agent tasks from 15 to 9 of 16, and remains bounded by reference integrity and probe coverage.

  • Problem

    Copyable descriptions cannot establish which implementation runs, and metadata binding at admission does not reveal later behavioral drift.

    Source: §1 · p. 1

  • Method

    ProbeID compares candidates’ canonicalized responses to schema-valid probes against an attested admission reference and abstains when no candidate matches or the decision is ambiguous.

    Source: §1 · p. 2, §3.2 · p. 4

  • Results

    0.956 strict-choice hit@1 on cached cases contrasts with description routers approaching 1/(k + 1), while verification cuts tainted agent tasks from 15 to 9 of 16.

    Source: §1 · p. 3

  • Takeaways & Limitations

    ProbeID reduces tainted agent inputs within the reported setting, while real-tool re-probing found 7 of 59 tools changed responses despite unchanged version pins.

    Source: §1 · p. 3, §5.5 · p. 8

    Verification relies on a trusted admission reference and probes that reach behavioral differences, and it cannot observe side effects that never reach a response.

    Source: §6 · p. 8, Ethics statement · p. 9

Abstract

from arXiv · show

Agents select tools by copyable descriptions, which cannot establish which implementation runs. ProbeID compares canonicalized responses to schema-valid probes with a trusted admission reference and abstains on ambiguity, so a squatter must copy probe-observable behavior. With the gold present, independent probes and a fixed margin, standard concentration arguments give a shared probe budget logarithmic in the candidate count and bound the joint probability of a wrong decision. On cached MCP-IdentityDrift cases, strict choice reaches \(0.956\) hit@1 while description routers fall near chance, but simpler exact comparisons tie it, MMD better tolerates heavy value noise, and a 32B behavioral judge is similarly accurate at higher cost. Random probes from agent banks and a prospective held-out study of 38 tools test theory-set thresholds. On the agent banks, equality refuses every reachable replacement from 64 probes and the bounded distance from 512, and sharper post hoc gates halve both. Two tools whose drift no bank probe reaches always pass. In an MCP-Bench agent, verification cuts tainted tasks from 15 to 9 of 16 at unchanged task success. Over eleven weeks, 7 of 59 real tools changed responses despite version pins. Verification depends on reference integrity, probe coverage, and response noise.

1 Introduction

Copyable tool descriptions cannot establish which implementation runs, and admission-time metadata binding does not reveal later behavioral drift. ProbeID instead compares candidate responses with a trusted admission reference, abstains when needed, and is evaluated on cached tools, agent banks, and real MCP servers.

Motivation

  • Copied descriptions can conceal different behavior, and metadata-based selection cannot recover the intended tool when descriptions are identical.Admission-time binding of definitions or provenance does not expose a later change in behavior.Source: §1 · p. 1

Approach

  • ProbeID records canonicalized trusted responses to schema-valid probes, then compares candidates’ paired responses against that reference at selection time.Behavioral identity requires probes to reach the difference and a trusted reference.Source: §1 · p. 2, Figure 1 · p. 2
  • At a fixed margin, the shared probe budget grows logarithmically with the candidate count, and the joint probability of a wrong decision is bounded by δ when the gold is present.The contribution states a matching lower bound on the leading budget term under independent candidate channels.Source: §1 · pp. 2–3

Evaluation overview

  • 0.956 strict-choice hit@1 for ProbeID contrasts with description routers approaching 1/(k + 1); an agent gate also cuts tainted tasks from 15 to 9 of 16.Random probes from agent banks and a prospective held-out study test where theory-set thresholds refuse replacements and where their resolution ends.Source: §1 · p. 3
  • 7 of 59 real tools departed from recorded responses under unchanged version pins during eleven weeks of re-probing.Three departures followed reinstalls at unchanged versions.Source: §1 · p. 3

2 Related work

Prior work studies tool retrieval, registry security, behavioral identity, formal selection certification, and related runtime checks. ProbeID is distinguished by comparing all candidates against trusted probe responses to select among tools that share a schema and description.

Tool retrieval and selection

  • Tool retrieval systems select by descriptions, so a copied description inherits the trusted tool’s retrieval score.Examples include ToolLLM [Qin et al., 2024] and MCP-Zero [Fei et al., 2025]; bridge models, rerankers, and documentation augmentation also remain description-based.Source: §2 · p. 3

Tool ecosystem security

  • Admission defenses bind definitions, registry trust, or provenance, while runtime detectors screen for deviation or injected content; ProbeID adds use-time selection against a trusted reference.ProbeID detects an admitted rug pull only when the reference and drawn probes expose the change, and cannot see exfiltration that leaves responses unchanged.Source: §2 · p. 3

Certification of tool selection

  • LLMCert-T certifies metadata-based selection and reports collapsing correctness under adaptive distractors; ProbeID instead gives a conditional guarantee from probe-observable behavior.Composing ProbeID with a metadata certificate is left open.Source: §2 · p. 3

Identity from behavior

  • Behavioral identity spans testing, query learning, fuzzing, scanning, software birthmarks, and model fingerprinting, but the cited model tests do not select among adversarially authored tools sharing a schema and description.ProbeID pairs a margin with a budget bound, a matching lower bound on its leading term, and an abstention rule with a joint error bound.Source: §2 · p. 3

Concurrent work

  • Concurrent MCP probing, skill-behavior measurement, endpoint stability checks, and TrustShiftProbe compare an artifact with its declaration or history; ProbeID probes all candidates against trusted responses to select among them.Source: §2 · p. 3

3 Behavioral tool identity

Behavioral identity is defined by agreement with a trusted reference on responses observable under schema-valid probes, not by endpoint identity. ProbeID builds an attested response bank, compares candidates on a secret subsample, and abstains when distance or ambiguity gates fail.

3.1 Problem setup

  • Behavioral identity recovery returns a candidate equivalent to the gold on the probe distribution, or abstains when no candidate is confidently identifiable.This notion cannot detect side effects absent from responses, behavior outside probe support, or a reference recorded from a compromised tool.Source: §3.1 · p. 4

3.2 ProbeID

  • At admission, ProbeID records each tool’s trusted behavior as an attested bank of schema-valid probes and canonicalized responses, then secretly subsamples bank probes at selection.Selection probes only current candidates, so no live reference endpoint is needed.Source: §3.2 · p. 4
  • Type-aware enumeration generates probes from boundary values, enums, and required or optional fields; the theorem covers i.i.d. probes or uniform sampling without replacement from the bank.Pure deterministic enumeration is outside the guarantee, and population margins transfer to the bank only if it was sampled from that population.Source: §3.2 · p. 4
  • Canonicalization drops named volatile fields and sorts mapping keys, while a bounded response-tree distance combines structural disagreement with status-code and state-delta channels.The analysis requires boundedness and symmetry, not that the chosen distance dominate other paired comparisons.Source: §3.2 · p. 4
  • ProbeID selects the candidate with the smallest estimated behavioral distance, but abstains if no candidate passes the distance gate or the nearest two are too close.A sole candidate faces only the absolute distance gate.Source: §3.2 · p. 4

4 Identity guarantee under squatting

Under a positive behavioral separation margin and independent probe observations, ProbeID’s nearest-fingerprint rule has a bounded joint error probability. Its guarantees depend on probe support and do not extend to stateful tools under the stated theorem.

Guarantee scope

  • The margin depends on the probe distribution: squatters indistinguishable on its support are equivalent under this comparison, and stateful tools fall outside the independent-probe theorem.Source: §4 · p. 5

Theorem 1

  • With the gold present, a margin γ > 0, and independent observations, the nearest-fingerprint rule returns a behaviorally correct tool with probability at least 1 − δ.The theorem assumes bounded per-probe distance and probes drawn i.i.d. from D.Source: §4 · p. 5

Decision rule and guarantee

  • For pools with at least two candidates, the stated gates bound the joint probability of deciding wrongly by δ; a sole candidate uses only the absolute gate.Among decided cases, the error bound is δ/P(decide), not δ; a sole candidate is refused with probability at least 1 − δ when its margin exceeds 2ϵ.Source: §4 · p. 5

Probe complexity

  • At fixed margin, k squatters affect shared probe inputs through log((n + k)/δ), while querying all candidates can require up to m(n + k) candidate calls.Source: §4 · p. 5

Variation and lower bound

  • When per-probe distance variance is at most σ2, Bernstein yields a variance-aware bound, and an independent-channel construction gives an Ω(σ2γ−2 log(n + k)) lower bound on the leading term.Source: §4 · p. 5

5 Experiments

The experiments evaluate ProbeID against description-based routing, test theory-set thresholds on random probe banks, compare behavioral metrics under noise, and assess verification on live tools and agent tasks.

5.1 Setup

  • The experiments use cached tool responses and schema-compatible squatters, with identity hit@1 conditional on gold tools being separable from a squatter.The experiments also test random-bank thresholds, comparison methods under noise, and effects on agent inputs.Source: §5 · p. 5, §5.1 · p. 6

5.2 Result 1: identity under registry drift

  • ProbeID reaches 0.956 strict-choice hit@1 while description routers approach chance under copied descriptions; behavioral comparisons tie on a clean cohort.A 32B judge is ahead at k ∈{2, 4} but behind at k ∈{1, 8}; the comparison depends on the operating point.Source: §5.2 · p. 6, Figure 2 · p. 7

5.3 Result 2: probe budget and error control

  • In the Bernoulli mismatch channel, hit@1 rises with probe budget, and measured live-server hit@1 tracks 1 − (1 − ϕ)^m against a near twin.The smallest budget reaching 0.99 hit@1 keeps m⋆γ^2 near constant across a fourfold range of γ, with larger products at greater gold self-distance.Source: §5.3 · pp. 6–7, Figure 2 · p. 7
  • With random bank probes, equality refuses every reachable drifted replacement from 64 probes, while structural distance requires 512; two unreachable drifts always pass.Sharper post hoc gates halve the refusal budgets, but one-in-twenty drift passes on 14 of 22 local tools in the held-out study.Source: §5.3 · pp. 7–8, Figure 2 · p. 7

5.4 Result 3: complementarity, and drift in real registries

  • ProbeID is complementary to routing: a dense router scores 1.00 and ProbeID 0.90 on distinct-description sibling groups, with ProbeID abstaining rather than misidentifying one pair.A constructed twin family differing only in description also shows why ProbeID does not replace routing.Source: §5.4 · p. 8
  • MMD performs best under heavier value jitter, while 0/1 equality uses fewer probes on deterministic agent tools; no comparison wins in every regime.Canonicalization can hide drift confined to volatile fields.Source: Figure 2 · p. 7, §5.4 · p. 8
  • Over eleven weeks, 7 of 59 tools changed responses beyond their own baseline noise despite unchanged recorded versions; none of the changes indicates malice.Three tools changed after reinstall, alongside legitimate content updates and a documentation recommender’s increasing distance.Source: §5.4 · p. 8

5.5 Result 4: verification in an agent

  • Verification reduces rug-pull-tainted MCP-Bench tasks from 15 to 9 of 16, while task success remains 1.000 in all four arms.The gate uses each tool’s whole bank of up to 512 probes; two changes missed by bank probes account for surviving corruption.Source: §5.5 · p. 8

6 Limitations

ProbeID’s guarantees and observed behavior are bounded by the reference, probes, and operating setting.

6 Limitations

  • A compromised admission reference can make ProbeID accept compromised behavior, and changes outside the probe distribution can pass unseen.The full-bank agent gate costs up to 512 probe calls and minutes on a slow remote tool; stateful tools fall outside the independent-probe theorem, and production scale is untested.Source: §6 · p. 8
  • Figure 3 distinguishes visible validation-error drift from tensor-value drift missed by bank probes, and reports per-tool distance, calls, and task outcomes.The figure also reports refused and corrupted calls by tool, and tainted tasks and probe calls under two gates.Source: Figure 3 · p. 9

7 Conclusion

ProbeID checks whether canonicalized probe responses match a trusted behavioral reference and abstains when evidence is inconclusive.

7 Conclusion

  • Across reported studies, calibrated gates refuse sufficiently separated replacements and reduce corrupted agent inputs without changing benchmark success, subject to reference integrity, probe support, and response noise.Source: §7 · p. 9

Ethics statement

The paper presents ProbeID as a defensive mechanism for agent tool use.

Ethics statement

  • Probe execution should be confined to read-only or sandboxed calls; the studies use cached responses, whitelisted read-only tools, or reset sandboxes.A compromised admission reference would cause ProbeID to accept the wrong behavior, and abstention does not remove that dependency.Source: Ethics statement · p. 9

Reproducibility statement

The paper documents how to reproduce its reported results and positions ProbeID within prior work on behavioral identification and software testing.

Reproducibility statement

  • Released scripts and artifacts make reported tables, figures, benchmark results, census replays, and agent-study counts recomputable.The releases include recorded tool calls and responses, replay verdicts, gate logs, and per-entry distances.Source: Reproducibility statement · pp. 9–10

Appendix

  • Appendices B–F prove Section 4 and define the distance, while later appendices cover adversaries, evaluation studies, census, agent replay, and implementation.Appendix T provides implementation and reproduction details.Source: Appendix · p. 14

A Behavioral identity in prior work

  • Prior work probes artifacts or compares them with their own histories, whereas ProbeID compares description-identical candidates with a trusted admission reference and bounds selection error.Differential testing and registry-side service testing also inform ProbeID’s use-time checks and calibrated abstention rule.Source: A Behavioral identity in prior work · pp. 14–15

B Proof of Theorem 1

This section bounds ProbeID’s probe budget and wrong-decision risk under explicit conditions, and explains how abstention, probe coverage, and response noise qualify those guarantees. It also addresses verifier-aware squatters and behavioral judging, including abstention on behavioral twins and the 32B judge’s higher cost and context limits.

B Proof of Theorem 1

  • With the gold present, a positive margin, and independent probes, nearest-fingerprint selection is correct with probability at least 1−δ using a shared budget logarithmic in the candidate count.For the deployed bank, sampling without replacement supports the concentration bound on the bank distribution; transferring a population margin requires the bank itself to be sampled i.i.d. from that population.Source: B Proof of Theorem 1 · p. 15

C The abstaining algorithm

  • The abstaining rule bounds the joint probability of a wrong decision and deciding by δ, and returns the gold when γ ≥ 4ϵ with no behavioral twin present.This is not a δ-level bound on error conditional on deciding; the sequential gate also preserves the joint guarantee and, on agent banks, refused reachable replacements after 12.5 probes on average.Source: C The abstaining algorithm · pp. 15–16

D Sources of the behavioral margin

  • A positive margin exists exactly when every squatter differs from the faithful deterministic gold on a set of positive probe mass.Zero margin means indistinguishability under the chosen distance and probe distribution, not necessarily identical behavior outside probe support or in ignored fields.Source: D Sources of the behavioral margin · p. 17

E The behavioral distance dY

  • The bounded behavioral distance compares canonicalized output, status, and state delta, with volatile keys dropped and structural differences measured in [0, 1].The probabilistic union prevents an output difference from being diluted when status and state delta stay constant.Source: E The behavioral distance dY · p. 17

F.1 Variance-aware budget

  • When per-probe variance is bounded by σ², Bernstein’s inequality yields a leading probe-budget term of O(σ²γ^-2 log((n+k)/δ)).A plug-in variance estimate provides a runtime diagnostic; the reported sweep found m⋆γ² growing with σ² as predicted.Source: §F.1 · p. 18

F.2 A lower bound matching the leading term

  • For independent candidate channels, the leading σ²γ^-2 dependence and log(n+k) factor are minimax necessary, matching the variance-aware upper bound’s leading term.The lower bound does not capture the additive O(γ^-1 log(n+k)) term, and its minimax claim does not cover correlated drift.Source: §F.2 · pp. 18–19

G.1 Randomized probes against committed-coverage adversaries

  • The live committed-coverage sweep had a 0.000 misidentification rate across 14,080 trials, while predictable fixed probes caused abstention on 1.000 of trials.With a secretly subsampled bank, detection rose with probe budget; for a committed mimicry region of coverage ρ, evasion occurs with probability ρ^m. These results have the gold present at distance 0.00 and do not establish the same outcome when it is replaced.Source: §G.1 · pp. 19–20

G.2 Independently authored and verifier-aware squatters

  • Against verifier-aware squatters, ProbeID’s hit@1 falls to 0.10 at k=8 as abstention rises from 0.58 at k=1 to 0.90 at k=8, with 0.00 misidentification.The benchmark includes model-written blind and verifier-aware attacks; against blind squatters, hit@1 stays 0.84–0.86 across k, with 0.00 misidentification. These attacks broaden the tested cases but do not cover all adversaries.Source: §G.2 · p. 20

H A behavioral LLM-judge baseline

  • At k=1, 2, 4, and 8, the Qwen2.5-32B judge reaches 0.996, 0.968, 0.967, and 0.950 hit@1, while ProbeID scores 0.952 for k ≥2.On the same truncated evidence, the 32B judge is ahead at k=2 and k=4; its decision cost is 3.40 seconds versus 3.527 milliseconds for ProbeID, excluding probe calls.Source: Table H.1 · p. 21, H A behavioral LLM-judge baseline · p. 22
  • The exact-match rule on judge-truncated responses scores 0.997 hit@1 at k=1 and 0.951 at k ≥2, versus 1.000 and 0.952 on complete responses.The 200-character cut hides the only distinguishing difference in 3 case-runs at k=1 and 1 case-run at each larger k.Source: Table H.1 · p. 21, H A behavioral LLM-judge baseline · p. 21
  • At k=8, counting unsent complete-response prompts as misses lowers the 32B judge to 0.783 hit@1; ProbeID scores at least 0.981 on those cases.The judge cannot send 73, 83, 108, and 154 of 900 case-runs at k=1, 2, 4, and 8 because of its 32,768-token context limit.Source: Table H.2 · p. 21, H A behavioral LLM-judge baseline · p. 22

G.2 Independently authored and verifier-aware squatters

  • On four live tools, blind squatters are caught at hit@1 1.00, while verifier-aware squatters reach 0.17 at k=8 with 0.00 misidentification.Across 103 benchmark cases driven to zero canonical divergence, ProbeID abstains on all behavioral twins; the theorem’s bound does not apply to these fixed probes.Source: §G.2 · p. 21

H A behavioral LLM-judge baseline

  • The Qwen2.5-7B judge reaches 0.973 hit@1 at k=1 and 0.844 at k=8, while ProbeID stays at 0.952 for k ≥2.At k=8, the judge’s 95% exact binomial interval is [0.819, 0.868], versus [0.936, 0.965] for ProbeID; the intervals treat case-runs as independent.Source: H A behavioral LLM-judge baseline · p. 21

H.1 Decision reproducibility and inference cost

Across these studies, ProbeID is insensitive to description similarity, identifies behaviorally distinct natural siblings against an admission reference, and abstains on behavioral twins. Its decision step is much faster than a behavioral judge, though that timing excludes probe execution.

H.1 Decision reproducibility and inference cost

  • Across tested replay configurations, the judge showed no reproducibility advantage over ProbeID; decision-step cost was 3.40 seconds versus 3.527 milliseconds.Both timings exclude probe calls, which can dominate end-to-end latency on real servers.Source: §H.1 · p. 22

I.1 Beyond identical descriptions

  • At k=4, ProbeID remains at 0.956 as description similarity falls from 1 to 0, while routers recover as descriptions diverge.The squatter behavior remains mutated throughout; dense retrieval recovers earlier than BM25, and routers reach ceiling only with clearly different descriptions.Source: §I.1 · pp. 22–23, Figure I.1 · p. 23

I.2 Complementarity on a constructed twin family

  • When every candidate has the gold behavior, the behavioral margin is zero, so the gap gate abstains and responses cannot identify the recorded tool.This constructed twin family complements the natural-sibling test: behaviorally equivalent candidates require routing or escalation rather than behavioral identification.Source: §I.2 · p. 23

I.3 Natural description collisions and a dense router

  • On listing siblings, ProbeID hits 1.00 versus routers’ 1/3 chance; on a deprecated-alias behavioral twin, it abstains while routers sit at 1/2 chance.The live filesystem study used vendor-authored descriptions; listing behaviors were widely separated, and routers lacked the admission reference, limiting direct comparison with intent-based retrieval.Source: §I.3 · pp. 23–24, Table I.2 · p. 24

I.4 Two real implementations, a behaviorally close natural case

ProbeID’s budget and abstention behavior are tested on naturally close implementations, controlled response noise, real recorded probes, and cached-case thresholds. The results show where comparison choice and probe coverage constrain identification.

I.4 Two real implementations, a behaviorally close natural case

  • At γ≈0.3, hit@1 rises from 0.29 to 1.00 as m grows from 1 to 16, while at ϕ≈0.15 it reaches 0.93 only at m=16, with no misidentifications.The CPython and V8 serializers agree on many inputs and differ on documented defaults; the remaining cases abstain.Source: §I.4 · pp. 24–25

J.1 The canonicalization ablation in full

  • With ten injected timestamp and id fields, canonicalization keeps the 150-gold cohort’s noise floor at 0.00 and hit@1 at 0.956; without it, gold self-distance rises to 0.73.Under noise the canonicalizer does not remove, continuous distance retains discrimination better than equality or hashing.Source: §J.1 · p. 25

J.2 Simple behavioral baselines

  • Under unlisted volatile fields, exact equality and response hash fall to 0.083, while structural distance and MMD score 0.950 and 0.951; at ρ=0.2 and k=8, MMD reaches 0.944 versus 0.881 for structural distance.The MMD advantage is empirical; the theorem’s probe-mean guarantee applies to bounded per-probe comparisons, not set-level MMD.Source: §J.2 · pp. 25–26, Table J.2 · p. 26

K.1 The probe budget on real recorded behavior

  • On real recorded probes, hit@1 rises with budget and tracks the conditional closed form; reaching 0.9 requires about 12 probes at k=4 and 16 at k=8 when γ=0.3.The curves are conditional on at least one separable squatter; counting excluded golds as misses lowers each measured point by at most 0.04.Source: §K.1 · p. 27, Figure K.1 · p. 27, Table K.1 · p. 27

K.2 Squatter counts beyond eight

  • At k=16, 32, and 64, ProbeID remains at 0.956 hit@1 on the noiseless cache, while BM25 and dense retrieval fall to or below their chance lines.This flat curve does not test the theorem’s logarithmic candidate-count dependence because the two mutation families are already separated.Source: §K.2 · p. 27

K.3 Abstention on fixed cached cases

  • At k≥2, all 20 of 450 strict-choice misses are ties on every available cached probe; a favorable threshold-grid point reaches 0.956 coverage but cannot establish error control on unseen cases.The threshold grid is retrospective, and 11 missed tools expose fewer than the full 12 cached probes.Source: §K.3 · p. 28

L.1 Probe generator ablation on a live server

Live-server studies test fresh schema-valid probes across several server types and adversary behaviors. ProbeID recovers identity when responses distinguish tools, but state, authentication, large outputs, and probe coverage bound what it can conclude.

L.1 Probe generator ablation on a live server

  • On live get-sum, M1 reaches 1.00 hit@1 by m=8 at every tested margin, while uniform probes track 1 −(1 −ϕ)^m at their realized divergence.M1’s boundary values over-cover the tested hash-defined drift region, a benefit that can make predictable boundaries vulnerable to enumeration.Source: §L.1 · p. 28

L.2 Diverse live servers

  • Across filesystem, memory, and everything server families, the same fingerprint rule recovers identity across three response modalities, while identical tool cards leave description routers at chance.Filesystem and memory squatters reuse the official binary with tampered backing data; this is weaker than an independent reimplementation.Source: §L.2 · p. 29, Table L.2 · p. 29

L.3 Re-implemented squatters

  • Format-faithful reimplementations with wrong values are caught at hit@1 = 1.00 with one probe across four tools, while exact twins cause abstention rather than misidentification.A near twin is identified only when probes reach its drift region; the study covers read-only deterministic tools with few side effects.Source: §L.3 · pp. 29–30, Table L.3 · p. 30

M.1 Stateful and side-effecting tools

  • With a shared session, the counter tool’s gold self-distance reaches 1.00 and ProbeID abstains on every trial; fresh sandboxes restore self-distance to 0.00 and hit@1 to 1.00.Probes remain sequential within each sandbox, so the independent-probe theorem does not cover this empirical result.Source: §M.1 · pp. 30–31, Table M.1 · p. 30

M.2 Side-effecting writes and authentication

  • Reset-sandbox probes identify both tested write-semantic drift and an authentication bypass at hit@1 1.00, with misidentification rate 0.00; an off-distribution backdoor remains outside the guarantee.The studies use controlled self-hosted tools and cooperative squatters, not adaptive or production-scale adversaries.Source: §M.2 · p. 31

N Behavioral distance under large outputs

O An in-the-wild census of real MCP servers

A read-only census and repeated replays measure response stability across real MCP servers, then apply a fixed gate to the saved replay. The observations include service-side drift and changes after same-version reinstalls, but do not establish attack or false-alarm rates.

Service-side drift over time

  • Four of 59 tools show genuine drift at 24 hours; the service-side count remains four at 22 days, while some response distances grow.Three affected tools belong to a time-varying weather server; the fourth is an AWS documentation recommender.Source: O An in-the-wild census of real MCP… · p. 33

Census findings

  • Across ten replays from day 22 onward, seven tools drifted while all 16 recorded server versions remained identical; four npm-server reinstalls at recorded pins changed three tools to distances 1.00, 1.00, and 0.80.Every changed response was a validation error string rendered by a library not pinned by the servers’ recorded versions.Source: O An in-the-wild census of real MCP… · p. 33

Fixed gate on the census

  • The retrospective equality gate accepts 55/59 tools and refuses 4/59 across 416 calls, but refuses only three of seven registry-side changes; three accepted changes had just one recorded probe.The census decisions are not adjudicated attacks or independently labeled benign updates, so they measure neither malicious-replacement risk nor false-alarm rate.Source: O An in-the-wild census of real MCP… · pp. 33–34

Agent impact

  • On MCP-Bench, verification reduces corrupted calls from 93 to 20 while refusing 102 calls instead, with zero refusals in the benign arm.The table reports the first of three runs; gold-server replay defines corrupted calls.Source: Table P.1 · p. 34

P The agent study and residual failures

In the 16-task agent study, verification reduced rug-pull-tainted tasks while task success stayed perfect; residual corruption came from drift that the probe banks could not reach. Repeated runs preserve the task-level reduction despite changing agent trajectories, and exact equality detected more drift at lower probe cost than structural distance in the reported comparison.

P The agent study and residual failures

  • The replay ground truth re-issued recorded calls against freshly started gold servers, excluding volatile leaves and image responses that could not be adjudicated.Every adjudicable benign call reproduced against the gold servers after masking volatile leaves.Source: P The agent study and residual failures · p. 34
  • Ten determinant calls and ten convert_batch calls remained corrupted because the banks did not reach the drifted behavior, leaving zero measured margin and no detection guarantee at any budget.The other two named tools were also allowed at m=32 but recovered when the full 512-probe bank was used.Source: P The agent study and residual failures · p. 34
  • 15 of 16 tasks were tainted in every ungated run, versus 9 of 16 in every gated run, while task success was 1.000 in all four arms.The paired task-risk reduction was 0.375, with a 95% task-resampling interval of [0.125, 0.625] in each run.Source: P The agent study and residual failures · pp. 35–36
  • Across repeated runs, taint counts stayed fixed while adjudicable calls and corrupted-call counts varied with the agent trajectory.The three ungated rug-pull runs tainted 15 tasks each; the gated runs tainted 9 each, with corrupted calls of 93/90/92 ungated and 20/16/17 gated.Source: P The agent study and residual failures · p. 35
  • At m=32, exact equality left 22 corrupted calls and 8/16 tainted tasks using 8,412 probe calls, versus 126,428 for structural distance at m=512, which left 20 and 9/16.Both benign configurations had no corrupted calls and no refusals in this comparison.Source: P The agent study and residual failures · p. 35
  • The study covers 16 tasks and uses a shared agent and judge model, so its scores are paired within the study and are not comparable to the published leaderboard.Greedy decoding and a swapped judge also limit comparison with that leaderboard.Source: P The agent study and residual failures · p. 36

Q Random probes from the attested banks

The bank study samples attested probe banks to test theory-set thresholds on their bank distributions. The checked bounds held in the observed draws, while replacement detection depended on both the comparison rule and whether probes reached the drift.

Q Random probes from the attested banks

  • Across 1,000 subsamples per tool and budget, no observed rate violated a checked bound in the premise cells.The bank means are exact expectations under the bank distribution, so each bound premise can be checked per tool, comparison, and budget.Source: Q Random probes from the attested banks · p. 36
  • At m=32, equality accepted reachable replacements at 0.006 on average, versus 0.750 for structural distance; equality accepted none from m=64, while structural distance accepted none at m=512.These rates are for the eight reachable tools among the ten whose squatters corrupted agent calls.Source: Q Random probes from the attested banks · p. 36, Table Q.2 · p. 37
  • No draw refused a gold or any of the 67 unaffected tools, while both comparisons let the two replacements whose drift was absent from the banks pass at every budget.Those two tools matched the gold on every bank entry, so the theorem makes no claim about their drift outside the bank.Source: Q Random probes from the attested banks · p. 36
  • At m=32, expected tool-level accuracy was 0.973 for equality and 0.896 for structural distance, against 0.870 for accepting every server.The two unreachable replacements cap accuracy for any gate that accepts every server matching the reference.Source: Q Random probes from the attested banks · p. 37

R A prospective held-out study

The prospective study tests fixed replacement policies on banks for 38 tools, then evaluates sharper gates on those same draws. Results vary with bank margin and response noise; full-bank checks remove within-bank sampling error but do not establish coverage beyond the bank.

R A prospective held-out study

  • Every candidate with bank margin above 0.2 was refused on every draw at full bank, while no gold was refused at full bank and smaller-budget gold refusals were at most 0.1%.The study found no observed rate beyond its bound in 7,227 premise cells.Source: R A prospective held-out study · p. 38, Table R.1 · p. 38
  • The noisiest weather tool accepted full value drift on every draw under the fixed gate, while the sequential gate left those checks inconclusive.A full-bank weather check took about 4.2 minutes at a median service latency of 1.6 to 1.7 seconds.Source: R A prospective held-out study · p. 38
  • Behavioral verification refused reworded error messages on all 16 affected tools and refused added fields on 5 of 7 affected local tools under dY.The format change was accepted wherever the gold was accepted; the added-field update passed on two file readers where it changed about one answer in a hundred.Source: R A prospective held-out study · p. 38
  • The prospective replacements were fixed transformations of open reference implementations, and local checks followed admission by only 22 to 23 seconds, so the study does not estimate wild replacement rates or slow drift.The policies and banks were fixed in advance; the study tests the rules on those banks and policies.Source: R A prospective held-out study · p. 38

S Exact and finite-population gates

  • On the agent banks, sharper equality refused a reachable replacement at 0.004 acceptance at m=8 and had no recorded acceptance from m=32, versus m=64 for the Hoeffding gate.The sharper gates were added after the Hoeffding results were known; the finite-population pair rule returned the gold on every draw at m=512 with the gold present.Source: S Exact and finite-population gates · p. 39, Table S.1 · p. 40
  • At full bank, the finite-population radius is zero, so the gate refuses candidates whose bank mean exceeds the admission self-distance, removing sampling error within that bank.It does not cover inputs outside the bank, and the sharper structural gate also refused every tested answer-changing legitimate update.Source: S Exact and finite-population gates · pp. 39–40

T Implementation and reproducibility

  • The cached benchmark keeps only cases with a separable squatter, and its gated thresholds are fixed configuration defaults rather than theory-derived values.The implementation description also notes that cached results average three registry seeds and that gold tools can recur across seeds.Source: T Implementation and reproducibility · pp. 40–41, Table T.1 · p. 40
  • For k=8, saved cases averaged 87.64 candidate-response comparisons, not measured live calls; querying the trusted reference live would add another m calls.The cached sweep took 18.8 seconds on one CPU core, excluding live tool latency, so it does not establish end-to-end speed.Source: T Implementation and reproducibility · p. 41

Related papers

Loading 2610.33633v1…