PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantifying Conversation Drift in MCP via Latent Polytope

Haoran Shi, Hongwei Yao, Shuo Shao, Shaopeng Jiao, Ziqi Peng, Zhan Qin, Cong Wang

arXiv:2508.06418v1cs.CL

TL;DR

MCP’s shared execution context creates security risks, while existing defenses do not adequately quantify conversational hijacking. SECMCP uses activation vectors and latent-polytope analysis to detect drift; its evaluation reports AUROC scores above 0.915 across three models and three benchmarks while preserving normal MCP functionality.

  • Problem

    MCP’s non-isolated execution context exposes systems to tool poisoning and indirect prompt injection, while existing defenses rely on static signatures, add computational overhead, or do not quantify conversational hijacking.

    Source: Introduction · p. 1

  • Method

    SECMCP models LLM activation vectors in a latent polytope to quantify conversation drift induced by adversarial external knowledge and assess inputs against learned client-specific access-control regions.

    Source: Introduction · p. 1, SECMCP Agent Design · p. 4

  • Results

    AUROC scores consistently exceeded 0.915 across Llama3, Vicuna, and Mistral on MS MARCO, HotpotQA, and FinQA while preserving normal MCP functionality.

    Source: Introduction · p. 2

  • Takeaways & Limitations

    SECMCP organizes MCP threats around hijacking, misleading, and data exfiltration and detects conversation drift associated with adversarial manipulation.

    Source: Introduction · p. 2

    The method assumes a stable query-response structure, lacks token-level attribution, and has an uninterpretable decision process, limiting use in asynchronous multi-agent settings and contexts requiring fine-grained control or transparency.

    Source: Limitations and Future Work · p. 7

Abstract

from arXiv · show

The Model Context Protocol (MCP) enhances large language models (LLMs) by integrating external tools, enabling dynamic aggregation of real-time data to improve task execution. However, its non-isolated execution context introduces critical security and privacy risks. In particular, adversarially crafted content can induce tool poisoning or indirect prompt injection, leading to conversation hijacking, misinformation propagation, or data exfiltration. Existing defenses, such as rule-based filters or LLM-driven detection, remain inadequate due to their reliance on static signatures, computational inefficiency, and inability to quantify conversational hijacking. To address these limitations, we propose SecMCP, a secure framework that detects and quantifies conversation drift, deviations in latent space trajectories induced by adversarial external knowledge. By modeling LLM activation vectors within a latent polytope space, SecMCP identifies anomalous shifts in conversational dynamics, enabling proactive detection of hijacking, misleading, and data exfiltration. We evaluate SecMCP on three state-of-the-art LLMs (Llama3, Vicuna, Mistral) across benchmark datasets (MS MARCO, HotpotQA, FinQA), demonstrating robust detection with AUROC scores exceeding 0.915 while maintaining system usability. Our contributions include a systematic categorization of MCP security threats, a novel latent polytope-based methodology for quantifying conversation drift, and empirical validation of SecMCP's efficacy.

Introduction

MCP expands LLM access to external information but its shared execution context creates security and privacy risks that existing defenses do not adequately quantify. SECMCP models activation shifts in a latent polytope to detect conversation drift, and is evaluated across several models and benchmarks.

Introduction

  • MCP’s non-isolated execution context exposes systems to tool poisoning and indirect prompt injection, which can cause hijacking, misleading information, or data exfiltration.Existing defenses rely on predefined signatures, add computational overhead, or fail to quantify the degree of conversational hijacking or hallucination.Source: Introduction · p. 1
  • SECMCP uses LLM activation vectors and a latent polytope to quantify conversation drift caused by adversarial external knowledge and support proactive detection.The authors link adversarial instructions, even when benign on the surface, to distinct latent neuron clusters and shifted generation trajectories.Source: Introduction · pp. 1–2
  • AUROC scores consistently exceeded 0.915 across Llama3, Vicuna, and Mistral on MS MARCO, HotpotQA, and FinQA while preserving normal MCP functionality.The evaluation uses simulated web-search and knowledge-database tools.Source: Introduction · p. 2
  • The paper categorizes MCP security risks as hijacking, misleading, and data exfiltration, and presents SECMCP as a detector of conversation drift with negligible usability impact.Its contributions also include evaluation across multiple LLMs and benchmark datasets.Source: Introduction · p. 2

LLM Attacks

  • Existing defenses include input-structure, authentication, and dynamic output-auditing approaches, but the reviewed methods operate at model interfaces rather than using internal activations.The authors state that no mature technique leverages internal model information for defense.Source: LLM Attacks · p. 2

MCP security

  • MCP security research is described as remaining at the level of technical guidance or engineering practice, motivating a systematic secure agent system.The cited overview and tool-security efforts are part of a research area the paper describes as still in its early stages.Source: MCP security · p. 2

MCP Architecture (Hou et al. 2025)

MCP connects an AI host to external services through a client and server, with transport and connection-lifecycle procedures governing their interactions. The section describes operation-phase attacks, the relevant threat actors, and three resulting risks: data exfiltration, misleading, and hijacking.

MCP Architecture (Hou et al. 2025)

  • The MCP host runs a local client to bridge to external services; the client dispatches requests and retrieves capabilities, while servers expose tools and return results.Client-server communication uses a transport layer and JSON-RPC 2.0, with initialization, message exchange, and termination stages.Source: MCP Architecture (Hou et al. 2025) · p. 3

Security and Privacy Risks in MCP

  • The operation-phase threat analysis covers tool poisoning and indirect prompt injection, with data exfiltration, misleading, and hijacking as the three resulting risks.The section introduces a threat model before defining the risks.Source: Security and Privacy Risks in MCP · p. 3

Threat Model

  • The threat model includes adversarial data-source providers and servers; servers can tamper with tool descriptions and manipulate agents toward unauthorized actions or disclosure of sensitive information.Data-source providers can embed malicious instructions in external data surfaced in AI dialogues.Source: Threat Model · pp. 3–4

Tool Poisoning Attacks

  • A malicious MCP server can embed instructions in tool metadata that the LLM uses to select tools, potentially bypassing system-level controls and disclosing sensitive information.Each tool is associated with metadata such as its name and description.Source: Tool Poisoning Attacks · p. 4

Data Exfiltration

  • Data exfiltration is an attempt to manipulate prompts to bypass LLM defenses and extract private information, such as PII, from the underlying database.Source: Data Exfiltration · p. 4

Indirect Prompt Injection Attacks

  • Indirect prompt injection occurs when malicious instructions embedded in external data are retrieved and processed, potentially leading to hijacking or misleading responses.The adversarial data-source provider can alter invoked external data to embed those instructions.Source: Threat Model · pp. 3–4, Indirect Prompt Injection Attacks · p. 4

Misleading

  • Misleading attacks inject deceptive information, such as fake news, which can distort the model’s understanding and produce inaccurate responses.Source: Misleading · p. 4

Hijacking

  • Hijacking injects segments intended to coerce the LLM into producing attacker-specified responses for certain queries, such as phishing redirects or biased political views.Source: Hijacking · p. 4

Our Methodology

SECMCP detects conversation drift by comparing query activations with benign references in latent space. It uses client-specific authorized regions and layer-wise distance matching to flag potentially malicious inputs.

Overview

  • SECMCP compares incoming-query activations with benign reference inputs and flags deviations that exceed a threshold.It computes deviations layer-wise to assess whether an input lies within the expected semantic region.Source: Overview · p. 4

Latent Representation

  • Benign and malicious queries show semantic divergence in activation representations, which the method uses to detect adversarial attacks.The paper reports that embeddings of malicious attacks differ significantly from those of benign requests.Source: Latent Representation · p. 4

SECMCP Agent Design

  • The SECMCP agent constructs client-specific access-control regions and assesses incoming representations in two stages: activation collection and unauthorized-access assessment.Inputs outside permitted boundaries are treated as potential malicious attacks.Source: SECMCP Agent Design · p. 4

Activation Collection

  • Legitimate queries provide anchor points that define a high-dimensional authorized region, and deviations are computed against those anchors.The method extracts the last-token activations across all layers; Act(q, l, θ) denotes an input’s activation vector at layer l under parameters θ.Source: Activation Collection · pp. 4–5

Risk Matching

  • Risk Matching embeds activation representations, compares each query with anchor points using squared Euclidean distance, and classifies by distance, computed separately for each layer.A larger distance indicates greater deviation from legitimate queries; the implementation uses a decision-tree classifier to assign query categories.Source: Risk Matching · p. 5

Experiment

SECMCP is evaluated on multiple models, datasets, and MCP attack settings, with comparisons to baseline defenses and ablations of activation deviations and anchor counts. The experiments report strong drift detection and examine robustness to synonym replacement.

MCP Setups

  • Experiments use Llama3-8B, Mistral7B, and Vicuna-7B with malicious servers embedding instructions in tool descriptions or retrieved content.The servers simulate tool poisoning and indirect prompt injection; attack methods also cover data exfiltration, misleading content, and hijacking.Source: MCP Setups · p. 5, Attack Method · p. 5, Setups · p. 5

Evaluation Metric

  • The evaluation spans FinQA, HotpotQA, and Ms Marco, and treats drift detection as binary classification measured by AUROC.AUROC measures area under the ROC curve formed by true-positive and false-positive rates.Source: Datasets · p. 5, Evaluation Metric · p. 5

Hyper-parameters

  • The default uses 1,000 anchor samples and top-5 retrieval, evaluating layers 0, 7, 15, 23, and 31 and reporting the best-performing result.The robustness test uses HotpotQA and replaces five randomly selected words in each original prompt with semantic alternatives.Source: Hyper-parameters · p. 5, Robustness · p. 6

Effectiveness

  • AUROC scores exceed 0.915 across the reported cases, average 0.98, and exceed 0.99 in several hijacking scenarios.Performance is comparatively lower on Ms Marco than on FinQA and HotpotQA, which the authors attribute to its broader topical diversity.Source: Effectiveness · p. 6, Table 1 · p. 6
  • SECMCP significantly outperforms the compared defenses; preventive methods have relatively low success rates, while known-answer detection misses a nonnegligible share of attacks.The comparison uses Sandwich Prevention, Instructional Prevention, and Known-Answer Detection across three LLMs.Source: Effectiveness · p. 6, Figure 3 · p. 6

Robustness

  • On HotpotQA, robustness is tested by replacing five randomly selected prompt words with semantically similar alternatives and comparing AUROC before and after perturbation.The supplied passages describe the test design but provide no perturbed-result values.Source: Robustness · p. 6, Table 2 · p. 6

Visualizations of the Activation Deviation

Number of Anchor Samples

  • Detection effectiveness generally increases as the anchor count rises from 200 to 2,000 in increments of 200.The study varies anchor counts using Llama3-8B across three datasets and attributes improved distinctions to more representative features.Source: Number of Anchor Samples · p. 7, Figure 5 · p. 7

Conclusion

SECMCP detects conversational drift in MCP agents using activation-vector deviations induced by malicious inputs. Experiments across datasets and risk types report high detection accuracy and robustness to adaptive threats.

Conclusion

  • SECMCP identifies conversational drift in MCP-powered agents through activation-vector deviations induced by malicious inputs.The authors state that this captures subtle semantic changes that traditional output-based or rule-based detectors often miss.Source: Conclusion · p. 7
  • Experiments across multiple datasets and risk types demonstrate high detection accuracy and robustness against adaptive threats.Source: Conclusion · p. 7
  • The method does not require prior knowledge of attack format, unlike approaches relying on predefined signatures or heuristics.Source: Conclusion · p. 7

Limitations and Future Work

The authors identify scope, attribution, and interpretability limits in SECMCP. Its stated setting assumes stable query-response structure, and its detection lacks token-level attribution and interpretability.

Limitations and Future Work

  • SECMCP assumes stable query-response structure and is not directly applicable to large-scale asynchronous multi-agent protocols such as A2A.In those protocols, conversation boundaries and speaker roles are fluid.Source: Limitations and Future Work · p. 7
  • The approach captures topic-level deviations but lacks token-level attribution, limiting use where fine-grained control is required.Source: Limitations and Future Work · p. 7
  • SECMCP's decision process lacks interpretability, limiting applicability in settings that require high transparency.Source: Limitations and Future Work · p. 7

1. General Paper Structure

The paper checklist marks the method's conceptual or pseudocode description, separation of opinion from objective results, and pedagogical replication references as present.

1. General Paper Structure

  • The checklist marks a conceptual outline or pseudocode description of the AI methods as present.Source: §1 · p. 9
  • The checklist marks opinions, hypotheses, and speculation as clearly delineated from objective facts and results.Source: §1 · p. 9
  • The checklist marks well-marked pedagogical references for replication background as present.Source: §1 · p. 9

2. Theoretical Contributions

The checklist reports no theoretical contributions; its questions about formal assumptions, claims, proofs, and related theoretical practices are marked not applicable.

2. Theoretical Contributions

  • The paper reports that it makes no theoretical contributions.The checklist marks the follow-up questions about assumptions, formal claims, proofs, intuitions, citations, empirical demonstrations, and code for theoretical claims as not applicable.Source: §2 · p. 9
  • The checklist marks theoretical-claim demonstrations and related experimental code as not applicable.These checklist items are listed separately from the paper's reported absence of theoretical contributions.Source: §2 · p. 9

3. Dataset Usage

The checklist indicates that the paper uses datasets and reports motivation for their selection. It also records dataset documentation, availability, and citation practices as affirmative.

3. Dataset Usage

  • The paper relies on datasets and reports motivation for conducting experiments on the selected datasets.The checklist also marks the data-appendix item for novel datasets as yes.Source: §3 · p. 9
  • The checklist indicates that novel datasets will be publicly available upon publication under a license allowing free research use.It marks citation and public availability of datasets drawn from existing literature as yes.Source: §3 · p. 9
  • The checklist marks detailed descriptions and explanations for any nonpublic datasets as provided.This item concerns datasets without public availability.Source: §3 · p. 9

4. Computational Experiments

The checklist reports computational experiments, with source-code and infrastructure documentation marked affirmative. It also flags gaps in reporting runs, distributional information, random seeds, and some parameter details.

4. Computational Experiments

  • The paper includes computational experiments and reports evaluation metrics with their motivation.The checklist marks code for preprocessing and conducting/analyzing experiments, public release, and implementation comments as yes.Source: §4 · p. 9
  • The checklist marks the random-seed description as no and the computing-infrastructure specification as yes.The infrastructure item covers hardware, software, memory, operating system, and relevant library/framework versions.Source: §4 · p. 9
  • The checklist marks reporting the number of runs and distributional performance information as no.The number-of-runs item asks how many algorithm runs were used for each reported result.Source: §4 · pp. 9–10
  • Significance testing is marked partial, while final hyperparameter reporting is also marked partial.The checklist separately marks the account of hyperparameter values tried and selection criteria as partial.Source: §4 · pp. 9–10

Related papers

Loading 2508.06418v1…