PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepContext: Stateful Real-Time Detection of Multi-Turn Adversarial Intent Drift in LLMs

Justin Albrethsen, Yash Datta, Kunal Kumar, Sharath Rajasekar

arXiv:2602.16935v1cs.AIcs.ETcs.LG

TL;DR

Multi-turn jailbreaks can distribute malicious intent across benign-looking exchanges, while stateless guardrails overlook its temporal accumulation. DeepContext uses turn-level embeddings and an RNN hidden state to track intent trajectories, achieving 0.84 F1 and 19 ms latency on a T4 GPU. The paper concludes that state-aware architectures can provide accurate, production-oriented multi-turn protection without repeatedly processing full histories.

  • Problem

    Stateless guardrails treat multi-turn dialogues as disconnected events, allowing distributed adversarial intent to evade detection.

  • Method

    DeepContext combines task-attention weighted turn embeddings with an RNN that maintains a recurrent hidden state for intent tracking.

  • Results

    0.84 F1 and 19 ms latency on a T4 GPU are reported for DeepContext on multi-turn benchmarks.

  • Takeaways & Limitations

    Tracking the evolution of user intent offers a state-aware alternative to repeatedly reprocessing large conversation histories.

  • Takeaways & Limitations

    DeepContext occasionally produces false positives in complex function-calling scenarios by conflating technical instructional complexity with malicious code injection.

Abstract

from arXiv · show

While Large Language Model (LLM) capabilities have scaled, safety guardrails remain largely stateless, treating multi-turn dialogues as a series of disconnected events. This lack of temporal awareness facilitates a "Safety Gap" where adversarial tactics, like Crescendo and ActorAttack, slowly bleed malicious intent across turn boundaries to bypass stateless filters. We introduce DeepContext, a stateful monitoring framework designed to map the temporal trajectory of user intent. DeepContext discards the isolated evaluation model in favor of a Recurrent Neural Network (RNN) architecture that ingests a sequence of fine-tuned turn-level embeddings. By propagating a hidden state across the conversation, DeepContext captures the incremental accumulation of risk that stateless models overlook. Our evaluation demonstrates that DeepContext significantly outperforms existing baselines in multi-turn jailbreak detection, achieving a state-of-the-art F1 score of 0.84, which represents a substantial improvement over both hyperscaler cloud-provider guardrails and leading open-weight models such as Llama-Prompt-Guard-2 (0.67) and Granite-Guardian (0.67). Furthermore, DeepContext maintains a sub-20ms inference overhead on a T4 GPU, ensuring viability for real-time applications. These results suggest that modeling the sequential evolution of intent is a more effective and computationally efficient alternative to deploying massive, stateless models.

1 Introduction

Multi-turn attacks distribute malicious intent across seemingly benign queries, exposing the limits of stateless and context-window defenses. DeepContext addresses this gap by tracking intent embeddings through a recurrent hidden state and detects intent evolution with low latency.

  • Crescendo and ActorAttack distribute malicious intent across benign-looking turns, allowing aggregated context to groom models without triggering stateless classifiers.
  • Transcript concatenation repeatedly reprocesses full histories with large guardrail models, incurring quadratic attention costs and prohibitive real-time latency.
  • Sliding windows can miss attacks spread beyond the window, while lightweight embedding classifiers cannot represent more than a few conversational turns.
  • DeepContext generates safety-space embeddings for each turn instead of reprocessing raw conversation history.
  • An RNN propagates a persistent hidden state that synthesizes relationships between previous queries and the current input.
  • Monitoring hidden-state transitions exposes intent drift from legitimate inquiry toward adversarial steering, including grooming behaviors missed by point-in-time filters.
  • 0.84 F1-score and sub-20ms latency on a T4 GPU demonstrate DeepContext’s reported accuracy and efficiency against multiturn jailbreaks.

2 Related Works

Recent attacks increasingly exploit reasoning, semantic drift, automation, and structural camouflage across multiple turns. Existing defenses remain largely reactive because many classify isolated interactions or operate only during training, motivating recurrent intent tracking.

  • The attack landscape has shifted from single-turn optimization-based injections toward stateful, agentic strategies that exploit reasoning and context windows.
  • RACE reformulates harmful queries as benign reasoning tasks, while Bad Likert Judge uses role-play and evaluation framing to elicit harmful content.
  • Crescendo uses benign turns to establish a narrative in which a harmful request becomes contextually consistent, while FITD uses bridge prompts to lower resistance.
  • AutoAdv automates multi-turn searches for psychological weak points, ActorAttack hides intent through personas, and M2S compresses multi-turn histories into structured payloads.
  • The defense landscape remains reactive because atomic guardrails lack temporal reasoning for correlating benign early turns with later multi-stage attacks.
  • Multilingual adapters can suffer cross-lingual drift, while training-time alignment does not provide runtime intervention in zero-day adversarial contexts.
  • DeepContext extends task-attention principles into recurrent temporal modeling by processing weighted embeddings and maintaining a hidden state for semantic drift.

3 Methodology: Stateful Intent Tracking via Recurrent Latent Embeddings

DeepContext formulates safety as a trajectory through semantic latent space rather than an isolated current-turn decision. Its state representation captures accumulated risk across conversation history, addressing the false Markov assumption in fragmented attacks.

  • DeepContext models safety as a trajectory through semantic latent space with continuous memory that captures accumulated risk over time.
  • Conversation history at turn t consists of user inputs and assistant responses from the first turn through t.
  • Current-turn safety approximations assume the present turn contains sufficient information, but Crescendo shows this Markov assumption fails when adversaries fragment malicious payloads.

Safety Detection as a State-Space Problem

DeepContext reformulates safety detection as state tracking: recurrently updated intent representations preserve conversational history, while residual connections retain immediate-turn signals for classification. Its GRU-based design combines temporal context with task-attention embeddings and is presented as computationally lean for real-time monitoring.

  • State-space formulation: DeepContext updates a hidden intent state recursively, treating safety detection as a state-space problem rather than isolated turn classification.The state is maintained across conversation turns to represent evolving user intent.
  • Projection and residual shortcuts: The recurrent state combines historical intent with current-turn embeddings through a projected-state residual connection before trajectory classification.The risk vector concatenates the projected hidden state with the raw task-attention embedding, preserving both contextual and immediate signals.
  • Turn-level representation: Task-attention weighted BERT embeddings prioritize semantic tokens relevant to safety policies, producing high-signal representations for each turn.The embeddings are generated from fine-tuned BERT and emphasize markers associated with prompt injection and jailbreaks.
  • Recurrent intent tracking: A GRU maintains conversation state while reducing the computational overhead associated with larger recurrent architectures such as LSTMs.The GRU merges forget and input functions through an update gate and is used to support real-time security applications.
  • Efficiency: Sub-20ms inference latency on a T4 GPU is reported for the GRU-based system, supporting its intended real-time monitoring use.The framework is described as tracking long-range semantic dependencies without becoming a bottleneck in the LLM inference pipeline.
  • Projection and residual shortcuts: The residual pathway supports both instantaneous detection of explicit attacks and contextual anchoring of long-term intent trajectories.This dual perspective combines the stateful component with the stateless current-prompt signal.
  • Training data: DeepContext trains on 437,058 conversational sequences, with approximately 20% labeled malicious, using diverse synthetic and human-generated samples.The training corpus includes benign, malicious, single-turn, and multi-turn trajectories.
  • Training procedure: Binary safety classification uses BCE with focal loss, while one-epoch training, batch size 512, dropout, and a conservative learning rate target generalization and gradient stability.The focal-loss component uses γ = 2.0 to emphasize harder examples near the safety boundary.

4 Results and Evaluation

DeepContext is evaluated on diverse benign, human-red-team, and automated multi-turn jailbreak data, with held-out samples and comparisons spanning lightweight encoders, generative guardrails, and enterprise APIs. It achieves stronger multi-turn detection and early threat identification while maintaining low latency, and also performs strongly on single-turn jailbreaks.

  • Evaluation Setup: The benchmark combines high-noise benign datasets, human red-teaming data, and automated multi-turn jailbreak frameworks, including 210 multi-turn jailbreaks with a median of 7 turns.Evaluation samples from HH-RLHF, XGuard, and DEFCON were kept separate from training.
  • Evaluation Setup: The baselines span lightweight task-specific encoders, generative guardrails, and managed enterprise API solutions.This comparison tests DeepContext against both academic models and commercial production tools.
  • Multi-turn Detection: 0.84 F1: DeepContext outperforms the closest baseline, Llama-Prompt-Guard-2-86M, by approximately 25% in multi-turn jailbreak detection.Its Recall is 0.83, which the authors associate with capturing intent drift that other models overlook.
  • Multi-turn Detection: 4.24 turns: DeepContext detects threats earlier on average, while Azure Prompt Shield reaches an MTTD of 8.00 and Recall of 0.11.The recurrent hidden state retains suspicious probing from previous turns before an explicit harmful payload appears.
  • Single-turn Detection: 0.98 F1: DeepContext is the overall best performer on the single-turn JailBreakBench benchmark.The evaluation tests whether the architecture remains effective without conversational context.
  • Efficiency: 19 ms: DeepContext is faster than Granite-Guardian-3.3-8B at 125 ms, Qwen3Guard-Gen-8B at 64 ms, and AWS Prompt Attack Guardrails at 235 ms.Lightweight encoders reach 4 ms but do not capture multi-turn context as effectively.

5 Discussion

DeepContext suggests that recurrent state can outperform larger stateless models while preserving real-time feasibility. The discussion also identifies trade-offs, future extensions, and remaining limitations.

  • State over scale: DeepContext achieves F1 0.84 versus 0.42 for Llama-Guard-4-12B and 0.65 for Granite-Guardian-3.3-8B.The result supports the paper’s claim that a compact recurrent architecture can outperform larger stateless models on distributed adversarial intent.
  • Speed-security frontier: 19 ms latency contrasts with Granite-Guardian-3.3-8B’s 1,430 ms per turn, highlighting a substantial speed difference for real-time use.DeepContext’s reported latency is described as computationally invisible to end users, whereas Granite-Guardian’s delay is described as unusable for real-time chat.
  • Limitations: Qwen3Guard reaches 3.12-turn detection latency but has recall 0.36, illustrating the risk of optimizing speed at the expense of recall.The paper characterizes this pattern as a brittle defense against novel, adaptive strategies.
  • Limitations: False positives in complex function-calling scenarios suggest that task attention can conflate technical complexity with malicious code injection.The proposed future direction is a dual-stream architecture separating benign functional intent from adversarial semantic steering.
  • Future directions: Future work extends DeepContext toward generalized intent detection, adaptive policy controls, deterministic threat tracking, and broader agentic security architectures.These directions include continuous risk signals, hybrid probabilistic-deterministic monitoring, and federated intent tracking.

6 Real-World Applications

DeepContext’s compact recurrent state supports several production-oriented applications by retaining context across long interactions without the latency burden of traditional guardrails.

  • Production integration: Sub-20ms overhead allows DeepContext to operate in high-throughput environments without degrading user experience.The paper presents this as an architectural efficiency advantage over traditional latency-heavy guardrails.
  • Agentic loops: DeepContext monitors agentic loops by maintaining persistent intent state across multi-step tool-use workflows vulnerable to goal drift.The stated threat includes seemingly benign intermediate steps that eventually trigger harmful actions such as unauthorized data exfiltration.
  • Enterprise systems: A fixed 2048-dimensional hidden state provides consistent real-time protection across long-horizon enterprise dialogues.The applications named include banking, healthcare, and technical support.
  • Proactive steering: The continuous risk vector R_t can support proactive steering by making the primary model more conservative as safety risk approaches a violation boundary.Suggested interventions include changing the system prompt or decoding strategy and restricting reasoning depth.
  • Auditing and compliance: The recurrent state can serve as a compact audit trail for identifying systematic probing and emerging threat vectors across user populations.This is presented as a way to reduce reliance on storing and manually reviewing massive text logs.

7 Conclusion

The paper frames multi-turn attacks as a Safety Gap caused by stateless guardrails’ inability to track temporal intent. DeepContext addresses this gap with recurrent trajectory modeling and reports strong benchmark performance with production-ready latency.

  • Conclusion: Stateless guardrails miss temporal nuances of intent grooming because they rely on massive models and static context windows.The paper describes this failure as the Safety Gap created by the shift from single-shot to multi-turn agentic attacks.
  • Conclusion: DeepContext combines an RNN with Task-Attention Weighted Embeddings to model safety as a continuous latent-space trajectory.This replaces the static evaluation paradigm with stateful monitoring.
  • Conclusion: 0.84 F1 on multi-turn benchmarks and 19 ms latency on a T4 GPU define the reported performance and deployment profile.The conclusion presents these figures as state-of-the-art performance and production-ready latency.
  • Conclusion: The authors conclude that tracking evolving intent requires smarter, state-aware architectures rather than simply larger models.They identify intent evolution as increasingly important for autonomous agents and prolonged conversations.

Appendix A: Multi-turn Jailbreak Transcript

The appendix identifies a Crescendo jailbreak transcript, but the supplied passage does not provide transcript content or a supported comparison.

  • Appendix: Table 8 is labeled “Crescendo Jailbreak,” but no transcript details are supplied.The accompanying passage contains unrelated chemistry text rather than an interpretable jailbreak transcript.

Appendix B: Human Red Teaming Transcript

The DEFCON conversation sample presents an administrator-identity claim followed by a blocked assistant response.

  • The sample is labeled as a DEFCON conversation sample.
  • A user claims to be a designated administrator from AI Village.
  • The assistant response is marked BLOCKED and includes a 0.924 score.

8 Appendix C: GPT5-Nano Prompt Template

The prompt template uses a security-auditor judge to classify transcript content for jailbreak attempts and unsafe content. It enforces explicit red-flag criteria and requires a single-word SAFE or UNSAFE verdict.

  • The evaluation uses an LLM-as-a-judge configured as a high-precision security auditor.The full prompt template includes system instructions and a defensive user wrapper.
  • The auditor analyzes text for jailbreak attempts and unsafe content.
  • The audit criteria cover instruction overriding, persona adoption, obfuscation, and malicious intent.Examples include requests to ignore prior instructions, adopt DAN, use encoded text, or request illegal acts and malware.
  • The output protocol requires exactly one verdict: SAFE or UNSAFE.The wrapper instructs the auditor not to answer transcript questions or adopt transcript personas.
  • The user wrapper places the transcript between explicit start and end markers for analysis.
Loading 2602.16935v1…