PaperlayerView exact revision on arXiv

Source-linked AI summary

Combating Instruction Conflict via Energy-Driven Latent Conflict Detection

Mingyu Ma, Yuxin Wu, Jingbo Wang, Tianxiao Huang, Leixin Sun, Xiaochuan Shi

arXiv:2609.08646v1cs.CL

TL;DR

LLMs can violate system constraints when conflicting user instructions cause Response Drift, while static input-side defenses may miss these final-response violations. ELCD detects drift after generation by modeling response hidden states with an energy-based composite representation and margin-ranking objective. Across five open-weight LLMs, it improves detection performance, including a roughly 30-point PR-AUC gain on Llama-2-7B and 2.67% FPR95 on Mistral-7B under IID conflicts.

  • Problem

    Conflicting user instructions can cause LLMs to violate system constraints, while static defenses may miss these response-level violations.

  • Method

    ELCD performs post-generation drift detection using energy-based modeling of composite response representations and pairwise margin ranking.

  • Results

    Across several open-source LLMs, ELCD improves Llama-2-7B PR-AUC by an average of 30 percentage points and reduces Mistral-7B FPR95 to 2.67% under IID conflicts.

  • Takeaways & Limitations

    The findings support post-generation verification as a complement to prompt-side defenses for reliable instruction-hierarchy monitoring.

  • Takeaways & Limitations

    ELCD requires target-model hidden states and is evaluated mainly on open-weight models with programmatically verifiable conflicts, limiting direct applicability to closed APIs and complex semantic policies.

Abstract

from arXiv · show

Large Language Models (LLMs) are increasingly deployed with hierarchical instructions, yet they remain vulnerable to conflicts in which user directives override system-level constraints. Existing defense mechanisms predominantly focus on static input inspection and therefore fail to detect Response Drift, a phenomenon in which the model's final response violates system-level constraints despite seemingly compliant inputs. To bridge this gap, we introduce ELCD, a response-level latent conflict detector for post-generation, pre-delivery verification. Given the full generated output, ELCD constructs a composite hidden-state representation by concatenating the final-token embedding with the mean-pooled response embedding. It then optimizes a pairwise margin ranking objective to separate compliant and drifting responses in latent space. Extensive experiments across five mainstream LLMs ranging from 1.5B to 14B parameters demonstrate that ELCD significantly outperforms competitive baselines. Notably, it improves the PR-AUC on Llama-2-7B by approximately 30 percentage points and reduces the False Positive Rate at 95% TPR (FPR95) on Mistral-7B to 2.67%. These results suggest that ELCD provides a promising approach for latent instruction-conflict detection in open-weight or self-hosted LLM deployments.

1 Introduction

Instruction conflicts can cause Response Drift: outputs may violate system constraints even when inputs appear compliant. ELCD addresses this gap by detecting latent response-level deviations after generation, with strong results across open-source models.

  • Problem: Static input-side defenses may miss Response Drift, where plausible responses gradually violate system-level requirements.The paper motivates checking completed outputs rather than relying only on prompt inspection.
  • Motivation: Energy-based modeling offers a measurable signal for detecting drift even when user instructions appear benign to static filters.The approach characterizes discrepancies between generated responses and distributions associated with the intended instruction hierarchy.
  • Approach: ELCD detects conflicts after generation by modeling response-level latent signals with an energy-based framework.The framework is designed for post-generation, pre-delivery verification.
  • Contribution: ELCD models Response Drift as an energy-space distributional divergence, targeting dynamic latent deviations that pattern-matching filters can miss.This reframes detection from static prompt inspection to output-level latent analysis.
  • Results: 30 percentage points: ELCD improves average PR-AUC on Llama-2-7B, while Mistral-7B reaches 2.67% FPR95 under IID conflicts and 16.59% under OOD conflicts.Experiments span open-source models ranging from 1.5B to 14B parameters.

2 Related Work

Prior defenses largely inspect prompts before generation, leaving a gap for violations that emerge during autoregressive response generation. ELCD addresses this gap by using internal hidden-state perturbations and energy-based detection.

  • Existing Defenses: Existing defenses include trained prompt-injection detectors, LLM self-evaluation, and heuristic or rule-based input methods.These approaches classify or inspect inputs before generation begins.
  • Research Gap: Static defenses can overlook Response Drift, in which user intent nudges generation from input compliance toward output violation.The paper identifies this transition as a critical safety gap.
  • ELCD: ELCD uses internal hidden-state perturbations to dynamically detect subtle Response Drift that static guardrails may miss.This shifts monitoring from prompt-side inspection toward response-generation dynamics.
  • Energy-Based Modeling: Energy-based models quantify compatibility with a predefined distribution and have been used to identify anomalous or out-of-distribution behavior.The paper connects this prior EBM role to monitoring LLM reliability.

3 Preliminary

The paper defines instruction-hierarchy compliance and Response Drift at the response level, then uses internal energy signals to separate their latent states. Energy distributions show distinct basins and a detection boundary even when surface text remains plausible.

  • Hierarchy: The model should generate a response satisfying user intent while adhering to system-level constraints.The hierarchy consists of system instruction Is and user instruction Iu.
  • Energy Representation: ELCD computes energy from a composite final-layer feature formed by concatenating the final-token state with the mean-pooled response representation.The resulting energy reflects alignment with system constraints or anomalous deviation.
  • Response States: Response Drift is the conjunction of an instruction conflict and a violation of system constraints.Compliance means the response satisfies system constraints regardless of user intent; drift occurs when conflicting user instructions prevail.
  • Detection Task: The detection task assigns a label y to each system instruction, user instruction, and generated response triplet using internal energy signals.The goal is to learn a scoring function S(Is, Iu, R).
  • Energy Landscape: Compliant responses occupy a low-energy basin concentrated between −6 and −3, whereas drift responses center between −2 and +2.These distributions indicate distinct latent states under the instruction hierarchy.
  • Detection Boundary: A spectral gap near E = −2.5 separates compliance from drift, providing a quantitative signal even when surface text appears plausible.The observed latent shift motivates contrastive optimization for response-level detection.

4 Methodology

ELCD detects Response Drift from completed responses by extracting response-aware hidden-state features, mapping them to energy scores, and separating compliant from drifting outputs with margin ranking. It operates after generation and before delivery, using a calibrated threshold for post-hoc verification.

  • Response-Level Detection: ELCD detects Response Drift from completed responses rather than relying only on input-prompt inspection.It treats compliant behavior as low-energy equilibrium and drift as departure from that equilibrium.
  • Composite Feature Extraction: Response-aware masking isolates the assistant response segment so prompt information does not dilute drift signals.The detector extracts hidden states from the response portion of the full system-user-response sequence.
  • Composite Feature Extraction: ELCD concatenates the final-token state with the mean-pooled response representation to capture terminal decisions and overall semantic trajectory.The final-token state captures late-generation shifts, while mean pooling improves stability against token-level noise and phrasing variation.
  • Energy Projection: The energy head maps the composite representation to energy scores, where lower energy indicates compatibility with system constraints and higher energy indicates response drift.A compact MLP with a progressive bottleneck performs the nonlinear projection.
  • Contrastive Optimization via Margin Ranking: Pairwise margin ranking pushes compliant samples toward lower energy and drifting samples toward higher energy.With m = 1.0 in the main experiments, drift samples are required to have energy at least 1.0 larger than compliant samples.
  • Post-Hoc Verification: During deployment, ELCD computes response energy after generation and labels outputs as drift when E(v; ϕ) > τ.The threshold is calibrated on held-out validation data, and inference adds a compact MLP head over hidden-state features.

5 Experiments

Experiments evaluate ELCD and complementary detectors across five open-source LLMs under IID and OOD instruction conflicts. ELCD achieves stronger detection, including substantially lower high-recall false-positive rates and robust performance under unseen conflict types.

  • Experimental Setup: Experiments evaluate detection across five open-source LLMs ranging from 1.5B to 14B parameters.The models include Llama-2-7B, Qwen2.5-1.5B, Qwen2.5-14B, Mistral-7B, and Phi-3-Mini.
  • Experimental Setup: The evaluation covers IID conflicts with shared conflict types across splits and OOD conflicts using held-out IHEval conflict types.IID data use 70%/10%/20% train, validation, and test partitions, while OOD evaluation uses disjoint conflict types.
  • Main Results: ELCD outperforms all baselines on PR-AUC and FPR95 across five model backbones.The comparison includes prompt-only, model-judgment, known-answer, and attention-based detectors.
  • Main Results: 30 percentage points: ELCD improves PR-AUC on Llama-2-7B under IID user–system conflicts.The result is reported for the IID conflict setting.
  • Main Results: 2.67%: ELCD reduces Mistral-7B FPR95 under IID conflicts, compared with 90.54% for the second-best result.Lower FPR95 indicates fewer false positives at 95% true-positive rate.
  • OOD Results: 16.59%: ELCD reduces Mistral-7B FPR95 under OOD conflicts, a 57.53-point reduction relative to Prompt-Guard.Baseline performance degrades under OOD conflicts, whereas ELCD maintains stable performance across backbones.
  • Ablation Studies: A margin of 1.0 provides the best trade-off, while larger or smaller margins produce worse detection behavior.On Llama2-7B, increasing the margin from 1.0 to 1.5 raises FPR95 from 15.70% to 25.70%.
  • Ablation Studies: 6.45%: concatenating mean-pooled and last-token features reduces Qwen2.5-1.5B FPR95, versus 27.21%–34.36% for either feature alone.The two feature types capture complementary global semantic and local generative signals.

6 Conclusion

The conclusion defines Response Drift as deviation from system constraints caused by conflicting user instructions and presents ELCD as an energy-based latent-dynamics detector. Across five LLM backbones, ELCD shows strong accuracy and generalization while reducing false-positive rates to practical levels.

  • Conclusion: ELCD detects Response Drift by analyzing internal latent dynamics through a composite representation of instantaneous intent and semantic trajectory.The framework is proposed for post-generation verification alongside prompt-side defenses.
  • Conclusion: Evaluations across five LLM backbones demonstrate strong accuracy and generalization for ELCD.The conclusion reports reduced false-positive rates and highlights internal-state monitoring for reliable AI systems.

Limitations

ELCD's scope is constrained by hidden-state access, programmatically verifiable conflict settings, and calibration tied to particular models and deployment conditions.

  • Scope and Deployment: ELCD requires hidden states, limiting direct applicability to closed-source API-only systems.The experiments focus on open-weight instruction-following models.
  • Scope and Deployment: The evaluated conflicts may not cover complex semantic policies, subjective safety requirements, or multi-turn interactions.The study uses programmatically verifiable constraints such as formatting, language, length, and structural requirements.
  • Scope and Deployment: Deployment with substantially different models, decoding settings, or task distributions may require detector recalibration.The detector is trained and calibrated on specific model backbones.

Ethical Considerations

ELCD is intended to reduce non-compliant outputs before delivery, but its learned detection may produce both false positives and false negatives. Deployment thresholds should therefore be calibrated to application risk.

  • ELCD is designed as a complementary post-generation verification module for reducing instruction-hierarchy violations before response delivery.It can be integrated with prompt-side defenses, policy checkers, or human review in high-risk settings.
  • Learned detection may falsely flag benign responses or miss constraint violations.
  • Deployment thresholds should be calibrated according to the application’s risk level.

A Dataset Details

The dataset covers programmatically verifiable instruction conflicts across multiple task domains, while the related-work analysis organizes detectors by the signals they use to identify violations.

  • Dataset Construction: Eight constraint families define in-domain conflicts using direct and permutation settings with balanced normal and conflict examples.Each setting contains 200 instances: 100 normal and 100 conflict examples.
  • Task Coverage: The task pool spans general knowledge, science, history, technology, procedural writing, literature, safety advice, and creative writing.Representative tasks are listed in Table 5, with conflict rows pairing system and user instructions.
  • Conflict Labeling: A formatting example labels the instance as conflict because plain-text user instructions violate the system’s JSON output constraint.Response drift is assigned only when the final response fails to satisfy the governing system constraint.
  • Compared Detectors: Input-side detectors inspect prompts before generation but cannot observe whether the final response follows the system-level instruction.LLM-based, known-answer, and attention-based detectors instead rely on model judgment, predefined answers, or internal attention patterns.

C Complexity Analysis

ELCD adds post-generation verification by pooling response hidden states and applying a compact energy head, with overhead dominated by response pooling when hidden states are available.

  • Detector Overhead: ELCD’s detector-side computation consists of response hidden-state mean pooling followed by projection through a lightweight energy head.The response length is Tr and the target model hidden dimension is d.
  • Detector Overhead: With fixed MLP widths, the additional computation is response-segment pooling plus a small feed-forward projection.The architecture is described as a compact detector-side component rather than additional autoregressive generation.
  • Deployment Cost: When hidden states are unavailable, ELCD may require a separate verification forward pass over the completed conversation.In that setting, total verification cost includes one target-LLM forward pass, denoted CM(T).
  • Deployment Cost: ELCD avoids additional autoregressive generation, external judge calls, and task-specific target-answer scoring.The paper characterizes the resulting overhead as acceptable for open-weight or self-hosted systems with hidden-state access.

D.1 Hardware & Training Configuration

The experiments use frozen open-weight backbones with a lightweight trained energy head, calibrated thresholds, and a post-generation deployment procedure based on response hidden states.

  • Hardware and Backbone Usage: Experiments use NVIDIA GeForce RTX 3090 GPUs, an Intel Xeon Gold 6240 CPU, PyTorch 2.0.1, and CUDA 11.7.ELCD does not update target-model parameters; the target model serves as a frozen feature extractor.
  • Energy Head: The energy head uses a 2d →256 → 128 →64 →2 MLP with dropout rate 0.2 and ReLU activations.Here d denotes the hidden size of the target language model.
  • Training Configuration: The energy score is defined so that larger energy indicates a higher likelihood of response drift.The head is trained with Adam for 20 epochs using learning rate 1 × 10^-4, batch size 64, and a 70/10/20 train-validation-test split.
  • Threshold Calibration: The decision threshold is selected on validation data, usually by maximizing F1, and then fixed for held-out test and OOD evaluation.High-recall and normal-score quantile calibration are also supported.
  • Deployment Procedure: Deployment generates a response, extracts response hidden states from the completed conversation, scores the composite feature, and flags responses exceeding the calibrated threshold.The response boundary is located through longest-prefix matching to support different chat templates.
Loading 2609.08646v1…