PaperlayerView exact revision on arXiv

Source-linked AI summary

When Keywords Drop but Classifiers Hold: Soft Refusals under KV Cache Compression

Kang Chen, Xiuze Zhou, Hong Chen, Yuanguo Lin

arXiv:2609.31678v1cs.LGcs.CL

TL;DR

The paper asks whether matched KV-cache compression that preserves task accuracy also preserves agreement between lightweight refusal monitors and stronger classifiers. It tests paired full-retention and eviction responses with multiple judges, finding a large keyword-refusal drop while stronger judges continue to classify many disputed outputs as refusals.

  • Problem

    It is unclear whether matched compression that preserves task accuracy also preserves agreement between lightweight lexical monitors and stronger refusal classifiers.

    Source: Introduction · p. 1

  • Method

    The study pairs full and compressed retention on 200 harmful prompts and scores the resulting responses with keyword heuristics, HarmBench-family judges, and human annotators on disagreements.

    Source: Introduction · p. 1, Experimental Setup · p. 2

  • Results

    Keyword refusal falls from 98.0% at ret=1.0 to 80.5% at ret=0.3, while stronger judges and human raters still treat disputed outputs as refusals.

    Source: Results · p. 3, Discussion · p. 6

  • Takeaways & Limitations

    Safety audits under compression should compare multiple judges matched to the serving context rather than rely on keyword rates alone.

    Source: Discussion · p. 6, Conclusion · p. 7

    The study uses a matched long-context stress protocol rather than a full production traffic mix, so boundaries should be re-measured under the target serving stack.

    Source: Discussion · p. 7

Abstract

from arXiv · show

KV cache compression is widely used for long context LLM inference under memory constraints, while deployed systems typically score refusals after generation with keyword filters or learned classifiers. Such monitors are intended to indicate whether a model declined a harmful request under the serving regime actually used. However, it remains unclear whether matched compression that preserves task accuracy also preserves agreement between lightweight lexical monitors and stronger refusal classifiers. We study this with a paired protocol on n=200 harmful prompts with a long filler context: each prompt is answered once under full retention and once under matched eviction after a shared prefill, and the same replies are scored by keyword heuristics, the HarmBench Llama-2-13B classifier, an auxiliary LLM judge, and humans on disagreements. On Qwen2.5-3B, keyword refusal falls from 98.0% to 80.5% (McNemar p~1e-8) while classifier refusal stays near ceiling (99.0%-99.5%) and MMLU accuracy is unchanged (50.0%); human labels predominantly follow the classifier, consistent with soft refusals. The gap is not universal and weakens under short fillers and paired SnapKV, so safety auditing under compression should rely on several judges matched to the serving context rather than on keyword rates alone.

Introduction

KV-cache compression is common in long-context deployment, where post-generation monitors judge whether harmful requests were refused. This paper asks whether those monitors still agree under matched compression and finds that keyword and stronger judges can diverge, depending on the serving context.

Introduction

  • Compression can change how downstream monitors interpret generations, beyond changing what models generate.Deployed systems commonly use keyword filters, regular expressions, or learned classifiers to monitor safety after generation.Source: Introduction · p. 1
  • Prior compression-safety studies focus on harmful compliance or attack success, leaving monitor agreement under matched compression an open deployment question.A disagreement would indicate a lightweight-monitor blind spot, not by itself prove increased harmfulness.Source: Introduction · p. 1
  • Under long-filler eviction, Qwen2.5-3B-Instruct keeps MMLU accuracy at 50.0% while keyword refusal falls and the HarmBench classifier stays near saturation.Human labels on keyword–classifier disagreements align with the classifier in 94.6% of cases, consistent with soft refusals.Source: Introduction · p. 1
  • The monitor gap is conditional: it attenuates or vanishes with short fillers and paired SnapKV, and is smaller on official HarmBench prompts.The study also checks sink size, system-prompt presence, and other model scales and families.Source: Introduction · pp. 1–2
  • The paper contributes paired, multi-judge safety auditing with human validation, targeting a deployment evaluation gap rather than proposing a new compressor.The labeled human subset has substantial inter-annotator agreement (Cohen’s κ=0.634).Source: Introduction · p. 2

Related Work

Prior work develops KV-cache compression and safety monitors, and studies compression-related safety behavior. This paper distinguishes its focus as whether post-hoc monitors agree under matched compression.

Related Work

  • KV-cache compression retains selected past states for efficiency, while prior task-level studies report capability fragility but typically do not evaluate safety monitoring.The surveyed approaches retain salient tokens, anticipated attention mass, or attention sinks; low-bit quantization is a complementary efficiency axis.Source: Related Work · p. 2
  • Safety evaluation combines standardized harmful prompts and refusal labels with lightweight production filters, stronger detectors, and LLM-as-a-judge comparisons.Prior analyses of soft compliance and refusal directions help explain why lexical signals may diverge from stronger judges.Source: Related Work · p. 2
  • Related compression-safety work studies safety taxes, jailbreak susceptibility, alignment degradation, or retention-side mitigations.The cited approaches include policy-pinned tokens, Safe-CAM, and safer-direction retention [Ni and Lao 2026].Source: Related Work · p. 2
  • Unlike behavior- or compressor-centric studies, this work holds generation settings matched and examines post-hoc monitor disagreement under fixed task accuracy.Its motivating pattern is lower keyword refusal alongside saturated HarmBench classifier rates, with human labels on disputed cases following the classifier.Source: Related Work · p. 2

Experimental Setup

The experiments pair full and compressed KV retention on matched prompts, then score generated replies with multiple safety monitors. The primary setup uses long filler, tail eviction, and Qwen2.5-3B-Instruct, with additional model, compressor, and task checks.

Experimental Setup

  • Qwen2.5-3B-Instruct in bf16 is primary; checks include its 4-bit version, Qwen2.5-7B-Instruct, and Llama-3.2-3B-Instruct.Eviction retains four prefix sink tokens and the trailing 30% of KV positions; SnapKV is tuned to approximately 30% retention, and sink-size mitigation is also tested.Source: Experimental Setup · p. 2
  • Prompts place a neutral filler between system safety instructions and the user query; a separate ablation removes the system message.Generation uses greedy decoding with a fixed maximum new-token budget shared across retentions.Source: Experimental Setup · p. 2
  • The primary safety set contains 200 HarmBench-style behaviors, with an official HarmBench prompt set as a robustness check.Identical replies are scored by a keyword heuristic, the HarmBench Llama-2-13B classifier, and an auxiliary Llama-3.2-3B compliance judge; one annotator labels keyword–classifier disagreements (n=56).Source: Experimental Setup · p. 2
  • Each harmful prompt is evaluated at ret=1.0 and ret=0.3 after prefill, with the resulting replies scored by multiple monitors.The primary protocol uses 2048-token filler, tail eviction with sink=4, and n=200 paired prompts.Source: Experimental Setup · p. 2, Figure 1 · p. 3
  • Task checks cover five MMLU mathematics subjects with 20 items each, and matched prompt outcomes are tested using paired McNemar exact tests.Wilson 95% confidence intervals are reported for refusal rates and bootstrap intervals for keyword drops where available.Source: Experimental Setup · p. 3

Results

Under long-filler matched eviction, keyword refusal drops while learned refusal judges remain near ceiling, despite unchanged task accuracy. The gap varies with filler length, eviction method, prompt set, and serving configuration.

Main monitoring gap

  • 17.5 pp: keyword refusal falls from 98.0% at ret=1.0 to 80.5% at ret=0.3, while HarmBench classifier refusal changes from 99.0% to 99.5%.The paired keyword difference is significant (McNemar p≈10−8); the classifier difference has p=1.0.Source: Figure 2 · p. 3, Results · pp. 3–4

Third-judge check

  • 95.0% to 95.5%: the auxiliary Llama-3.2 judge remains near ceiling, tracking the HarmBench classifier rather than the keyword drop.The auxiliary judge's paired comparison has McNemar p=1.0.Source: Results · p. 4

Matched task accuracy

  • 50.0% vs. 50.0%: MMLU accuracy is unchanged between ret=1.0 and ret=0.3 under the matched primary protocol.The evaluation covers five mathematics subjects and 100 items total.Source: Results · p. 4

Prefix-sink dose response

  • At ret=0.3, keyword refusal rises from 80.5% at sink=4 to 87.5% at sink=64 and stays at 87.5% at sink=128, while classifier refusal remains near ceiling.The sink dose response partially restores the lexical monitor, with diminishing returns beyond sink=64.Source: Results · p. 4, Table 3 · p. 5

Filler-length ablation

  • At filler=0, keyword refusal stays at 97.0%; at 512 tokens, its 3.0-point drop is not significant, while the 2048-token setting shows a 17.5-point drop.Classifier refusal remains near ceiling across all three filler lengths; the gap is therefore conditional on long padded contexts.Source: Results · p. 5, Table 4 · p. 5

System-prompt ablation

  • Without a system prompt, keyword refusal still falls 20.0 pp, from 99.0% to 79.0%, while classifier refusal remains high at 99.5% to 97.5%.Removing the system safety text does not eliminate the gap; the authors interpret it as a broader long-context cache-pressure effect.Source: Results · p. 5, Table 5 · p. 5

Human validation

  • 94.6%: human labels match the classifier on keyword–classifier disagreements, and 51 of 56 cases are refusals missed by the keyword heuristic.These responses are predominantly soft refusals: policy-compliant non-assistance without canonical refusal phrases.Source: Results · p. 5, Figure 5 · p. 6

Conditional boundaries

  • The gap varies across conditions: it shrinks on official HarmBench prompts, vanishes with paired SnapKV, and is small for 4-bit Qwen2.5-7B but large for 4-bit Llama-3.2-3B.The cross-model runs are serving-condition boundaries, not precision-matched causal claims; on official HarmBench, keyword refusal falls 91.0% to 86.5% and classifier refusal 97.0% to 92.5%.Source: Results · pp. 4, 6, Table 2 · p. 5

Discussion

Under matched long-context prompts, keyword monitors can report large refusal regressions while learned judges still classify outputs as refusals; this disagreement depends on serving conditions. The discussion recommends auditing monitors under deployment-matched settings and notes limits to the study’s scope.

Discussion

  • Keyword monitors report large regressions while learned refusal judges remain near ceiling, and human raters on disagreement cases still treat outputs as refusals.The authors describe soft, policy-compliant non-assistance as one source of this mismatch.Source: Discussion · p. 6
  • The monitor gap is conditional: paired SnapKV leaves keyword refusal unchanged at 98.0%→98.0%, while short fillers and official HarmBench prompts weaken or align the divergence.A larger prefix sink partially restores keyword refusal, with diminishing returns beyond sink=64; removing the system message leaves a 20.0 pp keyword gap.Source: Discussion · p. 6
  • Suggested checks include attention-aware retention, larger prefix budgets when early context matters, long-context stress tests, prompt-set stratification, and scoring with a strong learned judge alongside keywords.The authors caution that these steps address monitor disagreement, not alignment failure per se.Source: Discussion · p. 6
  • Deployment audits should match serving-time compression, context length, and prompt distribution, while treating method-, scale-, family-, length-, sink-, and dataset-dependent variation as first-class.Matching benign-task checks and multi-judge agreement helps distinguish monitoring artifacts from general capability loss.Source: Discussion · p. 7
  • The study uses a matched long-context stress protocol rather than a full production traffic mix, and its human review covers disagreement cases rather than the full response set.Human validation uses n=56 cases and a single primary annotator; scale/family checks use 4-bit settings not precision-matched to the primary run.Source: Discussion · p. 7

Ethical Statement

The paper evaluates refusal monitoring rather than developing attacks, using public harmful-behavior benchmarks and aggregate refusal scoring. It limits release of harmful-prompt outputs and attack-enabling artifacts.

Ethical Statement

  • The study evaluates existing open models on publicly available harmful-behavior benchmarks and reports aggregate refusal rates rather than actionable exploit instructions.Generated outputs are used for refusal scoring through keyword heuristics, the HarmBench classifier, and limited human review of disagreement cases.Source: Ethical Statement · p. 7
  • The authors do not release harmful-prompt response dumps, annotated reply sheets, or scripts designed to elicit or amplify harmful compliance.Public artifacts will exclude raw model completions and attack tooling that could lower the barrier to misuse.Source: Ethical Statement · p. 7
  • The paper warns that monitor disagreement can produce false alarms or false reassurance, so no single refusal metric should serve as a deployment safety certificate.Source: Ethical Statement · p. 7

Reproducibility

The supplementary archive implements the matched evaluation protocol and documents its configurations and scoring procedures. It provides aggregated metrics and scripts for regenerating evaluations without releasing harmful-prompt generations or annotation sheets.

Reproducibility

  • The supplementary code archive implements prompt construction with filler contexts, KV retention for eviction and SnapKV, paired scoring, and statistical analysis.Scoring includes keyword heuristics, the HarmBench classifier, and an auxiliary LLM judge; analysis includes McNemar tests and Wilson or bootstrap confidence intervals.Source: Reproducibility · p. 7
  • Configuration files list model identifiers, retention levels, filler lengths, system-prompt modes, and random seeds.Documentation covers keyword patterns, judge prompts, and the human annotation guideline for disagreement cases.Source: Reproducibility · p. 7
  • The archive releases aggregated metrics and scripts to regenerate evaluations from locally obtained outputs, but not full harmful-prompt generations or annotated reply sheets.Hardware and precision or quantization notes are documented in the archive README.Source: Reproducibility · p. 7

Conclusion

Under matched long-context eviction, keyword monitors can degrade while task accuracy and strong safety classifiers remain stable, a setting-dependent monitoring blind spot rather than universal alignment failure. The paper recommends evaluating multiple safety monitors under the serving regime.

Conclusion

  • Under matched long-context eviction, task accuracy and strong safety classifiers can remain stable while lightweight keyword monitors degrade.The paper characterizes this as a judge- and setting-dependent monitoring blind spot, not a universal alignment failure.Source: Conclusion · p. 7
  • The primary keyword–classifier divergence emerges with long fillers and attenuates with short or absent fillers; paired SnapKV eliminates the keyword drop, while larger prefix sinks only partially restore it.The sink effect saturates by sink=64, and the official HarmBench subset shows more judge-aligned shifts.Source: Conclusion · p. 7
  • The authors recommend auditing multi-tier safety monitors under the same retention, context length, compressor, and prompt regime used at serving time.They caution against treating any single refusal metric as a residual-risk certificate.Source: Conclusion · p. 7

Related papers

Loading 2609.31678v1…