PaperlayerView exact revision on arXiv

Source-linked AI summary

Overflip: Repetition-Induced Label Flips in Guardrail Models

Xu He, Chih-Hsuan Lin, Hung-Mao Chen, Junjie Xiong, Yan Zhai, Kun Sun

arXiv:2609.15013v1cs.AI

TL;DR

Prior guardrail evaluations assume decisions remain stable as input length increases. This paper studies repetition-induced Overflip, in which lengthening can flip a malicious prompt to benign while retaining its intent. Across nine guardrails, five flipped on a 100-prompt benchmark, with vulnerable models showing 8%–92% flip rates.

  • Problem

    Prior evaluations assume lightweight guardrails’ decisions remain stable as input length increases, leaving this stability assumption untested.

    Source: §1 · p. 2

  • Method

    The paper examines Overflip, repeatedly lengthening malicious prompts while preserving their intent to test whether guardrail predictions flip from malicious to benign.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Across 9 guardrail models, 5 exhibited MAL→BEN flips on 100 prompts; vulnerable-model flip rates were 8%–92%, with first flips at 2.6k–9.4k tokens.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Overflip’s low effort, semantic preservation, and black-box applicability make it a practical threat to guardrail-protected services.

    Source: §6 · p. 9

    The evaluation uses 100 prompts, and larger-scale testing may reveal additional edge cases.

    Source: §6 · p. 9

Abstract

from arXiv · show

Guardrail models are classifiers deployed to screen malicious prompts and responses in LLM-based services. To meet latency constraints, many lightweight guardrails adopt compact Transformer backbones (e.g., DeBERTa) that are trained with short context windows (typically 512 tokens) and rely on bucketed relative positional encodings to process longer inputs. Prior evaluations assume that a guardrail's decision is stable as the input is lengthened. We show that this assumption can fail. We identify Overflip, a repetition-induced instability where repeating a prompt causes the guardrail's prediction to flip (MAL$\to$BEN) as the sequence grows. We conduct experiments on 9 widely used lightweight guardrail models. Five exhibit MAL$\to$BEN flips on a benchmark of 100 prompts, with confidence margins shrinking steadily with repetition. Among these vulnerable models, flip rates range from 8% to 92%, with first flips occurring at roughly 2.6k--9.4k tokens. Our analysis suggests Overflip differs from traditional attention-dilution baselines, which aim to divert the model's attention away from tokens associated with malicious content, shifting it instead toward unrelated content, such as benign padding or shuffling. While Overflip preserves malicious content, it homogenizes token-level attention over repeated structure and induces a distinct, more gradual attention-dispersion trajectory than padding. Moreover, Overflip poses a greater threat to LLM services than traditional attention dilution methods. Because the bypassed prompt remains semantically intact and is still readily understood by downstream business LLMs, it can transmit malicious intent after passing the guardrail. These findings expose repetition as an attack surface for guardrail models and motivate length-robust evaluation and mitigation.

1 Introduction

The introduction identifies an untested stability assumption in guardrail evaluations and presents Overflip, where repetition can turn malicious prompts benign while retaining intent.

1 Introduction

  • Across 9 models, 5 exhibited MAL→BEN flips on 100 prompts; vulnerable-model flip rates were 8%–92%, with first flips at 2.6k–9.4k tokens.These flips were accompanied by shrinking confidence margins as sequences grew.Source: §1 · p. 2
  • Repetition preserves malicious content yet homogenizes attention over repeated structure, producing a more gradual dispersion trajectory than padding and shuffling.Source: §1 · p. 2
  • Beyond the 512-token boundary, bucketed relative positions can coarsely compress long-range distinctions, potentially undermining isolation of critical classification evidence.Source: §1 · p. 2
  • On GPT-4.1 Mini and Llama 3.1 8B, repetition better preserves attack semantics, while padding more often induces misinterpretations.Source: §1 · p. 2
  • Unlike traditional jailbreak attacks, Overflip is black-box, requires only guardrail API input-output access, and exploits a structurally different attack surface.Source: §1 · p. 2

2 Related Work

Related work frames the problem across lightweight guardrails, attention dilution, and positional extrapolation, locating Overflip as a distinct length-related stability issue.

2.1 Guardrail Models

  • Guardrail models are lightweight prompt classifiers used in production LLM services, often prioritizing low latency despite short-context training that makes length robustness relevant.Source: §2.1 · p. 3

2.2 Attention Dilution

  • Attention dilution describes degradation as attention spreads across long or complex contexts; unlike padding, repetition adds no novel benign content yet homogenizes attention over identical occurrences.Source: §2.2 · p. 3

2.3 Positional Encodings in Transformers

  • Positional encoding choices affect length stability, and lightweight classifiers’ extrapolation beyond training windows motivates length-aware stress testing.Source: §2.3 · p. 4

3 Problem Setup and Threat Model

The setup models a guardrail as a thresholded binary classifier and defines Overflip as a repetition-induced change in its prediction. The threat model limits attackers to intent-preserving text operations without access to model parameters or training data.

3.1 Task Definition

  • A guardrail maps a prompt to a benign (0) or malicious (1) label using its class probabilities.The model predicts by thresholding the malicious-class probability.Source: §3.1 · p. 4
  • An Overflip occurs when repetition lengthens an input and changes the guardrail’s prediction from its base label.Source: §3.1 · p. 4
  • The study focuses on MAL→BEN flips, where a malicious prompt is misclassified as benign and forwarded downstream.Source: §3.1 · p. 4
  • The attack repeats an already malicious prompt until the guardrail labels it benign, while the repeated instructions preserve its intent downstream.The repetition operator concatenates the prompt n times.Source: §3.1 · p. 4

3.2 Threat Model

  • Attackers may estimate token counts and observe guardrail outputs, but have no access to model parameters or training data.The threat model permits black-box output access or gray-box confidence scores and restricts manipulation to text-level changes.Source: §3.2 · p. 4
  • The evaluated guardrails span vendors, backbone architectures, parameter sizes, and detection targets; the table marks whether any prompt triggers a flip.Source: Table 1 · p. 5

4 Experimental Setup

The experimental setup evaluates repetition-related guardrail behavior across models, prompts, and lengthening strategies. It also tests whether bypassed prompts retain their effect on downstream LLMs and examines attention behavior as inputs grow.

4.1 Research Questions

  • The study asks about Overflip prevalence, whether bypasses preserve attack semantics downstream, and how attention behavior differs from traditional dilution strategies.Source: §4.1 · p. 5

4.2 Models

  • Nine lightweight guardrails span multiple architectures and objectives, while GPT-4.1 Mini and Llama 3.1 8B serve as downstream backends.For the backend tests, LPG-2-86M screens 100 malicious prompts; the lengthening strategies yield 172 bypasses: 71 by Overflip, 92 by padding, and 9 by shuffling.Source: §4.2 · p. 5

4.3 Dataset and Data Processing

  • The benchmark contains 100 validated malicious prompts, filtered to English and under 500 characters, with all verified unsafe by LPG-2-86M.Base prompts are kept below 512 tokens so initial predictions fall within each model’s native context window.Source: §4.3 · pp. 5–6
  • Flip Rate measures the fraction of prompts with a prediction flip; Flip Round and Flip Length measure the average first-flip iteration and input length.Source: Table 2 · p. 6
  • Nearly half of prompts bypassed exactly two models; LPG-2-86M and PIS had the most bypasses, while original prompt length did not significantly correlate with bypass count (r = −0.08, p = 0.418).Harmful Request and Instruction Override were the dominant prompt categories.Source: Figure 2 · p. 6
  • The comparison uses exact repetition, benign padding, and sentence shuffling to lengthen prompts while preserving semantic intent.Source: §4.3 · p. 6

5 Results

Repetition flips predictions in multiple lightweight guardrails, and the bypassed prompts retain practical impact downstream. The analyses link this behavior to attention homogenization and show different entropy dynamics under repetition, padding, and shuffling.

5.1 Overflip Prevalence (RQ1)

  • 5 of 9 evaluated models flipped; vulnerable models showed 8%–92% flip rates and first flips at 2.6k–9.4k tokens.The models span DeBERTa, ModernBERT, and Qwen; three jailbreak detectors without flips did not use bucketed relative positional encodings.Source: §5.1 · pp. 6–7

5.2 Impact on Downstream LLM (RQ2)

  • 41 vs. 28 backend attack success rates (ASR) on GPT-4.1 Mini and 33 vs. 6 on Llama 3.1 8B favored Overflip over padding, despite 71 vs. 92 guardrail flips.The repeated prompt retains its meaning for downstream LLMs, whereas padding adds incoherent content that can degrade comprehension.Source: Table 3 · p. 7, §5.2 · p. 7

5.1 Overflip Prevalence (RQ1)

  • Original prompt length showed no significant correlation with bypass count (r = −0.08, p = 0.418).This result supports the authors’ distinction between repetition-induced lengthening and prompt verbosity.Source: §5.1 · p. 7

5.3 Mechanism of Overflip (RQ3)

  • Entropy fell from 1.89 at k = 1 to 1.77 at k = 5, 1.65 at k = 20, and 1.58 at k = 100 as attention homogenized over repeated structure.The authors link structural anchors monopolizing attention to reduced localization of informative tokens and a destabilized decision boundary.Source: §5.3 · p. 7, Figure 3 · p. 8
  • Overflip entropy dropped from ≈0.95 to as low as 0.3 within one to five steps; padding declined gradually, while shuffling stayed near ≈0.85 and oscillatory.The authors associate Overflip’s collapse with the CLS token locking onto repeated structural anchors; shuffling displaces keyword positions.Source: Figure 4 · p. 8, §5.3 · p. 9
  • Overflip flip points ranged from ≈100 to several thousand tokens, while padding flip points clustered narrowly.The token-count variation depended on per-prompt structural properties.Source: §5.3 · p. 9

6 Discussion and Limitations

Overflip is a practical low-effort threat, but truncation can prevent straightforward attacks only when full context is not required. The authors recommend length stress tests and report that their evaluation is limited in scale and defense coverage.

Implications for safety evaluation and deployment

  • Length stress tests should report flip rates, first-flip lengths, and entropy trajectories alongside accuracy.The authors note that RAG pipelines, tool-use traces, and multi-turn dialogues can grow inputs beyond 512 tokens.Source: §6 · p. 9

Defenses and attack cost

  • 41 vs. 28 downstream ASR on GPT-4.1 Mini and 33 vs. 6 on Llama 3.1 8B show higher attack success than padding; automated repetition requires no adversarial expertise.Truncation to 512 tokens can prevent straightforward Overflip, but is infeasible when full context is required and can be bypassed by placing malicious content after the first chunk.Source: §6 · p. 9

Limitations

  • The evaluation covers 100 prompts; positional-encoding schemes are inferred from behavior, and defenses are not exhaustively evaluated.The authors also cite GPT-4.1 Mini API non-determinism as a possible constraint on end-to-end reproducibility.Source: §6 · p. 9

7 Conclusion

The paper concludes that repeating malicious prompts can exploit a structural vulnerability in lightweight guardrails, flipping their decisions beyond the 512-token boundary. The resulting bypass preserves prompt semantics and achieves higher downstream attack success rates than padding.

Conclusion

  • 5 of 9 guardrail models exhibit MAL→BEN flips, with rates of 8%–92% and first flips at 2.6k–9.4k tokens.The conclusion attributes these flips to repetition homogenizing attention over structural anchor positions and reducing sensitivity to malicious tokens.Source: §7 · p. 9
  • Backend ASR is 41 vs. 28 on GPT-4.1 Mini and 33 vs. 6 on Llama 3.1 8B for Overflip versus padding.The conclusion links the higher rates to the bypassed prompt remaining semantically intact.Source: §7 · p. 9
  • The findings identify input length as an under-examined attack surface and positional encoding design as a safety consideration.Source: §7 · p. 9

8 Ethical Considerations

The authors describe Overflip as a potential bypass risk for deployed LLM services and outline disclosure and release plans. They attribute the vulnerability to architectural choices rather than undisclosed exploits.

Ethical Considerations

  • The authors disclosed the risk to affected model providers and plan to release evaluation code and defensive guidance after institutional IP review.Source: §8 · p. 9
  • The stated vulnerability stems from positional bucketing, an architectural choice, rather than undisclosed exploits.Source: §8 · p. 10
  • The authors state that identifying the vulnerability publicly accelerates development of more robust guardrails and benefits the broader safety ecosystem.Source: §8 · p. 10

A Additional Results

Figure 5 compares Overflip, padding, and shuffling on flip outcomes, first-flip lengths, and attention entropy. Table 4 summarizes backend responses and reports differences in semantic retention and misinterpretation.

Additional Results

  • Padding achieves the highest flip rate near 1k tokens, Overflip flips occur at substantially longer lengths with more variable attention dispersion, and shuffling rarely flips.Source: Figure 5 · p. 12
  • Figure 5 plots MAL→BEN success rates, first-flip token counts, and normalized CLS-attention entropy, where higher entropy means more diffuse attention.Source: Figure 5 · p. 12
  • Table 4 reports acceptance, rejection, and misinterpretation outcomes; Overflip retains semantics most effectively, while padding causes the most misinterpretation in both models.Source: Table 4 · p. 12
Loading 2609.15013v1…