PaperlayerView exact revision on arXiv

Source-linked AI summary

FragToken: Amplifying LLM Inference Costs through Noncanonical Token Generation

Zihan Wang, Rui Zhang, Xinyuan Qian, Qingchuan Zhao, Hongwei Li, Guowen Xu

arXiv:2609.31552v1cs.CR

TL;DR

LLM resource-consumption attacks commonly increase inference cost through longer or repetitive outputs on attacker-controlled or triggered requests. FragToken trains models to generate longer noncanonical token sequences under ordinary prompts, combining self-distillation, capacity-aware budgeting, and BPE-Aligned Merging. Across four LLMs, its three-benchmark average TIRs range from 1.99 to 2.46 with minor utility degradation.

  • Problem

    Existing resource-consumption attacks typically rely on longer or repetitive outputs for attacker-controlled or triggered requests, while LLM inference costs recur as providers serve large volumes of requests.

    Source: §1 · p. 1

  • Method

    FragToken combines source-model self-distillation, capacity-aware filtering and budgeting, and BPE-Aligned Merging to induce fragmented generation under ordinary prompts.

    Source: §1 · pp. 2–3

  • Results

    1.99–2.46 three-benchmark average TIRs across four LLMs accompany largely preserved task utility.

    Source: §1 · p. 2

  • Takeaways & Limitations

    FragToken increases inference costs while preserving readable outputs and largely retaining task utility, and its fragmentation can coexist with output-lengthening attacks.

    Source: §7 · p. 10

    Combined output-lengthening attacks reduce stealthiness, and detector-based stealthiness of the GRPO extension remains untested.

    Source: §6 · p. 10

Abstract

from arXiv · show

As large language model (LLM) inference becomes increasingly expensive, resource-consumption attacks pose a growing threat to model providers. Existing attacks typically amplify cost by inducing abnormally long or repetitive outputs on attacker-controlled or triggered requests, making them easier to detect and limiting their deployment-wide impact when benign traffic dominates. In this work, we uncover a previously overlooked token-level attack surface arising from the many-to-one mapping from token sequences to decoded text. Although standard LLMs predominantly generate the canonical token sequences induced by their tokenizers, the same text can also be represented by substantially longer non-canonical sequences. This representational flexibility exposes a new avenue for resource-consumption attacks: an attacker can train the model to favor such sequences, systematically increasing the number of autoregressive decoding steps without a proportional increase in visible response length. However, we empirically find that directly maximizing token fragmentation substantially degrades model utility, producing conspicuous answer-quality failures that undermine attack stealthiness. To address this challenge, we propose FragToken, a training-time framework that combines source-model self-distillation, capacity-aware filtering and budgeting, and BPE-Aligned Merging to induce fragmented generation under ordinary prompts while largely preserving model utility. We evaluate FragToken on four LLMs across three benchmarks. Across the four models, FragToken achieves a three-benchmark average token inflation ratio (TIR) ranging from 1.99 to 2.46, while causing only minor degradation in model utility. Our work reveals a covert LLM supply-chain threat that increases inference cost without requiring large volumes of attack requests while largely preserving utility.

1 INTRODUCTION

FragToken targets recurring LLM inference costs by inducing longer noncanonical token sequences under ordinary prompts, avoiding proportional visible-text expansion. Its three design components aim to increase decoding costs while largely preserving utility, and evaluations report token inflation across four models and three benchmarks.

1 INTRODUCTION

  • Prior resource-consumption attacks amplify cost through longer responses or reasoning, with detectable anomalies and effects diluted by benign traffic.These limitations motivate overhead on ordinary traffic without detectable content expansion or runtime activation.Source: §1 · pp. 1–2
  • Noncanonical token sequences can decode to the same text as canonical sequences, creating an attack surface for fragmented generation under ordinary prompts.The proposed supply-chain scenario modifies an upstream model before delivery, avoiding runtime triggers and proportional visible content growth.Source: §1 · p. 2
  • FragToken combines source-model self-distillation, capacity-aware filtering and budgeting, and BPE-Aligned Merging to increase decoding costs while largely preserving response quality.The designs focus fine-tuning on familiar content, bound fragmentation budgets, and align merging with canonical-token spans.Source: §1 · pp. 2–3
  • 1.99–2.46 three-benchmark average TIRs across four LLMs accompany largely preserved task utility.Under evaluated defenses, detection remains comparable to benign behavior and lower than existing resource-consumption attacks; overhead persists when benign requests dominate traffic.Source: §1 · p. 2
  • The paper identifies a token-level attack that increases overhead without proportional visible-text growth while largely retaining task utility.Its evaluation spans four LLMs and three benchmarks, with average TIRs of 1.99–2.46 and only minor utility degradation.Source: §1 · pp. 2–3

2 PRELIMINARIES AND RELATED WORK

The tokenizer’s deterministic encoding produces canonical sequences, while decoding can map multiple token sequences to identical text. FragToken exploits noncanonical generation as a resource-consumption attack that operates under ordinary prompts rather than attacker-controlled inputs or triggers.

2.1 LLM TOKENIZERS

  • Byte-level BPE merges frequent adjacent byte patterns into single tokens, reducing the tokens and decoding steps needed for common text.Encoding applies deterministic merge rules; decoding concatenates token byte strings without enforcing those rules.Source: §2.1 · p. 3

2.2 NONCANONICAL GENERATION

  • A sequence is noncanonical when it differs from the canonical encoding of its decoded content, even though both sequences represent identical bytes.Reversing a BPE merge can turn a canonical single-token sequence into a longer sequence with the same decoded text.Source: §2.2 · p. 3
  • LLMs can generate noncanonical sequences because next-token prediction does not enforce canonical merge rules; FragToken fine-tunes for persistent noncanonical generation under ordinary prompts.This increases decoding overhead without changing the tokenizer or requiring adversarial inference-time inputs.Source: §2.2 · p. 3

2.3 RESOURCE-CONSUMPTION ATTACKS

  • Prior inference-time and training-time attacks rely on attacker-controlled or trigger-bearing inputs and amplify cost through longer responses or reasoning.Their additional costs can be diluted when benign queries dominate, whereas FragToken induces noncanonical generation under ordinary user prompts without proportional visible-text expansion or runtime triggers.Source: §2.3 · p. 3

3 THREAT MODEL

The threat model places the attacker upstream, as a fine-tuning contractor modifying a third-party LLM before delivery. After deployment, the modified model increases decoding overhead across ordinary user traffic without requiring continued access to the provider’s infrastructure or inference-time requests.

3 THREAT MODEL

  • A downstream provider deploys a third-party LLM, and an outsourced fine-tuning contractor modifies it before delivery while leaving the tokenizer unchanged.Source: §3 · p. 4
  • The compromised model generates more fragmented sequences for ordinary user requests, increasing decoding cost while largely preserving utility and visible text.Source: §3 · p. 4
  • The attacker needs no further serving-infrastructure access or control over inference-time requests, so overhead extends across benign traffic.Source: §3 · p. 4

4 METHODOLOGY

The section shows that maximizing fragmentation alone increases token use but damages utility, motivating label construction that controls content, fragmentation degree, and merge path. FragToken uses self-distilled responses, capacity-aware budgets, and BPE-aligned merges before fine-tuning on fragmented labels.

4.1 PROBLEM FORMULATION AND MOTIVATION STUDY

  • Full atomization raises average TIR from 1.00 to 4.11 but lowers average ACC from 81.88% to 72.41% on three benchmarks.The results are for Llama-3.1-8B-Instruct trained with Alpaca responses.Source: §4.1 · p. 4, Table 1 · p. 4

4.2 FRAGMENTED LABEL CONSTRUCTION

  • Source-model self-distillation supplies responses closer to the model’s predictive preferences as starting labels for fragmentation.Self-distilled responses had NLL 0.375 versus 1.382 for the original Alpaca references.Source: §4.2 · p. 4
  • Capacity-aware budgeting allocates a bounded fraction of each response’s attainable expansion, filtering low-capacity samples and capping high-capacity ones.A fixed expansion target can over-fragment low-capacity responses while under-fragmenting high-capacity responses.Source: §4.2 · p. 5
  • BPE-Aligned Merging restricts operations to canonical-token spans and preserves valid BPE recovery paths while changing token granularity.It avoids arbitrary cross-boundary merges and uses shared BPE-rank priority to select candidates.Source: §4.2 · p. 5
  • At comparable inflation, BPE-Aligned Merging consistently outperforms random merging on continuation ∆NLL, predictive KL divergence, and hidden-state similarity.The controlled diagnostic varies the preceding context’s token representation while keeping decoded content and canonical continuation fixed.Source: Figure 4 · p. 6, §4.2 · p. 6

4.3 FRAGMENTATION-AWARE TRAINING

  • Fine-tuning uses fragmented response token IDs as supervision, masks prompt positions, and leaves the tokenizer and inference pipeline unchanged.Source: §4.3 · p. 6

5 EXPERIMENTS

The experiments assess FragToken across four LLMs and three benchmarks, including serving overhead, defense detection, deployment mixtures, and component and hyperparameter ablations. FragToken increases serving time and token counts while maintaining overhead under benign-majority traffic; the ablations examine its inflation–utility trade-offs.

5.1 EXPERIMENTAL SETUP

  • The evaluation uses four LLMs and three benchmarks, with Alpaca prompts and source-model self-distilled responses for training.It reports TIR and accuracy, with serving analyses additionally reporting generated tokens and generation time.Source: §5.1 · p. 6

5.2 MAIN RESULTS

  • Table 2 presents the main results across four models; the supplied passages do not provide its individual result values.The evaluated benchmarks are GSM8K, PIQA, and Open-BookQA.Source: §5.1 · p. 6, Table 2 · p. 7

5.3 IMPACT ON SERVING EFFICIENCY

  • 2.44× the generation time and 2.56× the generated tokens of Original are reported for FragToken; visible response length falls by 10.3%.At a 10% attack-query share, FragToken retains 2.56× time overhead, while the baselines’ maximum latency amplification is 1.88×.Source: Table 3 · p. 7, §5.3 · p. 8

5.4 ROBUSTNESS AGAINST DEFENSES

  • 18% combined detection for FragToken compares with 22% for Original and 84–100% for four optimized inference-time attacks under the evaluated checks.FragToken’s repetition detection matches Original at 4%, versus 100% for P-DoS-R.Source: Table 4 · p. 8, §5.4 · p. 8

5.5 ROBUSTNESS ACROSS DEPLOYMENT SETTINGS

  • Figure 5 reports results across deployment settings, but the supplied passages do not state the figure’s outcomes.Source: Figure 5 · p. 8

5.6 ABLATION STUDY

  • Removing self-distillation reduces Avg. ACC by 4.57 percentage points and TIR by 0.22; removing capacity-aware budgeting lowers TIR from 2.46 to 2.00.Random merging yields Avg. ACC of 77.56% with TIR 2.44, while the complete method balances inflation and utility.Source: Table 5 · p. 9, §5.6 · p. 9
  • Increasing merge fraction lowers Avg. TIR from 3.17 to 1.85 and raises Avg. ACC from 76.29% to 81.11%; epoch 2 is reported as cost-effective.At the default β = 0.5, Avg. TIR is 2.46 and Avg. ACC is 80.40%; most training effect emerges by epoch 1, with later changes marginal.Source: §5.6 · p. 9, Figure 6 · p. 9

6 DISCUSSION

The discussion tests whether FragToken’s token-level fragmentation can combine with inference-time attacks that lengthen response content. Combined attacks increase token counts, but introduce anomalies on both input and output sides; detector-based stealthiness of the explored GRPO extension remains untested.

6 DISCUSSION

  • All five inference-time attack methods generate 1.53–3.08× their standalone token counts when combined with FragToken.Combined attacks retain TIR values of 2.74–4.98, while standalone counterparts remain near 1.00.Source: Table 6 · p. 9, §6 · p. 9
  • Combining the attacks increases effectiveness but reduces stealthiness through anomalous patterns on both input and output sides.The explored utility-aware output-lengthening extension uses GRPO, but its detector-based stealthiness remains untested.Source: §6 · p. 10

7 CONCLUSION

FragToken is a training-time attack using noncanonical token generation to increase inference costs while preserving readable outputs and largely retaining utility. The conclusion reports lower combined detection than four evaluated inference-time attacks, persistent overhead in evaluated traffic mixtures, and compatibility with tested output-lengthening methods.

7 CONCLUSION

  • FragToken increases inference costs through noncanonical token generation while preserving readable outputs and largely retaining task utility.The added work is less visible, and fragmentation can coexist with output-lengthening attacks.Source: §7 · p. 10
  • FragToken has a lower combined detection rate than the four optimized inference-time attacks evaluated in Table 4.Its overhead also persists in the evaluated traffic mixtures, and it is compatible with tested output-lengthening methods.Source: §7 · p. 10

AI USE STATEMENT

The authors used generative AI across manuscript preparation and research activities, while retaining responsibility for and manually reviewing the work.

AI USE STATEMENT

  • Generative AI assisted with drafting and editing, LaTeX formatting, code work, hypothesis refinement, methodological feedback, and interpretation of results.Source: AI USE STATEMENT · p. 10
  • The research used model-generated responses for self-distillation and an LLM-as-a-judge quality reward.These uses are described in the methodology and appendix.Source: AI USE STATEMENT · p. 10
  • The authors retained responsibility for the code, procedures, analyses, text, and claims, and manually reviewed AI-assisted code and outputs.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

The ethics statement describes authorized research use and cautions against misuse; the appendices document training, evaluation, serving, detection, and auxiliary-study procedures and results.

ETHICS STATEMENT

  • The authors conducted evaluations in authorized research environments, not against third-party online services or real users.They acknowledge possible misuse to increase inference costs and advocate use only in authorized security evaluations.Source: ETHICS STATEMENT · p. 10

A.1 TRAINING AND EVALUATION

  • Training uses source-model responses for selected Alpaca prompts and response-only SFT with BPE-Aligned labels; SFT uses canonical labels from the same per-model corpus.The appendix also specifies training limits and settings, baseline procedures, and benchmark prompting and scoring.Source: §A.1 · p. 13

A.2 SERVING AND DETECTION

  • Serving evaluates 50 Alpaca queries with greedy decoding; traffic mixtures add 200 or 450 benign requests for 20% or 10% attack shares.Five checks assess detection, and the RD results are in-sample diagnostics using reused samples from the same 50-query set.Source: §A.2 · p. 13

A.3 AUXILIARY STUDIES

  • Auxiliary studies specify a predictive-structure diagnostic, component ablations, attack-composition procedures, and GRPO-based utility-aware output lengthening.The diagnostic compares BPE-Aligned with random merging; ablations include disabling self-distillation, BPE-Aligned Merging, or capacity budgeting.Source: §A.3 · pp. 13–14

B.1 VISIBLE OUTPUT LENGTH

  • FragToken generates 2.56× as many tokens as Original while reducing mean response length from 1,501.2 to 1,347.3 characters, a 10.3% decrease.Other lengthening attacks produce 2,303.2–4,289.0 characters, while P-DoS-R reaches the output cap; Engorgio’s shorter response shows serving cost is not determined by visible length alone.Source: §B.1 · p. 14, Table 7 · p. 14

B.2 ID/OOD TRANSFER

  • FragToken reaches ID TIR 4.01 and OOD TIR 2.46, while Original remains at 1.00 on both; OOD/ID ratios are 61.4% and 100.0%, respectively.Inflation persists on OOD tasks but is lower than on ID prompts; broader training coverage is proposed as a future hypothesis to test.Source: §B.2 · p. 14

B.3 TRANSFERRED ATTACK COMPOSITION

  • Transferred prompts yield 1.11–3.02× as many tokens as the corresponding standalone attacks, with TIR of 2.39–5.38.Reoptimizing prompts on FragToken increases tokens for each of four optimized attacks, although TIR does not increase uniformly.Source: §B.3 · p. 14, Table 8 · p. 14

B.4 UTILITY-AWARE OUTPUT LENGTHENING

GRPO-based output lengthening adds visible text while retaining fragmentation, with a benchmark-utility trade-off and increased generation time in the separate Alpaca run.

B.4 UTILITY-AWARE OUTPUT LENGTHENING

  • The evaluation uses a merged FragToken reference with Avg. ACC 80.40% and Avg. TIR 2.46 on 3,657 examples.GRPO optimizes a FragToken checkpoint using decoded word count and a positive quality score.Source: §B.4 · p. 15
  • FragToken-G retains Avg. TIR 2.58 while Avg. ACC is 2.36 percentage points lower at 78.04% than FragToken on the three benchmarks.Mean response lengths are 507.5, 374.1, and 1,919.4 characters for Original, FragToken, and FragToken-G, respectively.Source: §B.4 · p. 15, Table 9 · p. 15
  • In the separate ten-query Alpaca run, time rises from 6.53 to 8.04 seconds per example from FragToken to FragToken-G, while TIR is nearly unchanged at 2.79 and 2.78.Original takes 2.30 seconds per example; response lengths are 1,321.8, 1,335.8, and 2,978.1 characters for Original, FragToken, and FragToken-G.Source: §B.4 · p. 15

C ADDITIONAL ABLATION STUDY

The appendix compares full atomization and additional component combinations, then illustrates fragmented traces against canonical tokenization and equal-budget re-merging.

C.1 COMPARISON WITH FULL ATOMIZATION

  • Avg. TIR 4.11 accompanies a substantial utility cost for full atomization: Avg. ACC falls from 81.88% to 72.41%, while FragToken reaches Avg. ACC 80.40% and Avg. TIR 2.46.FragToken’s Avg. ACC is 7.99 percentage points above Atomic; the comparison highlights a more favorable utility–overhead trade-off than maximizing token count alone.Source: Table 10 · p. 15, §C.1 · p. 15

C.2 ADDITIONAL COMPONENT COMBINATIONS

  • SD-only has the highest individual-component Avg. ACC at 76.97%, while BAM-only and CB-only yield Avg. TIR 2.17 and 2.27, respectively.Together with the full method’s Avg. ACC 80.40% and Avg. TIR 2.46, these configurations illustrate the utility–inflation trade-off across component choices.Source: Table 11 · p. 15, §C.2 · pp. 15–16

D.1 CANONICAL VERSUS FRAGMENTED TOKENIZATION

  • Recorded noncanonical sequences are compared with canonical re-encodings using a merged FragToken checkpoint, greedy decoding, and a 64-token output limit; these traces are separate from serving measurements.The trace setup uses Transformers, BF16 weights, eager attention, and batch size 2.Source: §D.1 · p. 16

A. User-visible response, fragmented server trace

  • The controlled example keeps the prompt and decoded response identical while changing the token path from 8 canonical steps to 19 FragToken steps.The additional server-side work is represented by the increase in autoregressive decoding steps.Source: A. User-visible response, fragmented… · p. 16

B. Equal-budget re-merging and span locality

  • With both re-merging rows using 21 labels, random fragments cross canonical-token spans while BPE-Aligned fragments retain local units such as “long—est” and “Afr—ica.”The comparison contrasts span-local structure with arbitrary cross-boundary fragments.Source: B. Equal-budget re-merging and span… · p. 16

D.2 INPUT AND OUTPUT VISIBILITY

A visibility example contrasts FragToken’s unchanged query and concise response with comparison methods that visibly perturb inputs or expand responses. The algorithm self-distills responses, constructs capacity-budgeted BPE-aligned labels, and trains on fragmented token IDs.

D.2 INPUT AND OUTPUT VISIBILITY

  • 19 tokens (TIR 2.38) produce FragToken’s concise response to an unmodified query, while its canonical encoding uses 8 tokens; comparison methods show visible input perturbations or response expansion.Table 12 pairs recorded queries with outputs and token overhead.Source: Table 12 · p. 16, §D.2 · pp. 16–17

E FragToken ALGORITHM

  • FragToken self-distills source outputs, screens and budgets samples by atomization capacity, constructs BPE-aligned labels within canonical-token spans, and applies response-only SFT to fragmented IDs.The tokenizer and inference pipeline remain unchanged.Source: E FragToken ALGORITHM · pp. 17–18
Loading 2609.31552v1…