PaperlayerView exact revision on arXiv

Source-linked AI summary

Detokenization Leaks: Reconstructing Local LLM Outputs From Cache Traces

Roy Weiss, Benyamin Konstantinov, Eitam Sheetrit, Tomer Simon, Yisroel Mirsky

arXiv:2609.06674v1cs.CRcs.AI

TL;DR

Existing cache attacks on local LLMs rely on deployment-specific assumptions, while detokenization provides a default-pipeline target with token-dependent CPU cache activity. The paper combines Flush+Reload synchronization, Prime+Probe capture, and clustering with language-model reconstruction to recover generated text, demonstrating applicability across diverse systems and real-world deployments. Its main scope boundary is process specificity: restarts limit profiling time but do not eliminate leakage because instances can be re-profiled.

  • Problem

    Prior cache attacks require deployment-specific features such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, limiting their applicability across local LLM deployments.

  • Method

    The attack detects detokenization with Flush+Reload on shared tokenizer code, captures token-dependent cache activity with Prime+Probe, and reconstructs text from clustered symbols using language models.

  • Results

    The attack reconstructs outputs across conversational, instruct, and code tasks, multiple datasets, hardware platforms, inference frameworks, model families, and real-world local deployments.

  • Takeaways & Limitations

    Detokenization is a practical and widespread leakage surface in local LLM serving, including systems built on commonly reused tokenizer libraries.

  • Takeaways & Limitations

    The reconstruction model is process-specific and does not transfer across independent instances; restarts limit profiling time but allow an adversary to re-profile each new instance.

Abstract

from arXiv · show

We present a new attack that reconstructs the text generated by locally hosted LLMs by observing CPU cache activity during detokenization. Unlike prior attacks that rely on deployment-specific assumptions, such as shared data memory, CPU offloading, or Mixture-of-Experts architectures, our approach targets the detokenizer, a component used in default LLM inference pipelines. To obtain clean signals, we use Flush+Reload on shared tokenizer code to detect when decoding occurs, which lets us perform Prime+Probe at the right moment and isolate token-dependent cache activity. We then apply a clustering-and-language-model pipeline to recover text from noisy cache observations. We evaluate the attack across multiple datasets, hardware platforms, inference frameworks, and model families, and show that it can recover semantically accurate outputs from real-world local LLM deployments, including agentic systems. This vulnerability is particularly significant because the most widely used tokenizer implementations are susceptible to the attack and are embedded in many popular local LLM products and agent frameworks, including systems such as OpenClaw (which we demonstrate), substantially broadening the practical attack surface.

1 Introduction

The paper targets a broadly applicable cache side channel against local LLMs by observing detokenization, avoiding assumptions tied to shared data, CPU offloading, or specific model architectures. It combines synchronized cache measurements with symbolic clustering and language-model reconstruction, and evaluates the attack across diverse tasks and real-world systems.

  • Prior cache attacks depend on deployment-specific features such as shared CPU/GPU memory, CPU offloading, or Mixture-of-Experts architectures.
  • The attack instead targets the detokenizer, whose CPU decode-table lookup maps each generated token ID to a word chunk and produces token-dependent cache activity.
  • Flush+Reload on shared tokenizer code aligns Prime+Probe measurements with detokenization, reducing noise and isolating the relevant cache activity.
  • Clustering noisy cache patterns into discrete symbols and applying a language model converts ambiguous trace sequences into reconstructed text.
  • Widely reused tokenizer libraries expose a broad attack surface across local LLM servers, agent frameworks, and downstream applications, enabling future-output reconstruction after profiling.
  • The attack covers conversational, instruct, and code tasks across three datasets, two hardware platforms, two inference frameworks, and two target models.

2 Background

This background describes tokenization and detokenization in local LLM inference, then introduces CPU-cache side channels and the Flush+Reload and Prime+Probe primitives used to observe memory activity. It also identifies the decode table and L1 cache as relevant structures for token-dependent leakage.

  • LLM Inference and Tokenization: LLM inference encodes prompts into token IDs and decodes generated token IDs into UTF-8 strings for human-readable output.
  • LLM Inference and Tokenization: Local LLM systems commonly keep long-lived inference backends active, streaming decoded responses to desktop interfaces, IDE plugins, or automation tools.
  • Cache Side-Channel Attacks: Shared CPU caches create contention between processes, allowing an adversarial process to infer victim memory accesses from cache behavior.
  • Cache Side-Channel Attacks: Flush+Reload monitors shared code or data lines by evicting and reloading them, whereas Prime+Probe infers accesses through contention without requiring shared memory.
  • Cache Side-Channel Attacks: L1 Prime+Probe is suited to short per-token decode operations because simultaneous-multithreading siblings share the private L1 data cache.
  • LLM Inference and Tokenization: The decode table maps token identifiers to UTF-8 byte sequences and is implemented using structures such as vectors or hash maps.

3 Attack Overview

The attack targets token-dependent detokenization accesses in long-lived local LLM deployments, using cache observations to reconstruct ordered outputs. It profiles stable trace–token mappings and exploits them under a threat model requiring co-residency, shared tokenizer code, and SMT sibling placement.

  • Measurement: Shared tokenizer code enables Flush+Reload timing of detokenization, while Prime+Probe detects the corresponding cache-set activity without shared data memory.The attack relies on shared executable library pages and uses the tokenizer routine as a timing trigger.
  • Threat model: An unprivileged co-resident process observes cache effects while interacting with a long-lived local LLM instance through its local interface.The adversary may run as a benign user-space application and does not require elevated privileges or shared model data.
  • Vulnerability: Each generated token is decoded through a fixed decode table, causing token-dependent cache activity that remains stable during execution.This stable mapping provides the leakage source for profiling and later reconstruction.
  • Attack workflow: The attacker collects labeled traces from benign queries, learns trace–token mappings, and reconstructs ordered future outputs from token-dependent trace sequences.The decode-table layout remains fixed, so labeled profiling traces can support exploitation against later outputs.
  • Challenges: Reconstruction is challenged by cache collisions, low signal-to-noise ratio, and many-to-one token-to-cache-set mappings.Single-shot generation prevents averaging repeated measurements, while coarse cache-set granularity creates aliasing among tokens.

4 Attack Implementation

The attack captures token-dependent L1 cache traces around detokenization, then converts noisy traces into symbolic sequences and reconstructs text with language models. Its implementation combines timing-triggered Prime+Probe measurements, clustering, and segmented semantic reconstruction.

  • Trace capture: Flush+Reload detects tokenizer decode execution, enabling Prime+Probe measurements to be timed around each token’s decode operation.The monitored tokenizer routine is loaded from shared executable pages, allowing decode events to be detected across evaluated frameworks.
  • Trace capture: 97.66% true positive rate with zero false positives was achieved over 1.2 million simulated decode invocations.The experiment used random 10–30 ms invocation intervals to reflect typical token-generation delays.
  • Trace capture: Prime+Probe monitors all N=64 L1 cache sets to obtain one latency trace per decoded token.The decode-table lookup is not shared, so the attack uses Prime+Probe rather than Flush+Reload for token-dependent accesses.
  • Trace capture: LLC monitoring is impractical because its larger, slower cache and broadly distributed decode-table entries cannot be covered within the short decode window.The implementation therefore focuses on L1 measurements despite noise from prefetching, limited capacity, and self-interference.
  • Trace symbolization: Individual traces provide weak token-level information, so the pipeline embeds traces, clusters token centroids, and replaces each trace with a learned symbolic cluster label.A direct MLP classifier achieved below 1% top-1 accuracy, motivating the intermediate symbolic representation.
  • Semantic reconstruction: The reconstruction model processes 32-token segments, using LLM_A for the first segment and LLM_B conditioned on prior segment text thereafter.During training, later segments use the ground-truth previous segment; during inference, they use the preceding prediction.

5 Evaluation

The evaluation tests reconstruction across datasets, models, frameworks, hardware, injected noise, profiling budgets, and a live OpenClaw deployment. The attack preserves semantic content across diverse settings, remains effective under noise, and transfers to a real-world agent despite end-to-end degradation.

  • Evaluation setup: The evaluation spans three datasets, two hardware platforms, two inference frameworks, and two target models, with reconstruction assessed on unseen traces and responses.The datasets cover textual and code outputs, while the experiments vary Phi-3-mini and Llama-3, Llama.cpp and HuggingFace Transformers, and laptop and desktop systems.
  • Attack generality: Across settings, full-paragraph reconstruction reaches 56–87% ASR on UltraChat and 78–93% on ChatDoctor while preserving semantic similarity.UltraChat reports semantic similarity 𝜙 of 76–87%, whereas ChatDoctor reports 61–76%; reconstructions often retain subtopics and local structure despite incorrect words.
  • Attack generality: Up to 62.92% of first-segment ChatDoctor reconstructions and 58.48% of full UltraChat responses achieve 𝜙≥0.9, demonstrating high-fidelity recovery beyond topic identification.For first segments, the corresponding UltraChat rate reaches 44.62%.
  • Attack generality: Performance is consistent across hardware and inference frameworks, while larger vocabularies and more diverse datasets reduce reconstruction performance.Llama underperforms Phi with its 4× larger vocabulary, whereas structured datasets such as ChatDoctor and Code-Alpaca outperform UltraChat.
  • Attack robustness: At injected noise probability 𝑝=0.15, LLM_A retains approximately 30% ASR and roughly 10% high-semantic-fidelity reconstructions.Performance degrades gradually as additional noise increases, but reconstruction remains effective under substantial injected noise.
  • Query performance tradeoff: Targeted profiling exceeds 55% ASR with 𝑄_t=250 queries in roughly 6 minutes, while 𝑄_b=10,950 benign queries reaches approximately 56% ASR near the 57.51% unlimited-profile baseline.Ordinary conversational queries therefore retain near-baseline performance with a substantially larger budget.
  • End-to-end OpenClaw attack: On live OpenClaw with local Phi-3-mini, the attack achieves 30.12% ASR and mean 𝜙=42.23%±20.28%, below the controlled setting but still effective.The controlled Llama.cpp, Phi, desktop, UltraChat setting achieves 61.55% ASR and 71.62% 𝜙, while OpenClaw adds timing variability and imperfect alignment.

6 Mitigation Strategies

The paper considers mitigations that disrupt profiling or cache observations, but each has practical costs or only limits rather than removes the leakage.

  • Decode-table randomization changes token-to-cache mappings, primarily increasing attack cost rather than eliminating the leakage channel.The attack relies on a stable mapping between token identifiers and their memory footprint during a victim process.
  • Periodic process restarts limit profiling time, but increase initialization latency and do not prevent re-profiling of new instances.The reconstruction model is process-specific because of ASLR, heap allocation differences, and randomized data structures.
  • Disabling SMT blocks L1 Prime+Probe by preventing physical-core sharing, but incurs significant performance costs for local deployments.
  • Cache partitioning and randomized cache indexing or eviction can reduce signal availability, although profiling may partially adapt.These defenses are generally uncommon on consumer systems because of support, performance, or deployment overhead.

7 Related Works

Prior local-LLM side-channel attacks exploit application, network, shared-data, or architecture-specific leakage, whereas this work targets detokenization as a broader leakage source.

  • Application-, network-, and cache-level attacks have inferred prompts, outputs, response structure, model data, architectures, and weights under different deployment conditions.
  • Table 4 compares deployment assumptions across local LLM side-channel attacks, distinguishing special non-standard resources from standard shared resources.
  • Prior attacks may require shared model data, CUDA unified memory, CPU embedding execution, or page deduplication to observe token-related accesses.
  • MoE attacks exploit expert routing and do not extend to standard dense models, while this work targets model-agnostic detokenization after each generated token.
  • Detokenization provides a weaker observation channel because each event is brief and touches few cache lines, unlike structured expert footprints or stable network observables.
  • The paper identifies the decode table as a previously unexamined leakage source and shifts the attack surface away from model-internal data or architecture-specific behavior.
  • The attack targets a default pipeline component, enabling cache-based output reconstruction across a broader range of local LLM deployments.

8 Conclusion

The paper presents a cache-based attack that reconstructs locally hosted LLM text from detokenization activity while avoiding deployment-specific assumptions. Its conclusion emphasizes practical leakage in default pipelines and the need for stronger defenses in privacy-sensitive deployments.

  • The attack reconstructs text generated by locally hosted LLMs by observing CPU cache activity during detokenization.
  • It combines Flush+Reload for decode detection, Prime+Probe for cache-trace capture, and clustering with language models for noisy-text reconstruction.
  • The authors identify detokenization as a practical and widespread leakage surface and highlight stronger isolation and defenses for privacy-sensitive deployments.
  • The evaluation uses publicly available models, frameworks, and datasets including Phi-3-mini, Llama-3, Llama.cpp, HuggingFace Tokenizers, UltraChat, ChatDoctor, and Code-Alpaca.
  • Experiments use local machines, open-source models, and controlled datasets without real user data or human subjects, with vulnerabilities responsibly disclosed to relevant maintainers.

C Data Cleaning

The data-cleaning procedure deduplicates prompts using 8-gram overlap and removes empty or generic UltraChat responses before evaluation.

  • An 8-gram deduplication procedure removes a sample when its prompt shares its first 8 words with a previous prompt or exceeds 70% 8-gram overlap.
  • UltraChat receives additional cleaning to remove empty or generic responses, including boilerplate refusals.
  • Table 5 reports dataset sizes before and after cleaning together with the percentage removed.
  • The training inputs use reconstructed response segments and symbolic-token sequences as inputs for language-model translation tasks.

E Additional Modeling Details

The reconstruction pipeline standardizes cache traces, learns token representations through contrastive sampling and clustering, and decodes symbol sequences into text with a language model.

  • Trace preprocessing and representation: Each trace is represented as a 64-dimensional latency vector, mean-centered per trace, and feature-wise standardized over the profiling set.
  • Trace preprocessing and representation: Training batches pair multiple traces from the same token with cross-token negatives for contrastive learning.
  • Clustering and symbolization: The pipeline averages traces into token centroids, measures centroid-consistent cluster assignments, and obtains approximately 75% mean clustering accuracy across trace datasets.
  • Reconstruction and decoding: The language model uses preceding text as context and symbol sequences as input, then reconstructs target text with four-beam search capped at 38 tokens.

F K Ablation Study

The ablation shows that symbol-alphabet size strongly affects reconstruction, with performance peaking at an intermediate number of clusters before declining from fragmented training signals.

  • K = 1 produces near-zero ASR, showing that an uninformative symbol alphabet cannot support text reconstruction.
  • ASR rises sharply and peaks around K = 48–64 before declining as larger K fragments the per-symbol training signal.
  • K = 48 and K = 64 perform within noise of each other, motivating the selected operating point.

G High-Fidelity Reconstructions

High-fidelity reconstruction is strongest for first segments, while complete responses also contain substantial cases with low edit distance and exact matches.

  • Table 6 reports high-semantic-similarity, high-lexical-overlap, low-edit-distance, and exact-match fractions across reconstruction settings.
  • 60.43% of ChatDoctor first segments reach ED ≤0.1, including 36.06% exact matches, for Phi-3-mini with Llama.cpp on a laptop.
  • 40.43% of UltraChat first segments reach ED ≤0.1, including 18.36% exact matches, for Phi-3-mini with Llama.cpp on a desktop.
  • 24.83% of complete UltraChat responses reach ED ≤0.1 under the same desktop configuration.

H SMT Coverage Across Intel CPUs

The attack relies on SMT co-scheduling and shared tokenizer code to observe cache activity, with coverage across representative Intel processors but a hybrid-CPU constraint requiring victim execution on P-cores.

  • SMT threat model: The threat model assumes SMT, allowing an adversary to share a core’s sibling logical processor and observe contention in the shared L1 data cache.
  • SMT coverage: Intel processors from the 6th–14th generations provide two logical processors per physical core in the representative coverage summarized by Table 7.
  • Hybrid processors: On hybrid processors from the 12th generation onward, SMT exists only on P-cores, so the attack requires the victim to occupy a P-core and yields fewer traces.
  • Tokenizer decode paths: The decode paths in Llama.cpp and HuggingFace Tokenizers contain token-dependent lookup operations used by the attack.
  • Reconstruction examples: Figures show successful and failed full reconstructions and paragraph-level reconstructions from Code Alpaca, UltraChat, and ChatDoctor.
Loading 2609.06674v1…