PaperlayerView exact revision on arXiv

Source-linked AI summary

JITterFlip: Uncovering Fault Attack Surfaces in JIT-Compiled LLM Serving

Tairui Wang, Zhi Zhang, Yansong Gao, Xin Zhang, Qingni Shen, Zhonghai Wu

arXiv:2608.29745v1cs.CR

TL;DR

Existing BFAs largely target model state or computation code, leaving host-side JIT serving decisions in GPU-based LLM inference insufficiently studied. JITterFlip uses decision-guided analysis to find CPU-resident control branches and demonstrates transferable gibberish and correct-output sponge attacks, including end-to-end Rowhammer effects across four LLMs.

  • Problem

    Existing BFAs mainly target model parameters or computation code, leaving the host-side JIT serving control plane of GPU-based LLM inference as an unexplored attack surface.

  • Method

    JITterFlip combines semantic localization, decision-guided static analysis, and binary-to-hardware fault mapping to target branches governing compiled-artifact selection and GPU-work submission.

  • Results

    Across four LLM workloads, faults transfer across models, produce 15.45× to 2.48×10^6× PPL ratios or 2.03× to 181.90× latency amplification, and bypass FaR and LM-Fix.

  • Takeaways & Limitations

    CPU-resident faults in JIT serving decisions can disrupt GPU-executed inference, yielding gibberish or severe latency amplification while preserving correct output.

  • Takeaways & Limitations

    The evaluation covers PyTorch 2.9.1 and one non-ECC DDR4 platform, leaving cross-framework, cross-backend, ECC, and newer DDR5 validation for future work.

Abstract

from arXiv · show

LLMs are widely deployed through cloud-hosted inference services, where Just-in-Time (JIT) compilation is used to reduce recurring framework and GPU-launch overhead. JIT serving introduces a host-side control plane that selects compiled artifacts and orchestrates their execution on the GPU. Meanwhile, the shared cloud setting has motivated a growing body of bit-flip attacks (BFAs) against LLM/DNN inference. Most existing BFAs target model parameters or weights and require model-specific knowledge. A smaller body of work reduces this dependency by faulting executable code, yet still corrupts code that directly implements model computation, limiting their attack effect to inference depletion. We present JITterFlip, the first BFA targeting the host-side JIT serving control plane of GPU-based LLM inference. By faulting CPU-resident serving decisions rather than model computation, JITterFlip enables both gibberish output generation and a correct-output sponge attack. To identify exploitable targets in a large JIT compiler stack, JITterFlip develops a decision-guided fault-vulnerable code analysis. Across four text and multimodal LLM workloads, the identified vulnerable code faults exhibit cross-model transferability, produce gibberish outputs with PPL ratios of $15.45\times$ to $2.48{\times}10^{6}\times$, and demonstrate correct-output sponge attacks with latency amplification of $2.03\times$ to $181.90\times$. JITterFlip also bypasses recent BFA defenses for LLMs while retaining both attack effects. Last, we demonstrate end-to-end Rowhammer attacks across four LLMs: a single bit flip in CPU-resident branch code propagates across the CPU-GPU boundary to disrupt GPU-executed inference without direct access to GPU memory, reaching up to $7.23{\times}10^{6}\times$ PPL amplification or $124.97\times$ latency amplification while preserving the exact generated output.

1 Introduction

JITterFlip shifts bit-flip attacks from model computation to the host-side JIT serving control plane, enabling both corrupted outputs and correct-output latency attacks. Its decision-guided analysis finds transferable vulnerable branches, and evaluations show severe effects across four LLM workloads and recent defenses.

  • Motivation: JIT compilation reduces recurring framework and GPU-launch overhead by reusing specialized compiled execution during LLM serving.The serving stack combines compiled-code caching, shape specialization, and CUDA-Graph execution.
  • Related gap: Prior executable-code BFAs corrupt model-computation code, limiting their demonstrated effects to inference depletion and requiring output-based detection.Examples include faults in OpenBLAS and compiler-generated model executables.
  • Contribution: JITterFlip targets host-side decisions that select compiled artifacts and submit GPU work, enabling either gibberish outputs or correct-output sponge attacks.Artifact-selection faults redirect execution to slower valid paths, while submission faults suppress or misdirect required GPU computation.
  • Method: The analysis reduces the PyTorch JIT stack from 181 files and 139.1K SLoC to 34 files and 34.7K SLoC of decision-relevant control logic, then ranks 36,861 conditional branches.Separate rankings identify branches associated with output corruption and latency amplification.
  • Results: 15.45× to 2.48×10^6× PPL ratios or 2.03× to 181.90× latency ratios occur across four text and multimodal LLM workloads, with cross-model transferability.JITterFlip also bypasses FaR and LM-Fix while retaining both attack effects.
  • Results: 287.3× to 7.23×10^6× PPL ratios or 97.38× to 124.97× latency amplification result from end-to-end Rowhammer attacks across four LLMs.The attacks flip CPU-resident branch code and affect GPU-executed inference without direct GPU-memory access.

2 Background and Related Work

The background distinguishes JIT serving and Rowhammer mechanics from prior BFA targets and defenses. JITterFlip instead attacks serving-control code, including a correct-output sponge effect that output-integrity defenses may miss.

  • Background: LLM serving uses GPU computation while the host CPU selects, prepares, and submits work; JIT compilation caches artifacts guarded by runtime conditions.Guard evaluation can trigger artifact reuse, recompilation, or fallback execution.
  • Rowhammer: Rowhammer repeatedly accesses aggressor rows to accelerate charge leakage in nearby victim rows and induce DRAM bit flips.Double-sided hammering alternates between two aggressor rows neighboring a victim row.
  • Related work: Prior BFAs primarily corrupt model parameters, weights, or computation-plane executable code, with end-to-end attacks often depleting inference.FrameFlip targets OpenBLAS control-flow instructions, while Compiled Models targets compiler-generated executable text.
  • Novelty: JITterFlip targets JIT-generated serving-control code and can preserve model outputs while significantly increasing inference latency.Its CPU-side faults can affect GPU-executed inference, matching GPU-dominant LLM serving environments.
  • Defenses: Output-integrity defenses can detect inference depletion but are insufficient for sponge attacks that preserve correct outputs while redirecting execution to expensive paths.Detecting the sponge effect requires monitoring serving-time execution behavior.
  • Defenses: BitShield protects compiler-generated DNN executables but does not cover JIT serving-control logic or sponge attacks without substantial redesign.The redesign would need to address attacks that do not change model semantics.

3 Threat Model

The threat model assumes an unprivileged co-resident attacker using Rowhammer against a GPU-based PyTorch JIT service. The attacker lacks victim-model knowledge but relies on publicly available compiler-stack knowledge and susceptible DRAM.

  • Attacker setting: The attacker is software-only, unprivileged, and co-resident with the victim on shared physical hardware.The OS and hypervisor are assumed correct and process isolation is enforced.
  • System assumptions: The attack assumes DRAM susceptible to Rowhammer-induced bit flips and targets memory-resident code in a GPU-based JIT-enabled LLM service.The stated deployment settings include shared servers, multi-tenant clouds, and ML-as-a-service platforms.
  • Knowledge assumptions: The attacker needs no knowledge of the victim model’s architecture, weights, training data, or prompts.The targeted PyTorch compiler stack is assumed known through publicly available source code.

4.1 Overview

JITterFlip studies CPU-side JIT control decisions that can be faulted without changing model computation. Its analysis maps serving semantics to vulnerable branches and Rowhammer-compatible faults that produce either gibberish or correct-output latency amplification.

  • Overview: JITterFlip targets two effects: gibberish-output generation and correct-output sponge attacks that preserve output while increasing inference latency.The attack is designed to work without knowing model architecture, weights, training data, or prompts.
  • Overview: The PyTorch torch.compile lifecycle spans host control code, generated GPU kernels and launchers, and CUDA-Graph replay, creating a fault surface in serving decisions.The targeted decisions concern compiled-artifact selection and GPU-work submission.
  • Analysis: Decision-guided analysis combines manual semantic characterization with automated static analysis to locate branches associated with inference-critical control behavior.It abstracts five recurring conditional-branch archetypes and searches for matching branches.
  • Fault mapping: Rowhammer exploitability requires matching profiled flippable DRAM bits and directions to binary-level branch-flip records.A matched branch inversion changes an execution-critical decision and determines the attack effect.

4.2 Execution-Critical Decisions in JIT LLM Serving

JIT serving relies on host-side decisions to select compiled artifacts and submit their work to the GPU. These decisions expose two execution-critical control families: compiled-artifact selection and GPU-work submission.

  • Execution pipeline: The CPU host-side control plane governs which computation executes and how it is issued, while computationally intensive operators run on the GPU.This CPU–GPU collaboration defines the execution-critical surface studied by JITterFlip.
  • Compiled-artifact selection: The host runtime selects whether and which cached compiled artifact serves each execution as runtime conditions change.It may reuse a valid artifact, try another, recompile, or fall back to a general execution path.
  • GPU-work submission: GPU-work submission prepares and issues the selected computation from the CPU to GPU kernels or CUDA-Graph replays.Generated wrappers and launchers prepare arguments and launch state before submission.

4.3 Decision-Guided Fault-Vulnerable Code Analysis

JITterFlip narrows a large JIT compiler stack to decision-relevant host control, then uses semantic branch archetypes and static analysis to identify fault-vulnerable conditional branches.

  • Decision-relevant source localization: Differential source analysis compares JIT and eager execution to identify JIT-dependent stages and their control responsibilities.Eager execution serves as a baseline for isolating JIT-dependent host-side control.
  • Decision-relevant source localization: The filtering procedure retains stages whose reconstructed responsibilities influence either target decision and resolves them to concrete source routines.ResolveImpl follows interfaces, call relationships, and JIT-specific state toward the decisions.
  • Decision-relevant source localization: Decision-guided filtering reduces the host-side JIT analysis scope from 181 files and 139.1K SLoC to 34 files and 34.7K SLoC.The retained routines implement control relevant to compiled-artifact selection or GPU-work submission.
  • Decision-relevant branch archetypes: The analysis abstracts branches by predicate provenance, successor-region behavior, and JIT control context to match recurring decision-relevant archetypes.These properties distinguish branches implementing compiled reuse, guard-result handling, and related execution choices.
  • Backward predicate provenance analysis: Backward analysis traces each branch condition to a predicate slice containing the computations that can affect it.The slice retains operations, constants, comparisons, and direct calls explaining the tested runtime value.

4.4 Memory Profiling

Memory profiling identifies physical bit locations that Rowhammer can flip and records the offsets and directions needed to match them to vulnerable branch bits.

  • Memory profiling: Rowhammer-flippable bits are sparse and hardware-dependent, so profiling records candidate physical locations before attack alignment.The process includes DRAM-address mapping, hammer-pattern fuzzing, and tests on candidate physical pages.
  • Memory profiling: A profiled bit is usable only when its in-page offset and observed flip direction match the selected branch inversion.Profiling records are independent of the victim model and victim process.

4.5 Targeted Runtime Bit Flip

The targeted runtime bit-flip procedure places a JIT compiler code page on a matched flippable physical page and then triggers the corresponding Rowhammer-induced opcode change.

  • Target placement: A vulnerable branch record is matched with a profiled physical location by in-page bit offset and flip direction.This matching is the prerequisite for placing and flipping the target branch bit.
  • Target placement: The branch’s file-backed executable code page is placed onto the matched physical page before the bit flip is triggered.Memory waylaying forces the operating system to reload the target code page onto the selected frame.
  • Runtime bit flip: The attack applies the profiled hammering pattern to aggressor pages, modifying the selected opcode bit and reversing the conditional branch.The target branch resides in the JIT compiler stack’s file-backed executable page.

5 Evaluation

JITterFlip evaluates fault-target search, attack severity and transferability, defense robustness, and end-to-end Rowhammer effects across four LLM workloads. Its ranked branch analysis finds vulnerable targets efficiently, while CPU-resident control-plane faults produce both severe output corruption and latency amplification, including under evaluated defenses and Rowhammer execution.

  • Fault-vulnerable Code Search Efficiency: 36,861 conditional branches were analyzed after localizing decision-relevant JIT control logic, and Top-20 candidates were tested for each attack objective under a 20-trial budget.The evaluation uses fixed text and multimodal workloads, with PPL and latency ratios normalized to clean inference.
  • Fault-vulnerable Code Search Efficiency: 0 vulnerable branches were found by sequential traversal and each random trial, whereas JITterFlip found 5 gibberish-capable and 10 latency-amplifying branches.The corresponding precisions were 25.0% and 50.0%, respectively.
  • Attack Severity and Transferability: 15.45× to 2.48×10^6× PPL ratios and 2.03× to 181.90× Latency ratios were achieved by vulnerable branches across the evaluated settings.The arithmetic means were 7.65×10^5× for PPL ratio and 85.48× for Latency ratio.
  • Attack Severity and Transferability: 2 of 5 output-corruption branches transferred to at least three settings, while 6 of 10 latency-amplification branches affected at least three settings.Figure 2 groups branches by the number of model–workload settings in which they achieve the corresponding effect.
  • Attack Effectiveness against BFA Defenses: 35,831.1× to 37,428.9× PPL ratios and 184.2× to 206.1× Latency ratios persisted across unprotected, FaR, and LM-Fix configurations.The gibberish and sponge branch faults kept the process alive; the sponge fault preserved the generated output.
  • End-to-End Rowhammer Attacks: 287.3× to 7.23×10^6× PPL ratios were produced by single CPU-resident branch-bit Rowhammer faults across all four models while completed responses remained available.The corresponding end-to-end sponge attacks preserved the generated token sequence while increasing latency by 97.38×–124.97×.

6 Discussion

JITterFlip motivates layered defenses because output corruption may be detectable, whereas correct-output sponges require latency or runtime monitoring. The paper also identifies limits in runtime coverage, JIT-stack portability, and Rowhammer-platform evaluation.

  • Mitigations: Output checks may detect gibberish, but correct-output sponges require latency monitoring or JIT runtime telemetry.Relevant telemetry includes unexpected recompilation, fallback execution, or kernel-submission activity.
  • Mitigations: Enforcement branches can themselves become fault targets when they remain in the same unprotected code path.
  • Mitigations: Branch-outcome checks and code-page integrity are more appropriate protections than defenses limited to illegal control transfers.Suggested mechanisms include duplicated conditional jumps, secure enclaves, code-page isolation, and code-integrity mechanisms.
  • Limitations: The analysis currently targets PyTorch 2.9.1, so new JIT versions, builds, or frameworks require rerunning localization, IR analysis, and fault mapping.Cross-framework and cross-backend validation remains future work.
  • Limitations: End-to-end Rowhammer evaluation covers one non-ECC DDR4 platform and does not evaluate ECC-protected DDR4 or newer DDR5 systems.The paper leaves validation on newer platforms to future work despite reported ECC and TRR bypasses.

7 Conclusion

JITterFlip is a model-agnostic single-bit Rowhammer attack against the host-side JIT serving control plane. Its faults target branches governing artifact selection and GPU-work submission, causing gibberish or correct-output latency amplification while preserving process liveness.

  • JITterFlip is a model-agnostic single-bit Rowhammer attack against the host-side JIT serving control plane of LLM inference.
  • Decision-guided analysis identifies vulnerable branches governing compiled-artifact selection and GPU-work submission.
  • A single CPU-DRAM fault can propagate to GPU-executed inference, causing gibberish output or correct-output latency amplification while preserving process liveness.
Loading 2608.29745v1…