Source-linked AI summary
LLMscope: Extracting LLM Assets from Edge AI Chips via Optical Probing
Dev Mehta, Lily Dukette, William Folan, Olivia Kochol, Noah Solomon, Shahin Tajik, Fatemeh Ganji
TL;DR
The paper examines the physical exposure of model and inference-state assets during LLM execution on edge accelerators. It uses EOFM to recover values from localized FPGA memories and compute structures, supplementing incomplete optical readout with algebraic and downstream-state methods. The experiments demonstrate complete targeted-value recovery and show that attack effort scales with asset size while shared hardware enables reuse.
Problem
Edge LLM accelerators repeatedly process model parameters and inference states on physically accessible chips, creating a need to assess their exposure to physical side-channel attacks.
Method
The paper characterizes FPGA LLM asset flows, uses EOFM for direct bit-level readout, and applies linear input-output or downstream-state constraints to complete incomplete recovery.
Results
The experiments demonstrate complete bit-level recovery of targeted model and inference-state assets and recovery of many incomplete weight configurations.
Takeaways & Limitations
Shared memories and compute boundaries can be reused across addresses, tiles, modules, and layers, so attack scaling does not require independently localizing every model bit.
Takeaways & Limitations
Hybrid recovery must account for quantization scales, zero points, group boundaries, packing rules, and rounding, and some nonlinear boundaries do not permit unique inversion from outputs alone.
Abstract
from arXiv · showhide
The move of LLM inference to edge AI accelerators introduces new physical vulnerabilities. During execution, model parameters and intermediate inference states are repeatedly loaded into and processed on the chip, making them suscep- tible to physical side-channel attacks. In this work, by deploying laser voltage imaging, we show that one can extract LLM assets during inference, namely embeddings, attention, and quantized MLP weights, activations, and other inference states, from localized memories and compute subcircuits. To validate our claims, we perform an attack on an FPGA-based LLM accelerator. Since such accelerators reuse the same buffers and compute subcircuits across addresses, tiles, modules, and layers, reading asset values comes down to probing different memories during inference. We demonstrate full recovery of the targeted values; however, we also establish a methodology to recover asset values even if some weights or bits remain unread. We further derive lower bounds that relate imaging effort to asset dimensions and show that even direct recovery scales linearly with the size of the targeted asset
1 Introduction
Edge-deployed LLM accelerators expose model and inference-state assets to physical probing because these values are repeatedly processed in on-chip memories and compute structures. The paper introduces EOFM-based direct bit-level recovery and complementary algebraic and downstream-state methods for incomplete readout.
- Motivation: Edge deployment places model parameters and user-dependent inference states on hardware that may be physically accessible to adversaries.The security boundary shifts from external storage to on-chip execution.
- Threat surface: LLM assets at risk include embeddings, attention and MLP projections, normalization parameters, vocabulary projections, activations, KV-cache entries, partial sums, and logits.Conventional power or electromagnetic side channels can obscure individual assets because many structures operate simultaneously.
- Approach: EOFM localizes FPGA structures and recovers complete binary representations of targeted model and inference-state values during controlled inference.Repeated execution exposes different addresses, streamed words, tiles, modules, and layers through the same localized resources.
- Recovery beyond direct readout: Incomplete optical coverage can be supplemented by Gaussian elimination for missing linear entries and exact downstream-state checks for unresolved bits.These observations serve as anchors across the deterministic inference graph.
- Scalability: The attack scales with asset dimensions, while hardware reuse amortizes localization across assets that traverse shared memories and compute subcircuits.The paper derives implementation-aware bounds incorporating simultaneous bit coverage, replay, and physical resource reuse.
2 Foundations and Asset-Level Security Model
The paper frames FPGA LLM inference as an asset-flow problem in which model parameters and inference-state values move through reused on-chip memories and compute structures. It defines the on-chip lifetime as the relevant protection boundary and combines direct EOFM readout with algebraic and consistency-based recovery when coverage is incomplete.
- Transformer Modules and Sensitive Assets: Model assets include embeddings, attention and MLP weights, normalization parameters, vocabulary projections, and representation metadata.Inference-state assets include Q, K, V, attention values, hidden vectors, partial sums, and KV-cache entries.
- Asset Flow in FPGA LLM Inference: FPGA LLM inference repeatedly transfers model parameters and inference state between external memory, on-chip storage, and compute structures.Implementations use memories and buffers such as BRAM, URAM, SRAM, FIFOs, and register files, with platform-specific hierarchies.
- Asset Flow in FPGA LLM Inference: The relevant protection boundary is the on-chip lifetime of an asset, including transient states that exist there only briefly.External-memory protection does not cover values after conversion into accelerator-specific representations and placement in buffers, registers, lookup structures, or datapaths.
- EOFM Observation and Adversary Model: EOFM can localize FPGA structures, identify physical bit positions, and recover complete binary representations of targeted model or inference-state assets under controlled execution.The paper defines an asset as EOFM-readable when its deployed representation can be reconstructed from recovered bits while present in an observable on-chip structure.
- Direct vs. Indirect Asset Recovery: For linear modules, directly recovered entries reduce the unknowns, and the remaining values can be solved from sufficiently independent exact input-output observations.If u_j entries remain unknown in a column, m ≥ u_j observations suffice in the noiseless full-column-rank case; quantized recovery must also account for scales, zero points, packing, and rounding.
- Direct vs. Indirect Asset Recovery: Hybrid recovery is advantageous when direct coverage of the complete weight matrix costs more than imaging the readable portion plus enough exact module input-output pairs.Its cost depends on the recovered fraction, boundary imaging cost, observed-input rank, and the accelerator’s arithmetic representation.
3 Asset-Level Security Assessment
The assessment maps how FPGA LLM accelerators expose model and inference-state assets at materialized memory and compute boundaries. It also shows that direct and hybrid recovery scale with asset size while hardware reuse and exact state observations can reduce recovery cost.
- Embeddings: Embedding vectors can be recovered directly from EOFM-readable staging structures by repeating executions with different token identifiers.Quantized or packed embeddings additionally require recovery of representation metadata.
- Attention: Attention projection weights and intermediate states can be read from reused transfer, buffer, register, and processing-element boundaries.Recovered context vectors and projected outputs can also supply the input-output pairs needed for hybrid completion of partially observed projections.
- MLP and elementwise state: MLP weight representations remain directly readable despite following nonlinear activations, while hybrid recovery requires linear pre-activation outputs for W1 and Wgate and activated inputs for W2.This makes MLP matrices a central hybrid-recovery target because their weights are large but their relevant vectors can be smaller.
- Vocabulary projection and logits: Vocabulary projections can be recovered tile by tile, and exact hidden vectors with corresponding logits can provide input-output pairs for missing matrix entries.The large vocabulary dimension makes complete direct coverage expensive, while tied embedding and vocabulary weights can reduce the distinct assets involved.
- Recovery scaling: After localization, direct coverage requires N(Z) ≥ R⌈S/w⌉, so imaging effort grows with target size and decreases when each replay state reveals more bits.The bound applies across addresses, streamed words, time-multiplexed tiles, and fields of view.
- Recovery scaling: A 4096 × 4096 projection quantized to 4 bits contains 67,108,864 bits, requiring at least 131,072 replay states on a fully packed 512-bit path.The corresponding 4096-element activation needs 32 states, illustrating why hybrid recovery can be attractive when compact exact state boundaries are repeatedly available.
4 Experiments and Results
Experiments demonstrate direct EOFM recovery from FF and BRAM storage boundaries, then test algebraic and downstream-state recovery when optical coverage is incomplete.
- Direct Asset Recovery from FFs: The Kintex-7 setup used backside EOFM to image FF and BRAM structures carrying LLM inputs, outputs, and buffered data.The experiments used 50× magnification and 12.5 MHz operation for FFs and BRAMs.
- Direct Asset Recovery from FFs: Complete input and output representations were reconstructed from EOFM maps of matrix-multiplier registers.Across four input conditions, decoded values matched the programmed identity matrix.
- Direct Asset Recovery from BRAMs: BRAM EOFM revealed logical-to-physical bit ordering, including exchanged bits 1 and 2 within each 4-bit group.The 0xFF, 0xAA, and 0x22 patterns exposed active positions and ordering.
- Direct Asset Recovery from BRAMs: A 16-bit BRAM output remained directly decodable across two 8-bit rows, recovering multiple asset bits from one EOFM map.Patterns 0x1111, 0x2222, 0x4444, 0x8888, and 0xFFFF identified ordering and active positions.
- Recovery Beyond Complete Readout: When input-output observations lacked diversity, all tested subsets constrained only sums of matrix entries rather than uniquely determining the matrix.Every stacked input matrix had rank one, while the resulting system had rank two regardless of retaining 25%, 50%, 75%, or 100% of observations.
- Recovery Beyond Complete Readout: Incomplete recovery nevertheless succeeded: 99.46%, 94.90%, and 82.87% of tested cases were uniquely recovered with two, four, and eight missing weight bits, respectively.These cases hid eight output bits per downstream state; the eight-bit case began with 256 weight completions.
- Recovery Beyond Complete Readout: A single downstream observation exactly recovered every tested input completion across 856 masking configurations, including bytes with all eight bits hidden.The all-hidden-byte case started with 256 possible values before downstream constraints were applied.
5 Discussion
The discussion frames the attack around reusable storage primitives rather than semantic LLM modules, while noting portability requirements and larger-memory targeting opportunities.
- Portability across FPGA families and embedded platforms: The recovery procedure can transfer across FPGA families when sensitive assets occupy comparable FF, register, BRAM-like, or SRAM-like structures.Physical locations and EOFM parameters still require characterization for each FPGA family.
- Portability across FPGA families and embedded platforms: URAM primitives offer 288 Kb capacity, eight times the 36 Kb capacity of BRAM, and dedicated columns that may simplify optical localization.This advantage is presented as a favorable targeting opportunity for newer FPGA families.
- Implication for larger LLM accelerators: AXI-loaded assets ultimately reach the same local storage boundaries, so the attack targets where values are staged rather than the transport protocol.The relevant structures include registers, FFs, BRAM, URAM, and other local memories.
- Implication for larger LLM accelerators: Characterized memory and register boundaries can be reused as different assets traverse them, reducing the need to localize every model bit independently.Algebraic and downstream constraints can recover information left incomplete by direct EOFM.
6 Conclusion
The paper demonstrates optical access to FPGA-resident LLM assets and combines direct EOFM readout with algebraic and downstream constraints for incomplete recovery.
- Conclusion: FPGA-based LLM inference creates a direct optical path to model and inference-state assets staged and processed on chip.Demonstrated targets include weights, embeddings, activations, KV-cache entries, and intermediate results.
- Conclusion: EOFМ recovers complete values from reusable FF and BRAM boundaries, while incomplete values can be recovered through rank-sufficient linear systems or exact downstream states.Repeated reuse across addresses, tiles, modules, and layers supports scaling the attack across assets.