PaperlayerView exact revision on arXiv

Source-linked AI summary

Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures

Mengjia Yan, Christopher Fletcher, Josep Torrellas

arXiv:1808.04761v1cs.DCcs.CRcs.LG

TL;DR

DNN architectures are confidential, valuable, and difficult to recover because they govern network structure and create an intractable search space. Cache Telepathy uses cache side channels to observe tiled GEMM behavior, map matrix parameters to architecture parameters, and prune candidate architectures. Across VGG and ResNet running OpenBLAS and Intel MKL, the attack substantially reduces the search space, including from more than 10^35 architectures to 16 for VGG with OpenBLAS.

  • Problem

    DNN architecture is confidential and valuable, but its many hyper-parameters make recovery through brute-force search difficult.

  • Method

    Cache Telepathy uses Prime+Probe and Flush+Reload to extract GEMM matrix parameters from cache behavior and reverse-engineer DNN architectures.

  • Results

    The attack substantially reduces architecture search spaces across VGG and ResNet on OpenBLAS and Intel MKL, including from more than 10^35 architectures to 16 for VGG with OpenBLAS.

  • Takeaways & Limitations

    Cache Telepathy makes otherwise intractable DNN architecture search spaces tractable enough to support architecture recovery.

  • Takeaways & Limitations

    The attack cannot determine exact matrix dimensions when a dimension is less than twice the block size, providing only a range.

Abstract

from arXiv · show

Deep Neural Networks (DNNs) are fast becoming ubiquitous for their ability to attain good accuracy in various machine learning tasks. A DNN's architecture (i.e., its hyper-parameters) broadly determines the DNN's accuracy and performance, and is often confidential. Attacking a DNN in the cloud to obtain its architecture can potentially provide major commercial value. Further, attaining a DNN's architecture facilitates other, existing DNN attacks. This paper presents Cache Telepathy: a fast and accurate mechanism to steal a DNN's architecture using the cache side channel. Our attack is based on the insight that DNN inference relies heavily on tiled GEMM (Generalized Matrix Multiply), and that DNN architecture parameters determine the number of GEMM calls and the dimensions of the matrices used in the GEMM functions. Such information can be leaked through the cache side channel. This paper uses Prime+Probe and Flush+Reload to attack VGG and ResNet DNNs running OpenBLAS and Intel MKL libraries. Our attack is effective in helping obtain the architectures by very substantially reducing the search space of target DNN architectures. For example, for VGG using OpenBLAS, it reduces the search space from more than $10^{35}$ architectures to just 16.

1. INTRODUCTION

DNN architectures are valuable but difficult to steal because they shape accuracy and performance while spanning a rapidly growing, infeasible search space. Cache Telepathy uses cache leakage from tiled GEMM operations to reduce that space and evaluates the attack across major DNNs and libraries.

  • Motivation: DNN architectures define network shape and broadly determine accuracy and performance, making them commercially valuable and costly to reproduce through hyper-parameter tuning.Known architectures also facilitate model extraction and membership inference attacks.
  • Challenge: A multitude of hyper-parameters and an expanding design space make brute-force architecture stealing unfeasible.
  • Approach: Cache Telepathy exploits cache leakage from tiled GEMM calls whose counts and matrix dimensions reflect DNN architecture parameters.
  • Evaluation: The attack targets general-purpose processors used for DNN inference and applies Prime+Probe and Flush+Reload to VGG and ResNet on OpenBLAS and Intel MKL.
  • Contributions: The study analyzes GEMM-to-architecture mappings, introduces a cache-based architecture-extraction attack, and evaluates it across VGG and ResNet with two libraries.

2. BACKGROUND

DNN architectures comprise layer structure, connections, layer-specific hyper-parameters, and activation functions, while weights determine computation values. Prior privacy attacks commonly require the architecture, and cache attacks exploit processor sharing to infer victim activity.

  • DNN Architecture: DNN architectures specify layer count and types, connections, per-layer hyper-parameters, and activation functions.Examples include neurons, filters, filter size, striding, shortcuts, branches, ReLU, and sigmoid.
  • DNN Usage: DNN weights specify the operands of multiply-accumulates, while training searches for weights and hyper-parameter tuning searches across architectures.
  • Prior Privacy Attacks: Model extraction, membership inference, and hyper-parameter stealing attacks assume knowledge of the victim architecture.
  • Cache Side Channels: Flush+Reload requires shared code or data, whereas Prime+Probe infers victim cache-line access using conflict addresses and access latency.
  • Assumptions: The attack assumes processor co-location with the victim and the ability to analyze the victim’s machine-learning framework and linear-algebra libraries.

3. ATTACK OVERVIEW

Naive architecture reverse engineering repeatedly trains candidate networks over an often intractable search space. Cache Telepathy narrows that space by extracting GEMM matrix parameters through cache behavior and mapping them back to DNN hyper-parameters.

  • Challenge of Reverse-Engineering DNN Architectures: Architecture stealing searches candidate networks trained on synthetic data and oracle outputs until one achieves sufficient prediction accuracy.
  • Challenge of Reverse-Engineering DNN Architectures: The search is compute-intensive and often intractable because training and validating many complex candidate networks can take hours to days.
  • Overall Cache Telepathy Attack Procedure: Tiled GEMM implementations expose matrix parameters through cache behavior because public or deducible block sizes let attackers count blocks.
  • Overall Cache Telepathy Attack Procedure: The GEMM-to-DNN mapping analysis is designed to apply across major machine-learning frameworks, including TensorFlow, Caffe, Theano, and MXNet.
  • Overall Cache Telepathy Attack Procedure: Cache Telepathy first monitors matrix multiplications, then reverse-engineers architecture parameters and prunes remaining hyper-parameter values.

4. MAPPING DNNS TO MATRIX PARAMETERS

The paper maps fully-connected and convolutional DNN computations to matrix multiplications, then links matrix-operation parameters to layer hyper-parameters. These mappings support architecture inference from observed GEMM execution.

  • 4.1 Layer Computations: DNN inference in fully-connected and convolutional layers can be represented using matrix multiplications.Fully-connected layers use matrix-vector or batched matrix-matrix operations, while convolutional layers can be transformed into a single matrix multiply.
  • 4.1 Layer Computations: A convolutional layer reshapes input subvolumes into matrix columns and filters into matrix rows, converting convolution into a matrix multiply.The transformed input has Di × Ri^2 elements per column, while the filter matrix has one row per filter.
  • 4.2 Mapping Hyper-Parameters: For batched convolutional inference, each convolutional layer performs B matrix multiplications, unlike fully-connected layers, which process the batch with one matrix multiplication.The batch size can therefore be inferred from repeated consecutive matrix multiplications with identical dimensions.
  • 4.2 Mapping Hyper-Parameters: One fully-connected layer performs one matrix multiplication, whose weight-matrix dimensions encode the layer’s neuron counts.The number of rows and columns in the weight matrix correspond to the neuron counts in adjacent layers.
  • 4.2 Mapping Hyper-Parameters: Convolutional-layer parameters map to GEMM dimensions: filter count becomes filter-matrix rows, filter width derives from matrix-row ratios, and padding affects input-matrix size.The mapping assumes equal filter width and height; input depth is obtained from the preceding layer or public image information for the first layer.
  • 4.2 Mapping Hyper-Parameters: Pooling presence and dimensions can be inferred from changes between convolutional output and subsequent input spatial dimensions, but non-unit stride is indistinguishable from pooling.Equal spatial dimensions indicate no pooling; otherwise, the reduced dimensions reveal the pool size under the stated procedure.

4.3 Connections Between Layers

The paper extends GEMM-based inference to connections between layers, including branches, shortcuts, and sequential links. Latency, dimensional matching, and integer constraints provide signals for identifying connection structure.

  • 4.3.1 Mapping shortcut/branch connections: Shortcut or branch sinks produce relatively longer inter-GEMM latencies because additional processing occurs between GEMM operations.The latency is generally related to the current layer’s output size and the next layer’s input size.
  • 4.3.1 Mapping shortcut/branch connections: A shortcut or branch connects layers whose output dimensions match, providing a dimensional condition for identifying non-sequential connections.The source and sink must have the same output dimension for their outputs to be combined.
  • 4.3 Connections Between Layers: The mapping learns filter spatial dimensions, but non-square filters increase the search space because different factor combinations can produce the same observed product.The analysis notes that filters in modern DNNs are nearly always square.
  • 4.3.2 Mapping consecutive connections: For consecutive convolutional layers, input and output matrix row counts must satisfy a square-of-an-integer constraint derived from integer filter dimensions.If the constraint is violated, the two layers are certainly not consecutively connected.
  • 4.3.2 Mapping consecutive connections: A second square-of-an-integer constraint applies to matrix column counts because pool and stride sizes are integers.Together, the row and column constraints help locate non-sequential connections.
  • 4.3 Connections Between Layers: Activation functions are not encoded in GEMM parameters, so cache monitoring of a sigmoid-function probe distinguishes sigmoid and tanh from relu activations.Nearly all convolutional layers are stated to use relu or a close variant.

5. ATTACKING MATRIX MULTIPLY

The attack extracts blocked GEMM matrix dimensions by monitoring cache-visible function invocations and execution times in OpenBLAS. It exploits loop structure, packing behavior, and known block sizes to infer matrix parameters, while remaining limited for dimensions smaller than twice a block size.

  • Attack objective: Cache observations of blocked GEMM enable reconstruction of matrix dimensions and, through the earlier mapping, DNN architectures.The attack targets matrix-multiplication parameters as an intermediate representation of DNN architecture.
  • OpenBLAS GEMM: OpenBLAS’s gemm_nn computes C = αA·B+βC for matrices with dimensions m × k, k × n, and m × n.The extraction target is m, n, and k.
  • Loop structure: The blocked implementation uses three nested loops whose iteration counts are iter3, iter2, and iter1, with Loop 3 innermost and Loop 1 outermost.Loop 3 is further split into two parts to improve cache locality.
  • Dimension extraction: The procedure counts dynamic-call-graph patterns, derives loop iteration counts, and uses block-size timing differences to estimate m, k, and n.OpenBLAS block sizes and UNROLL are known constants, while smaller final blocks reveal remainder information.
  • Invocation patterns: Each Loop 2 iteration invokes itcopy once, oncopy and kernel iter4 times, then itcopy and kernel iter3 −1 times; the sequence repeats iter1 × iter2 times.These invocation patterns provide cache-probe signatures for recovering loop counts.
  • Limitation: The attack cannot determine the exact dimension when a matrix size is less than twice its block size, providing only a range.In that case, no iteration operates on a full block.

6. GENERALIZATION OF THE ATTACK

Cache Telepathy generalizes beyond OpenBLAS by exploiting common blocked-matrix-multiplication structures, demonstrated on the closed-source Intel MKL library. The same dynamic-call-graph pattern and side-channel strategy support matrix-parameter extraction, with special cases handled using iteration counts and timing.

  • Scope: The attack generalizes to other BLAS libraries because they use blocked matrix multiplication and most implement Goto’s algorithm.Intel MKL is used as a challenging closed-source example.
  • MKL analysis: For MKL, side-channel-assisted analysis identifies packing and kernel functions, their invocation patterns, and the block sizes needed for extraction.The process combines binary analysis with cache-based probing.
  • MKL result: The MKL dynamic-call graph matches OpenBLAS’s pattern, so the OpenBLAS attack strategy also applies to MKL.The pattern is derived both manually and through side-channel probing.
  • Block-size extraction: Block sizes are recovered by increasing an input dimension until the observed iteration count increments, using the triggering stride as the block size.The method relies on the discrete linear relationship between matrix dimensions and iteration counts.
  • Special cases: For special cases, iteration counts reveal the largest dimension, while per-iteration timing estimates the product of the other two dimensions.The attack adjusts the Figure 5 strategy for these cases.

7. EXPERIMENTAL SETUP

The experiments evaluate Cache Telepathy on a four-core Intel Xeon workstation under Ubuntu, using representative VGG and ResNet models. The victim implementations use Keras with Theano as the backend.

  • Platform: The evaluation uses a Dell Precision T1700 with a 4-core Intel Xeon E3 processor and an 8MB shared last-level cache.The system also has private 32KB instruction and data caches and a 256KB second-level cache.
  • Platform: The experiments use a same-OS Ubuntu 4.2.0-27 scenario.The attack platform includes 8GB DDR3-1600 memory.
  • Models: VGG evaluation focuses on VGG-16, selected from VGG variants ranging from 11 to 19 layers.All VGG variants contain five blocks with replicated layer types within each block.
  • Models: ResNet evaluation focuses on ResNet-50, selected from variants whose depth ranges from 18 to 152 layers.ResNet variants use four module types replicated different numbers of times.
  • Software: Victim programs are implemented with Keras using Theano as the backend.

8. EVALUATION

The evaluation applies cache attacks to GEMM and DNN inference, extracting matrix parameters and using them with DNN constraints to reverse-engineer connections and shrink architecture search spaces.

  • 8.1 Attacking GEMM Using Prime+Probe: Prime+Probe traces reveal GEMM execution structure, including itcopy and oncopy calls, allowing loop iterations to be derived.One trace identifies 11 oncopy calls and three itcopy calls from victim activity intervals.
  • 8.1 Attacking GEMM Using Prime+Probe: Prime+Probe achieves accuracy comparable to Flush+Reload for extracting matrix parameters, so either attack can recover DNN hyper-parameters.The evaluation uses Flush+Reload for illustration after establishing Prime+Probe’s comparable accuracy.
  • 8.2 Extracting Parameters from DNNs: The attack extracts n, k, and m parameters for selected ResNet-50 modules and VGG-16 blocks, representing duplicate layers only once.Figures distinguish actual, detected, and DNN-constraint-deduced parameter values.
  • 8.2 Extracting Parameters from DNNs: The n dimension is determined with ignorable error across evaluated layers.The paper attributes this to the small subblock size used in the first iteration of Loop 3.
  • 8.2 Extracting Parameters from DNNs: The attack accurately determines m and k for most ResNet-M1, ResNet-M4, and all VGG blocks, but usually returns ranges for ResNet-M2 and ResNet-M3.The ranges arise because those layers’ m and k values are often smaller than twice the corresponding block sizes.
  • 8.3 Size of Architecture Search Space: DNN constraints help identify layer connections and eliminate parameter imprecision, reducing an intractable architecture space to a reasonable size.For ResNet-M1, inferred connections exactly match the actual connections; VGG with OpenBLAS shrinks from more than 10^35 architectures to 16.

9. COUNTERMEASURES

The paper considers library changes, smaller matrix dimensions, resource-isolation defenses, cache partitioning, and security-oriented cache mechanisms, while noting performance trade-offs and incomplete coverage.

  • Library modifications: Less aggressive BLAS blocking may hinder precise recovery of n and k, but abandoning blocking entirely is considered impractical because of poor cache performance.The proposed library modification removes an optimization while accepting tolerable performance degradation.
  • Reducing matrix dimensions: Reducing matrix dimensions can force range estimates, but this mitigation mainly protects later convolutional layers and not layers with large matrices.Linear quantization is given as one way to reduce weight and input precision and therefore matrix size.
  • Cache side-channel defenses: Disallowing server sharing has major throughput disadvantages, while disabling page sharing defeats Flush+Reload but not Prime+Probe.The countermeasures therefore differ in both deployment cost and attack coverage.
  • Cache side-channel defenses: Cache partitioning can protect code and data, but reduced LLC capacity causes performance degradation; the paper presents this as a possible security-performance trade-off.The GEMM block size must be adjusted to the reduced number of available LLC ways.
  • Cache side-channel defenses: Security-oriented cache mechanisms could mitigate the attack with moderate performance degradation if adopted in production hardware.The paper also suggests adding security features to performance-oriented partitioning proposals.

10. RELATED WORK

Prior work includes a hardware-accelerator attack requiring comprehensive memory-access monitoring, while existing cache side channels had not extracted all DNN hyper-parameters.

  • Architecture extraction: Hua et al.’s CNN-architecture attack targets hardware accelerators and assumes visibility into every victim memory access, including access type and address.Cache Telepathy uses a less privileged threat model than that attack.
  • Cache side channels: Before Cache Telepathy, cache side-channel attacks had not extracted all DNN hyper-parameters despite extensive use against cryptographic algorithms.The paper attributes the challenge to complex DNN computations, GEMM’s multi-level loops, and the number of parameters involved.

11. CONCLUSION

Cache Telepathy analyzes blocked GEMM and uses cache side channels to extract matrix parameters from complete DNNs, substantially reducing architecture search spaces.

  • 11. CONCLUSION: Cache Telepathy attacks VGG and ResNet running OpenBLAS and Intel MKL with Prime+Probe and Flush+Reload.The attack extracts GEMM input matrix parameters and scales the analysis to complete DNNs.
  • 11. CONCLUSION: For VGG using OpenBLAS, the attack reduces the architecture search space from more than 10^35 architectures to 16.The paper presents this as evidence that the attack substantially reduces the search space for obtaining target architectures.
Loading 1808.04761v1…