PaperlayerView exact revision on arXiv

Source-linked AI summary

Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware

Florian Tramèr, Dan Boneh

arXiv:1806.03287v2stat.MLcs.CRcs.LG

TL;DR

Outsourced ML needs integrity and privacy, but TEEs impose a performance cost relative to untrusted hardware. Slalom addresses this by selectively outsourcing DNN linear layers from a TEE to a faster co-located processor, achieving 6× to 20× higher throughput for verifiable inference and 4× to 11× for verifiable and private inference.

  • Problem

    Sensitive and security-critical ML workloads need integrity and privacy when outsourced, while TEEs provide these guarantees with substantial performance costs against untrusted alternatives.

  • Method

    Slalom selectively outsources DNN linear layers from a TEE to a faster untrusted co-located processor, using verifiable matrix multiplication and input blinding.

  • Results

    6× to 20× throughput increases are achieved for verifiable inference and 4× to 11× for verifiable and private inference across VGG16, MobileNet, and ResNet variants.

  • Takeaways & Limitations

    Slalom enables higher-throughput DNN inference in TEEs without compromising the evaluated integrity and privacy guarantees.

  • Takeaways & Limitations

    Slalom does not address private training, and its input-privacy precomputation does not apply when weights change after every training batch.

Abstract

from arXiv · show

As Machine Learning (ML) gets applied to security-critical or sensitive domains, there is a growing need for integrity and privacy for outsourced ML computations. A pragmatic solution comes from Trusted Execution Environments (TEEs), which use hardware and software protections to isolate sensitive computations from the untrusted software stack. However, these isolation guarantees come at a price in performance, compared to untrusted alternatives. This paper initiates the study of high performance execution of Deep Neural Networks (DNNs) in TEEs by efficiently partitioning DNN computations between trusted and untrusted devices. Building upon an efficient outsourcing scheme for matrix multiplication, we propose Slalom, a framework that securely delegates execution of all linear layers in a DNN from a TEE (e.g., Intel SGX or Sanctum) to a faster, yet untrusted, co-located processor. We evaluate Slalom by running DNNs in an Intel SGX enclave, which selectively delegates work to an untrusted GPU. For canonical DNNs (VGG16, MobileNet and ResNet variants) we obtain 6x to 20x increases in throughput for verifiable inference, and 4x to 11x for verifiable and private inference.

1 INTRODUCTION

The paper addresses how to preserve integrity and privacy for outsourced ML while reducing the performance cost of TEEs. Slalom selectively delegates DNN linear layers to a faster untrusted processor and substantially improves verifiable inference throughput.

  • Motivation: TEEs provide privacy and integrity for outsourced ML, but their isolation imposes substantial performance costs relative to untrusted GPUs and server CPUs.SGX also incurs memory overheads, has difficulty exploiting multithreading, and is limited to desktop CPUs.
  • Research question: The paper asks how to split ML computations between trusted and untrusted components to improve efficiency while preserving security.This challenge also seeks to minimize the Trusted Computing Base.
  • Approach: Slalom partially outsources DNN execution from a TEE to a co-located, faster untrusted device rather than relying on cryptographic or fully trusted execution.The co-located setting permits interactive outsourcing protocols with improved efficiency.
  • Approach: Matrix multiplication is the main DNN bottleneck and can be efficiently verified with Freivalds’ algorithm, while privacy can also be added in this setting.The TEE outsources CPU-intensive linear operations and runs the remaining steps itself.
  • Contribution: Slalom provides a TEE-compatible framework that outsources all linear layers to an untrusted GPU without compromising integrity or privacy.The framework is designed for TEEs including Intel SGX and Sanctum.
  • Results: 6× to 20× throughput increases are achieved for verifiable inference, versus 4× to 11× for verifiable and private inference, relative to running computations in SGX.The evaluation covers VGG16, MobileNet, and ResNet variants, alongside a formal security proof.

2 BACKGROUND

The background defines the security goals and baseline for outsourced DNNs, then motivates selective outsourcing of linear layers from TEEs to faster co-located processors. Slalom combines efficient integrity checks with symmetric input blinding, under practical communication and hardware constraints.

  • Security model: The outsourcing model has a client provide input while a server evaluates a DNN, with privacy, integrity, and sometimes model privacy as security goals.The model may belong to either the user or the server.
  • TEE baseline: TEEs isolate enclave computations from host software and support remote attestation, while outperforming cryptographic ML outsourcing under a different trust model.The simple enclave scheme is implemented in SGX and provides strong security guarantees.
  • TEE baseline: SGX has a performance gap against untrusted hardware because its 128 MB PRM limit causes severe paging overheads and its enclaves do not provide noticeable multithreaded speedups.The paper reports that SGX cannot compete with GPUs or server CPUs for DNN performance or energy efficiency.
  • Selective outsourcing: Slalom improves on the TEE baseline when the server knows the model, by combining untrusted-device execution with in-TEE verification at lower total cost than full enclave evaluation.The design targets settings such as cloud ML APIs or public models.
  • Practical constraints: The paper notes that SGX has faced side-channel attacks, although firmware and hardware updates and alternative enclave architectures may address this concern.Sanctum and separate security co-processors are cited as alternatives.
  • Selective outsourcing: DNN nonlinearities remain in the TEE because they are harder to outsource, while linear operators are delegated because they dominate computation and support efficient verification.For VGG16 inference on one CPU thread, about 1.5% of computation is spent on nonlinearities.
  • Integrity and privacy: Freivalds’ check uses 3n^2 multiplications and repeated checks achieve soundness error 1/|S|^k, while additive stream-cipher blinding protects outsourced linear-layer inputs.The privacy protocol precomputes f(r) offline and encrypts online inputs as x + r.
  • Practical constraints: The approach assumes co-location because each DNN inference can require over 50 MB of communication between the TEE and untrusted co-processor.Lower-communication cryptographic protocols may impose excessive prover overhead in this setting.

3 SLALOM

Slalom outsources DNN linear layers from a TEE to an untrusted co-processor while preserving verifiability and, with blinding, input privacy. It combines field-based quantization, Freivalds’ verification, preprocessing, and encrypted intermediate data to reduce trusted-side computation and memory.

  • Overview: Slalom quantizes inputs and weights into a field, outsources linear layers for verification, and encrypts layer inputs with precomputed pseudorandom streams for privacy.The framework supports fully connected, convolutional, separable convolutional, pooling, and activation layers, with extensions to residual networks.
  • Quantization: Fixed-point quantization represents x as round(2^l · x), scales layer outputs by 2^-l, and re-rounds them to integers.The method defines quantized weights and biases using corresponding fixed-point scales.
  • Verification: Freivalds’ verification checks outsourced linear outputs using random vectors, with batching reducing amortized verification toward one operation per input and output.For convolutions, the check can be applied directly, while batched verification may be prohibitive in SGX because of limited protected memory.
  • Preprocessing: Preprocessing reuses secret randomness and precomputes weight-dependent checks, achieving |x| + |y| verification operations for arbitrary linear operators and compressing weights.The scheme stores the random vector and its matrix product, and VGG16’s footprint decreases from 550MB to 36MB.
  • Input privacy: Privacy blinding regenerates factors from a TEE-held PRNG seed and uses encrypted unblinding factors to decrypt outsourced layer outputs.The process adds double-precision arithmetic, layer-by-layer data exchange, and substantial RAM or fast-disk requirements.
  • Security: Slalom provides privacy and t-integrity with t = n/|S|^k − negl(λ), and model privacy when the model belongs to the server.The result assumes a secure PRNG and, for remote-client outsourcing, a secure TEE.

4 EMPIRICAL EVALUATION

The evaluation measures Slalom on Intel SGX using micro-benchmarks and ImageNet DNN inference, comparing secure outsourcing against fully in-enclave execution. Across VGG16, MobileNet, and ResNet variants, outsourcing linear layers improves throughput, with gains affected by model architecture, preprocessing, and privacy costs.

  • Experimental setup: The evaluation uses Intel SGX with a co-located Nvidia TITAN XP GPU and measures throughput as forward passes per second.Workloads include synthetic matrix, convolution, and separable-convolution benchmarks plus ImageNet classification with VGG16, MobileNet, and ResNet models.
  • Micro-benchmarks: 4× to 8× faster verification is achieved for outsourced square matrix products up to dimension 2048.Larger matrices exceed SGX’s DRAM limit, causing expensive paging for both computation and verification.
  • Micro-benchmarks: Preprocessed Freivalds verification provides large savings for convolutions, with savings increasing as the number of channels increases.Verification is especially fast for separable convolutions without intermediate nonlinearities, while SGX’s small PRM limits effective batching.
  • Verifiable inference: 20.3× higher throughput is achieved for VGG16 with preprocessed verification, whose 500MB network otherwise exceeds SGX’s PRM.Storing preprocessed products for all network weights reduces the enclave memory footprint.
  • Verifiable inference: 3.6× to 6.4× speedups are achieved for verifiable MobileNet inference, with the larger value for the fused model.MobileNet gains less than VGG16 because it performs fewer operations per layer, leaving less computation to save through outsourcing.
  • Private inference: 10.7× and 4.1× speedups are achieved for private-and-integrity-protected VGG16 and fused MobileNet inference, respectively.Private-only outsourcing reaches 13.0× for VGG16 and 5.0× for fused MobileNet, while blinding and unblinding add costly enclave memory operations.
  • Overall results: Slalom’s improvements also hold for energy efficiency and become larger on standard CPUs without SGX or in multi-threaded environments.The reported comparison is against the fully in-enclave baseline.
  • ResNet inference: 6.6× to 14.4× speedups are achieved for verifiable ResNet inference, versus 4.4× to 9.0× when privacy is added.The quantization scheme decreases accuracy by less than 0.5% for the pretrained 50-layer ResNet.

5 CHALLENGES FOR VERIFIABLE AND PRIVATE TRAINING

Applying Slalom to DNN training remains an open challenge despite similarities between backward passes and forward-pass linear operators. Changing weights complicates quantization, verification preprocessing, and privacy protection.

  • Scope boundary: DNN training remains an open problem because changing weights undermine several techniques used for inference.Backward passes contain similar linear operators and could therefore be verified with Freivalds’ algorithm, but the paper does not solve training.
  • Quantization: Training requires more flexible quantization because weight magnitudes change substantially during optimization.The paper states that quantizing DNNs for training is harder than quantizing them for inference.
  • Integrity verification: Changing weights prevents indefinite reuse of the same preprocessed Freivalds vectors during training.The paper suggests that very large batches might enable simultaneous verification, but does not establish this as a solution.
  • Privacy: Input-privacy preprocessing does not transfer directly to training because weights change after every processed batch.Slalom also does not hide model weights from the untrusted processor, which may be required for private training.

6 CONCLUSION

Slalom studies secure DNN evaluation by splitting computation between a TEE and a faster co-located untrusted processor. The framework outsources linear layers, verifies them with Freivalds’ algorithm, and preserves input privacy through precomputed blinding factors.

  • Conclusion: Slalom securely outsources all DNN linear layers from a TEE to a GPU while retaining security protections.The framework is designed to work with any TEE and is evaluated using Intel SGX across canonical DNN workloads.
  • Conclusion: Freivalds’ algorithm verifies outsourced linear operators, while precomputed blinding factors encrypt inputs to preserve privacy.The approach is presented as a framework for efficient DNN evaluation rather than only a single model implementation.
  • Broader scope: Secure matrix-product outsourcing also has potential applications beyond DNNs, including non-negative matrix factorization and dimensionality reduction.The paper further identifies efficient verifiable training and other fast-verification tasks as future directions.

A DETAILS ON INTEL SGX SECURITY

The SGX security discussion distinguishes enclave isolation from side-channel protection. Slalom does not formally model TEE security and relies on data-oblivious DNN computations to reduce side-channel concerns.

  • SGX protections: SGX encrypts and authenticates enclave memory and isolates execution from other host processes, including a potentially malicious operating system.Memory decryption is handled when words enter CPU registers.
  • Side channels: SGX does not address a prominent class of side-channel attacks that can observe data-dependent enclave behavior.The paper considers standard DNN computations a minor concern because they are data-oblivious and apply the same operations regardless of input data.
  • Speculative execution: Spectre attacks damage SGX and may require architectural changes for a truly secure solution.The paper notes that mitigations are being developed and identifies Sanctum as a possible architectural direction.
  • Security model: The paper does not formally model SGX or other TEE security because its focus is outsourcing protocols in which the TEE acts as a client.It refers readers to other formalization efforts.

B FORMAL SECURITY DEFINITIONS AND PROOFS

The paper formalizes secure outsourcing between clients and servers, defining correctness, integrity, input privacy, efficiency, and optional model privacy. It proves Slalom’s integrity and input-privacy guarantees under a secure TEE and PRNG assumptions.

  • Definitions: Secure outsourcing combines offline preprocessing with an interactive online protocol in which the client outputs a result or aborts.Preprocessing generates data-independent state, while the online protocol takes the model, input, and state.
  • Security properties: The required properties include correctness, t-integrity, input privacy, and efficiency, with model privacy treated as an optional application requirement.Model privacy hides the model from the server or client depending on model ownership.
  • Model privacy: Slalom does not provide privacy against an adversarial server seeking to distinguish executions using different models.Meaningful model privacy toward the client additionally requires commit-and-prove behavior by the server.
  • Integrity proof: Freivalds’ checks bound an incorrect accepted output across n layers by n/|S|^k − negl(λ), even when secret check vectors are reused.The per-layer error is at most 1/|S|^k, and the proof applies a union bound.
  • Privacy proof: Precomputed random blinding makes outsourced inputs one-time-pad encryptions, yielding information-theoretic privacy in the hybrid and computational indistinguishability in the protocol.The TEE regenerates blinding factors from a PRNG seed and uses stored unblinding factors to decrypt outputs.
  • Security guarantee: Assuming a secure TEE, Slalom provides privacy and t-integrity with t = n/|S|^k − negl(λ), and additionally provides model privacy when the server owns the model.The remote-client protocol uses attestation to establish a secure channel and returns the output with a commitment to the model.

C PERFORMANCE COMPARISON OF DNN OUTSOURCING SCHEMES

The comparison frames a single-threaded SGX enclave as a secure baseline against faster but insecure GPUs and cryptographic outsourcing systems. It also evaluates throughput and energy efficiency when Slalom combines enclave and GPU resources.

  • Baselines: A TITAN XP GPU achieves over 50× the throughput of the single-threaded SGX baseline but provides no security guarantees.For MobileNet, throughput is 16 images/sec in the enclave versus 900 images/sec on the GPU.
  • Cryptographic comparisons: Cryptographic systems provide integrity or privacy but incur substantial overhead compared with enclave execution.SafetyNets reaches about 13 images/sec on its largest evaluated model, while the same model exceeds 3,500 images/sec in the baseline enclave.
  • Energy efficiency: Throughput alone is not a fair cross-hardware metric, so the paper also compares energy efficiency as a measure of recurrent server cost.The comparison involves a 30W SGX CPU baseline and an 85W GPU.
  • Slalom accounting: The GPU accounts for at most 10% of Slalom’s running time, making integrity checks and encryption or decryption the main bottleneck.The estimated combined power is 35.5W, and Slalom is about 3.4×–17.1× more energy efficient than the baseline.
  • DNN operators: Fully connected, convolutional, and separable-convolution layers expose matrix-multiplication structure that determines their multiplication costs.Convolution combines patch extraction with matrix multiplication, while separable convolution uses depthwise and pointwise stages.

E NEURAL NETWORK DETAILS

The evaluation models are characterized by ImageNet accuracy, parameter counts, and the communication and enclave-storage demands of their linear layers. Slalom implements its checks over a finite field using floating-point arithmetic and tuned soundness parameters.

  • Model accuracy: Table 3 reports top-1 and top-5 ImageNet accuracy with and without quantization, with quantization reducing accuracy by at most 0.5%.The evaluated models are pretrained Keras models.
  • Resource accounting: Model parameter counts matter for TEE memory, while linear-layer input and output sizes affect Slalom communication and preprocessing storage.These quantities capture two distinct resource constraints in the evaluation.
  • Arithmetic: Slalom performs Freivalds’ checks as modular inner products over Zp while using floating-point operations for efficiency.A matrix-product check is implemented as a set of inner products in the enclave.
  • Arithmetic: The quantization scheme embeds DNN values in Zp with p ⪅ 2^24, allowing the arithmetic to fit within standard floating-point representations.The implementation uses doubles for modular arithmetic and floats for efficiency-oriented integer representation.
  • Soundness: Targeting a per-layer soundness error of 2^-40 creates a tradeoff between Freivalds repetitions and the random-value set size.The evaluation uses k = 2 repetitions with S = [−2^19, 2^19].

G RESULTS ON A STANDARD CPU

On a standard CPU, verifying linear computations is much cheaper than recomputing them as problem size grows. Secure inference benchmarks show substantial integrity-only gains and smaller gains when privacy blinding is added.

  • Micro benchmarks: Figure 5 compares relative verification speedups with direct computation throughput across three linear operators.The dotted line denotes throughput for computing each operation.
  • Micro benchmarks: 1–2 orders of magnitude: verification becomes faster than computation for matrix multiplication, convolution, and separable convolution as the outer dimension grows.The benchmarks run outside SGX enclave mode on an untrusted CPU.
  • Inference results: 8.9× and 19.5×: integrity-only inference savings for MobileNet and VGG16, respectively.Batched verification also performs well without storing secrets in the enclave, though it could likely be improved further.
  • Inference results: Figure 6 compares baseline throughput with on-the-fly, batched, precomputed-secret, privacy-only, and privacy-plus-integrity strategies.The fused MobileNet model has no intermediate activation for separable convolutions.
  • Inference results: 3.9× and 8.1×: privacy-preserving inference speedups for MobileNet and VGG16 after adding blinding.These results combine privacy with integrity checking.

H PARALLELIZATION

SGX parallelization remains limited: multiple forward passes provide little speedup, while intra-operation parallelism is unsupported in the enclave and evaluated only on an untrusted CPU.

  • H PARALLELIZATION: SGX experiments used a single execution thread because enclaves cannot create threads.
  • H PARALLELIZATION: Less than 1.5× speedup was achieved for MobileNet with up to 4 threads, while VGG16 gained no parallel speedup.These results held for both direct computation and outsourced verification.
  • H PARALLELIZATION: SGX outer-parallelism is constrained because independent threads require extra memory that can exceed the PRM limit.
  • H PARALLELIZATION: The DNN library does not support intra-operation parallelism inside SGX, so its potential benefits were measured only on an untrusted CPU.Eigen multi-threading and custom OpenMP code were used for matrix products, convolutions, and dot products.
  • H PARALLELIZATION: With 4 threads, verifiable separable convolutions achieved close to linear speedups, but comparisons were omitted because direct depthwise-convolution threading was unavailable.The effect of similar intra-operation speedups inside SGX remains unclear because of memory-access overheads.
Loading 1806.03287v2…