PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepSecure: Scalable Provably-Secure Deep Learning

Bita Darvish Rouhani, M. Sadegh Riazi, Farinaz Koushanfar

arXiv:1705.08963v1cs.CR

TL;DR

DeepSecure addresses privacy-preserving DL inference when clients and model holders cannot reveal their data or parameters. It uses GC-optimized DL circuits with preprocessing and secure outsourcing, achieving substantially faster execution than prior privacy-preserving DL systems while retaining accuracy and confidentiality.

  • Problem

    DeepSecure addresses DL inference on sensitive individual data when both client data and trained model parameters must remain confidential.

  • Method

    DeepSecure uses Yao’s GC protocol, GC-optimized DL libraries, data and network preprocessing, and secure outsourcing for constrained devices.

  • Results

    More than 58-fold higher throughput per sample is achieved than the prior privacy-preserving DL solution, with preprocessing reducing MNIST execution from 9.67 seconds to 1.08 seconds without accuracy loss.

  • Takeaways & Limitations

    DeepSecure supports accurate, scalable privacy-preserving DL analysis for distributed clients, streaming settings, and constrained embedded platforms.

  • Takeaways & Limitations

    Evaluations against the prior-art system were available for only one benchmark because that system reported only one benchmark.

Abstract

from arXiv · show

This paper proposes DeepSecure, a novel framework that enables scalable execution of the state-of-the-art Deep Learning (DL) models in a privacy-preserving setting. DeepSecure targets scenarios in which neither of the involved parties including the cloud servers that hold the DL model parameters or the delegating clients who own the data is willing to reveal their information. Our framework is the first to empower accurate and scalable DL analysis of data generated by distributed clients without sacrificing the security to maintain efficiency. The secure DL computation in DeepSecure is performed using Yao's Garbled Circuit (GC) protocol. We devise GC-optimized realization of various components used in DL. Our optimized implementation achieves more than 58-fold higher throughput per sample compared with the best-known prior solution. In addition to our optimized GC realization, we introduce a set of novel low-overhead pre-processing techniques which further reduce the GC overall runtime in the context of deep learning. Extensive evaluations of various DL applications demonstrate up to two orders-of-magnitude additional runtime improvement achieved as a result of our pre-processing methodology. This paper also provides mechanisms to securely delegate GC computations to a third party in constrained embedded settings.

1 INTRODUCTION

DeepSecure addresses privacy concerns in DL inference by securely evaluating trained models on sensitive data while keeping both inputs and model parameters confidential. It combines GC-optimized computation with preprocessing and outsourcing mechanisms to improve scalability for distributed and constrained clients.

  • Sensitive applications such as medical, financial, and location analysis are hindered when DL inference risks leaking private client data.
  • Differential privacy supports DL training but does not directly address inference on individual data samples.
  • DeepSecure uses Yao’s GC protocol to analyze sensitive data while keeping both DL parameters and client data private.
  • DeepSecure supports streaming analysis and securely delegates GC computations to third parties for constrained embedded devices.
  • DeepSecure preprocesses data and DL networks to avoid unnecessary secure computation and communication without sacrificing accuracy or confidentiality.
  • The framework introduces GC-optimized libraries for DL computations, including implementations of Tanh and Sigmoid functions.

2 PRELIMINARIES

The preliminaries introduce DL architectures and the cryptographic tools underlying DeepSecure. They explain how neural-network layers map to Boolean circuits and how GC preserves input privacy through garbling, oblivious transfer, evaluation, and result merging.

  • 2.1 Deep Learning Networks: DL models learn hierarchical non-linear representations by mapping raw features through stacked processing layers to output labels.
  • 2.1 Deep Learning Networks: CNNs add convolutional layers to fully connected networks, making them suited to data with strong local connectivity such as visual measurements.
  • 2.1 Deep Learning Networks: DL inference commonly uses Sigmoid, Tanh, or ReLU activations, with Softmax typically determining the final output.
  • 2.1 Deep Learning Networks: CNN layers can be represented as Boolean circuits for GC, while ReLU maps accurately to a multiplexer rather than requiring high-order polynomials.
  • 2.2 Cryptographic Protocols: Oblivious transfer supplies Bob’s input labels without revealing his input to Alice or exposing both possible labels to Bob.
  • 2.2 Cryptographic Protocols: In Yao’s GC, Alice garbles a Boolean circuit and Bob evaluates its topologically ordered netlist using private input labels.
  • 2.4 Security Model: The framework assumes honest-but-curious parties that follow the protocol but may try to infer additional information.

3 DeepSecure FRAMEWORK

DeepSecure combines GC-optimized DL execution with transformations of the input data and network. Its workload depends on network size and input size, motivating preprocessing to reduce secure inference overhead.

  • DeepSecure has two main components: GC-optimized execution of the target DL model and data or network transformation.
  • The cloud server holds trained model parameters while the client owns a sample and seeks its inference label securely.
  • DeepSecure generates a Boolean netlist for the public DL architecture before executing the GC protocol.
  • GC computation and communication costs are governed by the number of neurons per layer and the input data size.
  • The framework applies data and network transformations as preprocessing to reduce secure DL inference overhead.

3.2 Data and DL Network Pre-processing

DeepSecure reduces secure-inference overhead through data projection and DL network distillation, preserving accuracy and confidentiality while shrinking the data and network footprint. The server performs offline retraining and releases a projection matrix for clients to transform streaming measurements.

  • DeepSecure pre-processing consists of data projection and DL network distillation.
  • Data Projection: Data projection represents high-dimensional training data as an ensemble of lower-dimensional subspaces for GC performance optimization.The server incrementally updates a dictionary and data embeddings while processing streamed training batches.
  • GC computation and communication overhead are governed by DL-layer neuron counts, motivating transformations that reduce the network and data dimensions.The approach targets communication workload while preserving accuracy and data confidentiality.
  • Data Projection: The server retrains the DL model using projected embeddings, then publicly releases a projection matrix for clients’ online data transformation.This retraining is a one-time offline process, so GC-time preprocessing involves matrix operations on client data.
  • DL Network Distillation: Network distillation removes connections below a weight threshold and retrains the condensed network to recover the initial model’s accuracy.Pruning eliminates unnecessary garbling and evaluation of noncontributing neurons.

3.3 Secure Outsourcing

DeepSecure securely outsources GC computation to a non-colluding honest-but-curious proxy server for resource-constrained users. XOR sharing reconstructs inputs inside the circuit, while Free-XOR keeps the added overhead almost unchanged.

  • DeepSecure supports secure outsourcing of GC computations to a proxy cloud server for severely resource-constrained users.The proxy is assumed honest-but-curious and non-colluding with the main server.
  • XOR sharing splits each client input into a random string and its XOR with that string, which two servers receive separately.An initial XOR layer reconstructs the true input inside the garbled circuit.
  • Almost the same computation and communication overhead as the original scheme results from adding only an initial XOR layer under Free-XOR.The client performs only random-string generation and a simple XOR operation.

3.4 GC-Optimized Circuit Components Library

DeepSecure uses industrial logic synthesis with custom GC libraries to minimize non-XOR gates in neural-network computation modules. The library provides optimized building blocks for the required DL operations.

  • GC optimization focuses on minimizing non-XOR gates because garbled tables are generated and transferred only for those gates.
  • The custom synthesis library provides GC-optimized MUX, comparator, adder, and multiplier blocks for neural-network layers.These blocks are added to the synthesis tool’s library to incur as few non-XOR gates as possible.

3.5 GC Memory Footprint

DeepSecure uses sequential circuits to reduce GC memory footprint and scale netlist execution across multiple clock cycles. Circuit folding reuses compact arithmetic modules instead of instantiating every operation separately.

  • Sequential circuits provide compact, scalable GC netlists through lower memory footprint than traditional combinational circuits.
  • Circuit folding runs shared MULT and ADD modules across multiple clock cycles instead of instantiating every matrix-multiplication operation.

3.6 DeepSecure Modular Structure

DeepSecure uses modular DL layers that can be connected into arbitrary end-to-end network architectures, with support for fixed-point and floating-point accuracy.

  • DeepSecure implements each DL layer as a separate module that can be stacked into arbitrary end-to-end network architectures.The framework supports composing layers such as convolution, nonlinear activation, and max pooling.
  • The GC-optimized library supports both fixed-point and floating-point accuracy.

3.7 Security Proof

DeepSecure proves security in the honest-but-curious model while identifying the information exposed by preprocessing and the non-collusion condition required for outsourcing.

  • DeepSecure’s security proof covers data preprocessing, DL network preprocessing, and secret shares used in secure outsourcing.GC optimizations preserve security because GC security is independent of Boolean-circuit topology.
  • The released projection matrix W reveals only the dictionary matrix’s column space, not its actual values, so D cannot be reconstructed.Reconstruction additionally requires the right-singular vectors V and singular values Σ; the preprocessing step can be skipped if revealing the subspace is unacceptable.
  • DL network preprocessing exposes the network sparsity map as public knowledge, but not the private network parameters.The sparsity map identifies components that do not contribute to the output.
  • XOR-sharing securely outsources client inputs when the two untrusted servers do not collude.One server receives a random string, while the other receives an XOR-masked input equivalent to one-time-pad encryption.

4 EVALUATIONS

The evaluation measures DeepSecure’s GC circuit costs and execution across DL benchmarks, then quantifies gains from data and network preprocessing. Results show preprocessing substantially reduces runtime while preserving accuracy, and performance depends strongly on communication overhead and workload size.

  • 4.2 Circuit Synthesis: DeepSecure implements DL nonlinearities with multiple circuit designs that expose speed/accuracy trade-offs, including LUT, approximate Tanh, piece-wise linear, and CORDIC realizations.TanhPL is almost 700 times less costly than Tanh, with a reported error rate of 0.22%; Tanh2.10.12 reports 0.01% error.
  • 4.2 Circuit Synthesis: Softmax uses optimized comparison and multiplexing blocks because its monotonicity preserves the input vector’s maximum-value index.Different nonlinear activation circuits can be selected according to application speed and accuracy criteria; experiments use CORDIC-based Tanh and Sigmoid.
  • 4.3 Performance Cost Characterization: Total execution time is dominated by garbled-table transfer, while GC computation and communication costs are determined by circuit gate counts, input size, and network structure.The evaluation characterizes XOR and non-XOR gate costs and reports benchmark-level communication, computation, and execution time.
  • 4.5 Empirical Analysis: DeepSecure is evaluated on four DL benchmarks spanning DNN and CNN models for visual, audio, and smart-sensing data.The MNIST experiments use handwritten-digit images, while the audio benchmark uses approximately 1.25 hours of speech from 150 speakers and a 3-layer fully-connected DNN.
  • 4.6 Data and Network Pre-processing: Up to 82-fold execution-time reduction is achieved through data and DL network preprocessing without reducing underlying DL accuracy.The preprocessing reduces the number of required XOR and non-XOR gates for particular DL models.
  • 4.7 Comparison with Prior Art Framework: More than 58-fold improvement in overall execution time per sample is reported over the prior privacy-preserving DL system, with MNIST reduced from 570.11 seconds to 9.67 seconds before preprocessing.Preprocessing further reduces the MNIST processing time per sample to 1.08 seconds without an accuracy drop or change in confidentiality.
  • 4.7 Comparison with Prior Art Framework: DeepSecure’s computational cost scales linearly with sample count, favoring streaming clients with small batches, whereas CryptoNet favors sufficiently large batches.The paper gives N_client ≤2590 as an example streaming regime and N_client = 8192 as an example where CryptoNet is better suited.

5 RELATED WORK

The related-work discussion contrasts DeepSecure with interactive secure-function-evaluation protocols and homomorphic-encryption-based DL. It positions DeepSecure’s GC approach and preprocessing as addressing privacy, connectivity, scalability, and gate-cost concerns identified in prior work.

  • Secure Function Evaluation: Earlier interactive secure-function-evaluation protocols repeatedly exchange encrypted intermediate results and may leak partial model-weight information while requiring constant client connectivity.Later protocols obscure the weights but still require a constant connection to the client for nonlinearity evaluation.
  • Homomorphic Encryption: Homomorphic-encryption-based DL introduces a privacy–accuracy trade-off and high computation overhead, limiting use to low-degree polynomials and 5–10-bit precision.The paper contrasts these constraints with its goal of scalable, fully secure DL evaluation while limiting client–cloud communication.
  • Preprocessing for Secure DL: DeepSecure introduces data and DL-network transformation to minimize non-XOR gates for privacy-preserving GC, extending projection and sparsity ideas beyond accuracy and physical efficiency.Earlier projection and sparsity work focused on feature extraction or model acceleration without addressing data privacy.

6 CONCLUSION

DeepSecure provides a practical, provably secure framework for distributed clients and cloud servers to jointly evaluate deep-learning networks on private assets. Its GC optimizations and preprocessing substantially improve throughput and runtime while supporting constrained devices.

  • DeepSecure enables distributed clients and cloud servers to jointly evaluate a DL network while keeping their private assets protected.
  • More than 58 times higher throughput per sample is achieved than with the prior art privacy-preserving DL solution.
  • Preprocessing techniques optimize GC computation and communication overhead, yielding up to two orders-of-magnitude additional improvements across different DL benchmarks.
  • Secret-sharing mechanisms support securely outsourcing GC computation when clients face severe resource constraints.
Loading 1705.08963v1…