PaperlayerView exact revision on arXiv

Source-linked AI summary

Low Latency Privacy Preserving Inference

Alon Brutzkus, Oren Elisha, Ran Gilad-Bachrach

arXiv:1812.10659v2cs.LGstat.ML

TL;DR

Private neural-network inference must balance accuracy, security, and computational complexity because prior HE methods have high latency and limited network width and depth. The paper introduces changing encrypted-data representations and transfer learning, achieving faster inference and support for wider or deeper models. Its transfer-learning approach is constrained when client-side computation is impractical.

  • Problem

    Prior HE-based private inference methods have high latency and limited support for wide and deep neural networks, restricting private prediction services.

  • Method

    The paper uses LoLa’s changing encrypted-data representations and transfer learning over encrypted deep features for private neural-network inference.

  • Results

    The methods provide over 10× lower latency, support wider networks, and enable private CalTech-101 inference in 0.16 seconds.

  • Takeaways & Limitations

    HE-based private inference can operate on more complex networks and reduce latency on networks previously studied.

  • Takeaways & Limitations

    Transfer learning is not useful for power-limited clients when most computation remains on the client side.

Abstract

from arXiv · show

When applying machine learning to sensitive data, one has to find a balance between accuracy, information security, and computational-complexity. Recent studies combined Homomorphic Encryption with neural networks to make inferences while protecting against information leakage. However, these methods are limited by the width and depth of neural networks that can be used (and hence the accuracy) and exhibit high latency even for relatively simple networks. In this study we provide two solutions that address these limitations. In the first solution, we present more than $10\times$ improvement in latency and enable inference on wider networks compared to prior attempts with the same level of security. The improved performance is achieved by novel methods to represent the data during the computation. In the second solution, we apply the method of transfer learning to provide private inference services using deep networks with latency of $\sim0.16$ seconds. We demonstrate the efficacy of our methods on several computer vision tasks.

1 Introduction

The paper targets private neural-network inference with Homomorphic Encryption, addressing prior methods’ latency and network-size limitations. It proposes LoLa’s data representations and transfer learning for faster, wider, and deeper private inference.

  • Private inference protects sensitive data by encrypting it before transmission and operating on encrypted data without access to raw data.The motivation covers applications in education, health, and finance.
  • Homomorphic Encryption offers strong security but restricts the operations and complexity practical inference can use.These constraints motivate the paper’s focus on latency, width, and depth.
  • 11.2× speedup reduces CryptoNets’ single-prediction latency to 2.2 seconds while preserving accuracy and 128 bits of security.LoLa changes data representations during homomorphic computation, encrypting entire layers rather than individual nodes.
  • LoLa processes CIFAR-10 predictions in 12 minutes using only a few Gigabytes of RAM, whereas CryptoNets requires 100’s of Gigabytes.The experiment demonstrates inference on wider networks that CryptoNets cannot execute because of its memory requirement.
  • Transfer learning enables private CalTech-101 predictions in 0.16 seconds with 81.6% class balanced accuracy.The approach encrypts deep features produced before private evaluation.

2 Related Work

Related work explores different trade-offs among computation, accuracy, and security. The paper distinguishes its HE-based methods from alternative HE schemes, MPC, hardware approaches, and concurrent private-inference methods.

  • Bourse et al.’s bootstrapping-based HE scheme incurs a linear penalty for additional layers but is slower per operation and less accurate in reported results.
  • Chou et al.’s MNIST method runs in 39.1s at 98.7% accuracy, compared with LoLa’s 2.2s at 98.95% accuracy.
  • On CIFAR, Chou et al.’s inference takes more than 6 hours at 76.7% accuracy, whereas LoLa takes less than 12 minutes at 74.1% accuracy.
  • Makri et al. also apply transfer learning to private inference, but their methods and threat model differ.
  • MPC, hardware-based, and hybrid approaches are faster or provide private predictions but rely on weaker security assumptions than HE.

3 Background

The paper uses Homomorphic Encryption to compute polynomial operations on encrypted vectors, with BFV rotations supporting structured representations. CryptoNets illustrate both the benefits and bottlenecks of this approach.

  • Homomorphic Encryption: Homomorphic Encryption permits addition and multiplication on encrypted data, enabling computation of polynomial functions without the secret key.
  • Homomorphic Encryption: The BFV scheme represents messages as n-dimensional vectors in Z_p and supports rotations of encrypted coordinates.
  • CryptoNets: CryptoNets replace unsupported ReLU and sigmoid operations with square activations and achieve 99% MNIST accuracy.
  • CryptoNets: 205 seconds per prediction and high memory usage arise from representing each feature as a separate message.
  • CryptoNets: CryptoNets encode feature values across message coordinates, allowing n input records to be processed simultaneously.
  • CryptoNets: CryptoNets cannot efficiently support deep networks because layer multiplications increase encrypted-message noise and size.

4 LoLa

LoLa accelerates private neural-network inference by changing encrypted-data representations across computation, reducing latency and memory use compared with CryptoNets. The approach supports wider networks and provides implementations for convolutional and feed-forward layers.

  • LoLa alternates encrypted-message representations during computation to improve latency and memory usage over CryptoNets’ single representation.CryptoNets encodes each feature as a separate message, whereas LoLa changes representations across layers.
  • 4.1 Linear Classifier Example: A single-message vector representation reduces a linear dot product to one multiplication plus log d rotations and additions, instead of d messages and d operations.The resulting vector contains the dot product in all coordinates.
  • 4.2.1 Vector representations: LoLa defines dense, sparse, stacked, interleaved, and convolution representations, with matrix-vector implementations that transform between representations.The methods use column-major or row-major weight layouts depending on the input and output representations.
  • 4.4 Secure Networks for CIFAR: 730 seconds: LoLa evaluates the larger CIFAR network using 12 GB of RAM, with the second convolution layer consuming 711 seconds.The network has more than 500,000 parameters, compared with fewer than 90,000 in the MNIST network.

5 Private Inference using Deep Representations

Homomorphic encryption faces noise and message-size growth on deep networks. Deep representations and transfer learning address these constraints, enabling private predictions with shallow encrypted models while retaining practical utility and clear scope limits.

  • Constraints: Noise and message-size growth limit homomorphic-encrypted evaluation of deep networks.Bootstrapping can mitigate noise growth but reduces performance, and even HEAAN cannot efficiently operate on deep networks.
  • Deep representations: Deep representations convert raw data with a standard network before encryption, producing smaller inputs for private evaluation.The paper gives AlexNet, VGG, and ResNet as examples of networks that can extract these representations.
  • Deep representations: Deep representations often support high accuracy with shallow networks, frequently making a linear predictor sufficient for fast homomorphic-encrypted evaluation.The approach is also useful when scarce data makes training a very deep network from scratch impractical.
  • Experiment: AlexNet features and a linear model were used to test private predictions on CalTech-101.The dataset was subsampled to 30 images per class, with 20 for training and 10 for evaluation.
  • Limitations: Transfer learning is unsuitable for power-limited clients because most computation remains on the client side.The technique can instead support private predictions when an institution wants to protect its model while serving predictions.

6 Conclusions

The study presents two homomorphic-encryption solutions for private inference that address limitations of earlier approaches. It demonstrates lower latency and operation on more complex networks, with performance gains mainly attributed to multiple representations during computation.

  • Conclusions: The two HE-based solutions address key limitations of previous private-inference methods.The study demonstrates both operation on more complex networks and lower latency on previously studied networks.
  • Conclusions: The performance gain is mainly due to using multiple representations during computation.The paper identifies encrypted-data training as a possible future application but leaves that direction for future study.

A Rings

This section defines the commutative rings used in the paper and introduces polynomial rings over modular integers. The BFV scheme uses a quotient ring whose elements have bounded-degree polynomial representatives.

  • Definitions: A commutative ring is a ring whose multiplication satisfies ab = ba for all elements a and b.The paper uses “ring” to mean commutative ring because all rings considered have this property.
  • Definitions: The integers Z and modular integers Zp are examples of rings, with elements of Zp represented by residue classes modulo p.A representative k denotes the class of integers of the form k + ap.
  • Polynomial rings: The polynomial set R[x] forms a ring when its coefficients come from a ring R.The section consequently introduces polynomial rings over Zp for the cryptographic construction.
  • BFV ring: The BFV scheme uses the quotient ring formed from polynomials modulo xn + 1.Its elements are equivalence classes of the form r(x) + q(x)(xn + 1).
  • BFV ring: Every element in this quotient ring has a representative in Zp[x] with degree at most n − 1.Distinct polynomials of degree at most n − 1 represent distinct elements.

B Parallel Scaling

The experiments evaluate how the solutions’ latency changes with available CPU cores. On MNIST, all methods scale linearly through eight cores, indicating that higher core counts could further reduce latency.

  • Experimental setup: Performance was tested with 1, 2, 4, and 8 cores on hardware with 8 cores.The experiments measure how latency varies with the available parallelism.
  • Results: On MNIST, all methods scale linearly with core count up to 8 cores.The authors report that the algorithm uses the available cores well and do not expect large gains from simultaneous multiple queries.

C Lola-Dense

LoLa-Dense applies homomorphic evaluation by changing data representations across the network, reducing encrypted operations while preserving the MNIST network’s accuracy. The full network runs substantially faster than CryptoNets.

  • Data representation: LoLa-Dense uses a single dense input message and converts it into convolution messages with vector masks.The masks extract pixels for convolution windows, while shifted masks support the convolution representation.
  • Data representation: The convolution outputs are rotated and summed into one message containing 845 values before squaring.The five messages each contain 169 results.
  • Activation evaluation: 845 activation multiplications in CryptoNets are reduced to one multiplication in LoLa-Dense.The representation requires only 4 rotations and 4 additions to consolidate the convolution results.
  • Latency: 7.2 seconds is required for the complete network, making it 34.7× faster than CryptoNets and 3.4× faster than CryptoNets 2.3.This result is reported on the same reference hardware.

D Secure CIFAR

The CIFAR network uses convolutional, pooling, activation, and fully connected layers. For inference, adjacent linear layers can be collapsed into a shorter convolutional structure.

  • Network architecture: The CIFAR network processes 3 × 32 × 32 images through convolutions, pooling, square activations, and a fully connected layer.The architecture includes convolutional layers with 128, 83, and 163 output maps, followed by a fully connected layer with 1024 outputs.
  • Representation and operations: Table 3 organizes LoLa-Dense’s MNIST inference by layer, message size, data representation, and operations.Input sizes are expressed as number of vectors × dimension.
  • Collapsed inference network: Adjacent linear layers are collapsed into two convolutions, two square activations, and a dense layer with 10 output maps.The collapsed convolutions use 8 × 8 × 3 and 6 × 6 × 83 filters.

E CalTech-101

The CalTech-101 method uses deep representations and tracks how data representations change across the private-inference pipeline. Dense-vector terminology is also applied to the pre-encryption AlexNet stage.

  • Representation changes: Table 4 reports data-representation changes for private inference on the CalTech-101 task using deep representations.The table compares different representations used by the proposed method.
  • Representation changes: The CalTech-101 representation pipeline is presented as a sequence of changes rather than a single fixed representation.Table 4 is specifically described as showing these representation changes.
  • Pre-encryption processing: Dense-vector terminology is used for the first AlexNet stage before encryption.This terminology applies before the encrypted evaluation stage.
Loading 1812.10659v2…