PaperlayerView exact revision on arXiv

Source-linked AI summary

nGraph-HE: A Graph Compiler for Deep Learning on Homomorphically Encrypted Data

Fabian Boemer, Yixing Lao, Rosario Cammarota, Casimir Wierzynski

arXiv:1810.10121v3cs.CR

TL;DR

Deploying deep-learning models on homomorphically encrypted data is labor-intensive because existing deep-learning and HE software ecosystems do not provide an integrated framework. nGraph-HE treats HE as a graph-compiler hardware target, adding HE-aware optimizations and framework integration. The paper demonstrates models and operations with 0.1% additional runtime overhead over direct C++ HE-library implementations.

  • Problem

    Building deep-learning models that operate on ciphertext requires coordinating deep learning, cryptography, and software engineering without a dedicated HE-oriented framework.

  • Method

    nGraph-HE extends Intel nGraph with an HE backend and privacy-preserving hardware abstraction layer for deploying plaintext-trained models to HE cryptosystems.

  • Results

    0.1% of total runtime was the additional software-layer overhead compared with implementing demonstrated operations directly in C++ using HE libraries.

  • Takeaways & Limitations

    Data scientists can use TensorFlow and other deep-learning frameworks to prototype and deploy encrypted-data models with minimal code changes while applying compiler optimizations.

  • Takeaways & Limitations

    MaxPool and ReLU are unsupported because HE provides only addition and multiplication, while limited computational depth constrains models in leveled HE.

Abstract

from arXiv · show

Homomorphic encryption (HE)---the ability to perform computation on encrypted data---is an attractive remedy to increasing concerns about data privacy in deep learning (DL). However, building DL models that operate on ciphertext is currently labor-intensive and requires simultaneous expertise in DL, cryptography, and software engineering. DL frameworks and recent advances in graph compilers have greatly accelerated the training and deployment of DL models to various computing platforms. We introduce nGraph-HE, an extension of nGraph, Intel's DL graph compiler, which enables deployment of trained models with popular frameworks such as TensorFlow while simply treating HE as another hardware target. Our graph-compiler approach enables HE-aware optimizations-- implemented at compile-time, such as constant folding and HE-SIMD packing, and at run-time, such as special value plaintext bypass. Furthermore, nGraph-HE integrates with DL frameworks such as TensorFlow, enabling data scientists to benchmark DL models with minimal overhead.

1 INTRODUCTION

nGraph-HE addresses the lack of a privacy-preserving deep-learning framework by extending a graph compiler to deploy trained models on homomorphically encrypted data. It combines framework integration with compile-time and run-time optimizations, while adding only 0.1% runtime overhead over direct C++ HE-library implementations.

  • Motivation: Homomorphic encryption enables computation on encrypted data, allowing data owners to receive encrypted results that only they can decrypt.The data processor operates without access to the secret key.
  • Motivation: Existing deep-learning frameworks and graph compilers hide implementation details, but comparable frameworks for privacy-preserving deep learning with HE were lacking.This gap made ciphertext-model development difficult amid diverse HE libraries, schemes, and APIs.
  • Framework: nGraph-HE extends Intel nGraph with an HE backend, enabling plaintext-trained models to be deployed to encryption systems through a privacy-preserving hardware abstraction layer.Modifying an existing TensorFlow model to use encrypted data can require only a single line of code.
  • Optimizations: The framework introduces HE-aware compile-time and run-time optimizations, including graph-level transformations, HE-SIMD packing, special plaintext-value bypass, and ciphertext-plaintext operations.These optimizations exploit computation structure, cryptosystem properties, and hardware parallelism.
  • Evaluation: nGraph-HE was demonstrated on GEMM, convolution-batch-norm operations, and MNIST and CIFAR-10 convolutional-network benchmarks using Python and TensorFlow.The experiments used different encryption parameters and evaluated both subgraphs and complete benchmark problems.
  • Evaluation: 0.1% of total runtime was the additional software-layer overhead compared with implementing the operations directly in C++ with HE libraries.The reported overhead was measured for the demonstrated operations.

2 BACKGROUND

Homomorphic encryption enables computation on encrypted data, but privacy-preserving deep learning remains constrained by cryptosystem limitations, computational costs, and software complexity. Existing privacy-preserving frameworks provide limited support across deep-learning frameworks and security models.

  • 2.1 Homomorphic encryption: Homomorphic encryption evaluates selected computations on ciphertext so that decryption recovers the corresponding plaintext result.A remote computer can process encrypted inputs without accessing the user’s unencrypted data.
  • 2.1 Homomorphic encryption: Semantic security prevents computationally bounded adversaries from distinguishing ciphertexts, including encryptions of identical plaintexts.Random encryption noise provides this indistinguishability.
  • 2.2 Challenges of homomorphically encrypted deep learning: HE schemes may restrict supported operations, computational depth, numerical representations, and ciphertext growth, limiting exact implementation of common neural-network functions.Addition and multiplication cannot directly express comparison-based operations such as Max, Min, and ReLU; leveled schemes also impose fixed depth.
  • 2.2 Challenges of homomorphically encrypted deep learning: HE computations typically require several orders of magnitude more CPU time and memory than plaintext computations.CryptoNets MNIST inference is cited as decreasing from 297.5s to 0.03s across prior work, with the latter using a hybrid scheme.
  • 2.2 Challenges of homomorphically encrypted deep learning: The diversity of HE libraries, schemes, APIs, and deep-learning libraries makes application-specific tradeoffs and ciphertext-model development difficult.The paper targets inference rather than training because of HE’s computational and memory overhead.
  • 2.4 Related Work: nGraph-HE differs from prior frameworks by operating on computational graphs, supporting multiple deep-learning frameworks, and requiring potentially only one line of code changes.Unlike PySyft and TF-encrypted, the cited systems use MPC and are tied to specific deep-learning frameworks.

3 NGRAPH-HE

nGraph-HE organizes homomorphic-encryption support around a unified API and maps it to supported cryptosystems. Its payload abstraction separates data transformations from cryptographic operations while supporting precomputed plaintext values for optimization.

  • 3 NGRAPH-HE: The framework maps its API onto BFV and CKKS cryptosystems implemented by the SEAL encryption library.The implementation discussion covers the API, cryptosystem mapping, compile-time and runtime optimizations, and support for additional cryptosystems.
  • 3 NGRAPH-HE: nGraph-HE exposes a cryptographic context, payload representation, and assembly language to unify differing HE APIs and supported operations.The cryptographic context stores scheme parameters, the payload stores data, and the assembly language describes supported functions.
  • 3 NGRAPH-HE: The cryptographic context records the polynomial modulus degree, plaintext and ciphertext moduli, security level, and HE scheme.Supported schemes omit bootstrapping, so the context must provide enough ciphertext moduli for the model’s multiplicative depth.
  • 3.2 Payload representation: The HETensor payload stores plaintext or ciphertext data together with an HEBackend pointer needed to obtain keys.The payload representation therefore connects data with the cryptographic backend used to process it.
  • 3.2 Payload representation: The payload abstraction represents data, encoding, encryption, decryption, and decoding as composable transformations between K, P, and C.Encoding and decoding may be identity mappings, allowing precomputed plaintext values to be stored for optimization.

3.3 Assembly language

nGraph-HE exposes HE computation through an nGraph assembly language and API, then applies runtime, parallel, and graph-level optimizations tailored to encrypted workloads.

  • Assembly language: The assembly language defines Add, Subtract, Multiply, and Negate over ciphertext and plaintext representations, with compound operations built on these primitives.Compound operations include AvgPool, Convolution, and Dot, while tensor manipulation operations include Broadcast, Concat, Pad, Reshape, Reverse, and Slice.
  • API components: nGraph-HE’s Backend, Tensor, and Kernel components organize cryptographic contexts, payload data, and operation implementations.Default implementations can be overridden by cryptosystem-specific kernels, reducing the effort required to add a new cryptosystem.
  • Runtime optimizations: Runtime plaintext-value bypass replaces selected ciphertext-plaintext operations with cheaper identity, negation, or freshly encrypted zero operations.The transformations cover p(0), p(1), and p(−1), reducing runtime and, for multiplication by zero, resetting the noise budget.
  • Parallel operations: HE-SIMD packing encodes vectors as single ciphertexts, while OpenMP parallelizes encryption, decryption, element-wise operations, GEMM, and convolution.HE-SIMD targets vectorized payload operations; OpenMP applies to non-mini-batch dimensions.
  • Graph-level optimizations: The compiler performs graph-level folding and constant-propagation optimizations that reduce multiplicative depth for selected pooling, activation, and batch-normalization patterns.AvgPool, activation, and batch-normalization folding reduce multiplicative depth Lf from two to one in the described cases.

3.5 Ciphertext-plaintext operations

nGraph-HE supports three privacy configurations for ciphertext-plaintext computation, trading privacy against runtime according to which operands remain encrypted.

  • Ciphertext-plaintext operations: Ciphertext-plaintext operations are typically faster than ciphertext-ciphertext operations, motivating three computing paradigms based on application privacy needs.The paradigms vary whether the data, model, or both are encrypted.
  • Ciphertext-plaintext operations: Encrypted data with an unencrypted model is the fastest paradigm because it permits the greatest number of C × P operations.This setting corresponds to private patient data processed by a remote server-held model.
  • Ciphertext-plaintext operations: Encrypting both the model and data provides the most privacy but has the slowest runtime among the three paradigms.Encrypting only the model forms the intermediate use case for deploying proprietary models to untrusted servers.

3.6 Adding a new cryptosystem

nGraph-HE currently supports BFV and CKKS through SEAL and provides default higher-level operations that developers can replace with cryptosystem-specific implementations.

  • Cryptosystem support: nGraph-HE currently supports BFV and CKKS, both implemented through the SEAL encryption library.Adding another cryptosystem requires implementing its storage model and low-level assembly-language operations.
  • Cryptosystem support: Developers can override default implementations of compound operations such as Dot and Convolution with more efficient cryptosystem-specific versions.The framework supplies defaults for higher-level operations while preserving implementation-specific extensibility.
  • Framework integration: TensorFlow integration requires only one added line of code, while models from MXNet, ONNX, and PyTorch must first be exported to nGraph’s serialized format.The TensorFlow path uses Intel nGraph-TensorFlow for integration.
  • Supported operations: MaxPool and ReLU are unsupported because HE’s addition-and-multiplication operations cannot express them exactly.nGraph-HE nevertheless supports most operations commonly found in neural networks.

4 EVALUATION

The evaluation demonstrates nGraph-HE’s optimizations, framework integration, and support for encrypted neural-network inference across GEMM, MNIST, and CIFAR-10 workloads.

  • Evaluation setup: nGraph-HE was evaluated on GEMM operations, convolution-batch-norm subgraphs, and MNIST and CIFAR-10 convolutional networks using TensorFlow and multiple encryption parameters.Experiments used SEAL’s CKKS implementation on a dual-socket Intel Xeon Platinum 8180 system.
  • GEMM operations: 50% and 80% of B entries set to 1 enabled special plaintext value bypass, producing significant GEMM runtime improvements.The experiment evaluated AB + C with encrypted A and plaintext B,C under N = 2^13 and N = 2^14.
  • Graph-level optimizations: BN folding reduced multiplicative depth L_f and enabled smaller encryption parameters, improving runtime by ∼4x and reducing memory usage.The direct Conv-BN runtime decrease was approximately 4%, while the smaller parameters produced the larger improvement.
  • Framework integration: The TensorFlow integration and graph-compilation overhead was less than 0.1% of overall runtime.The measured overheads were 0.02s and 0.03s in the compared execution columns.
  • HE-SIMD packing: HE-SIMD packing made runtimes independent of batch size per network step and reduced amortized MNIST inference to 4.1ms per image at batch size 4096.For N = 2^13, batch size 1 required 16.7s, whereas batch size 4096 achieved 4.1ms per image amortized.
  • Network benchmarks: Encrypting the model incurred a ∼3.2x runtime penalty, while encrypting both data and model incurred a ∼3.6x penalty; MNIST reached ∼99% accuracy and CIFAR-10 benefited from BN.The MNIST optimized runtime was 14.8s versus 16.7s for the original CryptoNets network, at reduced accuracy.

5 CONCLUSION AND FUTURE WORK

nGraph-HE provides a graph-compiler backend for deploying deep-learning models on homomorphically encrypted data with TensorFlow and minimal code or computational overhead. Future work includes automatic compile-time selection of HE parameters and additional HE matrix-operation optimizations.

  • Conclusion: nGraph-HE supports TensorFlow-based deployment of encrypted-data neural networks with minimal computational and code overhead.Demonstrated optimizations include plaintext bypass, HE-SIMD packing, graph-level transformations, and plaintext operations.
  • Future work: Future work would automatically select HE parameters at compile time according to the desired security level.The proposed extension would use computational, especially multiplicative, depth extracted from the computation graph.

A.1 Network Architectures

The appendix specifies three network architectures used in the evaluation: CryptoNets, Binarized CryptoNets, and a CIFAR-10 network, with their layers, activations, and tensor shapes.

  • CIFAR-10 network: The CIFAR-10 network combines convolution, average pooling, convolution, and fully connected layers with polynomial activation.The architecture progresses through 40 and 80 convolutional filters before a final n × 10 output.

A.2 CryptoNets Inference

The CryptoNets inference example shows how the network is represented in source code and intermediate form, requiring only one import-line change to enable nGraph-HE.

  • CryptoNets inference: Enabling nGraph-HE for CryptoNets requires adding the import ngraph_bridge line.Figure 7 presents the corresponding source code and intermediate representation.

B NGRAPH-HE ARTIFACT APPENDIX

The artifact appendix provides public code and runtime artifacts, detailed replication instructions, and the environment used for performance analysis.

  • Public code and runtime artifacts are available for replicating the reported runtime results.The artifacts are hosted in the he-transformer repository's v0.2-benchmarks-2 tree.
  • The benchmarks folder contains detailed instructions and the runtime results underlying the paper's tables and figures.
  • Performance analysis used a Xeon Platinum 8180 platform with 112 CPUs, 376GB of RAM, and specified nGraph-HE software versions on Ubuntu 16.04.4 LTS.The setup included 2 sockets and 2.5Ghz CPUs, with HE Transformer v0.2, nGraph-tf v0.9.0, and nGraph v0.11.0.
Loading 1810.10121v3…