PaperlayerView exact revision on arXiv

Source-linked AI summary

CrypTFlow2: Practical 2-Party Secure Inference

Deevashwer Rathee, Mayank Rathee, Nishant Kumar, Nishanth Chandran, Divya Gupta, Aseem Rastogi, Rahul Sharma

arXiv:2010.06457v1cs.CRcs.LG

TL;DR

Practical secure inference must handle nonlinear layers and fixed-point arithmetic efficiently without losing correctness or security. CrypTFlow2 develops new comparison, DReLU, and division protocols within OT- and HE-based 2PC systems, enabling secure ImageNet-scale inference while outperforming prior work in latency and scale.

  • Problem

    Secure inference for practical ML tasks remains challenging because realistic DNNs use expensive ReLUs and faithful fixed-point arithmetic is needed for correctness.

  • Method

    CrypTFlow2 combines new protocols for comparison, DReLU, and division with two inference systems using OT- or homomorphic-encryption-based linear layers.

  • Results

    CrypTFlow2 provides the first secure 2-party inference over ImageNet-scale DNNs and outperforms prior work by an order of magnitude in both latency and scale.

  • Takeaways & Limitations

    The framework makes secure, efficient, and correct inference practical for realistic DNN benchmarks, including ResNet50 and DenseNet121.

  • Takeaways & Limitations

    CrypTFlow2 considers only static semi-honest adversaries and maintains arithmetic shares across secure layer computations.

Abstract

from arXiv · show

We present CrypTFlow2, a cryptographic framework for secure inference over realistic Deep Neural Networks (DNNs) using secure 2-party computation. CrypTFlow2 protocols are both correct -- i.e., their outputs are bitwise equivalent to the cleartext execution -- and efficient -- they outperform the state-of-the-art protocols in both latency and scale. At the core of CrypTFlow2, we have new 2PC protocols for secure comparison and division, designed carefully to balance round and communication complexity for secure inference tasks. Using CrypTFlow2, we present the first secure inference over ImageNet-scale DNNs like ResNet50 and DenseNet121. These DNNs are at least an order of magnitude larger than those considered in the prior work of 2-party DNN inference. Even on the benchmarks considered by prior work, CrypTFlow2 requires an order of magnitude less communication and 20x-30x less time than the state-of-the-art.

1 INTRODUCTION

CrypTFlow2 addresses the efficiency and correctness challenges of secure 2-party inference for realistic DNNs. Its new comparison, DReLU, and division protocols support practical ImageNet-scale models through both OT- and HE-based linear layers.

  • 1 INTRODUCTION: Secure inference for practical DNNs is difficult because ReLUs are expensive to compute securely and fixed-point arithmetic must remain faithful to cleartext execution.Prior approaches either approximate ReLUs with accuracy loss, weaken security, leak activations, or sacrifice correctness in division.
  • 1.1 Our Contributions: CrypTFlow2 introduces new millionaires’ and DReLU protocols for securely evaluating nonlinear layers including ReLU, Maxpool, and Argmax.The millionaires’ protocol has approximately 3× better theoretical communication than the most communication-efficient prior protocol and runs in log ℓ rounds.
  • 1.1 Our Contributions: CrypTFlow2 supports two secure and correct inference protocols: SCIOT uses oblivious transfer for linear layers, while SCIHE uses homomorphic encryption.Both use the new activation and division protocols, with faithful divisions after linear layers.
  • 1.1 Our Contributions: The framework adds correct, efficient protocols for division by powers of two and arbitrary integers over both Z_L and Z_n.For 7 × 7 average pooling with 32-bit integers, the protocols use approximately 54× less communication than the compared garbled-circuit approach.
  • 1.1 Our Contributions: The system securely evaluates ImageNet-scale models including SqueezeNet, ResNet50, and DenseNet121, whereas prior 2-party work was limited to smaller DNNs and datasets.ResNet50 and DenseNet121 are at least an order of magnitude larger than the DNNs considered in prior work.
  • 1.1 Our Contributions: CrypTFlow2 evaluates the OT-versus-HE trade-off by implementing both protocol classes and benchmarking them on ImageNet-scale inference.HE reduces communication but requires more computation, while OT can be preferable when computation is less limiting.

2 PRELIMINARIES

The paper defines the sharing, oblivious-transfer, homomorphic-encryption, and security primitives used by its secure inference protocols.

  • Threat model and security: Security is defined against a static semi-honest PPT adversary corrupting either party at protocol start.Security follows the simulation paradigm, with real and ideal interactions indistinguishable to the environment.
  • Secret sharing: 2-out-of-2 additive sharing is used over Z2, ZL with L=2^ℓ, and arbitrary Zn.Shares over ZL and Zn are arithmetic shares, while shares over Z2 are boolean shares.
  • Secret sharing: Additive shares reconstruct by ring addition, while each individual share completely hides the shared value.The reconstruction algorithm sums the two shares in the corresponding ring.
  • Oblivious transfer: 1-out-of-k OT returns the receiver’s selected sender message, while COT returns correlated ring-element outputs.The cited OT and COT protocols execute in two rounds with communication 2λ+kℓ and λ+ℓ, respectively.
  • Supporting functionalities: B2A converts boolean shares into arithmetic shares, and MUX selects a shared value based on a boolean choice bit.B2A uses one COT invocation, while MUX can use two simultaneous OT calls with communication 2(λ+2η).
  • Homomorphic encryption: The homomorphic-encryption component requires additive homomorphism supporting addition and scalar multiplication.The paper uses BFV and optimized algorithms for homomorphic matrix-vector products.

3 MILLIONAIRES’ AND DReLU PROTOCOLS

The paper develops secure comparison and DReLU protocols using recursive block comparisons, oblivious transfer, and optimized ring-specific constructions.

  • Millionaires’: The millionaires’ functionality securely outputs shares of the predicate 1{x<y} for two parties holding ℓ-bit integers.The inputs are represented as blocks, and the final comparison is obtained from recursively combining block inequalities and equalities.
  • Millionaires’: Recursive comparison organizes block-level inequalities and equalities in a tree whose root gives the final comparison.Leaf comparisons use OT, while internal nodes combine results using the comparison identity.
  • Millionaires’: The protocol’s correctness is proved by induction on the depth of the comparison tree.The base case follows from the leaf functionality, and the induction step uses FAND correctness.
  • Millionaires’: The general comparison case handles nondivisible input lengths and non-power-of-two block counts with modified final blocks and recursion trees.Maximal perfect binary subtrees are connected recursively to cover the remaining leaves.
  • Millionaires’: The protocol combines OT and correlated-bit-triple optimizations to reduce communication, with a communication–computation trade-off controlled by block size.For ℓ=32, the stated minimum communication occurs at m=7.
  • DReLU: DReLU over ZL derives the sign from the most significant bit and a carry comparison, while general Zn uses ring-specific comparisons.For general rings, the optimized protocol reduces the number of FMILL calls to two.
  • DReLU: For Zn, DReLU is one when the ring value lies below ⌈n/2⌉, and the protocol computes the result through comparisons involving wraparound.The optimized construction is described as reducing FMILL invocations to two.

4 DIVISION AND TRUNCATION

The paper gives correct secure protocols for division and truncation over integer rings, using correction terms to preserve cleartext fixed-point behavior.

  • Overview: The division and truncation protocols are bitwise equivalent to the corresponding cleartext computations.The protocols cover division by positive integers and truncation by powers of two.
  • Division: Division of a secret ring element is decomposed into division of secret shares plus correction terms.The correction terms are expressed using inequalities on secret values, with truncation obtained as a simpler special case.
  • Truncation: Truncation for ℓ-bit integers is treated as division by 2^s and realized through a specialized functionality.The construction relies on the truncation corollary derived from the general division theorem.
  • Truncation: The truncation protocol for ℓ=32 and s=12 communicates 4310 bits, compared with 24064 bits for garbled circuits.These are concrete communication figures for the stated parameter setting.
  • General division: General division computes correction comparisons through DReLU calls over a larger ring chosen to avoid overflows and underflows.The protocol uses δ=⌈log 6d⌉ for the comparison domain in the described construction.
  • Special-case truncation: A special ring relation can omit one comparison and reduce the required comparison domain for truncation.When 2·n0≤2^s, the third comparison is omitted and δ=⌈log 4d⌉ suffices.
  • Special-case truncation: The homomorphic-encryption benchmarks use a prime modulus with n0=1 for 1≤s≤14, enabling the special truncation optimization for s≤12.The paper states that this optimization is used for truncation in SCIHE.

5 SECURE INFERENCE

CrypTFlow2 securely composes protocols for linear and nonlinear neural-network layers while preserving arithmetic shares across layers. Its OT- and HE-based variants support faithful fixed-point computation, including correct truncation and division.

  • Secure inference decomposes neural-network inference into linear layers, such as convolutions and matrix multiplication, and nonlinear layers, such as ReLU, Maxpool, and Argmax.
  • Arithmetic shares of each layer’s input are transformed into shares of its output over the same ring, enabling sequential composition of arbitrary supported layers.SCIOT uses Z_L, while SCIHE uses the prime field Z_n.
  • Faithful fixed-point multiplication and convolution compute ring operations followed by division-by-2^s truncation, producing bitwise-equivalent results to cleartext execution.CrypTFlow2 uses DIV in SCIHE for the required scaling step.
  • SCIOT evaluates linear layers with oblivious transfer over Z_L, whereas SCIHE uses homomorphic encryption over Z_n and applies the new activation protocols in each domain.
  • The Avgpool protocol uses local addition followed by secure division, and its execution is bitwise equal to the cleartext version.
  • New millionaires’ and DReLU protocols support secure ReLU, Maxpool, and Argmax computation over both power-of-two rings and general rings.Maxpool over d inputs uses d−1 sequential comparisons.

6 IMPLEMENTATION

The implementation integrates CrypTFlow2’s protocols into the CrypTFlow TensorFlow pipeline and optimizes both OT- and HE-based backends. Implementation choices balance communication, computation, and truncation costs.

  • CrypTFlow2 integrates SCIOT and SCIHE as cryptographic backends in CrypTFlow, allowing unmodified TensorFlow inference code to compile into secure protocols.Athos is modified to support faithful fixed-point arithmetic.
  • Re-keyed AES optimizations for OT extension, including key expansion and pipelining, improve AES_RK performance by roughly 6×.
  • The millionaires’ protocol exposes a parameter m that trades communication against computation; empirically, m=4 performs best despite m=7 minimizing communication for 32-bit inputs.At m=4, communication is about 30% higher while computation is approximately 3× lower.
  • SCIHE linear layers build on SEAL/Delphi, using parallelized convolutions, modulus switching, and strided convolutions to improve convolution performance.
  • Peephole optimization moves truncation after ReLU, eliminating redundant DReLU calls and adjusting addition scaling to preserve correctness.

7 EXPERIMENTS

Experiments evaluate CrypTFlow2 across garbled-circuit, Delphi, and ImageNet-scale benchmarks. The results show faster and more communication-efficient nonlinear computation, plus scalable secure inference with both OT and HE backends.

  • 7.3 Evaluation on practical DNNs: Backend performance depends on network conditions and bitwidth: SCIOT leads for some LAN benchmarks, while SCIHE leads for ResNet50 and WAN settings.ResNet50 requires 37-bit fixed-point values, compared with 32 bits for SqueezeNet and DenseNet121.
  • 7.1 Comparison with Garbled Circuits: 2×–25× speedups over garbled circuits occur for layers computing 2^13–2^20 ReLUs, with larger gains in WAN settings and for larger layers.The paper attributes WAN gains to amortized round costs and communication dominating runtime.
  • 7.1 Comparison with Garbled Circuits: Up to 9× and 21× faster runtimes than garbled circuits are achieved in LAN and WAN settings respectively for protocols over Z_n, with approximately 9× less communication.
  • 7.1 Comparison with Garbled Circuits: Avgpool evaluation achieves up to 51× lower runtime and 41× lower communication than garbled circuits.
  • 7.2 Comparison with Delphi: 20×–30× faster nonlinear-layer evaluation than Delphi is reported on MiniONN and ResNet32, with 12× and 9× communication reductions respectively.
  • 7.2 Comparison with Delphi: CrypTFlow2 is an order of magnitude more efficient than Delphi in online runtime because the online cost is concentrated in nonlinear layers.
  • 7.3 Evaluation on practical DNNs: Both SCIOT and SCIHE evaluate SqueezeNet in under a minute and scale to ResNet50 and DenseNet121.Across all three benchmarks, total time stays within 10 minutes on LAN and 20 minutes on WAN.

8 CONCLUSION AND FUTURE WORK

CrypTFlow2 provides secure, efficient, and correct 2-party DNN inference, extending evaluation to ImageNet-scale models. The conclusion identifies security, model-hiding, and secure-training extensions as future directions.

  • Conclusion: CrypTFlow2’s implementations outperform prior work by an order of magnitude in both latency and scale while providing secure and correct 2-party inference.
  • Conclusion: The system demonstrates ImageNet-scale secure inference previously addressed with 3-party protocols or leaked intermediate computations.
  • Future Work: CrypTFlow2 supports semi-honest adversaries, while malicious-adversary security remains a future direction.
  • Future Work: SCIHE hides filter sizes, strides, and whether a layer is convolutional or fully connected, but reveals more than fully homomorphic-encryption tools.
  • Future Work: Secure training at ImageNet scale is future work because it would require protocols that leverage specialized compute and networking hardware.
  • Supporting Protocols: The supporting protocols provide concrete communication costs, including 3298 bits for 32-bit DReLU and 5288 bits for the corresponding general-ring setting.

C PROOF OF DIVISION THEOREM

The proof decomposes secret division into share-wise division plus correction terms determined by inequalities. It verifies the correction expression by analyzing the possible values of the quotient-related term.

  • Case analysis: The wrap-bit w adjusts the share sum by subtracting w·n before division, yielding the expressions used for the correction analysis.The proof rewrites the represented value using the two shares and the wrap-bit, then derives the division form from that representation.
  • Correction decomposition: The proof expresses secret division using share-wise division and a correction term derived from the reconstructed share components.The correction combines terms involving corr, c0, and B to account for wrap-around and quotient adjustments.
  • Correction decomposition: The correction indicator C is computed from three inequalities on A: A < d, A < 0, and A < −d.The proof defines C0, C1, and C2 from these comparisons and sets C = C0 + C1 + C2.
  • Case analysis: The proof bounds A by −2d + 2 ≤ A ≤ 2d − 2, restricting c0 to {−2, −1, 0, 1}.Each possible value of c0 is matched to the corresponding interval for A and to 1 − C.

D PROTOCOL FOR GENERAL DIVISION

The general-division protocol implements the theorem’s correction terms securely using DReLU, oblivious transfer, and share conversion across ring domains. It supports both arbitrary rings and ℓ-bit integer domains with concrete communication bounds.

  • Protocol construction: The protocol computes correction terms by evaluating DReLU on A − d, A, and A + d, requiring three calls to F int,δ.These calls produce the inequality indicators needed to construct C securely.
  • Protocol construction: The protocol computes shares of corr over both Z_n and Z_Δ, where Δ = 2^δ, to support subsequent arithmetic operations.The protocol uses δ = ⌈log 6d⌉ and performs share conversion and correction-term computation across the required domains.
  • Protocol construction: The protocol sets B_b = idiv(a^0_b − x_b·n_0, d) for each party, incorporating comparison outputs into the share-wise quotient.The comparison bit x_b is computed from whether each share reaches the threshold n′.
  • Communication: 7796 bits is the best reported communication for DIV when η = 32 and millionaire invocations use m = 7.For Πint,32,49 DIV, setting m = 8 yields 5570 bits.
  • Communication: 5570 bits is the best reported communication for Πint,32,49 DIV with m = 8 in all millionaire invocations.The general DIV expression is bounded by a term proportional to (λ + 21)·(ℓ + 3δ).

F IMPROVEMENT TO GAZELLE’S ALGORITHM

The paper optimizes Gazelle’s output-rotations convolution method by grouping similarly misaligned ciphertexts before alignment. This reduces rotations and can make output rotations preferable when f2·c_i > c_o.

  • Baseline method: Gazelle’s output-rotations method accumulates c_i·c_o/c_n intermediate ciphertexts into tightly packed outputs after homomorphic convolution.Misaligned ciphertexts require rotations to align their slots before packing.
  • Optimization: Grouping ciphertexts by shared misalignment lets each subgroup be added first and aligned with one rotation.The groups contain c_i/c_n ciphertexts, reducing redundant alignment operations.
  • Optimization: The optimization reduces the rotation count by a factor of c_i/c_n to approximately c_n·c_o.This reduction applies to the output-rotations method’s ciphertext-alignment stage.
  • Comparison: Output rotations are better than input rotations when f2·c_i > c_o, which the paper says is usually the case.The comparison is driven by the filter size and the input and output channel counts.

G COMPLEXITY OF OUR BENCHMARKS

The benchmarks span DNNs with substantial convolution, activation, and pooling workloads, including ImageNet-scale ResNet50 and DenseNet121. Against garbled circuits, the protocols deliver especially large communication improvements across all three networks.

  • Benchmark complexity: ResNet50 has 53 convolution layers, 49 ReLU layers totaling 9,006,592 ReLUs, and a largest ReLU layer of 802,816 elements.Its peak output channel count is 2048, and it also includes Maxpool and Avgpool layers.
  • Performance comparison: DenseNet121 performs 176,640 divisions in the Avgpool evaluation, allowing protocol runtime improvements over garbled circuits exceeding 32× in LAN and 45× in WAN.The passage attributes the larger improvement to the number of divisions available for amortization.
  • Performance comparison: Communication improves by more than 19× on SqueezeNet, 27× on ResNet50, and 31× on DenseNet121 versus garbled circuits.These communication improvements hold in both protocols evaluated and across the reported settings.

I FIXED-POINT ACCURACY OF OUR BENCHMARKS

The fixed-point implementation matches the accuracy of the input TensorFlow code, enabling secure inference to retain fixed-point accuracy.

  • The fixed-point code matches the accuracy of the input TensorFlow code.Faithful truncation and division protocols ensure secure inference has the same accuracy as the fixed-point implementation.
Loading 2010.06457v1…