PaperlayerView exact revision on arXiv

Source-linked AI summary

OptiPrime: Optimizing Private Inference through Protocol-Hardware Co-design

Jiangrui Yu, Ye Yu, Si Chen, Chenqi Lin, Wenxuan Zeng, Junfeng Fan, Mingyu Gao, Meng Li

arXiv:2609.16898v1cs.ARcs.CRcs.LG

TL;DR

Hybrid HE-MPC enables private DNN inference but suffers from latency caused by HE computation and wireless ciphertext communication, limiting the benefit of HE accelerators. OptiPrime co-designs a communication-efficient convolution protocol with plaintext compression and ciphertext-reuse dataflow, outperforming Cheetah by up to 5.7× on CPUs and 4.2× with an accelerator.

  • Problem

    Hybrid HE-MPC inference remains bottlenecked by output ciphertext communication, while the denser protocol needed to reduce communication creates substantial plaintext and intermediate-ciphertext memory demands.

  • Method

    OptiPrime combines channel encoding and BSGS-based reformulation with sparse plaintext compression and a reuse-centric ciphertext dataflow.

  • Results

    OptiPrime outperforms Cheetah by up to 5.7× on CPUs and 4.2× with an accelerator.

  • Takeaways & Limitations

    Reducing transmitted ciphertexts together with memory traffic and off-chip accesses improves private inference across CPU and accelerator configurations.

  • Takeaways & Limitations

    OptiEncode reduces output ciphertexts by at most 128× and automorphisms by at most 128×, rather than eliminating communication or computation costs.

Abstract

from arXiv · show

Private deep neural network (DNN) inference based on hybrid homomorphic encryption (HE) and multi-party computation (MPC) can protect user data with a formal guarantee, but at the cost of significant latency overhead due to HE. Customized HE accelerators have been proposed and have achieved orders-of-magnitude speedup for individual HE operations. However, when directly applying a commercial HE accelerator to state-of-the-art HE-MPC frameworks, we observe only limited end-to-end performance gain. This is because HE-MPC frameworks often require wireless transmission of input and output ciphertexts for each HE operation, leading to a severe network communication bottleneck. To overcome this challenge, we introduce OptiPrime, a protocol-hardware co-optimization framework for efficient private DNN inference. OptiPrime features a novel HE protocol for convolutions that substantially reduces the number of transmitted output ciphertexts and mitigates the network communication bottleneck. Meanwhile, as the new protocol introduces complex computation for fewer output ciphertext, we observe new memory access challenges due to a high volume of weight plaintexts and intermediate ciphertexts. Hence, we further propose a lightweight compression system for the weight plaintexts, reducing memory traffic by 10 times, as well as a specialized dataflow to maximize on-chip data reuse of intermediate ciphertexts. Extensive experiments show that our framework outperforms the Cheetah baseline by at most 5.7 times on CPUs and 4.2 times with an accelerator.

I. INTRODUCTION

Hybrid HE-MPC protects private inputs and models but remains latency-limited by HE computation and ciphertext communication. OptiPrime addresses these bottlenecks through communication-efficient encoding, plaintext compression, and ciphertext-reuse dataflow, achieving substantial speedups over Cheetah.

  • I. INTRODUCTION: Hybrid HE-MPC preserves private inputs and model parameters by combining HE for linear layers with MPC for nonlinear functions.The framework involves a client holding input data and a server holding the private DNN.
  • I. INTRODUCTION: Accelerating individual HE operations yields only a modest 1.37× end-to-end reduction because wireless ciphertext communication becomes the dominant bottleneck.Linear layers dominate Cheetah latency, particularly through output ciphertext transmission after acceleration.
  • I. INTRODUCTION: OptiPrime reduces communication with a channel-encoding protocol, while OptiComp compresses sparse weight plaintexts and OptiFlow reorders computation to reuse intermediate ciphertexts on chip.OptiEncode addresses dummy coefficients; OptiComp and OptiFlow address the resulting plaintext-volume and ciphertext-working-set challenges.
  • I. INTRODUCTION: Up to 5.7× CPU and 4.2× accelerator speedups over Cheetah demonstrate OptiPrime’s end-to-end performance advantage.ResNet-18 and ResNet-50 inference latency falls to 2.9 and 14.6 seconds, respectively.
  • A. Homomorphic Encryption and Encodings: Coefficient encoding is MPC-friendly and convolution-efficient but wastes capacity on dummy coefficients, increasing the number of transmitted output ciphertexts.In the example, only two coefficients are useful while four outputs are distributed across two ciphertexts.

B. Hybrid HE-MPC Framework

The hybrid HE-MPC framework combines client-side encryption and server-side HE evaluation with MPC-based nonlinear processing. Its hardware implementation pairs CPU and wireless communication with an FPGA HE accelerator while retaining the stated honest-but-curious privacy model.

  • B. Hybrid HE-MPC Framework: The framework represents X as client and server additive shares satisfying X = ⟨X⟩c+⟨X⟩s mod t.The client and server initially hold separate shares of the activation tensor.
  • B. Hybrid HE-MPC Framework: The client encrypts an additive input share, the server reconstructs the encrypted input, evaluates the linear layer, and returns a masked output share for MPC processing.The server computes W·JXK−R = JWX −RK, while the client decrypts the returned value to obtain its output share.
  • B. Hybrid HE-MPC Framework: Both parties use CPUs and communicate over wireless networking, while the server additionally uses an FPGA-based HE accelerator.The accelerator contains HBM, an on-chip scratchpad, and compute clusters for NTT, automorphism, modular multiplication, and modular addition.
  • B. Hybrid HE-MPC Framework: OptiPrime retains the underlying privacy guarantees under the two-party honest-but-curious model because weights remain server-local and intermediate values remain encrypted or secret-shared.The protocol changes do not add secret-dependent control flow, messages, or interaction rounds.

A. Limitations of previous protocol

Cheetah’s convolution protocol leaves many dummy-filled output ciphertexts, so accelerator speedups are limited by wireless communication; OptiEncode addresses this through channel encoding and automorphism-based elimination.

  • A. Limitations of previous protocol: After hardware acceleration, wireless communication becomes the linear-layer bottleneck, limiting end-to-end speedup.Cheetah’s output ciphertext transmission is the primary communication bottleneck because its encoding produces many ciphertexts containing dummy coefficients.
  • B. Channel Encoding: OptiEncode encodes inputs and weights along the input-channel dimension to position useful coefficients at spaced polynomial degrees.When Ci is a power of two, useful coefficients occupy degrees such as {x0, xCi, x2Ci, . . .}, enabling subsequent dummy elimination.
  • A. Limitations of previous protocol: OptiEncode reduces output ciphertexts by at most 128× and, after reformulation, reduces automorphisms by at most 128× versus Cheetah.The reformulation also enables the baby-step-giant-step algorithm to reduce automorphism cost.
  • B. Channel Encoding: Automorphisms selectively preserve or negate polynomial coefficients, allowing dummy terms to cancel while valid terms remain.Summing a polynomial with its automorphic image cancels odd-degree terms; repeated automorphisms isolate coefficients at multiples of 2^r.
  • B. Channel Encoding: The elimination process preserves valid coefficients at degrees such as x0 and x4 while zeroing intermediate dummy coefficients.The outputs are then realigned with plaintext multiplication and recombined; the encoding is specifically designed to make this automorphism-based cleanup effective.

C. Reducing Automorphisms: a BSGS Approach

OptiPrime reduces automorphism cost by moving automorphisms to inputs, exploiting geometric progressions, and applying BSGS, while precomputing constant-weight terms.

  • C. Reducing Automorphisms: a BSGS Approach: The comparison table evaluates protocols on one ImageNet-ResNet18 layer with Co = 256, Ci = 256, H = 14, W = 14, and N = 8192.The caption defines f = h × w as the number of filter elements.
  • C. Reducing Automorphisms: a BSGS Approach: Geometrically progressing automorphisms can be represented as powers of one base automorphism, enabling the BSGS reduction.The example represents the required set as {σ2, σ}.
  • C. Reducing Automorphisms: a BSGS Approach: Automorphisms can be applied to input ciphertexts and plaintexts before multiplication, avoiding repeated operations on multiple intermediate outputs.The example has one input ciphertext but four intermediate output ciphertexts, motivating input-side transformation.
  • C. Reducing Automorphisms: a BSGS Approach: Precomputing constant-weight combinations reduces the example’s required automorphisms from eight to three.The reformulated dataflow defines q̂j from weighted plaintext terms before the final recombination.
  • C. Reducing Automorphisms: a BSGS Approach: 128× fewer rotations are achieved than without the BSGS optimization.The method reduces automorphisms from three to two in the running example and reports 128× savings generally.

A. Memory Bottlenecks of our protocol

Profiling shows that OptiPrime’s protocol is heavily memory-bound: plaintext traffic is large, ciphertexts thrash memory, and evaluation keys are not dominant.

  • A. Memory Bottlenecks of our protocol: Plaintexts create significant memory traffic because sparse encoding requires many weight plaintext polynomials.The profiling identifies plaintext volume as a direct consequence of the protocol’s sparse encoding scheme.
  • A. Memory Bottlenecks of our protocol: Evaluation keys are not the primary memory bottleneck because each key shrinks quadratically with level and BSGS reduces the number of keys required.The profiling gives 384KB for L = 1 as an example of the reduced key size.
  • A. Memory Bottlenecks of our protocol: 0.6 Ops/Byte classifies the workload as heavily memory-bound because plaintext and ciphertext movement dominates arithmetic intensity.The profiling attributes the low arithmetic intensity to large data movement for both data types.
  • A. Memory Bottlenecks of our protocol: Ciphertext thrashing is a separate bottleneck addressed by a reuse-centric dataflow that maximizes on-chip locality and reuse.OptiFlow reorders computation to improve intermediate-ciphertext reuse.

B. OptiComp: Plaintext Compression System

OptiComp compresses sparse weight plaintexts before storage and decompresses them on-chip, reducing memory traffic while preserving the plaintexts needed for HE computation.

  • Design trade-off: Storing compressed plaintexts in DRAM and decompressing them on-chip avoids both large data transfers and costly on-chip plaintext automorphisms.The approach is selected over storing evaluation-domain plaintexts or performing encoding and NTT entirely on-chip.
  • Compression design: OptiComp discards zero-runs and stores only non-zero blocks in fixed-size segments with metadata for each block’s position, length, and end marker.Segments address alignment and address-calculation problems caused by variable-length sparse sections.
  • System integration: The compiler packs annotated ComPlain() plaintexts into segments and emits compression information and decompression instructions for operations requiring full plaintexts.A dedicated on-chip unit reconstructs the full sparse polynomial from fetched compressed segments before computation such as NTT.

C. OptiFlow: A Reuse-Centric Dataflow

OptiFlow resolves the conflict between computation-efficient BSGS splits and memory-efficient execution by reordering work around on-chip reuse of ciphertext contributions.

  • Bottleneck: A large baby-step count is computationally efficient but creates Ni · 2^k1 temporary ciphertexts, causing scratchpad overflow and severe HBM thrashing.The temporary ciphertext working set is repeatedly fetched despite its high reuse potential.
  • OptiFlow: OptiFlow processes one input ciphertext at a time and computes its contributions to all outputs before advancing, avoiding materialization of the full temporary set.This combines computation efficiency with reduced memory traffic.
  • Dataflow: Output-stationary execution keeps partial sums on-chip while successive baby-step automorphisms update all dependent outputs.If partial sums exceed capacity, computation is tiled along the output dimension.
  • Dataflow: OptiFlow recomputes baby-step ciphertexts on the fly because hoisted automorphisms cost less than a single HBM access.The design therefore trades recomputation for avoiding off-chip temporary-ciphertext traffic.

V. EVALUATION

The evaluation measures OptiPrime across CNNs and examines both end-to-end performance and the contributions of its protocol and memory optimizations.

  • Evaluation scope: The evaluation covers different CNN networks on ImageNet and combines end-to-end benchmarking with detailed analysis of protocol and memory-efficient optimizations.The reported evaluation proceeds from overall network results to component-level analysis.

A. Methodology

OptiPrime is evaluated on CPU and FPGA-accelerated platforms under LAN and WAN conditions against multiple HE-MPC encoding baselines, showing lower normalized latency across configurations.

  • Implementation and setup: OptiPrime is implemented on the LattiSense platform with host-side cryptographic and networking operations and accelerator instructions generated from a Python-defined HE graph.Experiments use an Intel Xeon CPU, an AMD Alveo U55C FPGA, and simulated LAN/WAN conditions.
  • Methodology: The benchmarks use ImageNet with ResNet, VGG, and MobileNetV2, comparing Cheetah, Hyena, and an adapted Orion protocol under common cryptographic parameters.Results are normalized to the Cheetah+CPU baseline.
  • End-to-end results: OptiPrime achieves normalized latencies of 0.02×–0.3× on CPU in LAN and 0.03×–0.56× in WAN, relative to Cheetah+CPU.With accelerators, normalized latencies are 0.4×–0.78× in LAN and 0.29×–0.65× in WAN.
  • Baseline comparison: OptiPrime is 1.4–1.6× faster than Orion under WAN and 1.3–1.5× faster under LAN.The comparison attributes the difference to fewer HE operations and less communication from coefficient encoding.
  • Latency breakdown: Compared with Cheetah, OptiPrime reduces network communication latency, while Hyena’s overall performance is limited by computational cost over 10× higher.Orion also has low network latency but remains slower because its per-layer HE computation is higher.

C. Evaluation of OptiPrime’s Protocol

OptiPrime reduces communication and computational costs through channel encoding, while hardware co-design addresses the resulting plaintext-memory and intermediate-ciphertext access overheads.

  • Protocol efficiency: 0.11×–0.59× network communication and 0.07×–0.46× output ciphertexts versus Cheetah demonstrate the protocol’s communication advantage.The number of input ciphertexts remains similar across schemes.
  • Protocol efficiency: 0.05×–0.07× CPMults and 0.04×–0.07× automorphisms versus Hyena show the computational efficiency of BSGS and coefficient-based encoding.Without BSGS, OptiPrime still requires only 0.13×–0.19× Hyena’s automorphisms.
  • Protocol efficiency: The larger plaintext count trades memory traffic for fewer automorphisms and substantially lower communication, which is favorable when acceleration makes computation relatively inexpensive.Hyena uses fewer plaintexts but more computationally intensive operations and remains inefficient over wireless networks.
  • Hardware co-design: 0.7×–0.85× baseline memory access after plaintext compression demonstrates reduced traffic before applying the ciphertext dataflow.The evaluation uses evaluation-domain plaintexts and the computation-efficient BSGS baby-step selection as its baseline.
  • End-to-end evaluation: 1.7–2.4× LAN and 1.2–1.7× WAN speedups from acceleration are followed by up to 3.3× WAN speedup from channel encoding in the ablation.On LAN, channel encoding yields 1.0–1.6× because extra computation partly offsets communication savings.

F. Generality of OptiPrime

OptiPrime generalizes across accelerator platforms, Transformer models, network conditions, layer shapes, HE parameters, and quantized CNN inference while retaining performance and accuracy benefits.

  • Transformer models: 0.67×/0.23× of Iron’s LAN/WAN latency and 0.57×/0.69× of BumbleBee’s latency show OptiPrime’s applicability to BERT-base linear layers.Each matrix multiplication is implemented as a 1 × 1 convolution.
  • Accuracy and validity: Under W8A8 quantization, OptiPrime matches prior protocols’ ImageNet top-1 accuracy with only negligible quantization loss versus FP32.The measured accumulator peaks at 19.3 bits, below the plaintext modulus t = 221.
  • Network and parameter sensitivity: 1.71–6.12× and 2.04–6.69× speedups over Cheetah are achieved on ResNet-18 and ResNet-50 across the evaluated bandwidth range.OptiPrime retains 1.71× and 2.04× speedups even at 1 GB/s.
  • Network and parameter sensitivity: OptiPrime consistently outperforms Cheetah across the tested network conditions, while its accelerator-side optimizations continue reducing non-network latency even at high bandwidth.The network study varies bandwidth from 10 MB/s to 1 GB/s and RTT from 1 ms to 300 ms.
  • Network and parameter sensitivity: WAN speedup grows from 2.4× to 12.9× on ResNet-18 and from 2.7× to 13.6× on ResNet-50 as polynomial degree N increases from 4096 to 32768.The speedup holds as log q increases from 64 to 128 bits.
  • Overall evaluation: The framework’s overall scope is summarized by up to 5.7× CPU and 4.2× accelerator speedups over Cheetah.OptiPrime combines communication-efficient channel encoding with plaintext compression and ciphertext-friendly dataflow.
Loading 2609.16898v1…