PaperlayerView exact revision on arXiv

Source-linked AI summary

Helen: Maliciously Secure Coopetitive Learning for Linear Models

Wenting Zheng, Raluca Ada Popa, Joseph E. Gonzalez, Ion Stoica

arXiv:1907.07212v2cs.CRcs.LG

TL;DR

Organizations need collaborative model training over sensitive, complementary datasets, but privacy regulations and business competition prevent plaintext sharing. Helen combines maliciously secure MPC with a linear-model training protocol for this coopetitive setting. It achieves up to five orders of magnitude of performance improvement over a state-of-the-art malicious secure baseline while tolerating corruption of m −1 out of m parties.

  • Problem

    Organizations cannot safely share sensitive datasets in plaintext, while existing secure training approaches often rely on passive attackers or non-colluding servers.

  • Method

    Helen is a maliciously secure coopetitive-learning platform that trains regularized linear models using ADMM-based protocols and cryptographic techniques.

  • Results

    Up to five orders of magnitude: Helen improves performance over a state-of-the-art malicious secure framework baseline.

  • Takeaways & Limitations

    Helen provides a practical approach to collaborative linear-model training when each party must trust only itself.

  • Takeaways & Limitations

    Helen does not protect against attacks that recover training data from the released public model, so parties decide whether to share that model.

Abstract

from arXiv · show

Many organizations wish to collaboratively train machine learning models on their combined datasets for a common benefit (e.g., better medical research, or fraud detection). However, they often cannot share their plaintext datasets due to privacy concerns and/or business competition. In this paper, we design and build Helen, a system that allows multiple parties to train a linear model without revealing their data, a setting we call coopetitive learning. Compared to prior secure training systems, Helen protects against a much stronger adversary who is malicious and can compromise m-1 out of m parties. Our evaluation shows that Helen can achieve up to five orders of magnitude of performance improvement when compared to training using an existing state-of-the-art secure multi-party computation framework.

1 Introduction

Organizations want to train models on combined sensitive data, but privacy rules and business competition prevent plaintext sharing. Helen addresses this coopetitive-learning setting with malicious security and techniques designed to make training practical at scale.

  • Motivation: Coopetitive learning combines organizations’ sensitive datasets for shared model training without requiring plaintext data sharing.The setting can support applications needing complementary data, such as cross-bank money-laundering detection or geographically diverse medical data.
  • Threat model: The security goal is to tolerate malicious behavior by m −1 of m parties, so each organization needs to trust only itself.Prior approaches often assume passive attackers or non-colluding outsourced servers.
  • Contribution: Helen is a platform for maliciously secure coopetitive learning focused on regularized linear models, including least squares, ridge regression, and LASSO.These models are described as statistically robust and interpretable, with applications including cancer research, genomics, and financial risk analysis.
  • System setting: Helen targets a setting with fewer than 10 organizations, each holding hundreds of thousands or millions of records and tens or hundreds of features.The intended workloads have many samples but relatively few features.
  • Performance: 3 months versus less than 3 hours: for four parties with 100K points and 90 features each, Helen outperforms an SPDZ-based SGD baseline for linear regression.The baseline is an estimated training time, while Helen trains the same model in under three hours.
  • Techniques: Helen combines ADMM, sample-independent cryptographic computation, and decentralized local optimization to improve scalability under malicious security.These techniques target the linear scaling of training with samples while reducing expensive cryptographic work and preserving local computation.

2 Background

The paper formulates regularized linear learning for multiple parties and uses ADMM to separate local optimization from global coordination. Helen’s secure protocol protects private inputs and intermediate values while iterating toward a shared model.

  • Preliminaries: Each party holds a feature matrix X_i and label vector y_i, represented as fixed-point integers for cryptographic computation.The data dimensions are n samples per party and d features.
  • Cryptographic primitives: Partially homomorphic encryption permits limited computation on ciphertexts, including additive operations in the Paillier example.Helen also uses the scheme as a homomorphic commitment.
  • Cryptographic primitives: Threshold encryption splits the secret key across parties so all parties must participate before Helen decrypts a ciphertext.The public key is shared, while each party receives a secret-key share.
  • Cryptographic primitives: Zero-knowledge proofs let a party prove a statement about secret information without revealing that secret.Helen uses modified Sigma-protocols for properties of local computation.
  • Malicious MPC: SPDZ represents values with shares and authentication data, and its offline phase prepares function-independent values for online execution.The global authentication key remains hidden until the protocol ends.
  • Learning and convex optimization: Regularization functions such as L1 and L2 penalties improve prediction accuracy on high-dimensional data, corresponding to LASSO and ridge regression.Helen focuses on squared-error linear models with these regularization choices.
  • ADMM: ADMM splits the model into party-local variables constrained to equal a global model z, then optimizes an augmented Lagrangian.The penalty scaled by rho discourages local models from deviating from z.
  • ADMM: Each ADMM iteration performs local model updates, global-weight coordination, and local Lagrange-multiplier updates until a fixed point or fixed iteration count.Helen uses a fixed number of iterations as its stopping condition.

3 System overview

Helen enables organizations to jointly train regularized linear models while keeping their datasets encrypted, under a threat model allowing malicious compromise of up to m−1 of m parties. Its protocol combines threshold encryption, commitments, zero-knowledge proofs, and ADMM-based computation, but requires all parties to participate and does not prevent model-level or poisoning attacks.

  • System scope: Helen supports regularized linear models, including least squares regression, ridge regression, LASSO, and elastic net.The paper focuses its explanation on LASSO and also provides ridge-regression update equations.
  • Threat model: Up to m−1 of m parties may be maliciously compromised, with security formalized against arbitrary protocol deviations.Theorem 6 states secure evaluation against a malicious adversary statically corrupting up to m−1 parties.
  • Security boundaries: Helen does not prevent poisoning attacks or attacks that recover training data from the released public model.It binds parties to consistent inputs during computation, but parties remain responsible for deciding whether to share the model; differential privacy is described as complementary.
  • Protocol phases: Helen’s protocol uses threshold encryption, encrypted input commitments, zero-knowledge consistency proofs, and iterative ADMM computation.Initialization creates shared decryption keys; input preparation commits encrypted summaries; model computation alternates local encrypted computation with MPC coordination.
  • Model release: All parties jointly decrypt and release the final model, so a nonparticipating or misbehaving party can prevent model production.Helen uses a threshold structure requiring all parties to participate in decryption, and it does not provide fairness in the malicious-majority setting.

4 Cryptographic Gadgets

Helen uses modular zero-knowledge gadgets to verify encrypted matrix computations without revealing private matrices. The main optimization reduces matrix-multiplication proof cost by checking a randomly projected relation instead of every individual product.

  • Gadget framework: Helen decomposes its cryptographic design into modular gadgets that prove knowledge and correctness of encrypted matrix computations.The gadgets are composed into the overall protocol and use zero-knowledge proofs to avoid revealing private values.
  • Gadget 1: Gadget 1 proves that an encrypted output represents the product of a plaintext matrix and an encrypted matrix without revealing the plaintext matrix.The prover demonstrates both knowledge of the plaintext and correct homomorphic computation.
  • Gadget 1: For l×l matrices, the naïve proof costs l^3 because it proves every individual product.Helen reduces this cost by asking the prover to establish tZ = (tX)Y for a randomly chosen vector t.
  • Gadget 1: The randomly projected proof has exponentially small soundness error when a prover attempts to construct an incorrect projected product.The reduced relation is then proved using Paillier ciphertext multiplication proofs.
  • Gadget 2: Gadget 2 proves encrypted matrix multiplication when the prover knows both input matrices and the output matrix.It adds a proof-of-knowledge component for the second input matrix.

5 Input preparation phase

Helen prepares encrypted summaries for each party’s data and uses an SVD-based representation to make malicious-input consistency proofs depend on feature dimension rather than sample count. This replaces costly commitments to full datasets with commitments to compact matrices and vectors.

  • Summary commitment: Each party precomputes encrypted data summaries and commits to them because malicious parties could otherwise provide inconsistent summaries during training.The reused summaries include A_i = (X_i^T X_i + ρI)^−1 and b_i = X_i^T y_i.
  • Baseline approach: Directly proving summaries from the full dataset scales linearly in n and requires heavy matrix inversion inside SPDZ.This approach would commit to the entire input or process data-dependent computations through MPC.
  • SVD formulation: Helen instead uses an SVD formulation whose committed matrices scale linearly in d while remaining equivalent to proving properties derived from X_i and y_i.The formulation uses V, Θ, Σ, and y* to represent the required summary properties.
  • SVD formulation: The SVD constraints require V to be orthogonal and Θ to have diagonal entries 1/(σ_i^2 + ρ).The diagonal entries of Σ contain the singular values σ_i, while ρ is public.
  • Verification: Helen commits only to matrices of sizes d×d or d×1, removing n-dependent scaling from input preparation.The parties encrypt V, Σ, Θ, y*, A, and b, then use zero-knowledge and interval proofs to verify the required relations and numerical ranges.

6 Model compute phase

Helen’s model compute phase runs iterative ADMM training over encrypted weights, combining local optimization with MPC-based coordination. Conversion gadgets and proofs connect homomorphic encryption with SPDZ while checking consistency and valid authenticated shares.

  • Phase overview: The model compute phase generates an encrypted weight vector through initialization, ADMM training, and model release, after which participants jointly decrypt the model parameters.Training alternates local optimization and coordination before releasing the plaintext model.
  • Initialization and local optimization: Helen initializes weights and ADMM variables to zero, then each party locally updates its weights using encrypted data and proves the computation was performed correctly.Local updates use plaintext scaling and plaintext-ciphertext matrix multiplication; Gadget 1 produces the proof.
  • Coordination: During coordination, Helen converts encrypted weights into arithmetic shares, computes the soft-threshold update in SPDZ, and converts the resulting shares back into encrypted form.MPC also reduces scaling factors accumulated during fixed-point matrix multiplication.
  • Verification and release: After MPC, parties re-encrypt shares of the weights, z, and dual variables, publish interval and MAC proofs, and jointly release the model only after consistency checks succeed.The global MAC key remains unrevealed until the protocol’s final verification steps.
  • Conversion verification: Gadget 4 verifies that encrypted values match SPDZ input shares and that those shares have valid MACs, using masked differences to test equality modulo p without exposing the underlying value.The protocol uses interval proofs, random masks, joint decryption, and a final divisibility check.

7 Extensions to Other Models

Helen’s techniques extend beyond LASSO to other linear models by adapting their regularization and coordination updates. The paper specifically describes ridge regression and notes that elastic net combines the LASSO and ridge components.

  • Supported models: Helen’s techniques apply to ordinary least-squares linear regression, ridge regression, and elastic net in addition to LASSO.The paper uses LASSO as a running example but states that the techniques generalize to these linear models.
  • Ridge regression: Ridge regression changes LASSO’s regularization from L1 to L2, while retaining a similar local update and using a linear coordination update.The paper provides ridge regression update rules and describes its optimization objective over the feature matrix and prediction vector.
  • Elastic net: Elastic net can be implemented by combining the L1 regularization term from LASSO with the L2 regularization term from ridge regression.Its coordination procedure therefore combines the corresponding model components.

8 Evaluation

Helen is evaluated against a maliciously secure SPDZ-based SGD baseline using synthetic scalability tests and two real-world datasets. It scales slowly with sample count, preserves similar test errors, and achieves large runtime improvements over the baseline.

  • Evaluation setup: Helen is compared with a maliciously secure SGD baseline implemented using SPDZ, with experiments on synthetic and two real-world datasets.The evaluation uses EC2 instances and excludes Helen’s one-time Paillier key setup phase.
  • Evaluation caveat: Helen’s performance advantage remains substantial even if the secure baseline’s online phase is assumed infinitely fast.The evaluation notes that SPDZ’s online implementation is not parallelized, but this does not remove Helen’s measured gains.
  • Theoretic performance: Helen’s cryptographic computation is dominated by d^2-scaled matrix-multiplication proofs in SVD proofs and d-scaled MPC conversion proofs in model compute.The plaintext SVD computation is excluded from the cryptographic-computation breakdown.
  • Scalability: Helen’s input preparation scales slowly with samples because only plaintext SVD depends on sample count, while its proofs and MPC offline generation do not.Its model compute phase remains constant under the fixed iteration count, whereas SGD scales linearly with samples in both phases.
  • Runtime results: 20689x improvement is achieved by Helen over the baseline’s total runtime on gas sensor data at 1 million samples per party.On song prediction, the corresponding improvement reaches 911x at 100K samples per party.
  • Model quality: Helen achieves similar test errors to sklearn models trained on plaintext data on the gas sensor and song prediction datasets.Errors are compared using L2 and mean average error after post-processing.

9 Related work

Prior coopetitive systems generally weaken security assumptions by relying on passive attackers or non-colluding servers, while trusted hardware introduces additional trust and leakage concerns. Helen targets malicious security without those outsourcing assumptions, but offers narrower functionality than some weaker-security systems.

  • Coopetitive systems: Prior training systems generally do not provide malicious security and often outsource computation to two non-colluding servers.Some systems instead offer richer functionality, such as neural-network support, under weaker security models.
  • Other coopetitive systems: Existing works on coopetitive prediction and analytics address related applications but differ from Helen’s multi-party training setting.Prediction systems typically combine one party’s model with another party’s input.
  • Trusted hardware: Trusted-hardware approaches require additional trust and are vulnerable to side-channel leakage.
  • Attacks: Malicious MPC protects protocol execution but does not prevent attackers from choosing incorrect training inputs through data poisoning.Input-range checks and other mitigation methods can complement Helen’s approach.
  • Attacks: Malicious MPC also does not prevent learning information from the released model, so differential privacy can be added before model release.Helen does not integrate more complex differential-privacy techniques that modify training itself.

10 Conclusion

Helen is a coopetitive system for training linear models while defending against malicious participants, so each party needs to trust only itself. Compared with a malicious secure baseline, it achieves up to five orders of magnitude of performance improvement.

  • Helen trains linear models in a coopetitive setting while defending against malicious participants.
  • Helen requires each party to trust only itself under its stronger threat model.
  • Up to five orders of magnitude of performance improvement are achieved over a state-of-the-art malicious framework baseline.

A ADMM derivations

The appendix derives ADMM updates for regularized linear models using global-variable consensus. The w_i updates reuse LASSO techniques, while the z update is linear but still requires MPC for fixed-point scaling reductions.

  • Ridge regression: Ridge regression uses L2 regularization and is formulated over a feature matrix X and prediction vector y.
  • ADMM formulation: The derivation introduces an augmented Lagrangian and replaces the dual variable v with the scaled variable u = (1/ρ)v.
  • ADMM formulation: Because the loss is decomposable across data blocks, the generic global-variable consensus ADMM algorithm supplies the update equations.
  • Update equations: The w update is derived from the consensus ADMM formulation.
  • Update equations: The w_i update matches LASSO techniques, while the z update is linear but requires MPC to reduce fixed-point multiplication scaling factors.

B.1 Definitions

This appendix defines the real, ideal, and hybrid MPC models, their security relation, and supporting protocols and lemmas. It also describes the zero-knowledge transformation and cryptographic building blocks used in Helen’s proofs.

  • MPC models: The real-world model uses an open-broadcast network with a static, active, rushing adversary that may corrupt a majority of participants.
  • MPC models: The ideal-world model gives inputs to a trusted party that computes the function and returns outputs, while the adversary can replace corrupted parties’ secret inputs.
  • Security definitions: A hybrid execution replaces selected ideal functionality calls with trusted-party evaluations, and secure evaluation requires no more adversarial information than ideal execution.
  • Security definitions: The modular composition theorem allows secure subprotocols to replace ideal calls while preserving security in the resulting protocol.
  • Cryptographic tools: The appendix lists Schwartz–Zippel and smudging lemmas alongside Paillier knowledge, multiplication, and encryption-interval proofs.
  • Cryptographic tools: Helen transforms honest-verifier Sigma protocols into full zero knowledge, non-malleability, and concurrency, using simulation and extraction without rewinding.

B.2 Proofs

Helen’s proof develops zero-knowledge arguments for plaintext knowledge and encrypted matrix products, then establishes security against malicious corruption of up to m −1 out of m parties. The security proof uses simulation to show Helen’s execution is indistinguishable from an idealized protocol.

  • Theorem 2 establishes an honest-verifier zero-knowledge proof that a party knows the plaintext of a committed matrix.The proof uses ciphertext proofs of plaintext knowledge applied element-wise to the matrix.
  • Gadgets 1 and 2 prove knowledge and correctness of encrypted matrix products under standard cryptographic assumptions.Gadget 1 proves Z = XY with a public committed X, while Gadget 2 handles committed X, Y, and Z.
  • The matrix multiplication proof reduces matrix-matrix verification to matrix-vector polynomial testing using the Schwartz-Zippel lemma.The protocol checks tZ = tXY for a random vector t, with an inequality passing only with bounded probability.
  • The simulator for Gadget 3 has a distribution statistically indistinguishable from the real-world execution.This supports replacing the real proof execution with a simulated one without changing the observable distribution.
  • Theorem 6 proves fADMM secure in the (fcrs, fSPDZ)-hybrid model against a malicious adversary statically corrupting up to m −1 of m parties.The proof constructs simulators for input preparation and model computation, then compares their distributions with the real execution.
Loading 1907.07212v2…