PaperlayerView exact revision on arXiv

Source-linked AI summary

Implementing a White-Box Undetectable Backdoor for Random Fourier Features

Michael Collins, Jada Cumberland, Brianne Dunn, Ross Gore, Samuel Jackson, Sachin Shetty

arXiv:2609.16403v1cs.CRcs.LG

TL;DR

Goldwasser et al. show that CLWE-based RFF backdoors can evade even full white-box detection, but their construction lacks an implementation and its practical realizability is unclear. This paper implements the core using ordinary numerical libraries and tests weight-space and functional indistinguishability. It reports no evidence of detectable difference across sparsity ratios, while noting that tested parameter scales do not establish cryptographic hardness.

  • Problem

    Whether the theoretically described white-box CLWE-RFF backdoor can be realized with ordinary numerical tools remains unclear, despite its ability to evade full model audits.

    Source: §1 · p. 2, §3.2 · p. 4

  • Method

    The paper implements the CLWE-RFF backdoor core, including branch-free activation, and evaluates weight-space and functional indistinguishability across sparsity ratios.

    Source: §4.1 · p. 6, §5.3 · p. 8

  • Results

    Across a range of sparsity ratios, the construction empirically satisfies white-box and black-box indistinguishability criteria, with no evidence that the gap widens as sparsity decreases relative to ambient dimension.

    Source: §7 · p. 11

  • Takeaways & Limitations

    The implementation indicates that ordinary scientific-computing competence suffices to realize the construction's core algorithmic machinery without specialized cryptographic tooling.

    Source: §7 · p. 11

    The tested dimensions were chosen for computational convenience and are not validated against a known CLWE hardness threshold, so the results do not establish cryptographic meaningfulness at those parameter values.

    Source: §6.1 · p. 11

Abstract

from arXiv · show

Goldwasser et al. showed that undetectable backdoors can be planted in machine learning models trained with the Random Fourier Features (RFF) algorithm, under a hardness assumption tied to the Continuous Learning With Errors (CLWE) problem. Under standard cryptographic assumptions, even a full white-box audit of a model's weights cannot detect this class of backdoor. The construction is stated in terms of cryptographic reductions and probabilistic lemmas, without a reference implementation, and relies on secondary machinery such as the Sparse Gaussian Pancakes distribution and a homogeneous CLWE conditional density. Its realizability in ordinary numerical code is not obvious from the paper alone. This paper implements the white-box CLWE-RFF backdoor construction end to end using only numpy and scipy, to test whether this threat is realizable with commodity scientific-computing tools or requires specialized cryptographic infrastructure. We give two samplers for the core $GP_d(b_k)$ distribution. The first is a rejection-sampling proxy. The second is an exact closed-form sampler derived from the homogeneous CLWE density and verified against its own analytic form. Using this implementation, we run statistical indistinguishability tests, covering both weight-space and functional black-box comparisons. We find no evidence of detectable difference between backdoored and clean models across a range of sparsity ratios $\rho = d_{\text{sparse}}/D$. We report which parts of the construction were straightforward to realize, which required derivation not spelled out in the paper. We also highlight which parts we did not attempt to reproduce, including the underlying lattice hardness reduction. We see this work as a contribution to understanding the practical realizability of the Goldwasser white-box CLWE core, not as a new theoretical result.

1 Introduction

Goldwasser et al. [1] give a CLWE-based RFF backdoor that remains indistinguishable from clean models even under full white-box access, raising the practical question of whether it can be built with ordinary numerical tools. This paper documents an implementation and statistical validation of the construction, including parts that required derivation and parts left unreproduced.

1 Introduction

  • Goldwasser et al. [1] show that CLWE-based RFF backdoors can remain computationally indistinguishable from clean models despite full white-box access.Their strongest result relies on the conjectured hardness of CLWE.Source: §1 · p. 2
  • The paper also situates its work amid empirical backdoor-detection cycles and formal cryptographic guarantees, while outlining the paper's sections.It describes the work as a reproducibility study rather than a new theoretical result.Source: §1 · p. 2
  • The practical question is whether ordinary numerical libraries suffice to realize the construction or whether it requires specialized cryptographic tooling.The paper notes that either successful realization or implementation gaps can inform defenders about practical difficulty.Source: §1 · p. 2
  • The paper implements and statistically validates the Goldwasser et al. [1] white-box CLWE-RFF construction using standard numerical libraries, rather than presenting a new attack or theorem.The work is framed as a reproducibility study of whether the construction works as described and how hard it is to implement.Source: §1 · p. 2
  • The implementation covers two GP_d(b_k) samplers, branch-free activation, and weight-space and functional-space indistinguishability tests, including a sparsity-ratio sweep.The exact sampler is derived from the homogeneous CLWE density of Bruna et al., while the rejection sampler is a proxy.Source: §1 · p. 2

2 Background

This background connects lattice-based hardness to CLWE, then to RFF and the CLWE-RFF backdoor. The backdoor hides a sparse trigger in feature sampling while using a shift to change the classifier's prediction.

2.1 Lattices and Learning With Errors

  • Lattices are discrete subgroups generated by integer combinations of basis vectors, and worst-case lattice hardness underpins cryptographic assumptions.Regev's reduction connects worst-case lattice problems to average-case LWE hardness.Source: §2.1 · pp. 2–3
  • Figure 1 contrasts smooth Gaussian samples with hCLWE density concentrated in evenly spaced bands along the secret direction and parallel stripes in the 2D view.The left panel compares one-dimensional marginal densities for two β values at γ = 8; the right shows samples in the secret-direction plane.Source: Figure 1 · p. 3

2.2 Continuous LWE

  • CLWE replaces discrete LWE samples with Gaussian samples either unmodulated or secretly modulated along a hidden direction, with lattice reductions supporting its hardness.Bruna et al. gave a quantum reduction, and Vafa et al. [3] a classical-LWE reduction.Source: §2.2 · p. 3

2.3 Random Fourier Features

  • RFF maps inputs through cosine features generated from random Gaussian rows and trains a linear classifier, approximating shift-invariant kernels at lower computational cost.The untrained randomness in G is the feature that Goldwasser et al. exploit to hide structure in the backdoor construction.Source: §2.3 · p. 3

2.4 The CLWE-RFF backdoor construction

  • Goldwasser et al. [1] replace Gaussian feature rows with Sparse Gaussian Pancakes, which hide a sparse key while remaining computationally indistinguishable under CLWE hardness.Shifting an input by the key drives each feature-row inner product near a half-integer, flipping cosine-feature signs and the classifier output.Source: §2.4 · p. 4

3 White-Box Versus Black-Box Undetectability

Goldwasser et al. distinguish black-box auditing by query access from white-box auditing of the complete model. The CLWE-RFF construction targets the stronger setting by camouflaging the feature matrix while embedding a trigger, and the implementation tests both weight and functional behavior without key-dependent branching.

3.1 Two levels of auditor access

  • Black-box auditing observes query outputs, whereas white-box auditing exposes all parameters and training data and demands indistinguishability of the complete model.Goldwasser et al. [1] prove the stronger white-box guarantee specifically for the CLWE-RFF construction under CLWE hardness [1].Source: §3.1 · p. 4

3.2 Why white-box is the harder case to realize

  • White-box undetectability requires the entire feature matrix G to resemble a clean Gaussian matrix while retaining a usable trigger, a dual constraint met through Sparse Gaussian Pancakes.Unlike a black-box construction, the backdoor cannot simply hide a decision rule in exposed weights.Source: §3.2 · p. 4

3.3 What this means for our implementation and tests

  • The implementation avoids secret-key checks or branches in the forward pass, keeping activation branch-free as required for the white-box argument.The trigger shift is ordinary coordinate-wise addition in R^D, assuming a continuous representation where the perturbed input remains meaningful.Source: §3.3 · p. 4
  • The evaluation separates weight-space tests on G from functional-space tests on model outputs, since passing only one would not test both claims.Figure 2 maps the Backdoor pipeline and Activate trigger, including reuse of the trained classifier and feature matrix.Source: §3.3 · p. 5, Figure 2 · p. 5

4 Implementation

The implementation separates sparse-key generation, feature sampling, and activation in a standalone package. It provides both a rejection proxy and an exact closed-form GP_d(b_k) sampler, with branch-free activation through the learned classifier's structure.

4 Implementation

  • The standalone package implements the CLWE-RFF core separately from downstream demonstrations and organizes it into secret sampling, feature generation, and activation.Figure 2 maps those components onto the Backdoor and Activate stages of the scheme.Source: §4 · p. 5

4.1 Secret and feature sampling

  • The construction samples a scaled sparse secret, embeds it on a random d-coordinate support, then draws feature rows from GP_d(b_k) with conditional on-support and Gaussian off-support coordinates.The distribution is specified to be computationally indistinguishable from a standard Gaussian under CLWE hardness while concentrating key inner products near one-half modulo 1, except with probability δ.Source: §4.1 · pp. 5–6

4.2 Two samplers for GP_d(b_k)

  • The rejection sampler accepts Gaussian proposals when a noisy secret-direction projection falls within τ of one-half, approximating the conditional distribution at an acceptance rate of about 2τ.Because it discards most proposals, its cost grows as τ shrinks, making it problematic for small target failure probabilities.Source: §4.2 · p. 6
  • The exact sampler draws a discrete-Gaussian mixture layer, samples the secret-direction coordinate from its layer-specific Gaussian, and uses independent standard Gaussians orthogonally.The completing-the-square identity agreed numerically with the raw density to within approximately 3 × 10^-16.Source: §4.2 · p. 6

4.3 Branch-free activation

  • The model uses branch-free cosine features and a linear classifier; adding the secret key shifts inputs so the learned weights and CLWE structure determine whether the output flips.The implementation defaults to rejection sampling, with the exact sampler opt-in; reported Section 5 results use the default unless noted otherwise.Source: §4.3 · p. 6, §4.2 · p. 6

5 Empirical Results

The evaluation verifies both samplers, then tests whether trained backdoored models remain effective and indistinguishable from clean models across sparsity ratios. Enforcing b = 3 raises trigger firing while the indistinguishability tests continue to pass.

5.1 Sampler verification

  • At d = 10 and D = 64, the exact sampler matches its analytic secret-direction projection density (KS ≈ 0.004, p ≈ 0.54); half-integer concentration is 1.000 versus 0.960 for rejection sampling.Acceptance rates are 1.0 for the exact sampler and ≈0.198 for the rejection proxy; off-support marginals remain clean N(0, 1) under both.Source: §5.1 · p. 7

5.2 Backdoor efficacy and indistinguishability

  • At the representative setting, clean accuracy is 0.688 for the backdoored model versus 0.684 for the clean model, while real-key and fake-key flip rates are 0.869 and 0.489.The fake-key rate is described as indistinguishable from chance.Source: §5.2 · p. 7
  • The default parameter search stops at b = 1, where √m d^-b ≈ 3.46; across the sweep, flip rates rise from 0.754 to 0.968 as this quantity falls.The paper attributes the gap from Theorem 6.4’s certainty guarantee to choosing the smallest valid b rather than a larger one.Source: §5.2 · p. 7
  • None of the six weight-space or functional-space tests rejects indistinguishability at α = 0.05, including the null-comparison replication and both random seeds.The weight-space tests examine G, while functional-space tests compare behavior on held-out inputs.Source: §5.2 · pp. 7–8, Table 1 · p. 8

5.3 Robustness across sparsity ratios

  • Across ρ = 0.078–0.500, KS-statistic slopes are +0.00051 for weight-space tests and −0.0179 for functional-space tests; neither increases.Minimum p-values are 0.120 and 0.076, respectively, and flip rates stay above 0.75 throughout.Source: §5.3 · p. 8, Figure 3 · p. 9

5.4 Enforcing a larger b closes the flip-rate gap

  • At D = 64, m = 1200, and d_sparse = 10, constraining b = 3 raises real-key flip rates to 0.998 with rejection sampling and 0.999 with exact sampling; fake-key rates remain 0.49–0.50.All six indistinguishability tests pass for both samplers at this setting.Source: §5.4 · p. 9, Table 3 · p. 9
  • Across the b = 3 sparsity sweep, real-key flip rates reach 0.999–1.000 versus 0.738–0.963 at b = 1, while weight-space p-values stay above 0.05.The minimum weight-space p-value is 0.086, with no trend across ρ.Source: §5.4 · p. 9, Table 4 · p. 10

6 Discussion

The discussion distinguishes implementation of the construction’s mechanics from establishing its cryptographic hardness. It also identifies evaluation boundaries involving parameters, adaptive and operational detection, omitted experiments, and fidelity to the formal secret distribution.

6.1 The hardness reduction itself

  • The implementation does not reproduce the lattice-to-CLWE hardness reduction, and the tested dimensions were chosen for convenience rather than validated against a hardness threshold.The reductions are asymptotic; the results do not establish cryptographic significance for the tested parameter values.Source: §6.1 · pp. 10–11

6.2 Adaptive and adversarial detection

  • Both indistinguishability test suites use fixed, non-adaptive clean-input distributions, leaving adaptive query-based detection unevaluated.Evaluating an adaptive adversary would require a concrete attack strategy.Source: §6.2 · p. 11
  • Repeated use of the same trigger offset could create a detectable pattern in retained input logs, a risk outside the model-level guarantee studied here.The guarantee concerns a model snapshot, not activation logs accumulated over time.Source: §6.2 · p. 11

6.3 Persistence and immunization

  • The evaluation omits backdoor persistence under further training and randomized-smoothing immunization, both of which would require additional experiments.Persistence testing would retrain and rerun efficacy and detectability checks; immunization testing would measure its effect on the real-key flip rate.Source: §6.3 · p. 11

6.4 Fidelity of the secret distribution

  • The secret distribution uses a random unit vector scaled by 2 d on a random sparse support, but the implementation does not verify every detail of its formal definition.A fuller treatment would derive and check the complete specification.Source: §6.4 · p. 11

7 Conclusion

The paper shows that the core white-box CLWE-RFF backdoor mechanics can be implemented in ordinary Python, with an exact sampler for the homogeneous CLWE density. Its tests find no evidence of distinguishability across sparsity ratios, while cryptographic hardness and further detection questions remain unresolved.

7 Conclusion

  • The core Goldwasser et al. [1] white-box CLWE-RFF backdoor machinery, including sparse-secret sampling, conditional feature generation, and branch-free activation, is realizable in ordinary Python.The implementation uses numpy and scipy and derives and verifies an exact closed-form sampler for the homogeneous CLWE density.Source: §7 · p. 11
  • The construction empirically satisfies white-box and black-box indistinguishability criteria [7] [8] across tested sparsity ratios, with no observed widening of the gap as sparsity decreases relative to ambient dimension.The authors take this as evidence that specialized cryptographic tooling is not required to realize the threat.Source: §7 · p. 11
  • Undetectability still rests on the conjectured hardness of CLWE [3], which no numerical implementation can establish on its own; this assumption remains a target for future cryptanalytic scrutiny.Source: §7 · pp. 11–12
Loading 2609.16403v1…