PaperlayerView exact revision on arXiv

Source-linked AI summary

The Price of Peeking: Anytime-Valid Leakage Detection on ML-KEM EM Traces

Georgios Feretzakis, Alexandros Papaspyridis

arXiv:2609.33597v1cs.CRcs.LG

TL;DR

Leakage tests are often inspected during acquisition, but fixed-horizon screening does not guarantee error control for that monitored decision rule. The paper adapts swap-pair e-processes to leakage assessment and compares them with matched fixed-horizon tests. The anytime procedures cost more traces to detect leakage in matched tests, but can stop early relative to conservative budgets under stated assumptions.

  • Problem

    Monitoring leakage tests during acquisition adds an error source that fixed-horizon screening does not quantify.

    Source: §1 · p. 1

  • Method

    The paper adapts SKIT-type swap-pair e-processes for leakage assessment under a stated conditional symmetry null.

    Source: §1 · p. 2

  • Results

    First crossing needed 1.68–2.38× the traces of matched fixed-horizon testing on degraded recordings and 1.68–2.00× on synthetic streams.

    Source: §1 · p. 2, §8 · p. 17

  • Takeaways & Limitations

    The study recommends stating the null and decision rule before testing, matching comparators on the same payoffs, and calibrating detection pipelines with designed nulls on evaluators’ own recordings.

    Source: §8 · p. 17

    All recordings come from one device and probe setting, and the study makes no claim that its results transfer across devices or campaigns.

    Source: §7 · p. 16

Abstract

from arXiv · show

Side-channel evaluators routinely inspect leakage tests while acquisition is still running, and extend or stop the campaign based on what they see. Fixed-horizon screening such as the Welch $t$-test with threshold $|t|>4.5$ gives no error guarantee for this monitored decision rule. We study anytime-valid leakage detection based on testing by betting: SKIT-type swap-pair e-processes whose false-alarm probability is controlled uniformly over time under an explicit conditional symmetry null. In matched comparisons that share the frozen witness, rows and payoff, first-crossing detection needed 1.68-2.00$\times$ the traces of a fixed-horizon randomization test at 80% detection on synthetic streams, and 1.68-2.38$\times$ on degraded recordings from an open ML-KEM electromagnetic dataset with the primary Ridge witness at $\alpha=0.05$. With the same primary witness and level, on undegraded reference and pqm4 recordings the monitored procedure stopped early: its median stopping point was 62-72 and 146-316 evaluation traces, i.e. 2-8% of a conservative 4096-trace budget. Under exact designed nulls on the recorded backgrounds, repeated-look $|t|>4.5$ screening over all 13000-20000 samples raised a false alarm in 2.7-12.9% of replicates, against 0.0-4.7% for terminal-only screening and no rejection by a sample-wise e-Bonferroni process, which in a prespecified follow-up detected natural-label associations in 4 of 4 backgrounds after 840-3288 traces. All recordings come from one device, and natural-label results are descriptive; we state the assumptions each claim requires.

1 Introduction

Leakage tests are often monitored during acquisition, but fixed-horizon thresholds do not guarantee error control for that adaptive decision rule. This paper adapts betting-based e-processes to leakage assessment and measures their cost and practical benefits under explicit null assumptions.

1 Introduction

  • Matched comparisons use the same frozen witness, rows, and payoff, while designed nulls randomize labels on recorded backgrounds to check implementation under real noise, drift, and dependence.Natural-label results are descriptive because the acquisition process cannot be verified from the archive.Source: §1 · p. 2
  • The study adapts SKIT-type swap-pair e-processes and states the conditional symmetry null required for their time-uniform guarantee.It also uses a known-law likelihood-ratio process for designed nulls.Source: §1 · p. 2
  • 1.68–2.00× more traces on synthetic streams and 1.68–2.38× on degraded recordings were needed for first crossing versus matched fixed-horizon testing.The recorded comparison used the primary Ridge witness at α = 0.05.Source: §1 · p. 2
  • On undegraded recordings, the primary Ridge procedure at α = 0.05 stopped after a median of 2–8% of a conservative 4096-trace budget.Source: §1 · p. 2
  • Repeated-look |t| > 4.5 screening raised false alarms in 2.7–12.9% of designed-null replicates, compared with 0.0–4.7% for terminal-only screening.The tests covered full traces of 13000–20000 samples on recorded EM backgrounds.Source: §1 · p. 2
  • The recordings come from one device, and natural-label error rates require an undocumented conditional symmetry assumption; the study makes no claim about masking, key recovery, or absence of leakage.The contribution is evaluation design, implementation, and empirical evidence under explicit null assumptions; the betting construction itself is not new.Source: §1 · pp. 2–3

2 Background and related work

The background covers the ML-KEM operation targeted by the recordings, established leakage-assessment approaches and their error-control limits, and betting-based e-processes. The study focuses on monitored leakage assessment rather than key recovery or fixed-sample certification.

2.1 The target operation in ML-KEM

  • The target is the first pairwise multiplication in ML-KEM's NTT domain; the Donjon dataset records the operation on a Cortex-M4, and this study uses unmasked reference and pqm4 captures.Labels are Hamming weights of stored 16-bit a0 and b0 operand representations, not identified register intermediates.Source: §2.1 · p. 3

2.2 Leakage assessment and its error control

  • TVLA-style t-tests and standardized variants dominate leakage assessment, while related approaches address fixed-sample detection, power analysis, certification, or timing tests rather than time-uniform monitoring.BLAQ uses randomized block designs and bootstrap confidence intervals, but its reported analysis does not supply a time-uniform guarantee for repeated inspection.Source: §2.2 · pp. 3–4

2.3 Testing by betting

  • Testing by betting builds e-processes as wealth from predictable stakes; SKIT pairs observations and bets on the score difference after swapping labels.Witness extensions, stake optimization, and combinations of e-processes are established variants discussed in the background.Source: §2.3 · p. 4

3 Method

The method freezes a witness and uses paired-label swap payoffs to build anytime-valid wealth under conditional swap symmetry. It also defines designed-null and fixed-horizon comparisons, plus persistent 80% detection summaries.

3.1 Witness, swap payoff and wealth

  • Under conditional swap symmetry, stakes chosen from completed pairs make the wealth process a nonnegative martingale, allowing inspection after every pair and stopping at any data-dependent time.The guarantee requires conditional label symmetry, not Gaussian or independent traces.Source: §3.1 · p. 5

3.2 Designed nulls and the known-law likelihood ratio

  • Designed nulls use independently generated labels after freezing the background and witness, enabling an exact known-law likelihood-ratio process that is not used for natural labels.For natural labels, an estimated denominator would not preserve exact validity.Source: §3.2 · p. 5

3.3 Matched fixed-horizon comparator

  • The fixed-horizon comparator randomizes pairwise payoff signs at each prespecified horizon; its guarantee requires joint invariance, which sequential conditional symmetry alone does not imply.The procedures share the frozen witness, rows and payoffs, so differences reflect the decision rule.Source: §3.3 · pp. 5–6

3.4 Summaries

  • N80 is the first grid horizon where rejection reaches 0.8 and stays there at larger horizons; boundary-limited and right-censored values are excluded from ratios.Designed-null rejection fractions are conditional on the background and witness, with pointwise Wilson intervals.Source: §3.4 · p. 6

4 Synthetic evaluation

In synthetic streams, anytime procedures maintained nominal-level validity across non-IID backgrounds but required more traces than a matched fixed-horizon test for 80% detection. The evaluation spans several signal forms and reports that neither stake rule uniformly dominated.

4.1 Design

  • Synthetic streams combined four non-IID backgrounds with linear, nonlinear and second-order signals, using a matching witness frozen after training on 2048 independent rows.Positive scenarios used 256 streams and null scenarios 512, on a geometric grid up to 16384 traces.Source: §4.1 · p. 6

4.2 Validity under non-IID backgrounds

  • At the largest horizon, anytime rejection fractions were at or below nominal on every tested background, including backgrounds violating conventional independence and normality assumptions.The fixed-horizon randomization test was exact at each horizon and fluctuated around nominal, as expected.Source: §4.2 · p. 7

4.3 The matched cost

  • First crossing required 1.68–2.00× the fixed-horizon grid N80 at α = 0.05 across synthetic signal types; terminal exceedance cost slightly more.The curves compare distinct detection events on shared evaluation streams and a frozen witness.Source: §4.3 · p. 7, Table 3 · p. 7, Figure 1 · p. 8
  • Neither ONS nor the plug-in stake rule uniformly dominated, and a scale-adaptive payoff transformation did not improve efficiency.The weak-signal Gaussian idealization's predicted terminal ratio of about 1.903 was not treated as a prediction because its formal conditions did not hold.Source: §4.3 · p. 7

5 Recorded electromagnetic backgrounds

On recorded electromagnetic backgrounds, anytime detection paid a trace-efficiency cost in matched comparisons but often stopped far below a conservative budget. Designed-null checks characterized validity and monitored full-trace Welch screening produced more false alarms than terminal-only screening.

5.1 Data allocation and development

  • Evaluation segments were 4096 traces, and variable-key captures used the first 8000 rows to fit witnesses and the next 2000 to tune them.The primary label threshold used the development median Hamming weight of the stored key-side operand; a 17-sample window fed Ridge and MLP witnesses, and Gaussian noise created pilot-selected degraded conditions.Source: §5.1 · pp. 8–9, Table 5 · p. 8

5.2 Designed nulls: validity on real backgrounds

  • Under designed nulls, the likelihood-ratio process rejected near nominal levels on many backgrounds, while swap processes were more conservative, especially on reference fixed-key backgrounds.The authors conjecture that the variable-key-trained witness yields nearly constant scores on fixed-key data; this explanation was not tested, and conservatism costs power rather than validity.Source: §5.2 · p. 9, Figure 2 · p. 10

5.3 Natural labels: the matched cost on real leakage

  • 1.68–2.38×: on degraded recordings, first crossing required this many traces as the matched fixed-horizon test; terminal exceedance required 2.00–2.83×.On undegraded recordings, almost every procedure reached 80% detection at the 256-trace grid boundary, so the cost could not be resolved; the MLP and α = 0.01 showed the same pattern with more censoring.Source: §5.3 · pp. 9, 11, Table 7 · p. 11, Figure 3 · p. 12

5.4 Natural labels: stopping early under a conservative budget

  • 62–72 reference traces and 146–316 pqm4 traces: median stopping points were 2–8% of the conservative 4096-trace budget.Every prespecified segment stopped before 4096 traces; these are descriptive segment results, not expected savings for another campaign.Source: §5.4 · p. 11, Figure 5 · p. 13

5.5 Repeated-look threshold screening on full traces

  • 2.7–12.9% of designed-null replicates triggered repeated-look Welch screening, versus 0.0–4.7% for terminal-only screening and none observed for e-Bonferroni.Zero observed e-Bonferroni rejections still allowed a pointwise 95% Wilson upper bound of 1.48% per background; the Welch threshold had no assigned nominal level, and the procedures differed in calibration and multiplicity handling.Source: §5.5 · p. 11, Table 9 · p. 14, Figure 6 · p. 15
  • In a prespecified follow-up, e-Bonferroni detected natural-label associations in all four backgrounds after 840–3288 traces.Welch screening rejected on all four at its first look after 1000 traces; these observations establish detection on these backgrounds, not faster detection or general power.Source: §5.5 · pp. 11, 14

6 Discussion and practical guidance

The discussion frames anytime-valid monitoring as a trade-off: matched fixed-horizon tests used fewer traces at the reported threshold, while anytime procedures permit data-dependent stopping under their stated null. It recommends matching the procedure to the evaluation protocol and using designed nulls for pipeline checks.

When to use which procedure

  • When leakage strength is unknown or intermediate inspection is part of the protocol, anytime procedures can stop early while retaining their stated conditional-null guarantee.The reported matched fixed-horizon comparator was more trace-efficient at its prespecified detection threshold; the null and decision rule should be stated before testing.Source: §6 · p. 15

Witnesses are part of the cost

  • Reported evaluation-trace budgets exclude development, tuning, and any pilot cost used to select degradation levels; freezing the witness prevents selection on evaluation outcomes.Likelihood-ratio validity requires a predictable proper probability mass function in the numerator and the conditional null law in the denominator, not a calibrated witness.Source: §6 · p. 15

Designed nulls as an evaluation tool

  • Independent labels on recorded traces create designed nulls with realistic noise, drift, and dependence, making them a routine calibration check for any detection pipeline.They require no additional acquisition, but generating labels and running checks has computational cost.Source: §6 · p. 16

Open questions

  • Natural-label validity needs documented acquisition randomization or a design that establishes it; better stake rules or payoffs may reduce matched cost, but tested alternatives did not do so here.The authors identify randomized interleaving with a blind test harness as one possible design; neither ONS nor a scale-adaptive payoff reduced cost in this study.Source: §6 · p. 16

7 Limitations

The evidence is bounded by a single-device archive and assumptions for interpreting natural-label results. Resolution, unstudied settings, and unverified acquisition properties further constrain the conclusions.

Scope

  • All recordings come from one device and probe setting, and segments from one archive are not independent sessions; transfer across devices or campaigns is not established.The archive order is used as replay order without verified timestamps, and duplicates across captures have not been ruled out.Source: §7 · p. 16

Assumptions for natural labels

  • Natural-label validity requires conditional pairwise label symmetry; interpreting an association as data-dependent leakage additionally assumes inputs were generated independently per execution.Neither assumption can be verified from the archive, and an operand-label association does not separate secret-dependent from public-input-dependent leakage.Source: §7 · p. 16

Resolution and prediction

  • Undegraded recordings were too easy for the lower grid, and pointwise Monte Carlo intervals do not provide intervals for grid N80 or its ratios.The synthetic budget prediction uses the evaluation's generator family and is not validated on recorded data.Source: §7 · p. 16

Scope

  • The study excludes masked implementations, key recovery, and classifier-based comparators; non-rejection is not evidence that a device does not leak.Source: §7 · p. 16

8 Conclusion

The conclusion contrasts time-uniform betting procedures with monitored fixed-threshold screening on recorded ML-KEM traces, then qualifies the trade-off and recommended practice.

8 Conclusion

  • Repeated-look |t| > 4.5 screening raised false alarms in up to 12.9% of designed-null replicates on recorded ML-KEM traces.The reported range across backgrounds was 2.7–12.9%.Source: §8 · p. 16
  • Under the stated conditional null, betting procedures support arbitrary monitoring but required 1.68–2.38× the matched grid budgets on degraded recordings and 1.68–2.00× in simulation.They also allowed early stopping relative to conservative operational budgets; natural-label comparisons are descriptive.Source: §8 · p. 17
  • The authors recommend stating the null and decision rule before testing, matching comparators on the same payoffs, and calibrating pipelines with designed nulls on the evaluator’s own recordings.The natural-label comparisons do not establish a universally favourable trade-off.Source: §8 · p. 17

Data and code availability

The appendix describes the artifact and the conditional assumptions, approximations, and executable protocol used to support the paper’s analyses. Its simulation-based terminal prediction is a consistency check, not validation for recorded data.

Data and code availability

  • The public artifact provides portable scripts, configurations, split definitions, aggregate results, tests, and an integrity manifest, but the full experiments have not had a clean-environment rerun.Reported empirical table values and figure data can be regenerated from saved results without dataset access.Source: Data and code availability · p. 17

A.1 Pair-swap process; A.2 Known-law likelihood ratio

  • Pair-swap validity requires the next pair’s conditional law to be invariant to label exchange, while the known-law process uses the true conditional label mass and a predictable proper model.Designed-null labels are generated independently after the background is frozen; general sequential symmetry does not itself imply the joint sign-flip invariance required by the fixed-horizon comparator.Source: §A.1 · pp. 17–18, §A.2 · p. 18

A.3 Multiplicity and odd transformations

  • Multiplicity is handled with fixed positive e-Bonferroni weights, while odd payoff transformations preserve sign symmetry only when the scale path is unchanged under sign flips.A scale computed from past absolute payoffs has this property; one selected from signed future values generally does not.Source: §A.3 · p. 18

B Terminal approximation and simulation-based prediction

  • The terminal approximation begins with a constant-stake, weak-payoff idealization, but the familiar weak-signal growth approximation requires diagnostic support rather than following as an identity.A scale-adaptive payoff has different moments and cannot inherit raw-payoff moments without a new calculation; the 1.903 reference is not an empirical target.Source: B Terminal approximation and… · pp. 18–19
  • The executed prediction estimates terminal log-wealth moments from independent simulation streams at each horizon and maps them to a Gaussian terminal exceedance curve.Its finite-grid errors do not provide a certified stopping-time interval, and the estimand remains conditional on the frozen witness.Source: B Terminal approximation and… · p. 19

B.1 Simulation-based terminal prediction

  • In the synthetic prediction check, 16 independent streams per horizon estimated mean and variance; 20 of 24 prespecified comparisons were finite on both sides and appear in Figure 7.Because prediction and evaluation streams use the same generator family, the exercise checks consistency rather than validating the method on recorded data.Source: §B.1 · p. 19, Figure 7 · p. 19

C Executable protocol details; C.2 Seeds, grid, and development

  • Executable protocol details distinguish internal pre-execution freezes from public preregistrations and specify independent seed namespaces, development splits, and preprocessing order.Pilot repetitions reuse a finite prespecified pool, so the pilot curve selects a condition but does not certify its population detection probability.Source: C Executable protocol details · p. 19, §C.2 · p. 20

C.3 Randomization and censoring; C.4 Prespecified integrity gate

  • The protocol reports horizon-specific randomization decisions using 999 sign draws plus the observed statistic, and its integrity gate compares flagged-background counts with a frozen upper binomial quantile.The gate’s independent label streams across background IDs underpin its binomial reference; it is not a simultaneous scientific confidence statement.Source: §C.3 · p. 20, §C.4 · p. 21

C.5 Secondary sample-wise family

The secondary sample-wise family streams Welch statistics and e-process states over audited columns, with equal e-Bonferroni allocation across the prespecified family. Campaign accounting separates access types and preserves explicit row and segment boundaries.

C.5 Secondary sample-wise family

  • Each audited column receives a univariate standardized Ridge score fit on primary-target development rows, while Welch statistics and predictable plug-in e-process states are streamed on the same rows and designed labels.Equal e-Bonferroni allocation covers the prespecified sample family; completing this secondary study was not required for the primary execution.Source: §C.5 · p. 21

D Exposure ledger and campaign boundaries

  • Rows are tracked by implementation, capture, chunk, and index, with metadata, waveform, and trace–label/model-selection access recorded separately; pairs and segments follow prespecified archive-order boundaries.Chunks and segments are not treated as sessions, and reserved later pqm4 variable chunks remain excluded.Source: D Exposure ledger and campaign… · p. 21

E Complete prespecified comparisons

The supplied passages identify tables retaining prespecified comparisons and integrity-gate families, but provide no cell values or outcome comparisons.

E Complete prespecified comparisons

  • Table 10 retains every prespecified natural condition at both alpha levels, including boundary and censored entries.Its caption specifies the scope and notation, not cell outcomes.Source: E Complete prespecified comparisons · p. 21, Table 10 · p. 22
  • Table 11 lists all integrity-gate families, with FAIL defined as strictly more flags than the limit.The caption says the gate is not a simultaneous scientific claim.Source: Table 11 · p. 22
  • Table 12 contains all undegraded finite-stop summaries, while full budget-specific fractions and ratios remain in the hash-bound JSON.The supplied caption does not state specific stopping outcomes.Source: Table 12 · p. 23

F Reproducibility and evidence

The reproducibility account describes a scoped public artifact, staged execution records, and verified regeneration, while noting that a clean-environment rerun was not performed. The practical guidance emphasizes freezing assumptions and procedures before evaluation, preserving execution integrity, and matching reported claims to their evidence and limits.

F Reproducibility and evidence

  • The public artifact contains derived scripts, configurations, split definitions, aggregate results, tests, and a scoped manifest, while excluding raw data and private audit details.The main real-background run used 10181.9 CPU seconds, 4156.5 wall-clock seconds, and 662.9 MiB peak memory with no GPU; Table 13 excludes synthetic and follow-up costs.Source: F Reproducibility and evidence · pp. 23–24, Table 13 · p. 24
  • Regeneration of reported empirical table values and figure data was verified, but a clean-environment rerun of the full experiments was not performed.Regeneration checks used scikit-learn 1.6.1 rather than the recorded 1.8.0; portable derived runners are not presented as byte-identical historical producers.Source: F Reproducibility and evidence · p. 24

Before acquisition or replay

  • Before acquisition or replay, define the label, representation, physical interpretation, null law, and prediction rule, and verify that the frozen witness cannot access current or future labels.Record row identities and campaign boundaries, reserve data before selecting windows, recipes, degradation, or seeds, and decide whether the archive supports physical inference or only descriptive comparison.Source: Before acquisition or replay · pp. 24–25

Before computing the first test result

  • Before testing, freeze the score, probability mapping, stake, bounds, initialization, and update order, and specify the validity assumptions for swaps, randomization, or likelihood ratios.Also fix the horizon grid, stopping event, multiplicity family, censoring rule, alpha unit, and fitting and tuning budgets.Source: Before computing the first test result · p. 25

During execution

  • During execution, use immutable outputs and checked identities, reset state between prespecified repetitions, count failures, and do not retune witnesses or segments after evaluation.Preserve raw and degraded condition identities rather than treating them as new acquisitions.Source: During execution · p. 25

When reporting

  • When reporting, match each claim to its assumptions, distinguish conditional Monte Carlo intervals from capture variability and transfer uncertainty, and state what non-rejection and association cannot establish.Report sensitivity limitations alongside false-alarm results, and distinguish changed physical setups from transformed waveforms or randomized labels against fixed backgrounds.Source: When reporting · p. 25

Related papers

Loading 2609.33597v1…