Membership Inference via Pairwise Likelihood Ratios

• Low-FPR auditing is essential because false membership findings can themselves be harmful, and model outputs reflect query difficulty as well as membership.An intrinsically easy non-member may receive high confidence, while a difficult member may receive low confidence.Source: §1 · p. 1
• Existing approaches do not jointly extract membership signal, adjust for query heterogeneity, and preserve the strength of pairwise evidence.LiRA uses a Gaussian likelihood ratio but a global threshold misses query heterogeneity; RMIA calibrates against population points but uses binary comparisons and does not explicitly exploit variance contraction.Source: §1 · p. 2
• PL-MIA combines a Gaussian likelihood ratio, population calibration, and Cauchy combination of continuous pairwise p-values.The p-values preserve comparison strength that binary voting discards.Source: §1 · p. 2
• The paper characterizes how GLR captures variance contraction and how calibration and evidence aggregation contribute to attack power.Population calibration centers non-member scores around neutral baselines; the analysis attributes PL-MIA's advantage over LiRA and RMIA to exploiting variance contraction while reducing query-level heterogeneity.Source: §1 · pp. 2–3
• Extensive experiments report consistent performance improvements across datasets and substantial gains in the critical low-FPR regime.The introduction summarizes the experiments as showing gains without specifying a numeric value here.Source: §1 · p. 3

• A membership inference attack predicts whether a query point was used to train the target model, with membership represented by a bit in the formal game.The challenger trains a target model, samples a member or non-member query, and the adversary predicts its status.Source: §2 · p. 3
• The adversary operates in a worst-case black-box setting with access to the data distribution and a replica of the architecture and training algorithm, but not the target's training set or parameters.The adversary can query the target and observe its outputs.Source: §2 · p. 3
• The attack assigns a membership score and thresholds it to target an FPR; PL-MIA uses an analytic threshold rather than additional held-out data.Its Cauchy-combined pairwise p-values yield an analytic threshold for nominal FPR control.Source: §2 · pp. 3–4
• For the black-box test, the observable signal is the target model's true-label confidence transformed to a logit, with Gaussian distributions assumed conditional on membership.The paper motivates the Gaussian assumption by empirical approximations and says it makes the hypothesis-testing problem analytically tractable.Source: §2 · p. 4

3.1. A Unified Framework of MIA.

• The unified framework decomposes an attack into a pointwise statistic and population calibration, with calibrated evidence generated and aggregated from query–population comparisons.The comparison table organizes representative attacks by pointwise statistic and, when applicable, evidence generation and aggregation.Source: §3.1 · pp. 5–6, Table 1 · p. 5

3.2. Designing PL-MIA.

• PL-MIA adopts GLR as its pointwise statistic and compares each query's statistic with that of sampled population points.The pairwise difference is the basic unit of membership evidence; under membership it tends to be positive, while under the non-member null it is approximately symmetric around zero.Source: §3.2 · p. 6
• One-sided pairwise p-values retain comparison strength, unlike binary voting that reduces evidence to a hard-threshold decision.The p-values are computed against an approximately centered Gaussian null for pairwise differences; smaller values indicate stronger membership evidence.Source: §3.2 · pp. 6–8

3.3. Why PL-MIA Is More Powerful.

• The Cauchy combination can let a small subset of strong comparisons dominate rather than dilute them among weak comparisons.PL-MIA also avoids tuning the binary comparison threshold γ and, under the idealized Cauchy null, has an analytic decision threshold; the paper reports conservative finite-sample FPR control when exact-null conditions are only approximate.Source: §3.2 · p. 8, §3.3 · p. 9
• GLR uses both mean shift and variance contraction, while population calibration places query scores on a relative scale to address intrinsic heterogeneity.The paper's theoretical analysis says GLR can provide stronger separation than BLR when variance contraction or nuisance variation is present, and calibration centers non-member scores around neutral baselines.Source: §3.3 · p. 8

4.1. Pointwise Statistic Distributions.

• Under the paper's Gaussian setup, GLR retains a variance-contraction signal that the first-order BLR mean separation lacks.The GLR signal grows as O(1/ρ^2), whereas the stated first-order BLR expression is independent of ρ.Source: §4.1 · p. 9

4.2. Pairwise Difference and Population Calibration.

• Population calibration gives non-member scores neutral baselines: average win probability 0.5 for binary voting and median-zero aggregated evidence for Cauchy combination.The Cauchy baseline is described by its median because Cauchy evidence has no finite mean.Source: §4.2 · p. 10

4.3. Attack-Power Expressions and Advantage Analysis.

• For the compared attacks, PL-MIA's low-FPR characterization depends on exceptionally small pairwise p-values, unlike the common CDF-based power expression for LiRA, RMIA, and PL-MIA+.The paper relates the member-query event to at least one of the pairwise comparisons producing strong evidence.Source: §4.3 · pp. 10–11
• At fixed α, PL-MIA has greater attack power than PL-MIA+ when |Z|r_α > -log(1 - TPR_PL-MIA+(α)).PL-MIA+ uses GLR with binary-voting calibration; PL-MIA replaces that voting with Cauchy combination.Source: §4.3 · pp. 10–11
• The analysis decomposes PL-MIA's advantage into pointwise signal extraction, population calibration, and evidence-sensitive aggregation.Its component comparisons include conditions where the GLR-based PL-MIA+ exceeds RMIA and where calibration preserves LiRA's attack power.Source: §4.3 · p. 11

5.1. Experimental Setup.

• Experiments use balanced member and non-member queries across three image datasets, compare online and offline attacks, and report AUC and TPR at 0.01% and 0.0% FPR.The main comparison uses Wide-ResNet models and 254 reference models; the table averages results over 10 random target models.Source: §5.1 · pp. 11–12, Table 2 · p. 12

5.2. Main Results.

• At TPR@0.0%FPR, online PL-MIA improves over the strongest baseline by 22.94%, 27.29%, and 27.65% on CIFAR-10, CIFAR-100, and CINIC-10, respectively.Offline improvements on CIFAR-100 and CINIC-10 are 62.74% and 50.69%, respectively.Source: §5.2 · p. 12, Table 2 · p. 12

5.3.1. Ablation and Sensitivity Analyses.

• Across datasets and modes, binary population calibration improves GLR, while continuous pairwise p-values with Cauchy combination improve further; full PL-MIA avoids tuning γ.The complete method achieves the strongest performance in all reported ablation settings.Source: §5.3.1 · p. 13, Table 3 · p. 13, Figure 1 · p. 13
• CIFAR-100 offline PL-MIA stays stable across η and ρ: AUC ranges from 90.12 to 91.22 and TPR@0.0%FPR from 12.14 to 12.78, with η = 0.3 and ρ = 0.9 best.This evaluation approximates the IN-distribution parameters without explicit IN reference models.Source: §5.3.1 · p. 14, Table 4 · p. 14

5.3.2. Cost Analysis.

• PL-MIA is sensitive to small reference-model budgets because Gaussian variance estimates are unstable; pooled variance raises AUC from 61.71 to 66.32 at #Ref = 4.The pooled-variance benefit largely disappears as the reference-model budget grows.Source: §5.3.2 · pp. 14–15, Table 5 · p. 15
• More augmentations benefit all attacks, with PL-MIA's gap growing at zero FPR; at |Z| = 25, it remains competitive where RMIA's voting underperforms LiRA.With more population points, PL-MIA surpasses both baselines at low-FPR operating points.Source: Figure 2 · p. 15, §5.3.2 · p. 16

5.3.3. Statistical Properties of PL-MIA.

• Non-member pairwise p-values are near-uniform while member p-values skew small; despite dependence and mild null deviations, analytic thresholds were conservative across datasets.The finite-sample deviation from the nominal FPR becomes more pronounced in the extreme low-FPR regime.Source: §5.3.3 · pp. 16, 18, Figure 3 · p. 17, Figure 4 · p. 17

5.3.4. Generalization to Tabular Data and Different Model Families.

• 76.15%: at max_depth=7, PL-MIA's TPR@0%FPR is 21.86 versus RMIA's 12.41 on Purchase-100 GBDT; it leads at all depths.PL-MIA also achieves the highest AUC and strongest stringent-FPR performance across the tested depths.Source: §5.3.4 · p. 18, Table 6 · p. 18, Figure 5 · p. 19

• PL-MIA’s low-FPR gains arise from combining mean shift and variance contraction, population calibration against query-level heterogeneity, and Cauchy combination of continuous p-values.The combination preserves the contribution of strong pairwise comparisons and is most relevant to separation in the non-member distribution’s tail.Source: §6 · p. 19
• Training many independent reference models may be infeasible for large-scale models, motivating alternatives such as checkpoints, smaller proxy models, or token- and representation-level reference distributions.The paper also identifies optimal allocation of a fixed computational budget across reference models, population points, augmentations, and audit queries as an open problem.Source: §6 · p. 19
• The supplementary material covers additional training details, experiments on reference-model budgets, augmentation aggregation, Gaussianity, and MLP models, plus proofs of the main theoretical results.Source: §6 · p. 19

• The appendix supplies additional model-training protocol and hyperparameter details; the overall experimental setup is described in the main manuscript.Source: Appendix A · p. 19
• Each target and reference model is trained on a 50% data subset; the other half serves as non-members for each target.Each independently sampled reference model includes a query with approximately 50% probability, yielding approximately balanced IN and OUT reference sets.Source: Appendix A · p. 20
• Wide-ResNet is used across CIFAR-10, CIFAR-100, and CINIC-10, with SGD, cosine learning-rate decay, and warm-up over the first 1% of epochs; training uses no data augmentation.Source: Appendix A · p. 20
• Table 7 summarizes training hyperparameters and average training and test accuracy for the image-dataset experiments.Source: Table 7 · p. 20

APPENDIX B: SUPPLEMENTARY EXPERIMENTAL RESULTS

• The appendix section presents supplementary experimental results that complement analyses in the main manuscript.Source: Appendix B · p. 20

B.1. Performance under Limited Reference-Model Budgets.

• At #Ref = 4, pooled-variance PL-MIA raises AUC from 67.55 to 86.03, while RMIA has the strongest low-FPR performance at #Ref ∈{4,8,16}.At #Ref = 32, PL-MIA surpasses RMIA in AUC and TPR@0.0%FPR; at #Ref = 64, standard and pooled-variance PL-MIA outperform RMIA across reported metrics.Source: Table 8 · p. 20, §B.1 · p. 21

B.2. Effect of Augmentation Aggregation Strategies.

• Mean aggregation has the strongest overall performance, achieving the highest AUC and TPR@0%FPR; Voting is marginally higher at TPR@0.01%FPR (4.49 versus 4.45).Averaging retains augmentation-specific signal magnitude, and the paper uses mean aggregation throughout its experiments.Source: Table 9 · p. 21, §B.2 · p. 21

B.3. Gaussian Assumption Diagnostics and Robustness.

• The diagnostics apply Shapiro–Wilk tests separately to empirical IN and OUT distributions of logit-scaled true-label confidence, with 127 observations in each distribution.The Gaussianity score is the minimum of the IN and OUT Shapiro–Wilk statistics, so both distributions must be close to Gaussian for a query to receive a high score.Source: §B.3 · p. 22
• AUC falls by only 1.35 percentage points from the most to least Gaussian query third; PL-MIA’s AUC is 73.03 in the most Gaussian third and 71.68 in the least Gaussian third.In the least Gaussian third, it still achieves TPR@1%FPR of 14.19 and TPR@0.1%FPR of 7.45.Source: §B.3 · pp. 22–23, Table 10 · p. 23
• The Gaussian assumption is a practical approximation rather than a strict empirical requirement: moderate departures do not prevent useful membership-signal extraction, and more flexible density estimators are a future direction.The Gaussian approximation estimates the pointwise IN/OUT likelihood ratio, while final evidence aggregates across population comparisons.Source: §B.3 · p. 23

B.4. Multi-Layer Perceptron on Purchase-100.

• On Purchase-100 with a four-layer MLP, PL-MIA achieves AUC 84.23 versus LiRA’s 83.23 and RMIA’s 83.71, plus 3.70 TPR@0.01%FPR and 2.07 TPR@0%FPR versus RMIA’s 3.20 and 1.59.Source: Table 11 · p. 23, §B.4 · p. 23

APPENDIX C: PROOFS

• Appendix C derives the GLR and BLR statistics, then establishes pairwise-difference distributions, population-calibration baselines, and attack-power expressions and ordering.These proofs correspond to Lemmas 4.1–4.4 and Theorems 4.5–4.6 in the main manuscript.Source: Appendix C · pp. 23–24

C.1. Proof of Lemma 4.1.

• When ρ < 1, GLR mean separation exceeds (η(d))^2 and grows as O(1/ρ^2) as ρ decreases, capturing variance contraction as an additional membership signal.The proof derives Gaussian forms for the GLR and uses them in the pairwise-difference analysis.Source: §C.1 · p. 25

C.2. Proof of Lemma 4.2.

• BLR’s first-order mean separation is unaffected by ρ, although ρ affects its first-order variance term.The proof expands the normalizing term using reference models balanced between IN and OUT populations.Source: §C.2 · pp. 25, 27

C.3. Proof of Lemma 4.3.

• For a member query, the proof derives the distribution of the pairwise difference using the Gaussian GLR forms and the variance rules for a difference.The variance of the difference is the sum of the two variances.Source: §C.3 · p. 27
• For a non-member query, both points follow their respective non-member distributions, providing the null comparison underlying the symmetry argument.Source: §C.3 · p. 27
• Under the null, swapping the i.i.d. non-member points reverses the pairwise difference without changing its distribution, making the marginal null distribution symmetric about zero.The conditional means need not themselves be zero.Source: §C.3 · p. 27

C.4. Proof of Lemma 4.4.

• Exchangeability and continuity make pairwise differences equally likely to be positive or negative under the null, establishing the neutral baseline for binary-voting population calibration.Source: §C.4 · p. 28
• Under independence, averaging Cauchy-transformed null pairwise p-values gives an exact median-zero Cauchy score, while the shared query point motivates a score-level approximation in the pairwise setting.The median is used because Cauchy evidence has no finite mean.Source: §C.4 · p. 28

C.5. Proof of Theorem 4.5.

• For the Gaussian attack-power expression, a larger κ_A gives a higher TPR at the same FPR, and binary population calibration preserves the rejection region.The proof derives the power expression for LiRA, RMIA, and PL-MIA+.Source: §C.5 · pp. 28–29
• In the low-FPR approximation, the Cauchy-combined score’s upper-tail event is approximated by at least one population comparison producing exceptionally small p-value evidence.The proof relates this event to member queries being more likely to produce such evidence because larger pairwise differences correspond to smaller p-values.Source: §C.5 · p. 29

C.6. Proof of Theorem 4.6.

• PL-MIA+ and LiRA have the same rejection event and attack power after null-percentile calibration, which expresses heterogeneous raw GLR scores on a common scale.The calibration maps each raw GLR score to its percentile under the non-member distribution while preserving ordering and power.Source: §C.6 · p. 30
• GLR’s bounded-interval upper rejection region differs from BLR’s half-line, so the Neyman–Pearson argument gives GLR a strict power advantage.The proof assumes W has a strictly positive standard-normal density and that population calibration preserves both rejection regions.Source: §C.6 · pp. 30–31
• The PL-MIA+ versus RMIA analysis fixes ρ = 1 and η(Q) = η0 > 0 to isolate ξ-heterogeneity, then uses attack-power monotonicity to establish Part II.The proof states that ξ(Q) > 0 under Assumption 2.2.Source: §C.6 · pp. 31–32
• For the PL-MIA versus PL-MIA+ comparison, the low-FPR power approximation treats success as at least one of the |Z| comparisons producing exceptionally small p-value evidence.The proof uses a product approximation over the |Z| population comparisons.Source: §C.6 · p. 32