Token-Level Likelihood-Array Regression for Membership Inference and AI-Generated Text Detection

Problem setting

• LAR addresses membership inference and AI-generated text detection as binary classification problems based on conditional token probabilities.Membership inference tests training-set inclusion, whereas AI-generated text detection distinguishes human-written from model-generated text.Source: §1 · p. 2

Motivation and gap

• Existing likelihood methods usually compress token probabilities into a few prespecified scores, leaving context-length and token-position dependence insufficiently modeled.The paper also identifies variable document lengths, nuisance variation, and unknown informative regions as central statistical challenges.Source: §1 · p. 3

Method

• LAR evaluates each target token under nested left-context windows, aligns the resulting variable-length arrays, and models class log odds using location and likelihood-derived features.LAR-1 aggregates learned cell contributions, while LAR-2 adds same-token second-order products across context lengths.Source: §1 · pp. 3–4

Empirical results

• Across five scoring models, LAR achieves mean held-out AUCs of 0.914–0.983 for membership inference and 0.985–0.996 for AI-generated text detection, substantially outperforming likelihood-based procedures.Restricted-array analyses show that shorter-context likelihoods add information beyond full-context likelihoods, while second-order augmentation mainly improves membership inference.Source: §1 · p. 4

Theory

• For the within-path quadratic model, the paper establishes matching minimax lower and upper bounds and conditions under which an oracle spectral sieve attains the minimax rate.The analysis also connects prediction error to AUC convergence.Source: §1 · p. 4

2.1 Likelihood-based detection

• Likelihood-based membership and AI-generated-text detectors calibrate or aggregate token probabilities using contexts, perturbations, ranks, curvature, scoring models, and related constructions.These methods show that evidence depends on token difficulty, predictive vocabulary distributions, and evaluation context, while usually reducing token-level probabilities to one or a few scalar scores.Source: §2.1 · p. 5
• Labeled human-versus-AI detectors can perform strongly, but adapting them to new generators or domains may require substantial labeled data and further fine-tuning.Zhou et al. (2026) fine-tuned a language model on a large labeled corpus and calibrated its output for statistical error control.Source: §2.1 · p. 5

2.2 Regression with structured covariates

• LAR-1 and LAR-2 share first- versus second-order effects with functional regression, but each text supplies multiple context-indexed target-token paths rather than one trajectory.LAR-2 restricts second-order products to evaluations of the same target token, distinguishing its covariate structure from standard functional regression.Source: §2.2 · p. 6
• Unlike conventional fixed-dimensional arrays, raw likelihood arrays are triangular, vary in dimension and resolution with document length, and require cross-text alignment while preserving target-token paths.This structure motivates the proposed regression framework and the analyzed within-path quadratic model.Source: §2.2 · p. 6

3 Likelihood-Array Regression

• LAR evaluates each target token under nested left-context windows, aligns variable-size arrays across texts, and models likelihood-derived cells for classification.The scoring model is fixed, while texts are the independent sampling units and may contain arbitrary within-text dependence.Source: §3 · p. 7

3.1 Likelihood arrays and alignment

• Each likelihood-array cell records a target token’s log probability at a context scale and relative position, with interpolation mapping paths onto a common coordinate grid.The context coordinate uses a logarithmic scale, giving finer resolution to shorter contexts, while target position is normalized from the first to final predicted token.Source: §3.1 · pp. 7–8

3.2 Likelihood-derived cell features

• The five cell channels capture observed-token log probability, context contrast, standardized versions of both, and local change in standardized predictability as context increases.Context contrast compares the observed token with and without preceding context, helping isolate the incremental contribution of available context; the framework also permits other fixed, label-independent channels.Source: §3.2 · pp. 9–10

3.3 First- and second-order likelihood-array models

• LAR-1 aggregates learned contributions from aligned cells, whereas LAR-2 augments this additive structure with second-order features formed within each target-token path.The shared contribution function depends jointly on context scale, relative target position, and channel values, allowing likelihood-pattern effects to vary across scale and document location without requiring linear channel effects.Source: §3.3 · p. 11
• LAR-2 adds within-path second-order features to capture how a target token’s likelihood changes across context scales.Pairs are restricted to evaluations of the same target token, combining cross-scale associations and same-scale quadratic features; comparisons with LAR-1 assess predictive value rather than uniquely identified interactions.Source: §3.3 · pp. 12–13
• Unlike conventional scores that prespecify full-context aggregation or lower-tail rules, LAR learns how to combine likelihoods across context scales, target positions, and channels.Average loss uses full-context log probabilities, while Min-K% and Min-K%++ retain document-specific lower tails of full-context likelihoods.Source: §3.3 · p. 13

3.4 Finite-dimensional estimation

• Finite-dimensional LAR uses tensor-product basis expansions and reduces to ridge logistic regression on document-level first- and second-order features.Second-order coordinates use random rank-one projection directions to avoid directly estimating the full quadratic coefficient matrices, while transformations are fixed from training data and applied unchanged to validation or test texts.Source: §3.4 · pp. 13–15

4.1 Population formulation and assumptions

• The population formulation represents first-order path effects and same-token second-order interactions with position-sensitive Hilbert-space features and symmetric tensor products.The quadratic feature forms pairwise products within each target-token path before position-sensitive averaging, preserving the same-token restriction.Source: §4.1 · pp. 16–17
• Under bounded feature maps and spectral decay, the within-path quadratic model has a product covariance spectrum, without requiring different target-token paths within a text to be independent.Variation in symmetric-tensor directions indexed by (e_j, e_k) is controlled by products of covariance eigenvalue decay terms.Source: §4.1 · p. 18

4.2 Minimax prediction rate for the within-path quadratic model

• Theorems 1 and 2 establish matching minimax lower and upper bounds for prediction, with ridge estimation attaining the minimax rate over the assumed class.Squared prediction loss is equivalent up to constants to excess population logistic risk, and the matching logarithmic factor cannot be removed uniformly because of the pair-indexed quadratic spectrum.Source: §4.2 · pp. 19–20

4.3 Finite path sieves and squared projections

• The finite-dimensional analysis separates path-sieve approximation error from estimation using squared projections, which reduce unrestricted quadratic coefficients while preserving within-path structure.An unrestricted quadratic model has d_v d(d+1)/2 coefficients, whereas the squared-projection representation uses d_v R coefficients; Theorem 3 provides the corresponding approximation result.Source: §4.3 · p. 21
• The finite-sieve error bound separates path-sieve approximation, regularization bias, estimation error, and squared-projection approximation error.The squared-projection term concerns the scalar quadratic prediction score rather than approximation of an unrestricted coefficient matrix in Frobenius norm.Source: §4.3 · pp. 21–22
• The oracle spectral sieve achieves expected prediction error of order {log(en)/n}^ν/(ν+1).The spline and random channel bases used numerically need not coincide with the population covariance directions; Theorem 3 also applies to those and other basis families.Source: §4.3 · p. 22

4.4 Implications for AUC

• Under Assumption 3, with class probabilities bounded below by cY > 0, Proposition 1 establishes expected AUC convergence for score estimators; the condition permits class overlap and holds with τ = 0 without further restrictions.Source: §4.4 · pp. 22–23

5 Numerical Studies

• The studies evaluate detection across scoring models and applications, then probe shorter-context information, within-path second-order augmentation, labeled sample size, and numerical resolution.Primary comparisons use held-out-text AUC over repeated 80/20 splits, while simulation, same-supervision comparisons, transfer results, and sensitivity analyses are reported in the Supplement.Source: §5 · p. 23

5.1 WikiMIA membership inference

• LAR-1 and LAR-2 achieve higher mean AUCs than every comparison procedure across all five WikiMIA scoring models, with LAR-2 improving over LAR-1 by 0.023–0.039.ReCaLL is the strongest comparison procedure, with mean AUCs from 0.765 to 0.871; LAR-2 reaches 0.958 for Pythia-6.9B versus 0.90 for Fine-tuned Score Deviation.Source: §5.1 · pp. 25–26

5.2 AI-generated text detection

• The generated-text benchmark uses 1,200 matched human and GPT-4o texts spanning fiction, news, essays, and reviews, with shared sources, opening contexts, and lengths to limit topic-based detection.The five scoring models are open-weight models distinct from GPT-4o, testing whether likelihood-array features remain informative when probabilities come from an independent scorer.Source: §5.2 · pp. 26–27
• For GPT-4o detection across four domains, LAR-1 and LAR-2 outperform every comparison procedure for each scoring model, with LAR-1 AUCs ranging from 0.985 to 0.996.RADAR and StatDetectLLM obtain mean AUCs of 0.839 and 0.980, respectively, while corresponding LAR-2 AUCs differ from LAR-1 by at most 0.005.Source: §5.2 · p. 28
• Generated-text separation is captured primarily by the first-order functional, because second-order augmentation adds little information when its weak signal cannot offset estimation and projection costs.The larger product-spectrum feature class incurs additional finite-sample estimation and projection error, which can outweigh its small approximation gain.Source: §5.2 · p. 28

5.3 Sources of detection information

• The largest cumulative AUC increase comes from the BOS contrast for WikiMIA and standardized channels for generated-text detection, whereas the local context-response channel changes AUC little.Coefficient contrasts vary over context scale and text position, change sign across the array, and differ between tasks, so regional contributions can cancel in global summaries.Source: §5.3 · pp. 29–30
• Shorter-context cells achieve AUCs of 0.907 and 0.988 for LAR-1, versus 0.802 and 0.937 for full-context cells, across WikiMIA and generated-text detection.For LAR-2, the corresponding AUCs are 0.952 and 0.989, versus 0.893 and 0.945; shorter-context results are close to the complete array, while second-order gains are pronounced for WikiMIA but small for generated-text detection.Source: §5.3 · p. 29

5.4 Effect of sample size

• With 100 labeled texts, median AUCs are 0.638, 0.684, and 0.714 for full-context-only, LAR-1, and LAR-2 on WikiMIA, versus 0.860, 0.956, and 0.955 for generated-text detection.The shorter-context likelihood array improves ranking at the smallest labeled sample size; LAR-2 adds an improvement for WikiMIA, while LAR-1 and LAR-2 remain nearly indistinguishable for generated-text detection.Source: §5.4 · p. 31

• Shorter-context likelihood cells add ranking information in both applications, while within-path second-order features substantially improve membership inference.LAR preserves variation across context scale, target position, and likelihood-derived channels that conventional full-context summaries discard.Source: §6 · p. 32
• Under product-form covariance decay, the within-path quadratic model achieves matching minimax prediction bounds while finite path sieves and squared random projections incur additional approximation costs.The analysis characterizes these finite-dimensional and random-projection effects within the theoretical framework.Source: §6 · p. 32
• Practical deployment requires calibration for threshold decisions, and constructing likelihood arrays is more computationally demanding than full-context scoring.These limitations motivate adaptive selection or subsampling of context windows to reduce cost while preserving relevant likelihood information.Source: §6 · p. 32
• Future work could combine multiple scoring models and localize membership or generated-text evidence within documents.Source: §6 · p. 32

• The WikiMIA benchmark is available from its Hugging Face repository, while AI-generated text-detection data are available from the AdaDetectGPT repository.Source: Data availability statement · p. 32

• The section is labeled as supplementary material for Token-Level Likelihood-Array Regression addressing membership inference and AI-generated text detection.Source: Supplementary Material for Token-Level… · p. 36

S1.1 Notation and preliminary covariance bound

• The framework bounds text-level likelihood-array features uniformly, allows arbitrary dependence among within-text paths, and controls covariance for first- and second-order feature blocks.Conditional responses follow a logistic model with bounded intercept and coefficient norm; the covariance argument includes cross-covariance between feature blocks.Source: S1.1 Notation and preliminary… · pp. 36–37

S1.2 Spectrum and effective dimension

• Product-spectrum counting yields effective-dimension growth of order λ^-1/ν log(e/λ), with matching lower bounds under the two-sided eigenvalue condition.The ordered off-diagonal product eigenvalues and the first-order, diagonal, and off-diagonal covariance contributions are combined to obtain the effective-dimension bounds.Source: S1.2 Spectrum and effective dimension · pp. 38–40

S1.3 Proof of Theorem 1

• Theorem 1’s proof constructs bounded Rademacher path features and a logistic hypercube whose adjacent alternatives have controlled separation and indistinguishability for an Assouad lower bound.The construction verifies the model-class constraints at text level without requiring independent paths within a text, then converts testing difficulty into prediction error.Source: S1.3 Proof of Theorem 1 · pp. 42–45
• The lower-bound argument chooses δ ≤ 1, ensuring the resulting right-hand side remains bounded below by a positive constant.Source: S1.3 Proof of Theorem 1 · p. 45
• Using h2 = δ/(nρbn), absorbing fixed δ into c, and ρ2bn/ρbn ≍ 1 yields the stated lower-bound expression.Source: S1.3 Proof of Theorem 1 · p. 45
• Because the hypercube is a subset of Pν(M), the same lower bound applies to the supremum over Pν(M), completing Theorem 1.The proof also uses bn/n ≍ {log(en)/n}^ν/(ν+1).Source: S1.3 Proof of Theorem 1 · p. 45

S1.4 Proof of Theorem 2

• Bounded-score curvature makes excess logistic risk equivalent to squared prediction error under bounded features, radius constraints, and centered features.The logistic second derivative is uniformly bounded below on the relevant score interval, while centering removes the intercept–slope cross term.Source: S1.4 Proof of Theorem 2 · p. 46
• Rademacher contraction with a deterministic offset transfers one-Lipschitz loss increments into a symmetrized empirical-process bound.The logistic loss derivative is bounded by one, enabling Lemma S5 to control the localized supremum after ghost-sample symmetrization.Source: S1.4 Proof of Theorem 2 · pp. 46–48
• The localized empirical-process lemma bounds the scalar and Hilbert-space components using covariance effective dimension, without requiring the reference score to be the true conditional log odds.The proof completes the square in the Hilbert-space term and expresses the resulting quantity through the covariance operator’s eigenvalues.Source: S1.4 Proof of Theorem 2 · pp. 47–48
• Applying the estimator’s basic inequality, localizing its empirical term, and substituting the covariance bound yields the asserted Theorem 2 rate.The penalty difference is controlled by a quadratic inequality; the resulting effective-dimension term is proportional to λ, and the 1/n term is absorbed for 0 < λ ≤ 1.Source: S1.4 Proof of Theorem 2 · pp. 49–50

Quadratic feature and projection construction

• Theorem 3 compares the complete finite-sieve within-path quadratic feature with LAR-2’s finite collection of squared projections.The proof represents quadratic features as symmetric matrix blocks with a direct-sum Frobenius structure and constructs the corresponding random projection map.Source: S1.5 Proof of Theorem 3 · pp. 50–51

Random projection approximation

• The squared-projection representation is unbiased, and its variance is controlled uniformly in the sieve dimension through spherical fourth- and eighth-moment bounds.The inverse fourth-moment correction operator is uniformly bounded, yielding mean-square projection error of order O(R−1) for norm-bounded quadratic coefficients.Source: S1.5 Proof of Theorem 3 · pp. 51–54

Effective dimension after squared projection

• Random projection does not replace covariance-based estimation complexity by the nominal number dvR of projected coordinates.The effective-dimension lemma bounds the projected covariance complexity using the original covariance operator plus a positive perturbation of rank at most dv.Source: S1.5 Proof of Theorem 3 · pp. 54–56

Ridge oracle bound under feature reduction

• A ridge oracle inequality applies when the response law depends on the complete feature but estimation uses a centered bounded reduced feature.The argument conditions on the complete feature, so the true score need not be measurable with respect to the reduced feature; boundedness and logistic curvature provide the comparison control.Source: S1.5 Proof of Theorem 3 · pp. 56–57

Proof of Theorem 3

• Combining the projection approximation with the conditional reduced-feature oracle bound yields a projected ridge comparator whose approximation contribution is O(R−1).The proof conditions on the independently generated projection directions, applies the oracle inequality with the projected feature covariance, and uses uniform score bounds.Source: S1.5 Proof of Theorem 3 · pp. 58–59

Projection bound

• Under the covariance decay assumption, averaging conditional projection bounds and absorbing fixed-dimensional terms proves the inequality asserted in Theorem 3.The proof uses the effective dimension based on the fitted feature Z_R, while the target norm remains P_Φ because the true log odds depend on the complete feature.Source: S1.5 Proof of Theorem 3 · pp. 59–60

Path-sieve approximation

• The finite path sieve’s approximation error is bounded by projecting first- and second-order coefficient blocks onto the first d covariance eigen-directions.Omitted first-order coordinates have path indices above d, while omitted second-order coordinates include at least one such index, yielding the stated approximation bound.Source: S1.5 Proof of Theorem 3 · p. 60

Resolution and rate

• With d ≳ (n/L_n)^(1/(ν+1)), regularization λ ≍ r_n and the fixed-position term are both bounded by r_n, so the estimator attains the minimax rate.The covariance-projection eigenvalue comparison and effective-dimension bound ensure every term in the error expression is of order at most r_n.Source: S1.5 Proof of Theorem 3 · pp. 60–61

Proof of Proposition 1

• AUC admits a pairwise representation whose pointwise maximum is achieved by ranking observations according to q(Φ), equivalently the strictly increasing logit score η0.The representation uses independent feature copies and the class-probability condition keeps the normalization denominator bounded away from zero.Source: S1.6 Proof of Proposition 1 · pp. 61–62
• The AUC regret of the fitted score is bounded by the expected absolute posterior difference restricted to events where the fitted and optimal rankings disagree.Specifically, AUC(η0) − AUC(bη) ≤ [2P(Y = 1)P(Y = 0)]^-1 E{|q − q′|1(E)}.Source: S1.6 Proof of Proposition 1 · p. 62
• Splitting disagreements by whether |q − q′| exceeds a threshold and controlling them with posterior estimation errors yields the proposition’s bound after optimizing the threshold and averaging over the training sample.The proof uses ∆ ≤ e + e′ on disagreement events, the inequality 1(∆ > t) ≤ (e + e′)^2/t, and Jensen’s inequality.Source: S1.6 Proof of Proposition 1 · pp. 62–63

S1.7 Additional theoretical remarks

• The theory’s problem-specific ingredients are conditional averaging for dependent token paths, quadratic-product spectral ordering, a matching path-level lower submodel, and analysis of squared projections.The Rademacher construction defines only a least-favorable submodel, while arbitrary within-text path dependence remains allowed through covariance transfer by conditional expectation.Source: S1.7 Additional theoretical remarks · p. 63
• Theorem 3 accommodates explicit spectral-sieve approximation error d^-ν for the spline and random-channel bases used numerically.The stated bounded-predictor condition is uniform in d and R, including the normalization ∥a_r∥_2 = … as given in the supplied passage.Source: S1.7 Additional theoretical remarks · p. 63
• The bounded-predictor condition supplies uniform logistic curvature but does not follow from Assumption 1 alone or establish that λ = 0.01 is asymptotically optimal.The condition is stated separately because increasing d does not imply pointwise boundedness of projected predictors.Source: S1.7 Additional theoretical remarks · p. 63

S1.8 Equivalent empirical-measure formulation

• LAR-1 and LAR-2 have equivalent formulations as normalized empirical measures built from unit point masses over aligned array cells.The path-level and document-level measures are probability measures with equal mass across aligned scales and equal total mass across target tokens.Source: S1.8 Equivalent empirical-measure… · pp. 63–64
• The document-level measure retains context scale, target position, and channel value for LAR-1, while path-specific measures preserve target-token grouping for LAR-2.Thus the measures provide compact notation for the same finite aligned arrays and their cellwise and second-order contributions.Source: S1.8 Equivalent empirical-measure… · p. 64
• Conventional full-context scores can also be written as functionals of empirical distributions and document-specific quantiles.For channels ℓ and z, these quantile-based functionals correspond to Min-K% and Min-K%++, rather than special cases of LAR-1’s shared cellwise function.Source: S1.8 Equivalent empirical-measure… · p. 64

S2.1 Operating characteristics

• The supplement reports normalized partial AUC over false-positive rates [0, 0.10] and empirical true-positive rates at three operating points from five-fold out-of-fold LAR-2 scores.These characteristics use the primary representation and fitting specification.Source: S2.1 Operating characteristics · p. 65, Table S1 · p. 65

S2.2 Comparison under common supervision

• Under common supervision and regression specifications, complete-array LAR-1 beats full-context LAR-1 across all ten scoring models, while LAR-2 does so for WikiMIA and four of five generated-text settings.Supervised summaries improve on many fixed scores, and full-context LAR further improves through flexible modeling of the full-context channel distribution.Source: S2.2 Comparison under common supervision · pp. 65–66, Table S2 · p. 65, Table S3 · p. 66

S2.3 Additional text generators

• Across both Claude-3.5-Haiku and Gemini-2.5-Flash, LAR-1 and LAR-2 outperform comparison procedures for every scoring model, with AUCs of 0.978–0.996 and 0.979–0.992, respectively.The two variants differ little, with the better variant changing across scoring models; results extend beyond GPT-4o and suggest first-order array information explains most separation.Source: S2.3 Additional text generators · p. 67, Table S4 · p. 67, Table S5 · p. 67

S2.4 Values underlying the restricted and sample-size analyses

• Restricted-analysis tables provide numerical results for cumulative channel and separate array-support restrictions, while coefficient-contrast surfaces show spatial patterns across five likelihood-derived channels.The supplement also tabulates labeled-sample-size curves using one full-sample fit and 20 repeated training subsamples evaluated on a common test set.Source: S2.4 Values underlying the restricted… · pp. 68–69, Figure S1 · p. 68, Table S6 · p. 69

S2.5 Sensitivity analysis

• In sensitivity analyses, LAR-2 remains above the strongest comparison across displayed settings, with seed-based AUCs of 0.947–0.958 for WikiMIA and 0.988–0.990 for AI-generated text detection.Ridge penalty and channel-basis changes have little effect, while position-basis dimension and squared projections vary WikiMIA AUC more.Source: S2.5 Sensitivity analysis · pp. 69–70, Figure S2 · p. 70

S2.6 Simulation

• Simulations show LAR-1 detects marginal mean, scale, and variable-length shifts, whereas under dependence-only shifts LAR-1 stays near 0.5 and LAR-2 rises from approximately 0.53 to 0.70.The dependence-only setting preserves one-point Gaussian margins, isolating the quadratic term; the null control has mean AUC 0.49 for both models.Source: S2.6 Simulation · pp. 70–71, Figure S3 · p. 71

S2.7 Transfer across length groups and text domains

• Cross-length AUC is 0.979 when shorter groups train and longer groups test, but 0.836 in the reverse direction; leave-one-domain-out differences are 0.002–0.021.The within-group AUC of 1.000 for the 256-token group is descriptive because that group contains only 82 texts.Source: S2.7 Transfer across length groups and… · pp. 71–72, Table S8 · p. 72, Figure S4 · p. 72