PaperlayerView exact revision on arXiv

Source-linked AI summary

Trustworthy Method Comparison with AI Judges: Estimation and Design under Order, Batch, and Aggregation Effects

Tianxi Li, Jie Ding

arXiv:2610.07755v1stat.MLcs.LGstat.APstat.ME

TL;DR

AI-judge evaluations often batch items, where sequential dependence complicates whether aggregated scores recover the judge’s conclusion. The paper models those ratings with a Markov GLMM and finds different implications for leaderboard ranking and group comparison: naive group averaging can be biased, while model-based inference is supported, including empirically with higher-order memory.

  • Problem

    When batched ratings depend on preceding ratings, it is unclear whether randomization and aggregation recover the judge’s own conclusion rather than introduce measurement error.

    Source: §1 · pp. 1–2

  • Method

    The paper uses a Markov GLMM working model for sequential ratings and studies estimation and design for leaderboard ranking and group comparison.

    Source: §1 · p. 2, §2.1 · p. 4

  • Results

    Marginal averaging is consistent for leaderboard selection under mild separation, but can be structurally biased for group comparison; GLMM inference is valid for both tasks and empirically robust to higher-order memory.

    Source: §1 · pp. 2–3, §7 · p. 17

  • Takeaways & Limitations

    The paper recommends GLMM inference with a random-shuffle batch design for reliable inference using AI judges.

    Source: §7 · p. 17

    The design theory is first-order, and the paper focuses on binary accept/reject outcomes rather than scalar scores.

    Source: §7 · p. 17

Abstract

from arXiv · show

Large language models (LLMs) are increasingly used as judges for automated AI evaluation. A common practice is to randomize prompt sequences and average the resulting scores, but its statistical validity remains unclear. We show that LLM evaluation mechanisms can be approximated by a class of Markov generalized linear mixed models (GLMMs), supported by out-of-sample predictions across three major commercial LLMs. Using a first-order Markov GLMM, we study leaderboard ranking and group comparison. For leaderboard ranking, randomize-and-average selection is consistent under a mild separation condition, and a Williams square design can improve efficiency when item qualities are close. For group comparison, naive averaging can yield inconsistent conclusions about differences in group-level quality because of the response model's nonlinearity. Empirical results further support the validity of the proposed model-based inference beyond the first-order theory, including settings with higher-order sequence memory. We illustrate the approach in an application where AI judges compare two graphical model estimation methods.

1 Introduction

The paper addresses whether aggregation recovers an AI judge’s own conclusion when ratings depend on order, taking the judge’s criterion as given. It proposes a Markov GLMM working model and analyzes when randomize-and-average procedures are reliable for ranking and group comparison.

1 Introduction

  • The measurement question is whether aggregation recovers the judge’s conclusion, distinct from whether its criterion aligns with human judgment.The introduction warns that distortion can increase with evaluation budget, making conclusions more confident without recovering the judge’s answer.Source: §1 · pp. 1–2
  • The analysis treats the judge’s criterion as given and addresses statistical failures through estimators and experimental designs, not prompt changes.Source: §1 · p. 2
  • Randomized order is commonly followed by marginal averaging, yet aggregation can systematically distort ratings in an order-dependent process.The paper notes that this second, estimator step has received less attention in the order-effect literature.Source: §1 · p. 2
  • The proposed working model is a Markov GLMM for binary batch ratings, assessed empirically with out-of-sample prediction.It includes serial, cold-start, and batch-level effects and is presented as an approximation, not a representation of the judge’s internal mechanism.Source: §1 · p. 2
  • Leaderboard selection by marginal averaging is consistent under mild separation, but group averages can be structurally biased; GLMM inference is valid for both tasks, including empirical settings with higher-order memory.A Williams Latin square can improve leaderboard selection without added computational or token cost.Source: §1 · pp. 2–3

2 A Working Statistical Model of AI Judges

The paper develops a Markov GLMM working model for sequential batch ratings and evaluates its fit across three AI judges. Out-of-sample checks support the model’s adequacy, while the fitted judges show serial dependence and batch variation.

2 A Working Statistical Model of AI Judges

  • For Gemini, acceptance was 73.0% after rejection versus 66.6% after acceptance, a −6.4-point difference suggesting a negative serial effect.The comparison pools positions 2–30; the opening slot has no predecessor.Source: §2 · p. 3, Figure 1 · p. 4

2.1 A working model for sequential AI judges

  • The Markov GLMM models each binary rating using item quality and a pre-item contextual shift on the logit scale, with serial, cold-start, and batch effects.The contextual shift depends on preceding ratings, position, and batch effect, but not the identity of the current item.Source: §2.1 · pp. 3–4
  • Exchangeable allocation alone is insufficient: inference also relies on independent randomization units, supplied by random shuffles across batches or independently randomized Williams-design blocks.Near-uniform acceptance or rejection can also cause separation, leaving an item effect without a finite maximum-likelihood estimate.Source: §2.1 · p. 5

2.2 Fitted models for three AI judges

  • All three judges had negative lag-1 contrasts; Gemini’s signal was confined to lag 1, GPT-5.4-mini also showed lag 3, and Claude showed negative contrasts at lags 1–3.Batch random effects were substantial for all three judges (σ^2_b = 0.446, 1.173, and 0.418).Source: §2.2 · pp. 5–6, Figure 2 · p. 6

2.3 Model adequacy by out-of-sample predictive evaluation

  • 34 of 36 held-out transition probabilities fell within 95% prediction intervals, and all three conditional entropies were covered.The two transition misses were GPT-5.4-mini’s lag-2 and lag-3 0→1 cells; the intervals were described as narrow relative to the true values.Source: §2.3 · pp. 6–7, Figure 3 · p. 7

3 Task 1: Leaderboard Ranking

For leaderboard ranking, the paper studies selecting the best item from a pool: naive win-rate averaging is consistent when the top items are sufficiently separated, and a balanced Williams design can improve selection performance. Simulations show that the improvement depends on the judge’s position profile and is modest in the favorable setting.

3 Task 1: Leaderboard Ranking

  • The symbol-randomized Williams design balances each item across positions and each ordered pair across consecutive slots, with item-to-symbol assignments redrawn independently per block.The square exists for even V, and the budget is a multiple of V; the paper’s setting is a Markov model, so classical design results do not directly apply.Source: §3 · p. 8

3.1 Theory for the naive estimator in leaderboard selection

  • Naive selection is consistent under either random shuffle or Williams design when the top-item win-probability gap exceeds |β|Λ1/V.The condition requires a sufficiently separated pair of top items under Assumptions 1 and 2.Source: §3.1 · p. 8
  • For qualities near a common value, every contrast has strictly smaller variance under Williams at every finite budget, while equal qualities make its design variance zero.The result assumes Assumptions 1 and 2 and a common value at which Assumption 3 holds for the flat configuration.Source: §3.1 · p. 9

3.2 Simulation evidence

  • Under Setting 2, the Williams design raises naive Top-1 accuracy by 1.2, 2.0, and 3.1 percentage points, while Setting 1 gains are −0.5, −0.1, and −0.5.Setting 1 has a nearly flat position profile; Setting 2 has an uneven profile. The simulation used V = 6 items and 10,000 replicates per cell.Source: §3.2 · p. 10, Table 1 · p. 10
  • The simulation’s Williams advantage is modest and profile-dependent, while the design does little for the model-based estimator, which already accounts for position.The paper attributes the modest gain to position-profile departures concentrated in one slot; it frames balancing as useful for small pools under tight budgets.Source: §3.2 · p. 10

4 Task 2: Group-Level Inference

Group comparison asks whether two collections differ in average judge-assigned quality. Because averaging nonlinear response probabilities can distort latent quality differences, the naive estimator can mislead even as its sampling uncertainty shrinks.

4.1 The population-level bias

  • The naive contrast averages nonlinear transforms of item quality, so differences in within-group spread can appear as quality differences.Its limit can shrink the true quality signal by at least a factor of four, and by an order of magnitude in harsh regimes.Source: §4.1 · pp. 11–12
  • When curvature drives a nonzero naive contrast under the matched-third-moment null, increasing the evaluation budget can make a test confidently reject a true null.The test’s rejection probability tends to one under the stated conditions.Source: §4.1 · p. 12

4.2 Simulation evidence

  • Under the null, the naive test’s false-positive rate rose from 18.1% at N = 30 to 54.9% at N = 150, while GLMM tests stayed controlled.Figure 4 reports the false-positive rates; the GLMM Wald test stayed at 3.8–5.5%, and its bootstrap test at 6.0–8.2% beyond the smallest budget.Source: Figure 4 · p. 13, §4.2 · p. 14
  • In the null setting, the naive estimator picked the higher-variance group A in 83.1% of replicates at N = 30 and 98.1% at N = 150; the GLMM stayed near 50%.When group A truly had higher quality, both estimators recovered the correct sign in at least 99.8% of replicates at every budget.Source: §4.2 · p. 13
  • For the true-gap setting, test power was 78.3%, 77.0%, and 75.9% at N = 30, and exceeded 99.7% for all three methods from N = 90.The methods were indistinguishable in this setting.Source: §4.2 · p. 14
  • The Williams square design helps little here: bias, not variance, is the main difficulty, and resampling its balanced rows destroys the design’s balance.Resampling blocks is possible only for very large N, which the authors say is impractical in this setting.Source: §4.2 · p. 14

5 Robustness to Deep Memory

Simulations test whether the first-order conclusions persist when the judge has lag-3 memory, although the fitted model remains first-order. The tested ranking and group-comparison patterns largely persist, providing evidence of robustness in these settings, not a guarantee.

5 Robustness to Deep Memory

  • The robustness evidence is limited to tested settings: the generating judge had lag-3 memory, while one fitted GLMM assumed lag 1.The authors explicitly characterize the results as empirical robustness evidence, not a guarantee.Source: §5 · p. 14
  • For leaderboard ranking at V = 6, Williams designs gave naive averaging a modest gain in the uneven-position setting, and lag-1 and lag-3 GLMMs performed comparably.The fitted lag-1 model is deliberately misspecified relative to the lag-3 data-generating judge.Source: §5 · p. 14
  • With lag-3-generated data, the naive false-positive rate rose from 26.0% to 70.5% across N, while both GLMM fits controlled it approximately for sufficiently large N.This group-comparison result uses V = 30 under the null; the table reports bootstrap inference with B = 200.Source: §5 · p. 14, Table 2 · p. 15

6 Data Example: Comparing Two Graphical Model Estimation Methods by AI-Judged Edge Quality

The application compares AI-judged edge quality for GNC-lasso and ordinary glasso using human assessment as the reference. Across the three judges, aggregation outcomes differ: the GLMM reaches the reference conclusion sooner for OpenAI, resolves Claude’s comparison where the naive rule does not, and returns an inconclusive result for saturated Gemini where the naive rule is confidently wrong.

6 Data Example: Comparing Two Graphical Model Estimation Methods by AI-Judged Edge Quality

  • The comparison uses the 30 distinctive pairs from two estimated graphs, excluding 10 shared pairs that judges scored near-unanimously.The three judges rated each pair as a meaningful association or not across 160 randomly shuffled batches; the human evaluation served as the reference answer.Source: §6 · p. 15
  • For OpenAI, both aggregation rules supported the positive contrast, but the GLMM reached a given confidence level in roughly half the batches.It reached the plotted scale’s floor by N = 40, while the naive rule needed about 100 batches.Source: Figure 5 · p. 16, §6 · p. 16
  • For Claude, the naive rule was never significant and its estimate turned negative at N = 60; the GLMM was significant from N = 20 onward.The GLMM p-value fell to approximately 2 × 10^-12.Source: Figure 5 · p. 16, §6 · p. 16
  • For Gemini, the naive rule became significantly negative from N = 40, while the GLMM remained nonsignificant at the full sample and indicated inconclusive evidence.The judge had an accept rate of 0.86 and 60% of pairs were unanimous, consistent with the reported loss of discrimination between the graphs.Source: Figure 5 · p. 16, §6 · p. 16

7 Discussion and Practical Guidelines

LLM ratings can depend on preceding items, so prompt engineering alone cannot establish what their averages represent. The paper uses a Markov GLMM working model to distinguish the statistical strategies needed for leaderboard ranking and group comparison.

7 Discussion and Practical Guidelines

  • The Markov GLMM is treated as an approximation, with its adequacy assessed through out-of-sample predictive checks across frontier models.Empirical results also support approximately correct error control when judges have memory beyond lag one, across the regimes considered.Source: §7 · pp. 16–17
  • For leaderboard ranking, the naive marginal average is consistent under a mild separation condition, and a Williams square can improve efficiency at essentially no computational cost.The model-based approach also provides valid inference for both ranking and group comparison.Source: §7 · p. 17
  • For group comparison, nonlinear response links can turn within-group variation differences into persistent bias in marginal averages.The authors recommend GLMM inference with a random-shuffle batch design for reliable inference using AI judges.Source: §7 · p. 17
  • The design theory covers first-order Markov kernels; optimal designs for higher-order carryover remain open, and putting all items in one batch can be infeasible for large pools under bounded context.The paper studies binary outcomes; extending to scalar scores requires different models and re-deriving arguments that depend on link curvature.Source: §7 · p. 17

Supplementary Material

The supplementary material collects additional analyses, experiments, and proofs that follow the main text’s organization.

Supplementary Material

  • Section A provides predictive-check protocols, transition results, interval widths, and conditional-entropy analyses.It also includes proofs in the supplementary material’s concluding sections.Source: Supplementary Material · p. 20
  • Sections B through D detail leaderboard simulations and group-comparison experiments, including small-pool and Williams-design analyses.These materials follow the order of their appearance in the main text.Source: Supplementary Material · p. 20
  • Sections E and F cover leaderboard ranking under deeper memory and the estimated networks used in the data example.Section G gives a GLMM estimation result, while Sections H and I collect definitions, proofs, and supporting theory.Source: Supplementary Material · p. 20

A Details on out-of-sample predictive evaluation

The supplement evaluates out-of-sample predictive adequacy, specifies leaderboard simulations, and reports a small-pool group-comparison check. Its predictive checks show close coverage, while the small-pool results retain the patterns seen with the larger pool.

A Details on out-of-sample predictive evaluation

  • Across three judges, the frozen GLMM covers 34 of 36 transition terms; Gemini and Claude each cover 12/12, while GPT-5.4-mini covers 10/12.The two misses are deeper-lag 0→1 cells for GPT-5.4-mini; parameter-estimation uncertainty is not propagated, so the authors treat this as a predictive check rather than a formal goodness-of-fit test.Source: A Details on out-of-sample predictive… · p. 20
  • All 9 held-out conditional-entropy statistics fall within narrow 95% prediction intervals, including order-2 and order-3 entropy for Gemini’s lag-1 model.Transition intervals are 2.5%–14.8% of observed values, and entropy intervals are under 2.5% of observed values.Source: A Details on out-of-sample predictive… · p. 21

B Leaderboard ranking simulation

  • The ranking simulation tests whether balancing helps when the equal-quality position profile is non-flat, using two settings based on fitted judges at V = 6.The settings span a nearly flat profile and one with substantial position variation; budgets are N ∈ {6, 12, 18}, with 10,000 replicates per cell.Source: B Leaderboard ranking simulation · p. 22

C Group comparison with a small item pool

D Williams squares in group comparison: an empirical demonstration

The Williams-square experiment finds that balancing does not correct the naive group-comparison estimator’s bias. Unlike ranking, where balancing can remove design-induced variance, group comparison is constrained by estimator bias under the equal-means null.

D Williams squares in group comparison: an empirical demonstration

  • The experiment compares the naive marginal average and GLMM logit-scale contrast under random shuffle and Williams designs, using matched quality panels and 2,000 replicates per cell.It includes V = 30 and V = 6 pools; Williams budgets use whole multiples of V and independently redrawn item-to-symbol assignments.Source: D Williams squares in group… · p. 24
  • No tests are computed for the Williams arms because resampling individual balanced rows would destroy the design’s balance.Valid block-bootstrap inference requires the number of complete squares to grow, which can impose substantial evaluation costs.Source: D Williams squares in group… · p. 24
  • Under the equal-means null, the naive average names the higher-variance group winner 98.1% of the time with random shuffling and 98.5% with a Williams square at V = 30.Across 40 matched cells, the largest Williams-versus-shuffle discrepancy is 2.9%, against a Monte Carlo standard error of 1.6%; no cell reaches a Bonferroni-corrected threshold.Source: D Williams squares in group… · p. 24, Table S1 · p. 25

E Leaderboard ranking under deeper memory

The ranking experiment tests whether the Section 3 results persist when the judge has lag-3 memory. Both ranking methods work, while the Williams design’s benefit depends on the setting and lag-1 GLMM fitting performs comparably to lag-3 fitting.

E Leaderboard ranking under deeper memory

F Estimated networks for the data example

The supplement shows estimated term graphs and develops theory for GLMM estimation and Williams-design ranking. Its results characterize when balanced designs preserve expected ranks and reduce order-induced variance.

F Estimated networks for the data example

G GLMM estimation theory

  • Under independent randomization units and regular mixed-model likelihood conditions, the GLMM estimator is consistent and asymptotically normal; weak item-effect penalization leaves its limit unchanged.The stated regularity conditions include identifiable item contrasts, positive-definite limiting information, and an interior true parameter with σ_b^2 > 0.Source: G GLMM estimation theory · p. 29

H Proofs for Section 3 (Williams designs)

  • The Williams design uses complete squares with position and first-order carryover balance, with item-to-symbol assignments independently redrawn across blocks.The design assumes even V and a budget N that is a multiple of V.Source: H Proofs for Section 3 (Williams… · p. 30

H.1 Additional definitions and interpretation

  • Shuffle and Williams design terms separate rating-sampling variability from ordering variability; near equal qualities, Williams’ design term is smaller.At equal qualities the Williams term is zero, and under N^-1/2 local alternatives it is O(1/N^2), versus Θ(1/N) for shuffle.Source: §H.1 · pp. 30–31

H.2 Preliminaries: the mixture CDF and the affine recurrence

  • A Williams square gives every item an equal number of first-position slots and each other item as immediate predecessor, supporting the recurrence analysis of sequential ratings.The recurrence expresses each rating probability from the prior rating and the current slot’s model terms.Source: §H.2 · p. 32

H.3 Williams preserves expected ranks

  • The naive ranking rule is consistent under either design when the top item’s marginal win-rate gap exceeds twice the maximum rank perturbation, which is O(|β|/V).The direction of the missing-self perturbation for β < 0 is heuristic; the proof uses only its magnitude bound.Source: §H.3 · p. 34

H.4 The design variance of the Random Shuffle

  • Random shuffle has a strictly positive Θ(1/N) design-variance term, while Williams’ term vanishes at equal qualities and is O(1/N^2) under local alternatives.The two designs share every per-batch marginal moment.Source: §H.4 · p. 34

H.5 Second-moment analysis: the rest of Proposition 2′

This section analyzes Williams and shuffle designs through their expected scores, covariance, total variance, selection behavior, and missing-self tolerance. It also gives finite-sample and worst-case bounds and examines the stated scaling relationships, noting limits of the bounds and exact symmetrization.

H.5 Second-moment analysis: the rest of Proposition 2′

  • With per-block symbol randomization, Williams batches have Random Shuffle’s uniform marginal, so expected scores and average conditional covariances agree exactly.Batches within a Williams block remain dependent through their shared symbol randomization.Source: §H.5 · p. 35
  • At the global null, the Williams design term vanishes while Random Shuffle contributes Θ(1/N) variance; Random Shuffle’s design variance is positive when the slot-mean function m is non-constant.The supplied lemma gives parameter conditions ensuring non-constant m, including γ ≠ 0 when β = 0 and β ≠ 0 when γ = 0; with both nonzero, exceptions are discrete in γ.Source: §H.5 · pp. 36–37

H.6 The finite-sample identity: proof of Proposition 2

  • Near flat configurations, the finite-sample identity implies Random Shuffle has greater contrast variance than Williams, with the positive scaled difference independent of N.At the flat configuration the Williams variance term vanishes, and continuity extends the strict inequality to a neighborhood, uniformly over contrasts.Source: §H.6 · p. 37

H.7 Worst-case bounds for a fixed (unrandomized) square

  • For a fixed unrandomized square, worst-case range bounds give a covariance discrepancy of at most κcov/N, but these bounds cannot certify fixed-square dominance.The refinement is eκ|β|(|β| + |γ|)/N; the passage states that the proven constants fail to certify dominance in essentially every regime of interest.Source: §H.7 · p. 38

H.8 Quadratic refinement via the exact binary recurrence

  • When θa > θb, Random Shuffle’s variance has a positive-definite quadratic expansion in (β, γ), implying νR(a, b) = Θ(β² + γ²) near zero.For |β| ≤ 2, the fixed-square discrepancy is bounded by 7|β|(|β| + |γ|), though the proven constants still do not certify fixed-square dominance.Source: §H.8 · pp. 38–40

H.9 Probability of correct selection: proof of Theorem 1(ii) √

  • For a fixed, unrandomized square, Williams’ variance gain need not overcome a construction-dependent mean shift, and pairwise comparisons alone do not establish exact joint selection superiority.The shift can be O(|β| + |γ|), while the variance gain is Θ(β^2 + γ^2); the randomized main-line comparison does not require this case-by-case condition.Source: §H.9 · p. 42
  • Under the same selection-consistency and non-degeneracy conditions for each k, Williams reduces the asymptotic union-bound probability of incorrect selection by a strictly positive amount.The result follows by summing the strict per-contrast inequalities.Source: §H.9 · p. 42
  • Under non-degeneracy and Lk > 0 for every k, Williams has a strictly higher limiting exact correct-selection probability than the shuffle.The limiting contrast correlations coincide, so the advantage follows from Williams’ higher standardized thresholds.Source: §H.9 · pp. 42–43

I.1 Regularity of the marginal win probability

  • The marginal win probability π is infinitely differentiable, with M3 ≤ 1/8 and M4 < 0.13.Its first three derivatives are expectations of the corresponding logistic derivatives under the fixed contextual-shift law.Source: §I.1 · p. 45

I.2 Consistency under the complete-block shuffle, and the missing-self correction

  • With the independent-unit condition, complete-block-shuffle item win rates converge to their marginal means πv; exchangeability alone does not ensure convergence to a nonrandom limit.The independent units may be batches or blocks of batches.Source: §I.2 · pp. 45–46
  • The complete-block shuffle’s missing-self correction is bounded by |ev| ≤ |β|Λ1/(2V), and by 2|β|/V when |β| ≤ 2.The correction arises because the conditional prefix excludes the item itself, producing an N-free perturbation.Source: §I.2 · pp. 45–46
  • Exact computations across V ∈ {6, 8} and a fourfold spread change support the 1/V and spread-squared scalings of (18), while its constant overshoots by roughly three orders of magnitude.The computations used Gauss–Hermite integration over u and full enumeration of orderings.Source: §I.2 · p. 49

I.3 Proof of Theorem 2 (attenuated signal plus curvature bias)

The proof decomposes the naive group contrast into curvature, third-moment, and remainder terms. Under the stated null conditions, its limit is strictly positive, so fixed-threshold tests can reject with probability tending to one.

I.3 Proof of Theorem 2 (attenuated signal plus curvature bias)

  • The expansion isolates a missing-self component Rms, bounded by Proposition 6; at δ = 0, its bound is 25r|β|Λ.For unbalanced splits, the quadratic part is multiplied by r = Vmax/Vmin.Source: §I.3 · p. 50
  • At δ = 0, the expansion reduces to curvature, third-moment, and O(ρ̄^4 + |β|ρ̄^2/V) terms; matching third moments removes the displayed π′′′ term.Source: §I.3 · p. 50
  • Under H0, the naive contrast has a strictly positive in-probability limit, so tests using a fixed threshold below that limit reject with probability tending to one.The corollary states this under its assumptions, matched third moments, and positive curvature.Source: §I.3 · p. 51
  • Away from H0, the curvature term contaminates the attenuated π′(μ̄)δ signal; depending on signs, it can reinforce or partly cancel that signal, and no threshold repairs the defect.Source: §I.3 · p. 51

I.4 Finite-sample concentration

Finite-sample bounds control the naive group contrast around its exact mean under random shuffling and block shuffling. Batch-level dependence is contained within independent batch contrasts, while sharing context weakens the concentration rate.

I.4 Finite-sample concentration

  • Each per-batch contrast ct has mean ∆∞ exactly for every t and N, with dependence from shared context and the lag chain contained within ct.The contrasts are i.i.d. across batches; no within-batch independence is assumed.Source: §I.4 · p. 51
  • Pr(|b∆Naive − ∆∞| > ϵ) ≤ 2 exp(−Nϵ²/2) under random shuffling, while block shuffling replaces N with N/B.The random-shuffle variance is νb/N, with νb := Var(c1) ≤ 1.Source: §I.4 · p. 51
  • The batched bound loses a factor of order V in the exponent because of context sharing, but the results do not assume stateless evaluation.The passage frames this loss as calibration of the constants, not a stateless-evaluation assumption.Source: §I.4 · p. 52
  • Bernstein’s inequality sharpens the batch-contrast bound when νb + 2ϵ/3 < 1.It replaces the exponent Nϵ²/2 with Nϵ²/(2νb + 4ϵ/3).Source: §I.4 · p. 52
Loading 2610.07755v1…