Source-linked AI summary
Interpolation Is Not Invariance: Pair Count Is Not Coverage in Transformation Audits
Mohammed Ahnouch, Lotfi Elaachak
TL;DR
Transformation audits often report pair count as coverage, although pairs from one semantic object are correlated and the count does not capture independent constraints or orbit-wide certification. The paper models audits as correlated Gaussian contrasts and orbit graphs, then combines rank-aware analysis, graph certificates, leave-one-orbit-out updates, and a source-disjoint confidence gate. For s < r, the squared projection of a fixed direction onto a Haar (r − s)-plane is beta-distributed; for s ≥ r, exact calibrated interpolation is infeasible.
Problem
Transformation audits often report pair count as coverage, although pairs from one semantic object are correlated and the count does not capture independent constraints or orbit-wide certification.
Source: §1 · p. 2Method
Results
For s < r, the squared projection of a fixed direction onto a Haar (r − s)-plane is beta-distributed; for s ≥ r, exact calibrated interpolation is infeasible.
Source: §3 · p. 5Takeaways & Limitations
Audit coverage should be judged by semantic objects, contrast rank, orbit connectivity and gap, and whole-orbit validation rather than pair count alone.
Source: §7 · p. 13The beta-law result assumes Gaussian latent innovations and fixed A, while the graph certificate requires a correct equivalence relation and a finite reversible graph.
Source: §7 · p. 13
Abstract
from arXiv · showhide
Counting equivalent pairs is a common way to report transformation-audit coverage, but it can substantially overstate the constraints imposed by an audit: pairs generated from the same semantic object are correlated, and complete orbit graphs contain algebraically redundant edges. We therefore distinguish four complementary quantities---edge count $m$, effective contrast rank $s$, population support rank $r$, and graph spectral gap $\eta$---and characterize their roles in audit coverage and deployment reliability. Under a rank-$r$ Gaussian contrast model, a population-invariant calibrated reader exists exactly when the anchor has a component in $\ker T$. When an audit has rank $s < r$, its unobserved risk is $R^\star/U$, with $U \sim \operatorname{Beta}((r-s)/2,s/2)$; when $s \ge r$, exact calibrated interpolation is infeasible. The same distinction appears in orbit topology: a spanning tree imposes the same exact-null constraints as a complete graph, while a sharp graph Poincare inequality propagates edge-level drift to an entire orbit at a cost proportional to $1/\eta$. Cyclic audits can additionally yield zero pair-level leave-one-out error without holding out any semantic object. To address these failures, we derive exact block-Woodbury leave-one-orbit-out updates and introduce a source-disjoint deployment gate over finitely many candidate readers. The gate retains the original reader unless uncertainty bounds certify lower drift within a prescribed clean-utility budget. etc..
1 Introduction
Transformation audits often count equivalent pairs as coverage, although pairs from one semantic object are correlated and complete orbit graphs contain redundant edges. The paper separates edge count, contrast rank, population support rank, and graph spectral gap to describe what an audit constrains and what it can certify.
1 Introduction
- A q-state orbit contributes q(q −1)/2 recorded pairs but only q −1 independent contrasts, so a complete graph adds no exact-null constraint beyond a spanning tree.The audit’s pair count can therefore exceed the independent representation constraints it imposes.Source: §1 · p. 2
- Zeroing audited contrasts is interpolation, not evidence of small drift on a fresh semantic object or on transformation states absent from the audited edges.The paper motivates the issue with paired translations and paraphrases [Yong et al., 2024, Deng et al., 2024, Wang et al., 2024] and controlled image experiments.Source: §1 · p. 2
- Audit strength separates into edge count m, effective contrast rank s, population support rank r, and graph spectral gap η.The first three characterize the audit’s constraints; η separately bears on whether local comparisons certify an entire orbit.Source: §1 · p. 2
- The paper combines a rank-aware finite-sample theorem, a graph Poincaré inequality, a cycle lemma, exact leave-one-orbit-out updates, and a finite-candidate confidence gate.It positions this combination as an orbit-aware audit and deployment rule, rather than a new graph inequality or matrix identity.Source: §1 · pp. 2–3
2 Audits are graphs, not bags of pairs
The audit is represented as a graph for each semantic object, with edges encoding which transformation states are compared. Its contrast blocks record audited score differences, while rank and spectral gap capture properties that pair counts alone do not.
2 Audits are graphs, not bags of pairs
- For each semantic object, the audit specifies transformation states and a weighted graph of compared states; the representation matrix and incidence matrix form a contrast block.The blocks are U_j = H_jB_j, and the audit design is b_TG = XX⊤.Source: §2 · p. 3
- A contrast block’s columns contain the reader’s audited score differences, and deleting an object removes its whole block without changing remaining weights.Source: §2 · p. 3
- Effective contrast rank s is the rank of the audit design, while population support rank r is the rank of the fresh-contrast second moment T.The graph spectral gap η is defined for the reversible state-transition graph used in the orbit certificate.Source: §2 · pp. 3–4
- Anchor calibration a⊤w = 1 fixes one affine degree of freedom but by itself neither preserves decisions nor bounds clean accuracy.Source: §2 · p. 4
- An exactly interpolating reader satisfies X⊤w = 0 in addition to anchor calibration.Source: §2 · p. 4
3 The effective-rank law
Under the rank-r Gaussian contrast model, audit constraints depend on effective rank s rather than raw pair count m. The theorem distinguishes a zero-risk kernel case, a beta-law risk when 1 ≤ s < r, and infeasibility when s ≥ r.
3 The effective-rank law
- Only the s-dimensional column space selected by A constrains interpolation, so correlated audit columns do not make m the effective constraint count.The model takes Z to have independent standard Gaussian entries and A to be deterministic with rank s.Source: §3 · p. 4
- If the anchor has a nonzero component in ker T, its normalized kernel projection is anchor-calibrated, annihilates the audit, and has zero population risk.This kernel escape may move far from the deployed reader, so anchor calibration alone is not a utility guarantee.Source: §3 · pp. 4–5
- When a0 = 0 and s ≥ r, no anchor-calibrated exact interpolator exists almost surely.The audit span fills the population support, forcing a zero coordinate that contradicts anchor calibration.Source: §3 · p. 5
- For a0 = 0 and 1 ≤ s < r, exact calibrated interpolation has risk R*/U, where U is the squared projection of a fixed direction onto a Haar (r − s)-plane and follows a beta law.The tail and first inverse moment give risk bounds and moments; as r and s grow with r/s → γ > 1, the minimum-risk ratio converges in probability to γ/(γ − 1).Source: §3 · pp. 4–5
- A complete graph and any spanning tree on the same connected orbit impose identical exact-null constraints despite having different edge counts.For a graph with q vertices and c connected components, rank(B) = q − c.Source: §3 · p. 6
- For approximate repair, unlike exact constraints, edge weights and graph conditioning determine how observed edge energy controls unobserved orbit behavior.The population transition drift can be propagated to an all-state orbit certificate using a graph Poincaré inequality, at a cost proportional to 1/η.Source: §3 · p. 6
4 From local edges to orbit coverage
Connected reversible transformation graphs let local edge drift certify behavior across an orbit, with the certificate weakened by a small spectral gap. Pair-level leave-one-out can be vacuous on cycles, so validation should remove whole semantic objects.
4 From local edges to orbit coverage
- For connected reversible graphs, edge transition loss controls all-state orbit variance with the sharp factor 1/η.An eigenfunction for the first nonzero eigenvalue attains equality.Source: §4 · p. 6
- Object-level validation combines Hoeffding bounds with the orbit certificate, while small η magnifies uncertainty and reused training objects require independent validation or a simultaneous bound.The finite-sample bound is stated with probability at least 1 − δ.Source: §4 · p. 7
- Disconnected audits have η = 0 and cannot control offsets between components; identity-only generator checks do not establish invariance under compositions.Generator edges must be sampled at orbit locations in a connected graph before the certificate applies.Source: §4 · p. 7
- Deleting one edge on a cycle leaves its contrast in the span of the remaining edges, so exact training annihilation also gives zero held-out residual.Pair-level leave-one-out can therefore report perfect prediction without removing independent information; delete the semantic object and all its orbit edges instead.Source: §4 · p. 7
5 Utility-aware repair and safe selection
The paper trades invariance repair against clean behavior through utility-aware ridge, then uses object-level refitting and source-disjoint validation to select repairs safely. The baseline reader is retained unless bounds certify improvement within the utility budget.
5 Utility-aware repair and safe selection
- A G-weighted edit cost lets λ > 0 ridge trade invariance repair against clean behavior without requiring the reader to lie in the exact-null plane.G measures edit cost, and the regularized solution may depart from exact nulling.Source: §5 · pp. 7–8
- Block-Woodbury updates from one full fit give exactly the reader refitted after deleting an entire semantic object and its orbit block.The inverse update is a negative rank-|E_j| update; substituting it into the reader formula yields the object-deleted refit.Source: §5 · p. 8
- A source-disjoint validation gate selects only candidates whose bounds certify lower orbit loss within the clean-utility budget; if none passes, it retains the baseline reader.The theorem bounds clean-accuracy loss at 0.6 percentage points and uses simultaneous bounds to support adaptive candidate selection.Source: §5 · pp. 8–9
6 Experiments
Across the experiments, effective rank and graph connectivity—not raw pair count alone—characterize audit coverage. Under an equal pair budget, graph choice trades source breadth against orbit connectivity, and intermediate designs can balance the two.
6.1 Rank collapse despite hundreds of edges
- 31, 32, and 496 recorded pairs yield effective rank 31 for tree, cycle, and complete audits, respectively; the empirical mean risk ratio is 1.1362 against the exact 1.1390.The three graph types share exact nullspaces, and Figure 2 shows normalized risks following the inverse-Beta prediction.Source: §6.1 · pp. 9–10, Figure 2 · p. 10
6.2 Frozen digits: equal budgets, different orbit graphs
- At an equal budget of 480 pairs, identity checks cover 120 sources, cycles 30, cubes 15, and complete graphs 4, trading source breadth against state connectivity.Identity checks leave composition states disconnected; the other three designs cover all states, with different gaps and edge multiplicities.Source: §6.2 · p. 11
- −0.1880 drift change (paired hierarchical 95% interval [−0.2949, −0.0969]) favors the cube over identity under utility-aware ridge.The contrast is pre-specified and directional; the figure also reports threshold disagreement and clean accuracy.Source: §6.2 · p. 11, Figure 3 · p. 12
- 0% of fits pass the joint gate for the cube contrast, so deployment returns the original reader rather than accepting a repair that misses the clean-utility criterion.The reported clean accuracy falls from 0.9637 to 0.4964.Source: §6.2 · p. 11
6.3 What to optimize under a pair budget
- A spanning graph maximizes source breadth, while a complete graph maximizes local connectivity; intermediate graphs can improve gap per added edge, but the greedy stopping rule has no global optimality guarantee.After per-orbit rank saturates, added edges should improve spectral conditioning rather than merely raise pair count.Source: §6.3 · pp. 12–13
- 2.9 × 10^-16 relative error separates block-Woodbury from brute-force refitting, and the source-level gate’s simultaneous event holds in 99.8300% of 10^4 repetitions at δ = 0.05.The numerical Poincaré matrix-slack check has minimum eigenvalue −9.3 × 10^-16, up to floating-point tolerance.Source: §6.3 · p. 13
7 Limitations and conclusion
The paper’s certification claims are bounded by modeling and deployment assumptions, while its closing guidance bases audit decisions on rank, orbit topology, and source-level validation—not pair count.
7 Limitations and conclusion
- The beta-law analysis assumes Gaussian latent innovations and fixed A, while the graph certificate requires a correct equivalence relation and a finite reversible graph.The digit transformations are mild and fixed-order, so they do not test noncommuting actions or a misdeclared relation.Source: §7 · p. 13
- A frozen linear head cannot recover absent representation information, anchor calibration only normalizes, and the deployment gate requires independent source orbits with bounded loss.The digit experiments probe orbit geometry, not multilingual guards or foundation models.Source: §7 · p. 13
- Count semantic objects rather than edges, report contrast rank and orbit gap, hold out whole orbits, and deploy only when a source-level bound beats the current reader within budget.Pair count alone certifies none of these.Source: §7 · p. 13