PaperlayerView exact revision on arXiv

Source-linked AI summary

DSReg: Provably Recovering Individual World Latents without Reconstruction

Yujia Zheng, David Klindt, Randall Balestriero, Bernhard Schölkopf

arXiv:2610.09457v1cs.LGcs.AIcs.ROstat.ML

TL;DR

Methods without reconstruction or auxiliary anchors have generally identified world latents only up to a linear transformation, leaving individual factors mixed. DSReg uses observation-dependency footprints to select individual latents under Structural Diversity, and the paper reports improved individual-latent recovery while preserving dense prediction. The guarantee is evaluated in simulated or rendered settings, not on a physical robot.

  • Problem

    Without reconstruction or auxiliary anchors, methods such as JEPAs identify latent states only up to a linear transformation, leaving individual world factors mixed.

    Source: §1 · p. 1, §3.1 · p. 4

  • Method

    DSReg searches rotations of a linearly identified representation for sparse observation-dependency structure, recovering individual latents under Structural Diversity.

    Source: §3.2 · p. 5

  • Results

    DSReg preserves dense prediction while improving individual-latent recovery and downstream use across the reported evaluation settings.

    Source: §1 · p. 2, §4.1 · p. 8, §4.2 · p. 8

  • Takeaways & Limitations

    Distinct dependency footprints can identify individual world latents up to signed permutation without reconstruction or a decoder.

    Source: §3.2 · p. 5

    The benchmarks are simulated or rendered, and DSReg has not yet been tested on a physical robot.

    Source: §5 · p. 9

Abstract

from arXiv · show

Methods that recover individual latent variables of the world, from nonlinear ICA to dictionary learning and causal representation learning, anchor the latents to observations through reconstruction, auxiliary supervision, or distributional asymmetries such as non-Gaussianity. Methods without these anchors, including joint-embedding predictive architectures (JEPAs), identify the latent state only up to a linear transformation, so individual latents remain mixed. We close this gap: individual world latents can be provably recovered with no reconstruction, no decoder, and no labels. The key condition is Structural Diversity: different latents leave distinct dependency footprints on observations, just as no two snowflakes are alike. Building on the linear identifiability that LeJEPA provides, we prove that under Structural Diversity, DSReg (Dependency-Sparsity Regularization) recovers individual world latents up to signed permutation, without reconstruction or a decoder. It applies post hoc to any linearly identified representation, reusing trained checkpoints at no loss over joint training, and establishes the first fully identifiable JEPA that recovers every world latent. Moreover, as a condition on dependency footprints, Structural Diversity is strictly weaker than all structural conditions of prior identifiable latent variable models. Across synthetic regimes, world model probes, learned visual encoders, and external renderers, DSReg preserves dense prediction while improving individual-latent recovery and downstream use with scales.

1 INTRODUCTION

A world model may encode the full latent state while mixing individual factors, limiting uses that depend on acting on one factor at a time. DSReg uses dependency structure to recover individual latents without reconstruction, labels, or a decoder.

1 INTRODUCTION

  • A full latent state can remain a dense mixture, limiting intervention, precise control, visual editing, and mechanistic inspection that use individual factors.Controllers and monitors often act through only a few variables at a time.Source: §1 · p. 1
  • JEPAs avoid reconstructing observations, but without a decoder, likelihood, or auxiliary signal, their non-collapsed representations can still mix world latents.Avoiding collapse is distinct from identifying individual latents.Source: §1 · pp. 1–2
  • LeJEPA identifies the latent state up to a linear transformation, but individual variables may remain mixtures of ground-truth factors (Klindt et al., 2026).The paper illustrates the practical consequence with a robot moving a cup while keeping a nearby knife fixed.Source: §1 · p. 2
  • Prior routes to component-wise recovery use auxiliary variables, constraints on mixing, distributional asymmetries, interventions, or counterfactual views.The cited approaches anchor latents through reconstruction, likelihood, or symmetry-breaking side information.Source: §1 · p. 2
  • Under Structural Diversity, DSReg proves recovery of each world latent up to signed permutation, without reconstruction, labels, or distributional asymmetry.It applies post hoc to a linearly identified representation and reuses trained checkpoints at no loss relative to joint training.Source: §1 · p. 2

2 PRELIMINARIES

The paper distinguishes recovering individual world latents from identifying a representation only up to an invertible transformation. In a stationary Gaussian JEPA setting, alignment with a Gaussian constraint yields an orthogonal ambiguity, not component-wise recovery.

2 PRELIMINARIES

  • Individual-latent identifiability means each estimated coordinate equals one world latent up to sign and relabeling.This is the paper’s signed-permutation notion of component-wise recovery.Source: §2 · p. 2
  • Many nonlinear ICA and causal representation approaches use auxiliary variables, mixing constraints, interventions, or multi-view data, while nearly all rely on reconstruction or likelihood.Those observation anchors are absent from the JEPA objective.Source: §2 · p. 3
  • A JEPA predicts target-observation embeddings from context-observation embeddings without a decoder, reconstruction loss, or observation likelihood.Linear identifiability guarantees only that the representation is an invertible linear transformation of the latent state.Source: §2 · p. 3
  • Under the stationary Gaussian transition conditions, LeJEPA alignment reaches its optimum exactly when the representation is an orthogonal rotation of the latent state.Thus each estimated variable may still mix true latents, and no distributional criterion can resolve that rotation in this setting.Source: §2 · p. 3

3 IDENTIFIABILITY FROM STRUCTURAL DIVERSITY

The section identifies the rotation ambiguity left by linear identifiability and uses observation-dependency footprints to resolve it. Under Structural Diversity and functional no-cancellation, DSReg recovers individual latents up to signed permutation, with an estimator whose guarantee tolerates Jacobian estimation error.

3.1 FROM LINEAR TO SIGNED-PERMUTATION IDENTIFIABILITY

  • Gaussian rotations preserve the latent distribution while mixing variables, so recovery needs a rotation-sensitive signal outside that distribution.DSReg takes this signal from how observations depend on the estimated latents.Source: §3.1 · p. 4

3.2 DEPENDENCY SUPPORTS AND STRUCTURAL DIVERSITY

  • A dependency footprint is the set of observed variables affected by a latent; Structural Diversity requires these footprints to be pairwise distinct.Footprints may overlap or be nested, but identical footprints cannot be distinguished from support patterns alone.Source: §3.2 · p. 4, Figure 2 · p. 4
  • DSReg searches over rotations of a linearly identified representation, minimizing dependency support because the Jacobian transforms by the candidate rotation.The observation-dependency bridge does not require an encoder invertible on observations.Source: §3.2 · p. 5
  • Under Structural Diversity and functional no-cancellation, every support-sparsity minimizer recovers individual latents up to signed permutation.This extends reconstruction-free world models beyond linear identifiability without a decoder, likelihood, or labels.Source: §3.2 · p. 5
  • Functional no-cancellation rules out constant-coefficient cancellations that hide active Jacobian edges and holds for almost every parameter in specified smooth observation families.It plays a faithfulness-type role in the identifiability argument.Source: §3.2 · p. 5
  • Structural Diversity is strictly weaker than prior structural conditions; nested footprints satisfy it even when they violate Non-Inclusion and both Structural Sparsity forms.Identical footprints mark the boundary: support alone then cannot distinguish the corresponding latents.Source: §3.2 · p. 6

3.3 THE DSREG ESTIMATOR

  • The practical estimator fits local observation-dependency maps at data anchors and minimizes their averaged ℓ1 relaxation over rotations; it can be fit jointly or post hoc.The decoupled schedule reuses checkpoints, and the schedules recover equally well.Source: §3.3 · pp. 6–7
  • When threshold plus Jacobian noise stays below the active-dependency strength and pattern margin, every thresholded-criterion minimizer lies within δ of a signed permutation.Exact recovery is impossible under noise; the stated guarantee is approximate, and the ℓ1 relaxation can select a different rotation from the support criterion.Source: §3.3 · pp. 6–7

4 EXPERIMENTS

The experiments test whether DSReg recovers individual latents across synthetic regimes, sparse-use probes, learned visual encoders, and external renderers. Across these settings, recovery improves while dense-span performance remains matched.

4.1 THE FULL METHOD RECOVERS INDIVIDUAL LATENTS END TO END

  • DSReg reaches near-ceiling recovery across tested latent dimensions while LeJEPA remains mixed on the synthetic benchmark.The benchmark uses an MLP observation map with nested, distinct footprints, a regime excluded by prior conditions.Source: §4.1 · p. 8, Figure 3 · p. 8

4.2 EVERY REGIME LANDS WHERE THE THEORY PUTS IT

  • Recovery is near-perfect wherever Structural Diversity holds, including nested footprints, and identical footprints limit recovery to the pair’s span.The sweep tests diverse, nested, minimal-difference, and identical footprint regimes.Source: §4.2 · p. 8

4.3 SPARSE MODULES CAN ACT THROUGH RECOVERED LATENTS

  • DSReg improves every sparse-use probe and wins in nearly every run, while dense-span checks remain matched.The probes cover visual editing, sparse control, short-horizon prediction, and transition-surprise detection.Source: §4.3 · pp. 8–9

4.4 THE GAINS SURVIVE LEARNED ENCODERS AND EXTERNAL RENDERERS

  • On external renderers, DSReg raises Gaussian 3DShapes from mixed to near-ceiling recovery at matched dense R2 and beats swept β-VAE and β-TCVAE on quarter-orientation dSprites.On dSprites, partial satisfaction of the premise sets the recovery ceiling.Source: §4.4 · p. 9
  • On pixel-trained encoders, DSReg matches the supervised Procrustes oracle; with dim z = 8, the match holds within 0.001 as dim ˜z widens to 32.The figure reports fifteen seeds for the oracle comparison and five for the widening-estimate comparison.Source: Figure 5 · p. 9

5 DISCUSSION

DSReg recovers individual world latents under the LeJEPA premise and Structural Diversity, using dependency-Jacobian sparsity without reconstruction. The paper frames this as a route to variables that can be intervened on and inspected individually, while noting that its benchmarks do not yet include physical robots.

5 DISCUSSION

  • Under the LeJEPA premise and Structural Diversity, DSReg recovers each individual world latent through sparsity regularization on the dependency Jacobian.The paper states that the condition is strictly weaker than prior structural conditions and that the guarantee is stable to estimation error.Source: §5 · p. 9
  • Individual variables can be intervened on and inspected one at a time, while dense readouts remain unchanged across the reported benchmarks.The paper notes that identical footprints cannot be distinguished from support alone and may require extra signals.Source: §5 · p. 9
  • The benchmarks are simulated or rendered, and DSReg has not yet been tested on a physical robot.The authors identify robot behavior and planning as future work.Source: §5 · p. 9

AI DISCLOSURE

The disclosure states that generative AI tools assisted with polishing and plotting, and that the authors reviewed the assisted work and take responsibility for the final content.

AI DISCLOSURE

  • Generative AI tools assisted with polishing and plotting; the authors reviewed the AI-assisted work and take responsibility for the final content.The disclosure includes text, claims, or artifacts produced with AI assistance.Source: AI DISCLOSURE · p. 10

A.2 IDENTIFIABILITY UNDER STRUCTURAL DIVERSITY

The identifiability argument turns dependency footprints into a criterion for resolving the orthogonal ambiguity left by linear identification. Under Functional no-cancellation and Structural Diversity, every minimizing rotation yields signed-permutation recovery.

A.2 IDENTIFIABILITY UNDER STRUCTURAL DIVERSITY

  • Functional no-cancellation ensures that a mixed candidate latent has the union of its source dependency footprints.For analytic observation families, the condition holds for almost every parameter when each row has a parameter value with a nonzero functional determinant.Source: §A.2 · p. 16
  • A row-wise Gram matrix provides an empirical diagnostic: rank below the recovered row-support size flags a possible spurious cancellation.The diagnostic is rotation-invariant across anchors.Source: §A.2 · p. 16
  • The proof shows that signed permutations minimize the support criterion, while Structural Diversity excludes non-permutation minimizers.If two latents have identical footprints, a non-permutation rotation can attain the same minimum.Source: §A.2 · p. 19
  • Theorem 2 states that under Structural Diversity and Functional no-cancellation, every minimizer of the support-sparsity criterion yields a signed permutation of the true latents.Recovery is therefore up to sign and relabeling.Source: §A.2 · p. 19

A.3 COMPARISON OF STRUCTURAL CONDITIONS

Prior structural conditions on dependency footprints imply Structural Diversity, but none of those implications reverses. A two-latent example satisfies Structural Diversity and Functional no-cancellation while violating the stronger prior conditions.

A.3 COMPARISON OF STRUCTURAL CONDITIONS

  • Structural Sparsity implies Non-Inclusion, which implies Structural Variability, and each condition implies Structural Diversity.The implication chains hold for both forms of Structural Sparsity.Source: §A.3 · pp. 20–21
  • For d = 2, footprints S1 = {1} and S2 = {1, 2} are distinct, so Structural Diversity holds although Structural Variability fails.Their symmetric difference has size 1, below the required minimum of 2.Source: §A.3 · pp. 20–21
  • The example’s observation functions realize those footprints and satisfy Functional no-cancellation, so Theorem 2 applies without modification.The active derivatives in observation row 1 are linearly independent in L2(µ).Source: §A.3 · p. 21
  • Because both structural-sparsity forms imply Non-Inclusion, the example also violates both forms, establishing that Structural Diversity is strictly weaker.The implication chains are stated for the intersection and overlap-rank forms of Structural Sparsity.Source: §A.3 · pp. 20–21

A.4 ROBUSTNESS TO JACOBIAN PERTURBATION

The robustness result shows that thresholded recovery remains close to a signed permutation when Jacobian estimation error and the threshold are small relative to a support-pattern margin. Its proof combines a population count gap with quantitative activity bounds.

A.4 ROBUSTNESS TO JACOBIAN PERTURBATION

  • Whitening an invertible linear representation produces an orthogonal representation, reducing its ambiguity to the class analyzed by the recovery result.For h(z) = Az with z ∼ N(0, I_d), W = Cov(h)^−1/2 gives Wh(z) = Q̃z with Q̃ orthogonal.Source: §A.4 · p. 21
  • Every orthogonal matrix at least δ from signed permutations has a strictly positive pattern margin ρ*(δ), provided Kδ is nonempty.The count of active entries exceeds the minimum s* for any such matrix.Source: §A.4 · p. 22
  • On the two-dimensional witness, σ ≈ 0.374 and ρ*(0.4) ≈ 0.28, making the recovery condition satisfiable at nontrivial noise levels.The witness meets every assumption.Source: §A.4 · p. 22
  • If τ + ε < σρ*(δ), every minimizer of the thresholded criterion lies within δ of a signed permutation.The result assumes Theorem 2’s conditions, d ≥ 2, and a bounded-row Jacobian perturbation of size at most ε.Source: §A.4 · p. 23
  • The proof bounds the thresholded support count at signed permutations and shows that rotations outside the δ-neighborhood have a strictly larger count.Quantitative activity converts the positive margin into entries that remain above threshold despite perturbation.Source: §A.4 · p. 23

B ADDITIONAL DISCUSSIONS

Prior identifiable latent-variable methods rely on observation anchors or additional structure, while JEPA theory supplies only linear identifiability. DSReg uses dependency-footprint structure to resolve the remaining rotation, with a practical objective based on estimated local Jacobians.

B ADDITIONAL DISCUSSIONS

  • Nonlinear ICA and self-supervised methods obtain identifiability through auxiliary variables, latent conditionals, mixing restrictions, or positive-pair structure, rather than prediction in representation space alone.With Gaussian latents, the distribution carries no component-wise signal, so separation must come from outside both the conditional and the distribution.Source: B ADDITIONAL DISCUSSIONS · p. 24
  • Sparse-structure methods often anchor recovery through a decoder or likelihood, whereas DSReg uses Structural Diversity and faithfulness for signed-permutation recovery without either.The support-pattern requirement is strictly weaker than the cited prior structural conditions.Source: B ADDITIONAL DISCUSSIONS · p. 24
  • LeJEPA identifies a Gaussian latent state only up to an orthogonal transformation; DSReg resolves that rotation through dependency sparsity without reconstruction or likelihood.The unknown rotation can leave individual world latents mixed.Source: B ADDITIONAL DISCUSSIONS · p. 24
  • DSReg sparsifies the dependency Jacobian’s support, not the code, and permits dense codes and nearly full observation footprints when the true footprints remain pairwise distinct.Rectified LpJEPA instead changes the target law to produce sparse, non-negative codes.Source: B ADDITIONAL DISCUSSIONS · pp. 24–25
  • The practical objective replaces exact support counts with local-Jacobian estimates and an anchor-averaged ℓ1 relaxation; whether combining it with IMA recovers latents remains open.IMA imposes orthogonal Jacobian columns, a condition that neither implies nor is implied by DSReg’s distinct-support condition.Source: B ADDITIONAL DISCUSSIONS · p. 25

C ADDITIONAL EXPERIMENTS

The appendix lays out the experiment scope and shared protocol, separating whether representations contain the latent state, resolve individual variables, and support downstream tasks. Its recurring single-latent probe restricts readout to one estimated coordinate.

C ADDITIONAL EXPERIMENTS

  • The appendix covers scaling to d = 8192, regime and criterion stress tests, sparse-use probes, learned visual encoders, external renderers, and latent-only rotation baselines.These experiments progressively remove elements of the authors’ own design while testing whether the gain persists.Source: C ADDITIONAL EXPERIMENTS · p. 25, §C.1 · p. 26

C.1 METRICS AND PROTOCOL

  • Experiments separately assess latent-state presence, individual-latent recovery, and downstream use, with dense readout, latent mean correlation, and task-specific probes answering those questions.The latent mean correlation score reaches one precisely when individual world latents are recovered in the paper’s defined sense.Source: §C.1 · pp. 25–26
  • The few-shot single-latent probe selects one estimated coordinate per ground-truth factor and fits only a scalar affine map, so it penalizes mixtures that dense readout can tolerate.Ground-truth latents define evaluation, but the dependency rotation is fit from estimated latents and observations rather than labels.Source: §C.1 · p. 26

C.2 SCALING THE DEPENDENCY CRITERION

The scaling study factorizes local Jacobians and uses unbiased row sampling to reduce memory and per-step cost while preserving the DSReg objective. On one GPU, the estimated procedure reaches d = 8192, with neighborhood size k > d required for recovery.

C.2 SCALING THE DEPENDENCY CRITERION

  • The exact factorization avoids materializing p × d Jacobians, reducing storage from O(mpd) to O(mkd) and saving nearly two orders of magnitude at the largest observed dimension tested.Factored evaluation computes the same objective exactly by streaming neighborhood and observed-coordinate blocks; row sampling instead estimates it from sampled anchor-coordinate pairs.Source: §C.2 · p. 27
  • Increasing sample count leaves recovery and sampled step time flat within seed noise, while doubling observed dimension holds step time flat and improves MCC from 0.806 to 0.884.The reported sweeps use separate baselines rather than the dimension-scaling table rows.Source: Table 1 · p. 29, §C.2 · p. 29
  • Recovery reaches d = 8192 on one GPU, while sampled step cost stays nearly flat as dimension grows and materialized cost rises sharply.At the largest dimensions, row sampling is the only mode that fits on the card; Figure 6 summarizes recovery and cost across scales.Source: §C.2 · p. 29, Figure 6 · p. 29
  • Recovery’s dimension trend is consistent across the tested footprint families, with nested and random-overlap constructions tracking the banded family without family-specific tuning.The practical frontier is d = 8192 on a 48 GB GPU; d = 16384 requires an 80 GB device and shifts the binding constraint to rotation-search compute.Source: §C.2 · p. 30
  • Neighborhoods must satisfy k > d: k < d makes the regression rank-deficient, while k = d amplifies noise and fails at every tested scale.Recovery returns from k = 1.25d at every tested scale; the recommended default is k = 1.5d.Source: §C.2 · pp. 29–30
  • Orthogonally invariant sketches cannot guide rotation optimization, and dense random projections destroy the local footprints; locality-preserving row subsampling retains them.The estimated-procedure testbed uses known footprints and an orthogonally mixed representation, scoring recovery by MCC across scales.Source: §C.2 · pp. 29–30

C.3 VERIFICATION AND ROBUSTNESS

The verification studies test DSReg across synthetic footprint regimes, perturbations, sparse-use probes, and pixel-trained encoders. Recovery follows Structural Diversity, while gains in individual-latent recovery and sparse use persist in learned visual representations.

C.3 VERIFICATION AND ROBUSTNESS

  • Across dimensions, recovery stays at ceiling for diverse, nested, and minimal-difference footprints, while identical footprints recover only their shared-subspace level.The sweep uses analytic orbits with nonlinear observations and exact Jacobians; the pair span is recovered at CCA ≥ 0.998.Source: §C.3 · p. 30, Figure 7 · p. 31
  • Recovery is insensitive to sparsity surrogate and anchor budget except that more anchors help minimal-difference footprints, and post-training fitting matches joint training after the final decoupled fit.A thresholded hard support count recovers at least as well as the ℓ1 default at roughly twice the cost, and refinement closes remaining gaps within seconds.Source: §C.3 · pp. 30–31, Table 2 · p. 31
  • At identical dense recovery (R2 = 0.988), the rotation improves individual-latent recovery, eight-label few-shot readout, and sparse editing on the MLP benchmark.The few-shot readout moves from below the held-out mean to near the dense-readout ceiling.Source: §C.3 · p. 32, Figure 10 · p. 32
  • Recovery stays at ceiling under tested Jacobian corruption, every cell satisfying Theorem 3’s stability condition succeeds, and success extends beyond the guaranteed region.The guarantee is sufficient rather than tight; the identical-footprint control remains at its shared-subspace level under corruption.Source: §C.3 · pp. 31–32, Figure 11 · p. 32, Figure 9 · p. 32
  • Recovery returns from the excluded all-linear observation stratum at γ∗ ≈ 0.1 across tested dimensions and both diverse and nested regimes.At γ = 0, where Assumption 2 fails, the criterion returns a mixed rotation; exact and estimated Jacobians closely agree.Source: §C.3 · p. 33, Figure 12 · p. 33

C.4 SPARSE-USE PROBES: ENVIRONMENTS AND PROCEDURES

  • The sparse probes test whether one-coordinate edits isolate factors and whether capped sparse models support control, rollout, and violation monitoring.The evaluation uses name-free procedures and shared data, model class, planner, thresholds, and budgets; the fitted rotation is the only difference between representations.Source: §C.4 · pp. 33–34
  • On the sparse rollout and monitoring probes, the mixed representation’s model is misspecified, while a sparse model’s large clean error can hide the same physical violation.The rollout test iterates a one-input-per-output model open loop for five steps; monitoring scores separation of clean and corrupted transitions by AUROC.Source: §C.4 · p. 34

C.5 LEARNED VISUAL ENCODERS

  • On held-out pixel-trained encoders, dense recovery remains unchanged while latent MCC, few-shot readout, and sparse-use scores improve; all three pixel probe families favor DSReg in every run.Figure 13 reports control error falling from 0.63 to 0.52, five-step sparse rollout R2 rising from 0.74 to 0.81, and transition-surprise AUROC rising from 0.995 to 0.999.Source: §C.5 · p. 34, Figure 13 · p. 35, Table 3 · p. 35

C.6 GAUSSIAN 3DSHAPES

On Gaussian 3DShapes, DSReg recovers factors from an external renderer, including factors that alter few pixels, without a decoder. On quarter-orientation dSprites, it improves over LeJEPA and both VAE baselines despite a partially attained linear premise.

C.6 GAUSSIAN 3DSHAPES

  • DSReg recovers scale, shape, and orientation, where VAE baselines collapse, while hue factors that affect larger pixel regions are captured to some degree by every method.The dependency criterion asks which features a factor touches, unlike reconstruction-driven objectives that weight factors by the pixels they explain.Source: §C.6 · p. 36
  • On Gaussian 3DShapes, DSReg improves latent MCC on every run while matching LeJEPA’s dense recovery.The benchmark uses the external DeepMind renderer; the VAE references’ low dense R2 reflects axis alignment rather than lost factor information.Source: Table 4 · p. 36, §C.6 · p. 36
  • Factor sweeps and matched correlation matrices show selective responses in DSReg coordinates and smeared responses in the unrotated LeJEPA coordinates, without a decoder.Each rendered-image sweep varies one factor while holding the others fixed; factor labels are used only for evaluation.Source: §C.6 · p. 36, Figure 16 · p. 37, Figure 17 · p. 37

C.7 QUARTER-ORIENTATION DSPRITES

  • The dSprites result tests recovery under a partially attained linear premise, with dense R2 as the rotation-invariant ceiling on recoverable factors.Without the quarter-turn restriction, the linear-identifiability premise fails for every method.Source: §C.7 · p. 37
  • On quarter-orientation dSprites, DSReg improves over LeJEPA and exceeds both VAE baselines, with one-sided Mann–Whitney p = 8 × 10^-5 and p = 4 × 10^-5.The twenty-run test restricts orientation to a quarter turn because full-turn orientation is not a function of the symmetric sprite image.Source: §C.7 · pp. 37–38, Figure 18 · p. 38

C.8 ROTATION BASELINES AND DENSE-USE SANITY CHECKS

Latent-only rotations remain at the unrotated level, whereas DSReg approaches the oracle; on learned representations, DSReg nearly matches the supervised Procrustes oracle. Dense readouts and retrieval stay unchanged while DSReg improves sparse access.

C.8 ROTATION BASELINES AND DENSE-USE SANITY CHECKS

  • On exact analytic orbits, PCA, Varimax, and FastICA stay at the unrotated level, while DSReg approaches the oracle and reaches its dependency-objective level.The residual gap reflects the ℓ1 relaxation, whose global minimizer can be a slightly mixed rotation even though the support criterion identifies the permutation.Source: §C.8 · p. 38
  • On learned, nonisotropic representations, the rotation baselines leave latents mixed while DSReg nearly matches the supervised Procrustes oracle.The observation-side dependency signal, unlike asymmetry left by learning, points toward the world’s variables on both learned visual suites.Source: §C.8 · p. 38
  • Table 6 reports dense R2 of 1/1 for LeJEPA and DSReg on every task, while sparse-control error drops when only a few estimated latents may be used.Dense readouts and retrieval are unchanged to two decimals on every task.Source: §C.8 · p. 38, Table 6 · p. 39
  • With both methods using the same estimated anchor Jacobians, orthogonal parameterization, and restarts, DSReg stays ahead of sparse ICA on both renderer benchmarks.The comparison attributes the advantage to the criterion rather than access to the dependency signal alone.Source: §C.8 · pp. 38–39

Related papers

Loading 2610.09457v1…