PaperlayerView exact revision on arXiv

Source-linked AI summary

Representation Learning via Composite Subspace Flow Matching

José Afonso, Mário Figueiredo

arXiv:2610.32256v1cs.CVcs.AI

TL;DR

Self-supervised visual representation learning must extract useful features from raw pixels whose nuisance variation can dominate semantic targets, while reconstruction and joint-embedding methods face different trade-offs. CSFM uses deterministic degradation across visual subspaces with a capacity-bounded encoder and style truncation. Across the reported datasets and architectures, it outperforms unguided flows and autoencoders in probing while retaining generative invertibility.

  • Problem

    Raw visual data have high pixel-space nuisance variance, while reconstruction-based and joint-embedding approaches trade off modeling pixel detail against semantic abstraction.

    Source: §1 · p. 1

  • Method

    CSFM pairs ordered deterministic projections across frequency, color, and illumination with a decoupled isotropic noise schedule and a capacity-bounded latent code.

    Source: §1 · p. 2

  • Results

    Across CIFAR-10, CIFAR-100, and STL-10, CSFM outperforms unguided flows and autoencoders in linear and few-shot probing while preserving an invertible, connected latent space.

    Source: §1 · p. 2, §6 · pp. 12–13

  • Takeaways & Limitations

    The reported evaluations support CSFM as a representation approach that combines discriminative transfer with generative reversibility across convolutional and transformer architectures.

    Source: §1 · p. 2, §6 · p. 13

    Adapting cumulative multi-operator degradation to representation learning is challenging because sequential noise injections require costly covariance inversion and introduce an irreducible target noise floor.

    Source: §1 · p. 2

Abstract

from arXiv · show

Distilling semantic meaning from raw image data remains a challenge in machine learning. Current self-supervised paradigms face a trade-off: joint-embedding architectures produce relevant features by aggressively discarding intra-class variance, irreversibly spatial and reconstruction. Conversely, generative models preserve spatial details but struggle to isolate semantics, as their optimization is dominated by pixel variations. To bridge this divide, we propose composite subspace flow matching (CSFM). Rather than corrupting data with unstructured isotropic noise, CSFM formulates representation learning as the variance-minimizing inversion of a deterministic degradation curriculum across spectral, color, and illumination subspaces. By bounding trajectory variance with hard temporal style truncation, a capacity-constrained encoder is driven to ignore transient style artifacts and encode persistent content. Empirical evaluations across CIFAR-10, CIFAR-100, and STL-10 show that CSFM yields competitive discriminative transferability while preserving generative synthesis across both convolutional and vision transformer architectures. By avoiding the collapse pathologies of non-generative methods, CSFM forms a smooth latent space that outperforms standard optimal transport flows on linear probing and few-shot classification, while enabling coherent synthesis and trajectory interpolation. These results confirm that steering the physical dynamics of generative flows offers a scalable paradigm for learning representations that are simultaneously linearly separable and generatively invertible.

1 Introduction

Self-supervised visual learning must extract semantic features from pixels dominated by nuisance variation, while balancing generative detail against discriminative abstraction. CSFM addresses this tension with deterministic degradation across visual subspaces and reports competitive probing alongside generative reversibility.

1 Introduction

  • Pixel-space nuisance variation and the conflicting trade-offs of reconstruction-based and joint-embedding methods motivate a representation that preserves semantics and spatial information.Pixel variance can be dominated by texture, illumination, and clutter; reconstruction models can focus on raw variance, while joint embeddings risk collapse and irreversible information loss.Source: §1 · pp. 1–2
  • CSFM replaces uniform corruption with ordered deterministic projections across frequency, color, and illumination, paired with a decoupled isotropic noise schedule.A capacity-bounded latent code captures content while the generative vector field handles transient spatial details.Source: §1 · p. 2
  • Across CIFAR-10, CIFAR-100, and STL-10, CSFM outperforms unguided flows and autoencoders in linear and few-shot probing while preserving an invertible, connected latent space.The reported evaluations use CNN and vision-transformer backbones.Source: §1 · p. 2

2 Background and Related Work

The background contrasts reconstruction and joint-embedding approaches, then motivates continuous flows as a route to representations that retain generative reversibility. It frames deterministic latent conditioning as a variance bottleneck and defines content by persistence under physical transport.

2.1 Paradigms of Representation Learning: JEAs vs. Generative Models

  • Reconstruction-based methods preserve spatial detail but can prioritize raw pixel variance, whereas joint-embedding methods discard spatial structure and can become irreversible.SIGReg is described as a linear-complexity projection-based alternative to quadratic second-moment decorrelation for regularizing latent representations.Source: §2.1 · p. 3

2.2 Continuous Normalizing Flows and Conditional Flow Matching

  • Continuous normalizing flows transport distributions along continuous paths, while conditional flow matching regresses a vector field against an analytical conditional target to avoid dataset-wide marginal integration.Trajectory intersections create irreducible target variance; conditioning on a latent representation can disambiguate crossing paths.Source: §2.2 · p. 3

2.3 Flow-Based Representation Learning and Deterministic Conditioning

  • A deterministic encoder output conditions the flow and, under a capacity bottleneck, learns sufficient statistics that reduce trajectory uncertainty; content is framed as temporal persistence under physical transport.This contrasts with joint-embedding architectures, which define content through invariance to hand-crafted augmentations.Source: §2.3 · p. 4

3 Methodology: Composite Subspace Flow Matching

CSFM defines orthogonal visual subspaces and a decoupled flow objective, then uses a temporally truncated style schedule to direct representation learning toward content. The method also specifies asymmetric gradient routing and forward-mode computation for implementation.

3.1 Composite Orthogonal Subspaces

  • The content and style subspaces are separated through orthogonal projections, with frequency, color, and illumination domains used to construct the decomposition.Commuting operators share a joint basis that enables evaluation in O(D log D) time.Source: §3.1 · pp. 4–5

3.2 Decoupled Flow Matching and Variance Nullification

  • A single isotropic noise path paired with composite deterministic drift makes the target velocity deterministic given the clean sample and state, eliminating the irreducible target noise floor.The decoupled path also lets content and style trajectories evolve independently without noise cross-talk.Source: §3.2 · p. 6

3.3 Generative Curriculum and Style Truncation

  • Style penalties vanish for t > t1 while content precision remains active across the trajectory, so a finite cutoff suppresses style encoding and permits semantic variance reduction to diverge.The schedule uses a decay rate κ and a style truncation boundary t1.Source: §3.3 · p. 6

3.4 Asymmetric Gradient Routing and Joint Objective

  • Near t = 0, encoder gradients vanish, so asymmetric routing truncates encoder backpropagation there while the vector-field regressor trains across the full interval.The stated example cutoff is t0 = 0.05; the joint objective also includes SIGReg weighted by λreg.Source: §3.4 · p. 6

3.5 Simulation-Free Forward-Mode Vector Field Computation

  • Forward-mode evaluation computes the time derivative without caching reverse-mode computation graphs, and the commuting operators permit joint diagonalization and coordinate-wise computation in the transform domain.The cascade and its derivative are accumulated jointly in a single forward pass for the proposed commuting operators.Source: §3.5 · p. 7

4 Theoretical Analysis: Optimal Representation Geometry

The analysis derives an optimal encoder geometry under a Gaussian surrogate and finite-capacity constraints. It shows that cross-subspace coupling is suboptimal and that hard style truncation directs capacity toward content.

4.1 Gaussian Surrogate Formulation and Exact Posterior

  • Under a Gaussian surrogate, the encoder’s conditional distribution is approximated by a regularized Gaussian, and its flow-matching risk depends on conditional first and second moments.The posterior covariance is given in closed form under the stated linear observation model.Source: §4.1 · pp. 7–8

4.2 Optimal Subspace Disentanglement

  • The surrogate-loss optimum requires zero cross-subspace coupling in the encoder precision matrix, yielding exact disentanglement of content and style coordinates.The analysis characterizes the optimum in the orthonormal basis spanning the two subspaces.Source: §4.2 · p. 8

4.3 Subspace Capacity Allocation via Truncation

  • Under the stated capacity constraints, the optimal encoder saturates its bottleneck with exactly c eigenvalues at the maximum bound and the remaining D − c at zero.The objective decreases strictly as each eigenvalue increases within its allowed range.Source: §4.3 · p. 8
  • Without style truncation, style variance reduction exceeds content reduction, so the optimal encoder assigns all active bottleneck dimensions to style.This result applies to the untruncated case t1 = 1 under the decay schedule in Theorem 7.Source: §4.3 · pp. 8–9
  • With a hard style cutoff t1 < 1, style variance reduction is bounded while content reduction diverges asymptotically, directing all active dimensions to content.Theorem 7 states this allocation for the specified content and style decay schedules.Source: §4.3 · pp. 8–9

5 Experiments

Across CIFAR-10, CIFAR-100, ViT/DiT, and STL-10, CSFM improves representation results over standard OT-Flow while retaining connected latent structure. The experiments test representation quality, model capacity, temporal boundaries, and scalability under specified benchmark settings.

5.1 Experimental Setup

  • The evaluation covers CIFAR-10, CIFAR-100, and STL-10 using frozen-feature linear probes, k-NN, few-shot classification, and silhouette scores.STL-10 uses 96 × 96 images; the supplied setup also describes convolutional and ViT/DiT backbones and single-GPU training constraints.Source: §5.1 · p. 9, §5.2 · p. 9

5.2 Representation Quality on CIFAR-10

  • CSFM exceeds pure autoencoding (58.07%), unguided OT-Flow (67.11%), and CDAE (57.27%) in the reported CIFAR-10 comparison.The passage attributes the contrast with CDAE, which shares the same marginal degradation states, to continuous velocity-field regression rather than pixel reconstruction.Source: §5.2 · p. 10, Table 1 · p. 10
  • 75.29% and 71.94% linear accuracy with 100% and 10% labels, respectively, exceed LeJEPA’s 72.49% and 71.02% under 40K training iterations.At 1% labels and on neighborhood metrics, LeJEPA performs better; CSFM retains a connected manifold with distinct semantic clusters.Source: §5.2 · p. 10, Figure 1 · p. 10
  • With a fixed 5.98M-parameter encoder, scaling the vector-field regressor to 59.10M parameters yields 78.51% linear probing and 63.45% 1% few-shot accuracy.The comparison uses an extended 120K-step horizon.Source: §5.2 · p. 11, Table 2 · p. 11
  • On the 20.81M baseline tier, severing gradients at t0 = 0.05 raises linear-probe accuracy from 69.09% to 69.52%; style cutoff t1 = 0.5 yields 75.12% linear and 57.40% 1% few-shot accuracy.Without truncation (t1 = 1.0), tuning κ and β yields at most 69.60% linear accuracy.Source: §5.2 · p. 11

5.3 Scalability Analysis: Cardinality, Architecture, and Resolution

  • On CIFAR-100, CSFM achieves 51.58% linear accuracy and 17.65% with 1% labels (5 samples per class), versus OT-Flow’s 42.27% and 10.66%, gains of 9.31 and 6.99 points.Source: §5.3 · p. 12
  • For ViT/DiT, CSFM improves from 53.30% at 40K steps to 61.49% at 80K steps, while OT-Flow rises from 51.84% to 53.27%.At 1% few-shot labels, CSFM reaches 49.69%, 8.63 points above OT-Flow.Source: §5.3 · p. 12, Table 4 · p. 12
  • On 96 × 96 STL-10, CSFM exceeds OT-Flow by 1.84 points in linear accuracy, 2.20 in 1% few-shot accuracy, and 2.29 in k-NN accuracy.The experiment uses a lighter decoder due to GPU-memory constraints and pre-trains on 105k uncurated images.Source: §5.3 · p. 12, Table 4 · p. 12

6 Conclusion

The conclusion presents CSFM as a continuous-flow representation method that combines deterministic subspace drift with a decoupled noise path. The appendix material describes commuting visual operators, variance decomposition, and the resulting computational structure.

6 Conclusion

  • CSFM repurposes continuous normalizing flows for representation learning and combines deterministic drift across visual subspaces with a decoupled noise path.The conclusion reports competitive linear-probe performance across CIFAR-10, CIFAR-100, and STL-10 while retaining generative invertibility across convolutional and transformer architectures.Source: §6 · pp. 12–13

A.1 Simultaneous Diagonalization of Physical Subspace Operators

  • Color, spectral-frequency, and illumination operators commute and share a basis formed from the Ohta color transform and 2D orthonormal DCT-II.The passage states that linear compositions of these operators evaluate in O(D log D) time.Source: §A.1 · pp. 16–17

A.2 Derivation of the Tripartite Variance Decomposition

  • The variance derivation decomposes velocity-prediction error around the conditional-mean optimal predictor, with the cross-term vanishing under conditioning.It then relates conditional velocity variance to the finer conditioning on the original image.Source: §A.2 · p. 17

A.3 Derivation of the Cumulative Variance Floor

The appendix derives conditions for residual target-velocity variance, decoupled subspace penalties, and CSFM’s training and inference procedures. It also supplies proof steps for posterior covariance and exact subspace disentanglement.

A.3 Derivation of the Cumulative Variance Floor

  • For the cumulative corruption model, residual target-velocity variance is nonzero whenever the derivative of the noise map has components in the null space of the noise map.The setup represents the observation as xt = Mtx0 + Btϵ with Gaussian ϵ.Source: §A.3 · p. 18

A.4 Derivation of Decoupled Penalty Weights

  • Orthogonal content and style projectors make the squared drift operator block diagonal, and the truncated style schedule assigns zero style penalty after t1.The schedule has a jump at t1, which has zero probability under the continuous time-sampling density.Source: §A.4 · p. 18

B Algorithmic Workflows

  • CSFM training generates target states and velocities, routes encoder gradients asymmetrically by a t0 mask, regresses the vector field, and regularizes latent codes with SIGReg.The total objective combines flow loss and weighted latent regularization.Source: B Algorithmic Workflows · p. 19

C Complete Proofs for Section 4

  • The analytical target-flow routine computes the composite state and its derivative, then forms xt = Mt(x0) + t·x1 and ut = Ṁt(x0) + x1.It supports either a joint spectral-basis computation or sequential operator application.Source: C Complete Proofs for Section 4 · p. 19

C.1 Proof of Lemma 4 (Closed-Form Posterior Covariance)

  • The posterior-covariance proof combines observation-likelihood and prior precision, then substitutes the content/style drift and uses orthogonal projector identities.The inference routine initializes from noise and integrates the vector field backward, optionally conditioning on a reference image.Source: §C.1 · pp. 19–20

C.2 Proof of Theorem 5 (Exact Subspace Disentanglement)

  • The exact-disentanglement proof concludes that the global optimizer has zero cross-coupling between content and style blocks.The argument uses positive-definite block structure and equality conditions for the trace bounds.Source: §C.2 · p. 20

C.3 Proof of Lemma 6 (Capacity Saturation)

The capacity-saturation proof establishes that the surrogate objective decreases with each encoder eigenvalue, so the optimum uses the available bottleneck capacity at its upper bound.

C.3 Proof of Lemma 6 (Capacity Saturation)

  • At the optimum, exactly c eigenvalues reach ωmax and the remaining D − c eigenvalues are zero.This follows from strict monotonic decrease of the objective over the permitted eigenvalue range under the stated capacity constraint.Source: §C.3 · p. 21

C.4 Proof of Theorem 7 (Subspace Capacity Allocation via Truncation)

Without style truncation, variance reduction favors style and the encoder allocates its bottleneck to style; truncating the style trajectory instead makes content the optimal allocation.

C.4 Proof of Theorem 7 (Subspace Capacity Allocation via Truncation)

  • With an untruncated style schedule, style variance reduction exceeds content reduction, so the optimal encoder allocates its bottleneck exclusively to style.The proof compares the style and content integrands pointwise for t ∈ (0, 1).Source: §C.4 · p. 22
  • When t1 < 1, style variance reduction remains bounded, while content reduction diverges as ϵ0 → 0+, making content the optimal bottleneck allocation.The content reduction scales as O(1/√ϵ0), or O(1/δclamp) under boundary clamping.Source: §C.4 · p. 22

C.5 Hyperparameter Details and Training Configurations

Table 5 specifies the default CSFM degradation curriculum and its optimization hyperparameters.

C.5 Hyperparameter Details and Training Configurations

  • Table 5 details the default curriculum schedules, operator-specific exponential decay rates, and optimization parameters used in the experiments.The supplied text identifies the table's scope but does not give individual settings.Source: §C.5 · p. 22, Table 5 · p. 22

D Baseline Objectives and Discriminative Setup

The baseline setup compares task-specific objectives and specifies LeJEPA's projector and asymmetric views. Its SIGReg ablation selects λreg = 3 × 10−2; the extended CIFAR-10 evaluations cover synthesis, trajectory curvature, parameter sweeps, and manifold continuity.

D Baseline Objectives and Discriminative Setup

  • The baselines balance task-specific losses with a structural regularizer, and Table 6 defines the task objectives across four paradigms.AE reconstruction is forced through the latent bottleneck; CDAE predicts x0 from intermediate noisy states, and OT-Flow uses a linear path with constant target velocity.Source: D Baseline Objectives and… · pp. 22–23, Table 6 · p. 23
  • LeJEPA uses a 3-layer MLP projector that maps 512 backbone features through a 2048-dimensional hidden layer to a 256-dimensional representation, with batch normalization and ReLU in its first two layers.Its asymmetric views share common augmentations but use different blur probabilities: 0.999 for view 1 and 0.1 for view 2.Source: D Baseline Objectives and… · p. 23
  • LeJEPA's SIGReg optimum is λreg = 3 × 10−2; values ≤1 × 10−3 yield 18.62% linear accuracy, while values >3 × 10−2 favor feature spreading over semantic view clustering.The selected value is used as the default across LeJEPA benchmarks.Source: D Baseline Objectives and… · p. 23, Table 7 · p. 23

E Extended CIFAR-10 Empirical Results and Generative Validation

  • The extended CIFAR-10 evaluation covers synthesis metrics, ODE trajectory curvature, physical-decay and regularization ablations, and manifold continuity.The supplied passage describes the evaluation scope without reporting its individual outcomes.Source: E Extended CIFAR-10 Empirical Results… · p. 23

E.1 Latent Space Structural Geometry: SVD Spectrum and Feature Covariance

Across the extended CIFAR-10 evaluations, truncated CSFM distributes variance across more latent dimensions than the cited baselines, and the analyses report regularization, degradation-schedule, synthesis, and interpolation outcomes.

E.1 Latent Space Structural Geometry: SVD Spectrum and Feature Covariance

  • d90% is 37 for truncated CSFM, versus 31 for OT-Flow and 9 for LeJEPA, indicating broader variance distribution while retaining capacity for reverse-ODE synthesis.d90% is the minimum number of principal components capturing 90% of embedding variance.Source: Figure 2 · p. 24, §E.1 · p. 24
  • CSFM (t1 = 0.5) has d90% = 37 and mild structured feature correlations, while OT-Flow has d90% = 31 and near-zero off-diagonal correlations and LeJEPA has d90% = 9 with dense block structure.The section also reports that truncated CSFM preserves global manifold connectivity, whereas LeJEPA clusters are separated by low-density regions and OT-Flow has diffuse, overlapping boundaries.Source: §E.1 · pp. 24–25, Figure 3 · p. 24

E.2 Sensitivity to Latent Regularization (λreg)

  • At λreg = 10−3, CSFM reaches 69.09% linear accuracy and 46.41% 1% few-shot accuracy; omitting regularization still yields 68.98% linear accuracy.Regularization at λreg ≥0.1 degrades downstream separability.Source: Table 8 · p. 25, §E.2 · p. 25

E.3 Physical Degradation Parameter Sensitivity

  • With t1 = 0.5, linear probing reaches 76.10% at κ = 3.25 and β = 0.7, and performance remains robust from κ = 10−4 to 6.0; without truncation, it stays below 69.61%.Performance decreases at extreme decay stiffness, κ ≥8.75.Source: §E.3 · p. 25, Table 9 · p. 26

E.4 Generative Synthesis Fidelity and ODE Trajectory Curvature

  • Truncated CSFM (t1 = 0.5) yields conditional FID1k 87.88, unconditional FID1k 95.76, and integration error 0.04393, nearly halving the un-truncated model's 0.08340 error at N = 10.OT-Flow has unconditional FID1k 93.57 and conditional FID1k 119.04; un-truncated CSFM has conditional FID1k 48.84 and unconditional FID1k 125.97.Source: §E.4 · p. 25, Table 10 · p. 26

E.5 Latent Manifold Smoothness, Interpolation, and Extrapolation

  • Truncated CSFM (t1 = 0.5) produces continuous semantic transitions during interpolation and maintains coherent global structures during extrapolation.The comparison evaluates spherical interpolation and extrapolation across test-set latent anchors at 120K steps.Source: Figure 4 · p. 27, §E.5 · p. 27

Related papers

Loading 2610.32256v1…