PaperlayerView exact revision on arXiv

Source-linked AI summary

Conditional Flow Models Convert Epistemic Uncertainty into Apparent Aleatoric Variance

Navodita Sharma, Abhishek Shetty, Max Simchowitz, Pulkit Agrawal

arXiv:2610.29172v1cs.LGcs.AI

TL;DR

Conditional flow models are used even for point-prediction problems, raising the question of what their conditional distributions add beyond a point estimate. The paper studies how underfitting and smoothing can inflate their sample variance, and reports that this spread can support approximately calibrated predictive sets in the underfitting regime.

  • Problem

    Conditional flow models are increasingly used for point-prediction problems, raising the question of what information their conditional distributions provide beyond a point prediction.

    Source: §1 · p. 1

  • Method

    The paper studies smoothing bias in conditional vector-field learning through experiments and an idealized regularized-smoothing model.

    Source: §1 · p. 2, §5 · pp. 6–7

  • Results

    Underfitting can make CFM samples express unresolved conditioning dependence as excess variance, and predictive sets from those samples show approximate coverage before overfitting.

    Source: §1 · p. 2, §7 · p. 10

  • Takeaways & Limitations

    Before overfitting, predictive sets constructed directly from CFM samples can achieve approximately nominal coverage without explicit calibration training.

    Source: §1 · p. 2, §7 · p. 10

    The theory covers an idealized population-level regularized-smoothing model, not arbitrary neural-network training, and the studied coverage properties are empirical rather than formal guarantees.

    Source: §7 · p. 10

Abstract

from arXiv · show

Conditional Flow Models (CFMs) are increasingly being used in place of deterministic predictors across machine learning, including for tasks that were classically formulated as regression. Given their success in such tasks, we ask the question: what useful information do the conditional distributions produced by CFMs carry beyond what a point predictor would provide? We identify a systematic phenomenon: in many natural regimes, CFM samples routinely have larger variance than the true conditional distribution, even in nearly deterministic regression problems. This effect is most pronounced where the model has not resolved the dependence on the conditioning variable, and thus the additional variance can be viewed as converting epistemic uncertainty in fitting the ground truth conditional distribution into apparent aleatoric variance in the generated samples. Through controlled synthetic and real-data experiments, and comparisons with explicit smoothing estimators, we show that the induced variance tracks model underfitting of complex or hard-to-fit targets, and decreases with additional training or data. We provide a theoretical account of the phenomenon in a simplified setting and demonstrate a consequent practical use-case of conditional flow-matching: designing high-coverage predictive sets that cover the ground-truth regression target.

1 INTRODUCTION

Conditional flow models are used for point-prediction problems, but their conditional distributions can express uncertainty beyond a point prediction. The paper finds that underfitting can turn unresolved dependence on the conditioning variable into excess sample variance, which can support predictive sets with approximate coverage.

1 INTRODUCTION

  • The paper studies conditional generative models because they are increasingly used in settings where a single point prediction would suffice, including robotics and scientific estimation.The listed examples also include policy-value estimation in reinforcement learning.Source: §1 · p. 1
  • Smoothing bias in the learned vector field is the proposed mechanism: neural networks can oversmooth high-frequency structure and underfit the ground-truth dependence on the conditioning variable.The paper considers smoothing from early stopping and explicit regularization.Source: §1 · p. 2
  • Underfitting in a conditional flow model can make generated samples have greater conditional variance than the true distribution, even in nearly deterministic regression problems.The paper identifies this as converting uncertainty about the conditioning dependence into apparent aleatoric variance.Source: §1 · p. 2
  • The resulting sample spread can be used to form confidence sets for potentially deterministic target functions without conformal correction, provided the model does not overfit.The paper presents this as an uncertainty estimate alongside the point prediction.Source: §1 · p. 2
  • CFM-sample predictive sets are approximately calibrated across policy evaluation, cell perturbation prediction, and robot policy learning in the underfitting regime.The paper reports that this coverage holds before overfitting and that these sets can be competitive with or tighter than conformal-prediction baselines.Source: §1 · pp. 2–3

2 RELATED WORK

Prior work studies flow and diffusion models, their applications, and biases in learning. This paper focuses on how underfitting with respect to conditioning changes the conditional law, and on the distribution produced by integrating a smoothed conditional vector field.

2 RELATED WORK

  • Conditional flow matching learns a continuous-time transport indexed by side information, and CFMs have been applied in robotics and scientific modeling.Flow matching and related methods learn a time-dependent vector field along a path between base noise and data (Lipman et al., 2023; Liu et al., 2022; Albergo et al., 2025).Source: §2 · p. 3
  • Recent studies attribute diffusion or score-model generalization to several biases, but focus on unconditional generation rather than how conditioning-variable underfitting changes the conditional law.The passage cites variance in denoising targets (Vastola, 2025), architectural bias (Kamb & Ganguli, 2025), and selective score underfitting (Song et al., 2025).Source: §2 · p. 3
  • This work builds on smoothing and equivalent-kernel perspectives but studies the distribution obtained by integrating a smoothed conditional vector field, rather than point-estimator bias.The related work connects spectral bias (Rahaman et al., 2019; Xu et al., 2020), kernel gradient flow (Jacot et al., 2018), and classical regularized estimators.Source: §2 · p. 3

3 PRELIMINARIES

The preliminaries define conditional generation through a learned law of outputs given conditioning values, represented by a flow transporting an initial distribution. Flow matching trains the velocity field used for that transport.

3 PRELIMINARIES

  • Conditional generative modeling learns to generate outputs from the conditional distribution of X given a conditioning value C=c.The paper denotes the target conditional law by p⋆(· | c).Source: §3 · p. 3
  • A conditional continuous normalizing flow uses a conditional vector field and an initial distribution to define an evolving random process for each conditioning value.The process starts from the initial conditional distribution and evolves according to the vector field.Source: §3 · p. 3
  • The flow-map sends an initial state to its state at time t, and the corresponding conditional distribution marginalizes over initial states.The paper suppresses dependence on the initial distribution when it is clear from context.Source: §3 · p. 3
  • Flow matching constructs an interpolant between base noise and data, then trains a time-dependent velocity field by approximately minimizing an empirical flow-matching loss.The interpolant is Xt = αtX0 + βtX1, with endpoints specified by the coefficients.Source: §3 · pp. 3–4

4 VARIANCE INFLATION IN UNDERFIT CONDITIONAL FLOWS

Underfitting in conditional flow models inflates conditional variance, even for nearly deterministic targets; the effect tracks smoothing bias and also appears in high-dimensional policy evaluation. As training progresses or smoothing decreases, the learned distributions contract toward the target.

4 VARIANCE INFLATION IN UNDERFIT CONDITIONAL FLOWS

  • Underfitting produces excess conditional spread even for nearly deterministic targets, as the flow averages over local high-frequency oscillations that a smoothed mean fails to resolve.The resulting density can be broad and roughly bimodal where those oscillations fall within a smoothing neighborhood.Source: §4 · pp. 4–5
  • The excess spread reflects reducible uncertainty about unresolved dependence on the conditioning variable, and decreases as additional data or training helps the model resolve that structure.Within the learned distribution, this model-induced spread is indistinguishable from genuine aleatoric variation.Source: §4 · p. 5

4.1 VARIANCE INFLATION TRACKS SMOOTHING BIAS

  • Across early stopping, roughness penalties, and kernel ridge regression, learned conditional variance decreases monotonically as smoothing decreases.Variance is often largest where the learned mean is most biased, indicating that smoothing bias in vector-field learning—not flow matching alone—is the key object.Source: Figure 2 · p. 4, §4.1 · p. 5

4.2 BEYOND ONE-DIMENSIONAL SYNTHETIC REGRESSION: MONTE-CARLO POLICY EVALUATION

  • In high-dimensional state-action policy evaluation, early learned distributions are more diffuse than empirical targets, then contract and align more closely at later checkpoints.The task predicts scalar Q-values from state-action pairs in two OGBench environments; the figure shows learned and true distributions against the first two conditioning-variable principal components.Source: Figure 3 · p. 5, §4.2 · p. 6

5 AN IDEALIZED MODEL FOR VARIANCE INFLATION

The paper models imperfect conditional vector-field learning as smoothing along the conditioning variable. In this idealized setting, the learned endpoint distribution is a local mixture of true conditional distributions, yielding a bias–variance relation in which smoothing bias appears as additional generated spread.

5 AN IDEALIZED MODEL FOR VARIANCE INFLATION

  • The account idealizes neural-network smoothing bias through a regularized population-level flow-matching problem because the precise neural approximation is difficult to characterize.The kernel assumption is discussed as valid in regimes including small t, large path variance, or locally slowly varying conditional means.Source: §5 · pp. 6–7
  • The idealized model treats smoothing as acting on the conditioning dependence of the vector field while leaving its time and flow-state dependence flexible.The linear operator L_c can represent derivative, higher-order smoothness, or RKHS-type penalties.Source: §5 · p. 6
  • Under the stated kernel factorization and regularity assumptions, the learned endpoint distribution is a local mixture of true conditional distributions.The proof reduces the regularized solution to a density-weighted local average of the true velocity field, then takes moments of the resulting mixture.Source: §5 · p. 7
  • In the scalar case, average learned conditional variance is at least true conditional variance plus the squared bias of the learned conditional mean.The corresponding vector result holds direction by direction; Figure 5 compares squared bias and excess variance across smoothing estimators and synthetic distributions.Source: §5 · p. 7

6 COVERAGE PROPERTIES OF CONDITIONAL FLOW MODELS

The section tests whether CFM predictive sets provide meaningful coverage across policy evaluation, cell perturbation prediction, and behavior cloning. In the underfitting regime, coverage is approximately calibrated, while variance and set size exhibit useful relationships to prediction error and comparator methods.

6 COVERAGE PROPERTIES OF CONDITIONAL FLOW MODELS

  • The evaluation covers Monte-Carlo policy returns, perturbed-cell states, and robot action chunks, using central intervals for scalar outputs and sample-based KNN sets for high-dimensional outputs.The reported tasks include OGBench policy evaluation, scPerturb single-cell prediction, and PushT behavior cloning.Source: §6 · p. 8
  • The figures use training and validation losses to mark the regime before overfitting, and track coverage, predictive-set sizes, and task utility.Scalar-output tasks also compare pointwise predictive variance with squared prediction error.Source: §6 · pp. 8–9
  • Before training and validation losses diverge, validation coverage stays approximately at or above nominal levels, then dips afterward.Training-data coverage stays at or above nominal throughout training, including after overfitting begins.Source: §6 · pp. 8–10
  • Across scalar-output environments, larger flow variance tends to accompany larger squared prediction error for the flow mean, with positive Spearman correlation.The variance is larger where the model’s mean prediction is less accurate.Source: §6 · p. 10
  • Before overfitting, CFM coverage-set widths or KNN radii are tighter than split-conformal sets, competitive with CQR and other near-nominal baselines, and similar with or without conformalization.The section attributes split-conformal sets’ lack of local adaptivity as one reason their sets are wider.Source: §6 · p. 10

7 CONCLUSION AND LIMITATIONS

The paper concludes that smoothing bias in conditional vector-field learning can appear as excess predictive variance, and reports approximately nominal coverage from CFM sample-based sets before overfitting. Its theoretical account analyzes an idealized population-level regularized smoothing model, with an equivalent-kernel approximation subject to stated conditions.

7 CONCLUSION AND LIMITATIONS

  • Before overfitting, predictive sets constructed directly from CFM samples achieve approximately nominal coverage without explicit calibration training.The paper reports this as an empirical finding, not a formal coverage guarantee.Source: §7 · p. 10
  • The theory covers an idealized population-level regularized smoothing model, not arbitrary neural-network training, and does not establish formal predictive-coverage guarantees.The paper identifies stronger theoretical coverage guarantees as future work.Source: §7 · p. 10

A THEORETICAL MODEL OF VARIANCE INFLATION

  • The theoretical argument reduces regularized flow matching to a density-weighted local average, derives a local mixture of true conditionals, then decomposes covariance into true covariance and unresolved conditional-mean variation.An operator argument establishes that this unresolved variation dominates squared smoothing bias on average.Source: A THEORETICAL MODEL OF VARIANCE… · p. 14

A.1 ASSUMPTIONS: EQUIVALENT-KERNEL MODEL OF REGULARIZED FLOW MATCHING, AND REGULARITY CONDITIONS

  • The equivalent-kernel representation blends conditional probability paths according to how likely each condition is to produce the observed state, producing variance inflation unlike ordinary pointwise smoothing.The exact kernel approximation is expected to work best when its design density is locally stable and is weakest near t = 1 in low-noise regions.Source: §A.1 · pp. 15–16

A.2 FORMAL MAIN RESULTS

This section presents formal results on how regularization smooths conditional flows and affects generated variance, alongside predictive-region constructions and baselines expressed on a common k-NN radius scale. Its plots use three random seeds.

A.2 FORMAL MAIN RESULTS

  • Regularization makes the generated conditional a local mixture of true conditionals, whose covariance includes variation among nearby conditional means.This connects smoothing of the velocity field to variance inflation in the generated distribution.Source: §A.2 · pp. 16–17
  • On average over C, the scalar learned conditional variance is at least the true variance plus the squared bias of its learned conditional mean.The excess variance is the unresolved variation of the regression function over the smoothing neighborhood; as smoothing vanishes, it recovers the true conditional variance.Source: §A.2 · pp. 17–18
  • The vector-valued result extends the excess-covariance interpretation by applying the scalar argument along every output direction.The stated covariance comparison uses the Loewner order.Source: §A.2 · p. 18

A.3 DEFERRED PROOFS

  • The scalar and vector inequalities are averages over conditioning inputs, not pointwise guarantees for heteroscedastic problems.The text notes that empirical pointwise predicted variance nevertheless correlates strongly with squared error of the flow mean.Source: §A.3 · p. 19

A.4 VALIDITY OF THE EQUIVARIANT-KERNEL ASSUMPTION (ASSUMPTION 5.1)

  • In a synthetic task, the learned kernel approximation tracks the neural flow field over much of its trajectory, with weaker agreement in some regions near t = 1.The endpoint distributions remain very similar despite the late-time field error.Source: §A.4 · p. 20

B COVERAGE EXPERIMENTS

B.1 COVERAGE SET CREATION

  • For scalar outputs, flow predictive intervals transport base-distribution quantiles; for high-dimensional outputs, the appendix constructs sample-based KNN predictive sets.The KNN coverage metric uses k = 1.Source: §B.1 · p. 22

B.2 BASELINES

  • The baselines include matched deterministic regressors with split conformal, CFM-mean split conformal, conformalized quantile regression, and conformalized CFM sets.Conf CFM calibrates the boundary of the raw CFM set without changing its learned distribution.Source: §B.2 · pp. 23–24
  • At each condition, methods’ predictive-region sizes use the α-quantile of leave-one-out k-nearest-neighbor distances among K samples, averaged over conditions, with K and k fixed within each task.This preserves the CFM’s radius statistic while avoiding an ellipsoidal approximation to its predictive region.Source: §B.2 · p. 24
  • CFM radii use samples from the learned flow, while HGR radii use its predicted diagonal Gaussian and are computed separately at each condition.Euclidean k-NN distances depend on the full shape of HGR’s diagonal covariance, not only the geometric mean of its marginal standard deviations.Source: §B.2 · p. 24
  • The det + SC and CFM mean + SC baselines require no per-condition sampling; each conformal L2 ball is represented as a central α-mass region of an isotropic Gaussian around its point prediction.The isotropic Gaussian’s covariance uses the χ² quantile for output dimension D.Source: §B.2 · p. 24
  • Ball-based baselines are placed on the common k-NN scale using a reference radius simulated once for each (α, k, K, D) and reused across both baselines.Distance rescaling is exact under the isotropic-Gaussian representation; only the reference quantity is estimated numerically.Source: §B.2 · p. 25
  • Conf CFM reports the sample-based CFM threshold after the conformal correction, averaged over conditions.Source: §B.2 · p. 25

B.3 DATASETS AND TRAINING SETUP

  • All plots use three random seeds, with dataset-specific details and training setup provided below.Source: §B.3 · p. 25

B.3.1 BIOLOGY DATASETS

The biology experiments cast perturbation-response prediction as conditional generation from control cells to perturbed-cell states. They evaluate Norman, PBMC cytokines, and SciPlex3 using dataset-specific condition embeddings and PCA-reduced expression targets.

B.3.1 BIOLOGY DATASETS

  • Across all three benchmarks, control cells provide the source distribution and perturbation conditions specify the generated target-cell distribution.Conditions are pretrained embeddings for Norman and PBMC, and drug fingerprints with dose for SciPlex3.Source: §B.3.1 · p. 25
  • Norman uses K562 CRISPRa profiles across 229 perturbation conditions, with gene-protein embeddings as conditions and 50-component PCA expression targets.The experiments use the predefined training perturbations, split in-distribution cells 80–20 for training and validation, and draw source samples from 7,353 unperturbed controls.Source: §B.3.1 · p. 25
  • The Norman and SciPlex3 flows use a CellFlow-style three-branch MLP, while PBMC uses a related architecture with larger time and decoder dimensions.The source is control-pool samples plus Gaussian noise with σ = 1.0 for Norman and σ = 10.0 for PBMC and SciPlex3; evaluation uses 100 Euler steps for Norman and 1,000 for PBMC and SciPlex3.Source: §B.3.1 · pp. 25–26
  • PBMC cytokine experiments restrict the atlas to Donor 1 and balance 90 cytokine conditions plus PBS controls to 91,000 cells.Cytokine protein embeddings condition 100-component PCA expression targets; the 1,000 PBS controls form the source pool, and cells are split 80–20.Source: §B.3.1 · p. 25
  • SciPlex3 experiments use A549 cells across 703 in-distribution conditions covering 187 drugs, with drug fingerprints and log10 dose as conditions.The target is 100-component PCA expression; approximately 136,000 perturbed cells and 3,000 DMSO controls are split 80–20.Source: §B.3.1 · p. 26

B.3.2 POLICY EVALUATION

The policy-evaluation setup distills a trained FloQ policy’s returns into conditional flow predictors, while the behavior-cloning setup predicts action chunks from observation histories.

B.3.2 POLICY EVALUATION

  • For each OGBench environment, Monte Carlo averages of discounted on-policy returns provide stochastic targets for a flow predictor conditioned on state and action.The evaluated policy is a trained FloQ agent; nMC counts independent rollouts, H is the fixed horizon, and γ is the discount factor.Source: §B.3.2 · p. 26, Table 1 · p. 26
  • All three FloQ predictors use a four-layer 512-width ReLU MLP, a 32-dimensional time embedding, and an isotropic Gaussian prior in the one-dimensional target space.Training uses Adam with learning rate 3×10^-4 and batch size 1024.Source: §B.3.2 · p. 26

B.3.3 BEHAVIOR CLONING

  • The state-based PushT behavior-cloning task predicts an eight-step action chunk from two steps of observation history using a 10-dimensional condition and 16-dimensional target.Observations and actions are MinMax-normalized over the training set, with an 80–20 train-validation split and a standard Gaussian source.Source: §B.3.3 · p. 26
  • PushT uses an eight-block residual MLP with 512-dimensional embeddings, GELU activations, LayerNorm, and dropout 0.1, trained with Adam and gradient clipping.The setup is state-based and uses no image encoder.Source: §B.3.3 · p. 27

B.4 ADDITIONAL EXPERIMENTS

Additional OGBench and cell-perturbation experiments extend the paper’s coverage analyses beyond the main-paper environments. Their qualitative findings match those reported in Section 6.

Additional policy-evaluation environments

  • On Cube-Single and Humanoidmaze-medium, training coverage stays at or above nominal, while validation coverage is close to nominal during underfitting.Figures 12 and 13 show training loss and coverage at three nominal levels, plus variance-versus-error and mean interval widths for flow, split conformal, and CQR.Source: §B.4 · p. 27, Figure 12 · p. 27, Figure 13 · p. 27

Additional cell-perturbation benchmarks

  • On PBMC and SciPlex3, training coverage remains at or above nominal and validation coverage is close to nominal during underfitting, yielding approximately calibrated predictive sets.Figures 14 and 15 show training and validation flow-matching loss alongside KNN-based coverage at three nominal levels.Source: §B.4 · pp. 27–28, Figure 14 · p. 28, Figure 15 · p. 28

C VARIANCE INFLATION: ADDITIONAL SYNTHETIC EXPERIMENTS

Additional synthetic experiments report variance-inflation trends similar to Figure 2. Across the experiments, learned conditional variance rises with mean smoothing and falls as smoothing decreases.

C VARIANCE INFLATION: ADDITIONAL SYNTHETIC EXPERIMENTS

  • The additional synthetic datasets show variance-inflation trends similar to Figure 2.The reported experiments compare three estimators.Source: C VARIANCE INFLATION: ADDITIONAL… · p. 28
  • Across the experiments, learned conditional variance increases as the learned mean becomes smoother.The average learned conditional variance decreases monotonically as smoothing decreases across all three estimators.Source: C VARIANCE INFLATION: ADDITIONAL… · p. 28
  • For all three estimators, average learned conditional variance decreases monotonically as smoothing decreases.This pattern accompanies the reported increase in conditional variance as the learned mean becomes smoother.Source: C VARIANCE INFLATION: ADDITIONAL… · p. 28

Related papers

Loading 2610.29172v1…