Source-linked AI summary
DiffLTM: Continuous Diffusions Large Tabular Models for Conditional Generation
Alankrita Bhatt, Mukur Gupta, Andrea Montanari, Pulkit Tandon
TL;DR
Conditional generation for tabular data must handle mixed numerical and categorical entries, including high-cardinality or skewed categoricals, and is especially challenging for infrequent conditions. DiffLTM uses continuous diffusion for tabular generation and reports strong generation and conditional-generation capabilities, including recovery of econometric patterns.
Problem
Tabular conditional generation must handle mixed numerical and categorical data, often with high-cardinality or skewed categories, and is particularly challenging for infrequent conditions.
Source: §1 · p. 3Method
DiffLTM designs a tabular generative model based on continuous diffusions, using learned categorical embeddings and cross-entropy loss and supporting equality, range, and linear conditions.
Source: §4.1 · p. 10, §7 · p. 19Results
The model is reported to have state-of-the-art generation and conditional-generation capabilities and to reproduce econometric patterns including synthetic controls and the Mincer equation.
Source: §1 · p. 3, §7 · p. 19Takeaways & Limitations
The authors present recovered econometric patterns as complementary information for validating tabular generation.
Source: §1 · p. 3, §7 · p. 19The econometrics application lacks rigorous statistical guard-rails, which the authors identify as an avenue for future work.
Source: §7 · p. 19
Abstract
from arXiv · showhide
Conditional generation is a key capability to be supported by Large Tabular Models, enabling a number of applications. Accurate conditional generation has to contend with two key challenges: $(i)$ Data comprise a mixture of numerical and categorical entries, the latter often with high cardinality and heavily unbalanced proportions; $(ii)$ Conditional generation becomes increasingly difficult for rare sub-populations. The best models in this class use a hybrid of continuous and discrete diffusions. In this paper, we design a continuous diffusion model with strong generation quality. An important element is the use of cross-entropy loss for categoricals, for which we provide a first theoretical justification. We introduce metrics that focus on rare subpopulations. Further, we generalize conditional generation techniques to enable new types of conditioning. Finally, we show that our model (trained on labor data) can capture classical insights from econometrics. Code is available at https://github.com/granica-ai/DiffLTM
H Optimality of CE loss for continuous diffusions: · 1 Introduction
DiffLTM is a continuous diffusion model for tabular data that targets accurate conditional generation, including rare subpopulations, and introduces evaluation metrics and theoretical results on categorical loss. The paper also reports that it reproduces selected econometric patterns and extends conditioning to equality, range, and linear constraints.
1 Introduction
- The study trains on census datasets of up to 7M rows and evaluates conditional generation with newly designed faithfulness metrics.The introduction describes accurate conditional generation as especially challenging for infrequently occurring conditions.Source: §1 · p. 3
- DiffLTM introduces interpretable fidelity metrics emphasizing accuracy on rare subpopulations, addressing the risk that conventional metrics are dominated by majority classes.The paper argues that standard metrics such as FID and precision-recall can be hard to interpret or misleading for tabular data.Source: §1 · p. 3
- DiffLTM reproduces two econometric patterns: synthetic-control counterfactuals and the Mincer equation on returns to education.Figure 2 shows wage trends for Arizona and Pennsylvania, including a no-policy counterfactual for Arizona and a placebo prediction for Pennsylvania.Source: §1 · p. 3, Figure 2 · p. 5
- Figure 1 reports that DiffLTM-CFG achieves conditional Shape scores close to rejection sampling and Ideal, outperforming DiffLTM-SMC and Harpoon.Scores use matched synthetic and test samples for ACS Labour data (ntrain ≈ 4.8M).Source: §1 · p. 3, Figure 1 · p. 4
- DiffLTM uses purely continuous diffusion, with carefully chosen embeddings and loss, and demonstrates state-of-the-art unconditional generation among continuous diffusion baselines.The introduction frames this design against hybrid methods such as TabDiff and purely continuous approaches [MGF26].Source: §1 · pp. 3–4
H Optimality of CE loss for continuous diffusions
- Under a simple misspecification scenario, cross-entropy is mathematically optimal; the paper also proves bounded inefficiency for continuous diffusion on a simple discrete generation task.The authors additionally report empirical evidence that cross-entropy with learned embeddings is preferable, despite theoretical arguments favoring score matching.Source: §1 · p. 4
1 Introduction
- The paper adapts classifier-free guidance and inference-time interventions, including sequential Monte Carlo, for equality, range, and linear conditions.It reports state-of-the-art and near-optimal conditional-generation performance, with CFG and SMC significantly outperforming tabular conditional-generation baselines.Source: §1 · p. 4
1.1 Further related work
- Related work spans statistical-accuracy, downstream-utility, and latent-causal metrics [JSJ25], GANs, graph models, autoregressive models, latent diffusions, and variational flow matching [GCEVDM25].Recent work also addresses relational databases beyond the single-table setting [HXS+25].Source: §1.1 · p. 5
2 Metrics
DiffLTM is evaluated on ACS datasets using fidelity, conditional-fidelity, predictive-utility, and memorization metrics. It outperforms continuous-diffusion comparisons on most metrics and has results comparable to TabDiff on two of three datasets, with an advantage on ACSLabour.
2.1 Datasets
- The evaluated datasets are ACSPubCov and ACSMobility from Folktables, plus ACSLabour, which retains continuous log wage and broad labor and demographic columns.ACSLabour includes high-cardinality occupation, industry, geographic, and birthplace variables.Source: §2.1 · p. 5
2.2 Fidelity metrics
- DiffLTM outperforms TabRep and TabCascade on most metrics across all datasets, while matching TabDiff on two of three datasets and holding an advantage on ACSLabour.Table 1 compares unconditional-generation performance; the supplied text does not report its individual metric values.Source: Table 1 · p. 6, §2.2 · p. 8
- Shape [SDM23] assesses univariate marginals, while Trend assesses pairwise categorical marginals or numerical correlations; scores generally range from 0 to 1, with 1 best.The metrics compare generated samples with held-out target rows using plug-in estimates of distributional distances.Source: §2.2 · p. 6
- Conditional Shape measures mean shape within a subpopulation, weighting subpopulations equally regardless of frequency and thereby emphasizing rare groups.Conditional Shape (Rare) emphasizes rare subpopulations further, using a default frequency threshold τ = 0.05; Conditional Trend and its rare variant are analogous.Source: §2.2 · p. 7
- MLE Ratio evaluates predictive utility through synthetic-train/real-test performance relative to real-train/real-test performance, while DCR median ratio measures synthetic-to-training nearest-distance behavior.MLE Ratio uses AUC for classification and R2 for regression, with values near one indicating preserved predictive utility; DCR median ratios substantially below one indicate memorization.Source: §2.2 · p. 7
- DiffLTM’s MLP-only architecture yields an approximate 5× speedup in inference and training compared with TabDiff’s Transformer architecture.The paper also notes that entirely continuous diffusions have evidence of better parallelizability [CW26].Source: §2.2 · p. 8
3 Diffusion architecture and unconditional generation
DiffLTM embeds mixed-type rows in a continuous space and performs diffusion, denoising, and sampling there. Its experiments favor logit parameterization with cross-entropy for categoricals, with learned embeddings improving on one-hot embeddings for that model.
- Categorical values are mapped to continuous embeddings, which are combined with numerical columns to form each row’s continuous representation.The embeddings may be one-hot or learned; learned embeddings are normalized to the square root of their dimension.Source: §3 · pp. 8–9
- The forward process adds Gaussian noise to the continuous representation at a variance set by a monotonically increasing schedule, while the denoiser predicts numerical values and categorical outputs.For categoricals, it can predict clean embeddings or per-category logits, which induce posterior mean embeddings.Source: §3 · p. 9
- Training combines mean-squared error for numerical columns with either embedding-space square loss or cross-entropy on categorical logits.The total objective sums numerical and categorical losses and backpropagates through learned embedding tables when applicable.Source: §3 · pp. 9–10
- Generation starts from Gaussian noise and uses denoiser predictions in Euler updates across decreasing noise levels; final numerical values are returned directly and categoricals are decoded from embeddings or logits.Embedding mode selects the nearest category embedding, whereas logit mode selects the highest-probability category.Source: §3 · p. 10
- Table 2 reports that logits with cross-entropy outperform the other architecture variants, and learned embeddings improve performance over one-hot embeddings for this model.The three variants are representation with MSE, logits with MSE, and logits with CE.Source: §3 · p. 10
4 Conditional generation
DiffLTM supports conditional generation through classifier-free guidance (CFG) and sequential Monte Carlo (SMC), with CFG extended to range and linear conditions. The methods cover equality, range, and linear constraints, while SMC also enables conditioning on categorical predicates and arbitrary numerical conditions.
4 Conditional generation
- DiffLTM uses CFG and SMC for conditional generation, extending CFG to range and linear conditions compared with baselines in Figure 1.The supplied passages describe the comparison but do not provide its results.Source: §4 · p. 10
4.1 Train-time intervention: Classifier-free guidance
- The conditional denoiser accepts equality, range, and linear conditions, which can be imposed simultaneously, and replaces equality-conditioned noisy values with learned tokens.Condition masks indicate which constraints are present; the remaining architecture matches unconditional generation.Source: §4.1 · p. 10
- CFG training samples conditions by masking columns, converting selected numerical equalities to ranges, and optionally replacing selected numerical constraints with a linear condition.Range and linear bounds are sampled around the training row’s values to target the appropriate truncated distribution; linear weights are Gaussian-sampled and normalized.Source: §4.1 · pp. 10–11
4.2 Inference-time methods
- SMC reweights reverse-diffusion trajectories using approximated condition likelihoods, enabling categorical predicates and arbitrary numerical conditions beyond prior work such as Harpoon.Its approximations assume conditional independence of columns given the current state, use denoiser logits for categoricals, and model numerical columns as Gaussian around denoiser predictions; rejection sampling is unbiased but highly inefficient for rare conditions.Source: §4.2 · p. 11
5 Reproducing econometric patterns via conditional generation
DiffLTM conditional generation reproduces econometric patterns in counterfactual analyses and Mincer regressions, including patterns within specific subpopulations. The counterfactual findings are preliminary, while the Mincer experiments recover both population-level and subgroup-specific relationships.
5.1 Counterfactual analysis
- DiffLTM’s Arizona counterfactual is in rough agreement with synthetic control; for Pennsylvania, its predictions closely match observed post-2016 wages and are somewhat more accurate.Arizona’s observed tenth-percentile nominal hourly wage follows the statutory minimum almost exactly, but the counterfactual has no observable ground truth; the authors characterize this as a preliminary study.Source: §5.1 · pp. 12–13
- Conditional generation produces individual-level counterfactual records that can be stratified by occupation, revealing larger estimated effects for cleaning and sales workers than healthcare workers.The figure caption attributes the difference to cleaning and sales wages piling up at $11, while healthcare wages were already above that level.Source: Figure 3 · p. 12, §5.1 · p. 13
- For nonlinear outcome transformations, DiffLTM-CFG stays close to the ground truth while synthetic control becomes inaccurate; both match for the untransformed outcome.The reparameterization experiment uses AdaLN, which the authors say significantly improves robustness; they describe the broader counterfactual application as preliminary.Source: §5 · p. 11, §5.1 · p. 13, Figure 4 · p. 13
5.2 Mincer returns to education
- DiffLTM reproduces the Mincer earnings equation [Min74], with positive schooling and linear experience coefficients and a negative quadratic experience coefficient.These coefficient signs reflect higher earnings associated with more schooling and a concave wage-experience profile.Source: §5.2 · p. 15
- For people reporting no English proficiency, schooling coefficients fall from approximately 0.13 in the full dataset to 0.01, and experience coefficients from approximately 0.09 to 0.03; CFG and SMC recover this pattern.The subgroup represents approximately 0.95% of the dataset; the coefficients are averages across three seeds.Source: §5.2 · p. 15
- For China-born female private-sector workers, both CFG and SMC reproduce the ordering of earnings by education and the concave age–earnings profile.The comparison conditions on high-school, bachelor’s, and master’s education and jointly generates age and log wage; the figure plots median log wage across five-year age bins.Source: Figure 6 · p. 14, §5.2 · p. 15
6 Asymptotic theory
The analysis finds cross-entropy asymptotically variance-optimal for continuous diffusions with under-smoothed, time-varying denoisers. In the binary setting, its inefficiency relative to maximum likelihood is bounded, whereas MSE’s is unbounded.
6.1 Optimality of CE loss for continuous diffusions
- This revises MSE-optimality conclusions that assume denoisers at different noise levels are carefully tied; allowing parameters to vary with time can make MSE suboptimal.The paper identifies this time-varying parameterization as usual for denoising diffusions.Source: §6.1 · pp. 16–18
- Under under-smoothed estimation with time-varying denoiser parameters, generated class probabilities have an asymptotic normal limit whose covariance is minimized by cross-entropy.The undersmoothing condition corresponds to weak penalization of the time dependence and makes the denoiser estimates asymptotically unbiased.Source: §6.1 · p. 17
6.2 Comparing continuous to discrete sampling
- Continuous diffusion can still have strictly greater asymptotic variance than maximum likelihood, even for k = 2; the binary result establishes a bounded-factor guarantee, not equality.The comparison is between covariance matrices V for continuous diffusion and V_ML for probability estimation by maximum likelihood.Source: §6.2 · p. 18
- For k = 2 with one-dimensional ±1 embeddings and optimally chosen time-sampling density, cross-entropy has bounded asymptotic relative inefficiency versus maximum likelihood, while MSE’s is unbounded.Cross-entropy satisfies the theorem’s condition with α = 2, while MSE satisfies it with α = 4.Source: §6.2 · pp. 18–19
7 Discussion and limitations
The paper presents a continuous-diffusion tabular model with generation and conditional-generation capabilities, and demonstrates econometric pattern recovery on larger datasets. It identifies cross-dataset modeling and rigorous statistical guardrails for its econometrics application as future work.
- The continuous-diffusion model provides state-of-the-art generation and conditional generation, using learned categorical embeddings and cross-entropy loss with a mathematical rationale.The model was trained on datasets one order of magnitude larger than previous published work and recovered econometric patterns.Source: §7 · p. 19
- Cross-dataset modeling remains missing; preliminary training of a cross-dataset version is deferred to future work.Source: §7 · p. 19
- The econometrics application lacks rigorous statistical guardrails, which the authors identify as an important avenue for future work.Source: §7 · p. 19
AI use statement
The authors used AI tools for writing, literature review, code development, and proof drafting, while retaining responsibility for the final work and reviewing AI-assisted outputs.
AI use statement
- AI assistants supported writing polish, literature review, and code development, but not code for the work’s core methodological components.The authors manually verified references and reviewed and tested all AI-generated code for correctness.Source: AI use statement · p. 19
- Generative AI tools helped draft proofs in Appendix H, which the authors reviewed and edited.The authors take responsibility for the final content, including AI-assisted text, claims, and artifacts.Source: AI use statement · p. 19
Reproducibility statement
The paper documents its data, model configurations, and experimental variability to support reproducibility.
- Datasets are public or documented in Appendix A, architectures and hyperparameters in Appendices B–C, and experiments use multiple seeds with means and standard errors or standard deviations reported.Source: Reproducibility statement · p. 19
A Datasets details … B.2 Training
The appendix defines ACSLabour as a continuous-log-wage regression benchmark and rare-subpopulation stress test, then specifies the datasets’ preprocessing and DiffLTM’s experimental setup. It details the model choices, architecture, training configuration, and compute environment.
A Datasets details
- The datasets derive from ACS PUMS; ACSPubCov and ACSMobility use Folktables datasets concatenated across the 2014–2018 survey period.ACS PUMS contains de-identified person-level survey records with demographic, geographic, educational, employment, and income information.Source: A Datasets details · p. 24
- ACSLabour retains continuous log wage and broad labor and demographic columns, including high-cardinality variables, for regression and rare-subpopulation testing.Unlike ACSIncome’s binary income target, this dataset supports analyses such as regressions on continuous log wage.Source: A Datasets details · p. 24
- ACSLabour retains only employed civilians, records with observed positive WAGP, and records with WKHP > 0.The employed-civilian condition is ESR ∈ {1, 2}.Source: A Datasets details · p. 24
- Preprocessing fits transformations on training data only, quantile-transforms numerical columns, integer-encodes categoricals, and uses a 70/10/20 train/validation/test split.Classification splits are target-stratified; regression splits are unstratified.Source: A Datasets details · p. 25
B Unconditional generation: Experiment details
- The Table 1 model uses learned embeddings, logits mode, and cross-entropy loss.These architecture and training choices are the model used for comparison against baselines.Source: B Unconditional generation: Experiment… · p. 25
B.1 Architecture parameters
- The ACSLabour unconditional model has approximately 9.84M parameters and uses 16-dimensional embeddings, a 1024-wide five-layer SiLU MLP, and sinusoidal time embeddings.The diffusion time embedding is 1024-dimensional and passes through two 1024-wide SiLU layers before being added to the projected input.Source: §B.1 · p. 25
B.2 Training
- DiffLTM variants train for 2000 epochs with batch size 8192, AdamW, an initial learning rate of 3 × 10^-4, cosine decay, EMA decay 0.997, and validation-loss checkpoint selection.Unconditional and CFG-based conditional generation use 100 Euler steps; the noise schedule is (σmin, σmax, ρ) = (0.002, 80, 7). EMA follows earlier work.Source: §B.2 · p. 25
- Experiments use a single node with 192 CPU cores and eight NVIDIA A100 80 GB GPUs; three random seeds and bfloat16 precision are used, with each run under 12 GPU-hours.The environment uses Ubuntu 22.04 and CUDA 13.0.Source: §B.2 · p. 25
B.3 Baselines implementation … C.3 Evaluation methodology
The appendices specify the baseline training and evaluation, report DiffLTM’s faster gradient steps, and detail conditional-generation configurations and slice-based evaluation.
B.3 Baselines implementation
- The benchmark uses official TabRep, TabCascade, and TabDiff implementations, shared data splits and evaluation, 2,000 epochs, batch size 8,192, and lowest-validation-loss checkpoints.Each model generates ntrain synthetic rows, matching the real training-set size.Source: §B.3 · p. 26
- TabDiff’s default learning rate destabilized training, so the authors use 3 × 10−4 and gradient-norm clipping at 1.0 while retaining its original denoiser architecture.The authors report that these changes yield stable training and substantially better performance.Source: §B.3 · p. 26
- 78.03 ms per gradient step for TabDiff versus 34.98 ms for DiffLTM; BF16 reduces DiffLTM to 14.40 ms, a 5.4× speedup.The isolated profiling used 100 gradient steps after 20 warm-up steps, batch size 8,192, and an NVIDIA A100-SXM4-80GB GPU; values are averaged over five runs.Source: §B.3 · p. 26
B.4 Metrics computation
- For fidelity metrics (Shape, Trend, and Conditional Shape), methods generate ntrain synthetic rows, subsample ntest rows, and compare them with held-out real test data; Table 1 and Table 2 metrics report means and standard deviations across three training seeds.Source: §B.4 · p. 26
C Conditional generation: Experiment details
- Appendix C describes the experimental setup behind Figure 1 and points to additional conditional-generation results in Appendix F.Source: C Conditional generation: Experiment… · p. 27
C.1 CFG
- CFG changes only the denoiser input: masked condition parameters are concatenated per column, while the time-conditioned MLP and output heads remain shared.The CFG variant uses dropout probability 0.15, prng = 0.5, plin = 0.5, condition-width limits of 0.02–1.0 of the relevant span, and conditioning strength ω = 1.Source: Figure 7 · p. 27, §C.1 · p. 27
C.2 Inference-time methods
- Rejection sampling caps unconditional draws at 10 million and marks exhausted conditions missing; SMC uses ESS threshold 0.95 and estimates arbitrary-condition probabilities from 256 antithetic draws.Source: §C.2 · p. 28
C.3 Evaluation methodology
- Evaluation uses 50 shared equality, range, and linear slices per task, with each slice 1–3% rare in training and at least 100 test rows.Fidelity evaluation retains all generated rows, averages each slice metric over three training-seed checkpoints, and reports mean and standard error across slices within rarity buckets; the empirical ideal compares real test rows with an equal number of real training rows.Source: §C.3 · p. 28
C.4 Baseline implementation … E.2 Training and generation
The appendices compare Harpoon with CFG and SMC, detail Arizona counterfactual generation and subgroup analyses, and specify a synthetic reparameterization experiment. The Arizona analysis reports subgroup-specific wage effects, while the synthetic experiment evaluates how outcome transformations affect counterfactual estimation.
C.4 Baseline implementation
- With the stronger learnt-embedding, logits, and CE base model, Harpoon still performed significantly below CFG and SMC across datasets and metrics.The authors implemented Harpoon on the one-hot, MSE, representation-mode base model used in the original guidance comparison, extending it to linear conditioning with interval-based gradient guidance.Source: §C.4 · pp. 28–29, Figure 8 · p. 29
D Arizona proposition 206 experiment details; D.1 Dataset
- The Proposition 206 analysis uses 489,225 CPS observations of employed private-sector workers aged 16–64, retaining state, year, demographics, occupation, and log hourly wage.Wages were restricted to $2.00–$99.98 and normalized to 2019 units; the appendix provides setup details for studying the proposition’s effects and additional results.Source: §D.1 · p. 29, D Arizona proposition 206 experiment… · p. 29
D.2 Synthetic control counterfactual
- Synthetic-control weights match Arizona to a donor pool of 21 states that held the federal $7.25 minimum wage throughout 2010–2019, using 14 pre-2017 features.The target statistic is each year’s tenth-percentile hourly wage; features include pre-intervention wage percentiles and population characteristics selected for minimum-wage exposure, with weights minimizing standardized feature distance.Source: §D.2 · p. 30
D.3 DiffLTM counterfactual
- For Arizona and Pennsylvania, CFG-enabled DiffLTM generates 100,000 records per post-treatment year after excising the held-out state’s post-intervention rows.Generation conditions on state and year, jointly samples remaining worker attributes and wages, and estimates each year’s tenth-percentile wage from synthetic records.Source: §D.3 · p. 30
D.4 Subpopulation-level analyses
- Arizona’s KS-distance changes were +0.128 for high-school workers and −0.031 for bachelor’s workers; sub-$11 reductions were 20.5 and 7.3 percentage points, respectively.Across occupations, KS distance increased by 0.085 for healthcare, 0.089 for cleaning, and 0.262 for sales workers; corresponding sub-$11 reductions were 1.7, 11.1, and 36.6 percentage points. Observed data also had substantial mass at exactly $11, particularly among more exposed groups.Source: Figure 9 · p. 31, §D.4 · p. 32
E Reparametrization in synthetic control details
- The appendix motivates a controlled test of synthetic control’s brittleness, noting sharp performance degradation when the target is non-linearly transformed.Source: E Reparametrization in synthetic… · p. 32
E.1 Data distribution
- The synthetic experiment generates outcomes for 11 states and 11 periods, intervenes on state 6 in periods 8–10, and tests identity, log, and non-monotone outcome transformations.It draws 1,200 rows per state-period pair and compares medians from DiffLTM and synthetic control with observed outcomes.Source: §E.1 · p. 32
E.2 Training and generation
- The reparameterization experiments use a roughly 350,000-parameter model trained for 1,000 epochs, with three seeds and 100,000 generated samples per state-period condition.Categoricals use CE and numerical features MSE; the model uses learned embeddings, logits, standardized numerical columns, and AdaLN MLP blocks, with medians averaged across seeds and standard-error bars.Source: §E.2 · pp. 32–33
F Additional empirical results … H.5.1 Proof of Lemma H.1
The appendix supplements the paper with additional fidelity and conditioning results, generation pseudocode, and theoretical analysis of CE-loss optimality, asymptotic parameter and generated-class estimates, time-sampling choices, and terminal-flow properties.
F Additional empirical results
- DiffLTM outperforms the baseline robustly across fidelity metrics and datasets in the additional fidelity-versus-rarity results.The evaluation protocol matches Appendix C; the supplied passages do not provide table cell values.Source: Table 5 · p. 33, F Additional empirical results · p. 33
- SMC conditions on arbitrary numerical conditions, with shape scores reported against ideal values as means over three training seeds.The results are for the ACSLabour dataset and use the method described in Section 4.2.Source: F Additional empirical results · p. 33, Table 6 · p. 34
G Pseudocode
- DiffLTM generation iteratively denoises a noisy representation and decodes categorical columns by taking the most probable category.The SMC pseudocode weights particles against the condition and systematically resamples when effective sample size falls below a threshold.Source: G Pseudocode · pp. 35–36
H Optimality of CE loss for continuous diffusions: Proofs for Section 6.1
- The appendix formalizes the Section 6 results establishing CE-loss optimality in the undersmoothed regime and includes proofs of technical steps.It also characterizes the optimal sampling density in that setting as a byproduct.Source: H Optimality of CE loss for continuous… · p. 42
H.1 Setting and assumptions
- Under stated assumptions, probability-flow trajectories converge terminally to category embeddings almost everywhere, and CE and score matching satisfy specified regularity assumptions.The associated basins are Borel measurable and partition the input space up to a null set; the loss-specific verification has additional conditions for score matching.Source: §H.1 · p. 45
H.2 Asymptotics of parameter estimates
- The asymptotic analysis characterizes localized population bias and the first-order effect of parameter-estimation errors on sampling probabilities under probability-flow generation.The linear-response result considers bounded measurable perturbations supported on a compact time interval.Source: §H.2 · pp. 46–47
- The scalar mixture distribution divides the state space into two basins, A−(t) = (−∞, γ(t)) and A+(t) = (γ(t), ∞), with a spatial-derivative integral evaluated over a half-line.The supplied passage gives only a partial expression for γ(t) and the resulting integral.Source: §H.2 · p. 47
H.3 Undersmoothing regime
- With probability tending to one, shrinking-bandwidth empirical minimizers are unique and lie in int(K(t)) simultaneously on compact I.The proposition assumes Assumptions 1, 4, 5, and 3.Source: §H.3 · p. 48
- Generated class probabilities have a joint limit when endpoint contributions are negligible, with V(λ)1 = 0.Integrated parameter estimates also satisfy a joint central limit theorem for every finite collection of test functions.Source: §H.3 · pp. 48–49
- Corollary H.7 addresses cross-entropy efficiency relative to losses satisfying the stated local Fisher-consistency condition.The comparison concerns the asymptotic covariance for fixed λ under the corollary's assumptions.Source: §H.3 · p. 49
H.4 Choice of the time-sampling density
- The analysis gives an optimal time-sampling density and shows that optimal Gaussian-limit risk depends on the schedule only through its SNR range, making it invariant to monotone time reparametrization.The invariance result applies to cross-entropy or squared denoising loss when ρ(t) is strictly monotone and the improper integral is finite.Source: §H.4 · pp. 49–50
H.5.1 Proof of Lemma H.1
- Almost every trajectory has finite length and a terminal limit distributed as Pq; the proof also derives basin-set and basin-mass identities.The terminal-law conclusion follows from weak convergence of pt(z) dz to Pq as t approaches T.Source: §H.5.1 · p. 51
H.5.2 Proof of Proposition H.2 … I.4 Putting everything together: Proof of Theorem 2 √
The proofs establish convergence and regularity guarantees, characterize perturbation effects and terminal class masses, and analyze Gaussian limits, losses, and optimal sampling. They also show how the optimal variance ratio depends on α, being unbounded for 2 < α ≤ 4 and bounded when α = 2.
H.5.2 Proof of Proposition H.2
- Both losses have positive-definite Hessians and unique population minimizers; for cross-entropy, the conditional score covariance equals its Hessian.Cross-entropy is convex, while squared loss may be nonconvex and uses compact time-dependent domains; the losses and their first three parameter derivatives have polynomial envelopes.Source: §H.5.2 · pp. 51–52
H.5.3 Zero-bandwidth limit of the population risk
- For sufficiently small bandwidth, population minimizers are unique and interior, converge uniformly to the target curve, and have uniformly convergent Hessians.On intervals strictly inside the time domain, the bandwidth normalization also has a uniform asymptotic scaling.Source: §H.5.3 · pp. 52–53
H.5.4 Proof of Proposition H.3
- Kernel symmetry makes the localized minimizer an even function of bandwidth, so its first bandwidth derivative vanishes and a uniform second-order expansion follows.The implicit function theorem supplies a uniformly C2 solution near the zero-bandwidth target curve.Source: §H.5.4 · p. 53
H.5.5 Proof of Proposition H.4
- Perturbations induce a terminal class-mass derivative given by the response kernels, which are bounded and yield a uniform remainder in total variation.The derivative is continuous in time, and the class-mass variations sum to zero.Source: §H.5.5 · pp. 56–57
H.5.6 Proof of Proposition H.5
- Empirical minimizers are uniformly consistent and unique with high probability, and their deviation from the population minimizers is OP(en).The minimizers are interior; in the unconstrained convex case, strong convexity gives global uniqueness.Source: §H.5.6 · pp. 58–59
- Uniform empirical-Hessian bounds and Assumption 3 establish Proposition H.5’s final assertion.The proof replaces the averaged empirical Hessian by Hb(t) using bounds on the empirical and population Hessians.Source: §H.5.6 · p. 59
H.5.7 Proof of Proposition H.6
- Proposition H.6’s limit has joint Gaussian convergence for finitely many test functions.The Lindeberg–Feller argument establishes the limit, and applying it to linear combinations yields joint convergence.Source: §H.5.7 · p. 60
H.5.8 Proof of Theorem 3
- Theorem 3’s proof extends the limit to the estimator over (0, T), with covariance satisfying V(λ)1 = 0.The truncated estimator has a Gaussian limit, and the covariance integrals converge as the truncation endpoint tends to zero.Source: §H.5.8 · p. 61
H.5.9 Proof of Corollary H.7
- The cross-entropy sandwich operator is no larger than that for other losses in the comparison class, which includes squared denoising loss.The comparison follows from the shared volume-response kernels and the loss-specific sandwich operators.Source: §H.5.9 · p. 62
H.5.10 Proof of Proposition H.8
- The optimal sampling density is proportional to hW, while normalized basin quantities depend on time only through r = ρ(t).The optimal density result also covers cases where strict positivity requires approximation by positive regularizations.Source: §H.5.10 · p. 62, §H.5.11 · p. 63
I.1 Reduction to the rare-class limit
- Label symmetry reduces the analysis to rare-class p↓0, Fisher consistency fixes the score-moment tail exponent, and the variance integral localizes to the critical window around t = L.The localization lemma identifies t = L as the only contributing time scale.Source: §I.1 · p. 64, §I.2 · pp. 65–66
I.3 The critical-window calculation
- Critical-window asymptotics show a logarithmic displacement that produces an extra log L factor when α > 2, with the normalized integrands uniformly integrable.For 2 < α ≤ 4, the leading asymptotic constant depends only on the loss.Source: §I.3 · pp. 66–67
I.4 Putting everything together: Proof of Theorem 2 √
- For 2 < α ≤ 4, the variance ratio is unbounded; for α = 2, it is bounded on (0, 1).The α = 2 conclusion uses label symmetry at the endpoint and continuity on compact subintervals.Source: §I.4 · p. 68
Related papers
This paper cites
- Score-Based Generative Modeling through Stochastic Differential Equations2020
- Classifier-Free Diffusion Guidance2022
- FiLM: Visual Reasoning with a General Conditioning Layer2017
- Elucidating the Design Space of Diffusion-Based Generative Models2022
- Diffusion Posterior Sampling for General Noisy Inverse Problems2022