PaperlayerView exact revision on arXiv

Source-linked AI summary

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Sai Niranjan Ramachandran, Suvrit Sra

arXiv:2605.00414v2cs.LGcond-mat.stat-mechcs.AI

TL;DR

Decision trees and diffusion models lack a unified account despite both hierarchically refining information. This paper establishes a mathematical correspondence and shared optimization framework, yielding competitive tabular generation with 2X speedup and neural networks that match teacher performance within 2%.

  • Problem

    Decision trees and diffusion models are treated as disparate despite both performing hierarchical refinement of information.

    Source: §1 · p. 1

  • Method

    The paper formalizes a tree–diffusion flow correspondence and introduces Global Trajectory Score Matching as a shared optimization framework.

    Source: §1 · pp. 1–2

  • Results

    TREEFLOW achieves competitive tabular generation quality while being 2X faster, and DSM-TREE matches teacher performance within 2% on 4/5 datasets.

    Source: Summary of Contributions · p. 2, §5.2 · p. 10

  • Takeaways & Limitations

    The unified perspective supports hybrid models that combine discrete hierarchical structure with continuous dynamics.

    Source: §7 · p. 12

    The theory relies on continuous-path refinement and smoothness assumptions and focuses experiments on continuous feature spaces.

    Source: §7 · p. 12

Abstract

from arXiv · show

Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: \emph{Global Trajectory Score Matching (GTSM)}, for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: \treeflow, which achieves competitive generation quality on tabular data with higher fidelity and a 2\times computational speedup, and \dsmtree, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2\% on many benchmarks.

1 Introduction

The paper establishes a formal Tree↔Flow correspondence linking hierarchical decision trees with diffusion-associated deterministic flows, then derives GTSM as a shared optimization principle and develops two tabular-data algorithms from this unification.

1 Introduction

  • The Tree↔Flow correspondence identifies hierarchical decision trees and diffusion-associated deterministic flows as two perspectives of the same underlying generative and discriminative object.Decision-tree coarse-graining becomes a discrete-time Markov process whose continuous limit is a unique PF-ODE, while suitable forward diffusions induce canonical hierarchies through progressively merged data modes.Source: §1 · pp. 1–2
  • Global Trajectory Score Matching (GTSM) unifies greedy boosting and end-to-end score-based diffusion training as related solvers of a shared objective.The framework shows gradient boosting is asymptotically optimal as a discrete solver under the stated limiting analysis.Source: §1 · p. 2, Summary of Contributions · p. 2

Summary of Contributions.

  • TREEFLOW achieves competitive tabular generation quality, with highest TSTR accuracy on 3/5 benchmarks, lowest Wasserstein distance on 4/5, lowest correlation error on 3/5, and 2X faster computation.It conditions continuous flows using tree-based priors, targeting settings where standard diffusion models typically struggle.Source: Summary of Contributions · p. 2, §1 · p. 2
  • DSM-TREE distills complete hierarchical decision logic into neural networks, matching teacher performance within 2% on most benchmarks and exceeding it by 3.7% on Heart Disease.The method transfers full tree hierarchies rather than only leaf predictions.Source: Summary of Contributions · p. 2, §1 · p. 2

2 The Tree–Flow Correspondence

The section establishes a two-way correspondence: decision trees converge under dyadic refinement to deterministic PF-ODEs, while entropically homogeneous stationary SDEs induce hierarchical trees through moment-based mergers.

2 The Tree–Flow Correspondence

  • Decision-tree coarse-graining forms a discrete-time Markov chain whose entropy decreases from the uniform root toward the data distribution at the leaves.The conditional densities are piecewise constant and evolve by conditional-expectation transition operators.Source: §2 · p. 3
  • Under scale-consistent, locally fine dyadic refinement, the tree density path converges along a subsequence to a continuous-time process with a time-invariant generator.The limiting generator is locally Lipschitz on compact intervals.Source: §2 · p. 3
  • The continuous-time tree limit is a deterministic PF-ODE, because higher-order jump moments vanish and the coarse-graining operator contributes no diffusion.The resulting dynamics are ẋ = v(x, t), with velocity uniquely determined by the partitions.Source: §2 · p. 4

2.1 From Flows to Trees

  • Moment-based cluster mergers are irreversible because clusters that become statistically indistinguishable remain merged, producing a dendrogram whose hierarchy depends only on the SDE dynamics and initialization.Stationarity is stated to be necessary and sufficient for the tree to be rooted.Source: §2.1 · p. 5
  • An entropically homogeneous stationary SDE with well-separated initial modes induces a unique hierarchical clustering through moment-based merger times, fully characterized by its PF-ODE.The merger times satisfy an ultrametric inequality, making the resulting hierarchy well-defined.Source: §2.1 · pp. 4–5

3 Global Trajectory Score Matching

Global Trajectory Score Matching decomposes intractable path-space optimization into local score-matching objectives, unifying diffusion training and gradient boosting. Under stated regularity and richness conditions, zero continuous loss ensures full path matching, while greedy boosting is globally optimal for the discrete objective.

3 Global Trajectory Score Matching

  • GTSM replaces intractable global trajectory optimization with an integral of local score-matching errors across states, times, and infinitesimal trajectory steps.End-to-end score training minimizes the integral jointly, whereas boosting greedily reduces its largest remaining errors.Source: §3 · p. 5

3.1 The Continuous GTSM Objective

  • Zero continuous GTSM loss under any strictly positive time weighting is necessary and sufficient for matching the full path-space measures.The equivalence follows because path-space KL divergence reduces to the score-based loss through the reverse-time drift.Source: §3.1 · pp. 5–6

3.2 Boosting as a CGTSM Solver

  • In the continuous limit with sufficiently expressive weak learners, greedy boosting is globally optimal for the discrete GTSM problem.Net trees convert additive learner updates into monotonic structural refinements, whose Tree-to-Flow processes follow a discrete GTSM trajectory.Source: §3.2 · p. 6

4 Algorithmic Instantiations

This section introduces TREEFLOW and DSM-TREE, two GTSM-inspired algorithms that use hierarchical tree structure for tabular generation and neural distillation. Together, they provide competitive performance with computational advantages while transferring tree logic into differentiable models.

4 Algorithmic Instantiations

  • The two algorithms directly approximate the Global Trajectory Score Matching objective for tabular data, combining hierarchical structure with competitive performance and computational advantages.TREEFLOW targets generative modeling, whereas DSM-TREE performs discriminative distillation into neural networks.Source: §4 · p. 6

4.1 TREEFLOW: Tree-Conditioned Flow Matching

  • TREEFLOW achieves state-of-the-art fidelity with the lowest Wasserstein distance on 4/5 benchmarks and a 2× speedup over TabDDPM.It conditions Conditional Flow Matching on decision-tree path encodings, enabling partition-targeted generation and specialized flows for different data regions.Source: §4.1 · p. 7

4.2 DSM-TREE: Distilling Trees via Hierarchical Score Matching

  • DSM-TREE distills an entire decision trajectory into a neural network by supervising predictions at every hierarchy level rather than only leaf outputs.Each internal split provides a directional guide, allowing the network to learn the tree’s coarse-graining flow while retaining its tabular-data inductive bias.Source: §4.2 · pp. 7–8
  • DSM-TREE is supported by a finite-sample convergence guarantee, with asymptotic path reproduction when per-level decision errors are reduced.Its training follows score matching by design, and inference traverses the teacher tree using the network’s predicted decision at each level.Source: §4.2 · p. 8

5 Experiments

Experiments support the Tree-Flow correspondence by finding hierarchical structure and analogous information decay in diffusion models, while TREEFLOW and DSM-TREE demonstrate practical utility on tabular generation and distillation.

5.1 Verifying the Tree-Flow Equivalence

  • Trained diffusion models encode implicit hierarchical structure: time-domain clustering of forward SDE trajectories recovers dendrograms whose merger times represent cluster indistinguishability.The learned reverse PF-ODE shows clusters beginning to overlap at an internal node time, consistent with the recovered hierarchy.Source: §5.1 · p. 9, Figure 2 · p. 9
  • Decision trees and diffusion processes exhibit near-identical sigmoidal entropy increases, with tree prototypes progressively blurring toward the root as diffusion noise destroys digit structure.The comparison uses weighted-average class entropy by tree depth and an information-theoretic proxy for forward diffusion.Source: §5.1 · p. 9, Figure 3 · p. 10

5.2 Algorithmic Instantiations for Tabular Data

  • DSM-TREE matches teacher performance within 2% on 4/5 datasets and outperforms by 3.7% on Heart Disease while transferring complete hierarchical decision logic into neural networks.It distills the decision path of a Base Tree trained on RandomForest soft labels at every level.Source: §5.2 · p. 10
  • TREEFLOW leads multiple tabular-data metrics—highest TSTR accuracy on 3/5 benchmarks, lowest Wasserstein distance on 4/5, lowest correlation error on 3/5—and runs 2X faster than TabDDPM.The results aggregate five runs across utility, fidelity, structure, and efficiency axes.Source: §5.2 · p. 9, Figure 4 · p. 11

6 Related Work

This work connects diffusion-model theory, tree-model interpretations, and hybrid tabular architectures through a global architectural perspective. It establishes local-to-global score consistency, a formal tree-to-flow correspondence, and two complementary integration strategies: DSM-TREE and TREEFLOW.

Theoretical Understanding of Diffusion Models

  • GTSM equates full path-space matching with an integral of local score-matching errors under standard regularity conditions, making infinitesimal correctness sufficient for global correctness.CGTSM serves as a master objective for analyzing practical training schemes as principled approximations.Source: §6 · p. 11

Different Views of Trees

  • A fixed learned tree hierarchy can reduce in the limit to a deterministic PF-ODE flow, formally linking tree partitioning with continuous dynamics.This bridges deterministic boosting views and probabilistic tree-structure models such as Mondrian processes.Source: §6 · p. 11

Hybrid and Generative Models for Tabular Data

  • DSM-TREE distills complete tree hierarchies into differentiable neural networks, while TREEFLOW conditions flow matching on tree-based priors to inject hierarchical structure into generative models.DSM-TREE matches or exceeds its teacher on 4/5 datasets, including a +3.7% improvement on Heart Disease.Source: §6 · p. 12, Figure 5 · p. 12

7 Limitations, Future Work and Conclusion · A Notation

The paper unifies hierarchical decision trees and diffusion flows through GTSM, yielding TREEFLOW and DSM-TREE while acknowledging theoretical limits and future extensions. Appendix A summarizes the mathematical notation used in the theoretical appendices.

7 Limitations, Future Work and Conclusion

  • The theoretical framework depends on continuous-path refinement and smoothness assumptions, and evaluation focuses on continuous feature spaces to preserve theory–experiment alignment.The algorithmic approach is described as more general than the evaluated setting.Source: §7 · p. 12
  • Future work includes extending the theory to intrinsic discontinuities via Lévy processes or rough-path theory and developing hybrid foundation models for heterogeneous tabular and sequential data.Such models would combine trees’ adaptive partitioning with diffusion models’ expressive power to address multimodal irregularities.Source: §7 · p. 12
  • The framework formally unifies hierarchical decision trees and diffusion process flows through Global Trajectory Score Matching (GTSM).It connects discrete hierarchical structure with continuous dynamics.Source: §7 · p. 12
  • Gradient boosting is proved globally optimal as a greedy solver for the discrete GTSM objective, linking stage-wise construction to continuous score-based dynamics.Source: §7 · p. 12
  • TREEFLOW conditions diffusion flows on tree-based priors for strong generation quality and fidelity with improved speed, while DSM-TREE distills complete tree hierarchies into neural networks.These algorithms operationalize the proposed connection between trees and diffusion models.Source: §7 · p. 12
  • The work enables hybrid models that combine discrete hierarchical structures with continuous dynamics.Source: §7 · p. 12

A Notation

  • Appendix A provides tables summarizing the key mathematical notation used throughout the theoretical appendices.The notation summary spans the main table and its continuation.Source: A Notation · p. 18, Table 1 · p. 18

B A Visual Roadmap of the Theoretical Derivations … C.3 The Tree Hierarchy as a Homogeneous Markov Process

The appendix provides visual roadmaps for the paper’s theoretical derivations and formally develops the probabilistic, Markov, and tree-hierarchy foundations of the continuous-time flow. It maps tree levels to probabilistic states and identifies coarse-graining transitions as a homogeneous Markov process.

B A Visual Roadmap of the Theoretical Derivations

C Detailed Derivation of the Continuous-Time Flow

C.1 Probabilistic Formulation and Markov Process Foundations

  • The probabilistic formulation defines partitions of the feature space and associates each partition level with a joint density p(x, k).Here, k labels the partition level, while p(x, k) denotes the density conditional on that level.Source: §C.1 · p. 22

C.1.1 Foundations of Markov Processes

  • Markov-process foundations require future states to depend only on the present, with transition densities satisfying Chapman-Kolmogorov consistency.Time homogeneity makes transitions depend only on Δt and corresponds to a step-independent operator Mk; continuous paths enable the drift-diffusion SDE limit.Source: §C.1.1 · pp. 22–23

C.2 Formalizing the Tree Hierarchy as a Sequence of Probabilistic States

  • A tree’s nested levels induce partition states that progress from a structured, low-entropy leaf model to a maximum-entropy uniform root distribution.Each level coarsens the preceding partition, forming a trajectory of information loss and increasing entropy.Source: §C.2 · p. 23

C.3 The Tree Hierarchy as a Homogeneous Markov Process

  • Coarse-graining must erase finer partition boundaries while conserving probability mass within every region of the coarser partition.These requirements are expressed through measurability with respect to the coarser sigma-algebra and partial averages over its sets.Source: §C.3 · p. 23
  • The coarse-graining operator Mk is uniquely determined as the conditional expectation with respect to the coarser sigma-algebra.The resulting density sequence is a Markov chain because each state is a deterministic function of only its predecessor.Source: §C.3 · pp. 23–24

C.3.1 Emergence of Homogeneity via Subsequential Limits … D.1.2 Discretizing the Flow via the Characteristic Function

This section develops the continuous-time diffusion interpretation of hierarchical coarse-graining, characterizing subsequential limits, their generators, and associated reverse hierarchies. It also establishes the resulting merger structure and its dendrogram-like organization under stated regularity conditions.

C.3.1 Emergence of Homogeneity via Subsequential Limits

  • Although finite-data decision-tree operators are inherently inhomogeneous, dyadic refinement reduces operator step size and produces subsequential limits with locally Lipschitz generators and time-invariant subsequential limits.The construction uses compactification and diagonal subsequence arguments rather than assuming homogeneity at the discrete level.Source: §C.3.1 · pp. 24–25
  • The limiting process is governed by an effective time-invariant generator, with propagator error O(2^-n); balanced trees improve this approximation, while imbalanced trees retain smooth time dependence.Time invariance is asymptotic rather than exact, and the framework extends to time-dependent generators for imbalanced trees.Source: §C.3.1 · p. 26

C.4 The Continuous-Time Limit and its Governing Equations

  • The continuous-time limit is described by a Kramers–Moyal evolution equation derived from the Chapman–Kolmogorov equation using propagator moments.The derivation uses a displacement change of variables, Taylor expansion, and small-∆t approximation.Source: §C.4 · p. 26

C.4.1 Truncation of the Kramers-Moyal Expansion

  • Continuous-path Markov processes have all propagator moments D(n) identically zero for n > 2, so Pawula’s theorem yields exact truncation after the second Kramers–Moyal term.The refined coarse-graining limit is established as a continuous-path Markov process with an effective time-invariant generator.Source: §C.4.1 · pp. 27–28
  • Higher-order propagator moments vanish in the refinement limit, so the limiting Markov process has continuous paths and the Kramers–Moyal expansion truncates after second order.Time-invariance makes this path-continuity property uniform over x and independent of time.Source: §C.4.1 · p. 29

C.4.2 The Fokker-Planck Equation and Equivalent SDE

  • The continuous-time hierarchical process satisfies a Fokker–Planck equation formally equivalent to an Itô SDE with time-invariant drift and diffusion terms.Because coarse-graining uses deterministic conditional expectation, the actual Tree-to-Flow correspondence is a zero-diffusion Probability Flow ODE, while the general SDE form supports broader analysis.Source: §C.4.2 · p. 30

D The Implicit Hierarchical Structure of Continuous-Time Diffusion

  • The reverse construction requires diffusion convergence to a unique maximally entropic stationary distribution to produce a single rooted tree, and the Ornstein–Uhlenbeck process satisfies this structure.The induced hierarchy is determined by marginal densities rather than stochastic dynamics.Source: D The Implicit Hierarchical Structure… · p. 31

D.1 The General Construction: From Homogeneous SDEs to Dendrograms

  • Any well-behaved homogeneous SDE applied to clustered data induces a canonical hierarchy by tracking the temporal flow of entropy.The hierarchy depends only on marginal densities, so the SDE and its corresponding Probability Flow ODE induce the same structure.Source: §D.1 · p. 31, D The Implicit Hierarchical Structure… · p. 31

D.1.1 Coarse-Graining as a Flow of Entropy

  • For homogeneous diffusions with strictly positive-definite diffusion tensors, entropy is non-decreasing and strictly increases from any non-stationary distribution until stationarity, yielding forward information loss and reverse information gain.The entropy evolution depends only on marginals and therefore applies equally to the SDE and its Probability Flow ODE.Source: §D.1.1 · pp. 31–32

D.1.2 Discretizing the Flow via the Characteristic Function

  • Under smooth-density and isolated-mode assumptions, a suitable super-level set yields exactly K well-separated initial clusters before any mergers occur.For non-smooth densities, spatial partitions can initialize clusters, provided the conditional laws have the required finite moments.Source: §D.1.2 · pp. 32–33
  • Moment-based merger time is the first time conditional moment tensors fall within threshold ϵ; under the moment condition, this implies total-variation convergence and satisfies an ultrametric inequality.Because merger times depend only on conditional marginals, they are identical for an SDE and its Probability Flow ODE.Source: §D.1.2 · pp. 33–34

D.1.3 Main Construction Theorem … E.3.3 The Boosting Update as an Optimal Score Matching Step

This section develops the mathematical correspondence between diffusion processes and canonical decision-tree hierarchies, and formalizes gradient boosting as a coarse-to-fine trajectory through SDE tail-equivalence classes. It establishes the optimality of boosting’s residual update as a meta-level score-matching step and its equivalence with structurally supervised trajectory refinement under the coarse-to-fine learning assumption.

D.1.3 Main Construction Theorem

  • Entropically homogeneous diffusion processes induce a unique canonical hierarchical clustering, equivalent to a dendrogram, via ordered moment-based merger events.The construction uses temporal agglomerative clustering, and the ordered merger times define an ultrametric isomorphic to a dendrogram.Source: §D.1.3 · p. 34

D.2 The Role of Stationarity in Defining the Hierarchy’s Structure

  • Stationarity controls hierarchy structure: non-stationary diffusions yield infinitely deep, continuously evolving hierarchies, whereas stationary processes yield asymptotically stable hierarchies within a finite time horizon.Multiple stationary endpoints can instead produce a forest of disconnected dendrograms, and the finite-time tree remains threshold-dependent as ϵ →0.Source: §D.2 · pp. 34–35

D.3 The Condition for a Single, Rooted Decision Tree

  • Convergence to a unique maximally entropic stationary distribution produces a single rooted tree, and high-dimensional OU or VP diffusion satisfies this condition through its Gaussian stationary law.For d ≥5, N(0, I) is treated as effectively uniform on a sphere and therefore maximally entropic; reverse-time endpoint partitioning provides a practical construction method.Source: §D.3 · pp. 35–36

E Derivation of the Relationship between Boosting and Score Matching

  • The appendix argues that idealized gradient boosting is an optimal score-driven method for constructing coarse-to-fine SDE trajectories through residual fitting and Bellman-optimal stagewise updates.Its four-part derivation connects boosting, net-tree refinement, discrete flow trajectories, denoising score matching, and global trajectory optimality.Source: E Derivation of the Relationship… · p. 36

E.1.1 Gradient Boosting as Functional Gradient Descent

  • Gradient boosting minimizes a loss by iteratively fitting weak learners to negative functional-gradient pseudo-residuals rather than solving for the optimal function in one step.Each weak learner is added to the current model as an approximation to the steepest-descent direction in function space.Source: §E.1.1 · pp. 36–37

E.1.2 The Net Decision Tree: A Monolithic Abstraction

  • The net decision tree represents the ensemble as one piecewise-constant model whose partition is the common refinement of all weak-learners’ partitions and whose leaf values sum their predictions.This monolithic abstraction enables geometric analysis of the ensemble’s evolving decision boundaries.Source: §E.1.2 · p. 37

E.1.3 Monotonic Structural Refinement

  • Boosting generates a nested hierarchy in which each partition is a strict refinement of its predecessor, provided the current model is not yet optimal.The new weak learner’s nontrivial partition subdivides at least one existing region.Source: §E.1.3 · pp. 37–38

E.1.4 The Canonical Hierarchy and the Net Decision Tree

  • The boosting history selects a unique canonical hierarchy whose earlier structure is preserved, with each update refining only the current leaf partition.Reversing the temporally ordered refinement sequence defines the unique coarsening path from leaves to root.Source: §E.1.4 · p. 38

E.2 The Boosting Process as a Trajectory in SDE Space

  • Applying the tree-to-SDE mapping to each boosting tree reframes the boosting sequence as a structured trajectory through SDE space.Finer partitions capture lower-entropy information that diffusion processes lose as they approach the high-entropy stationary distribution.Source: §E.2 · p. 39

E.2.1 Formalizing Temporal Coarseness via Tail Equivalence

  • Tail-equivalence classes form a strictly nested backward filtration, providing the formal structure for defining coarse-to-fine SDE trajectories.Two SDEs are t-tail-equivalent when their laws agree on the future σ-algebra F≥t; decreasing t moves into smaller, finer classes.Source: §E.2.1 · p. 40

E.2.2 The Boosting Process as a Coarse-to-Fine Trajectory

  • Theorem E.13 proves that gradient boosting generates a coarse-to-fine SDE trajectory, with progressively finer partitions corresponding to strictly earlier approximation times.Each refinement captures more detail about the data distribution and extends faithful approximation of the ideal process to an earlier time t_m.Source: §E.2.2 · p. 40

E.3 Score Matching in the Space of SDEs: The Local Update Rule

  • The local update problem is interpreted as denoising score matching in SDE space, with process supervision and data supervision providing equivalent navigation perspectives.The framework distinguishes oracle-prescribed structural trajectories from residual-driven constructive trajectories and proves their equivalence under the central learning assumption.Source: §E.3 · p. 41, §E.3.2 · pp. 41–42

E.3.1 Formal Definitions of Score Matching

  • Score matching models gradients of log densities, while denoising score matching trains on known conditional noise scores to estimate noisy-data scores.The conditional score is tractable because the noise distribution p(˜x|x) is known.Source: §E.3.1 · p. 41

E.3.2 The Duality of Supervision in Trajectory Refinement

  • Under Assumption E.14, optimal data-supervised boosting and structurally supervised refinement produce identical model trajectories.The equivalence follows because residual-based updates optimally resolve the coarsest remaining temporal interval and match the ideal law on the next tail field.Source: §E.3.2 · p. 42

E.3.3 The Boosting Update as an Optimal Score Matching Step

  • For squared error, the residual is proportional to the optimal meta-score, making each boosting update a denoising score-matching step from S_m to S_{m+1}.The conditional Gaussian likelihood yields a meta-score proportional to r_{m+1} = y − F_m(x), which the weak learner fits.Source: §E.3.3 · p. 43

E.3.4 Connecting the Meta-Score to the SDE Score Error · E.4 Global Optimality of the Greedy Trajectory · E.4.1 The Global Trajectory Score Matching (GTSM) Objective

The analysis links boosting residuals to integrated SDE score error, then proves greedy refinement is globally optimal under a trajectory-level objective. It defines DGTSM as cumulative denoising score-matching loss and shows stage-wise boosting minimizes it.

E.3.4 Connecting the Meta-Score to the SDE Score Error

  • The correspondence is established by relating reverse-time SDE endpoint differences to integrated score-dependent drifts after forward drift cancellation.The proof starts both reverse processes from the same terminal point and uses the martingale property of Ito stochastic integrals.Source: §E.3.4 · pp. 43–44
  • The proof assumes a fixed known forward drift and diffusion process, while learning the score function that drives reverse-time generation.For t ≥ tm, the model approximates the ideal law and scores, making the score-error contribution negligible there.Source: §E.3.4 · p. 44
  • The expected boosting residual is an asymptotically unbiased estimator of integrated SDE score error, with bias vanishing as sample size grows and KL divergence decreases.The integrated error is effectively supported only where the model dynamics are invalid, t < tm.Source: §E.3.4 · pp. 44–45
  • The meta-score equals integrated SDE score error exactly when the model is realizable, the data distribution is matched, and boosting has converged.In practice, approximation quality depends on sample size N and residual variance at step m.Source: §E.3.4 · p. 45

E.4 Global Optimality of the Greedy Trajectory

  • The greedy, stage-wise boosting algorithm is provably globally optimal for the total trajectory objective via dynamic programming and the Bellman principle.This resolves the gap between local optimality at each boosting step and optimality of the complete SDE trajectory.Source: §E.4 · p. 46

E.4.1 The Global Trajectory Score Matching (GTSM) Objective

  • DGTSM sums denoising score-matching losses across the entire boosting trajectory, measuring cumulative squared error in following the optimal meta-score at each refinement stage.The weak learner hm+1 is the refinement operator selected at step m.Source: §E.4.1 · p. 46
  • Because the optimal squared-loss meta-score is proportional to the residual, DGTSM equals the sum of weak-learner squared residual-fitting errors minimized by stage-wise boosting.Thus the global trajectory objective is exactly aligned with the boosting algorithm’s optimization target.Source: §E.4.1 · p. 46

E.4.2 Provable Optimality via the Bellman Principle · F Implications of the GTSM Framework: A Unifying View of Score-Based Objectives · F.1 The Continuous GTSM Objective and Its Equivalence to Path Matching

The paper proves that greedy gradient boosting is globally optimal for the discrete DGTSM objective and frames it as a discrete counterpart to a continuous trajectory-matching framework. The continuous GTSM objective is equivalent to full path matching under Girsanov’s theorem and the Novikov regularity condition.

E.4.2 Provable Optimality via the Bellman Principle

  • The boosting process is formulated as a finite-horizon deterministic decision problem whose states are canonical-tree SDEs, actions are weak learners, transitions are deterministic, and costs are DSM losses.This formulation enables dynamic-programming analysis of the DGTSM objective.Source: §E.4.2 · pp. 46–47
  • Greedy gradient boosting is globally optimal for DGTSM, making its residual-fitting updates a principled solution rather than a heuristic.The proof uses Bellman’s principle and backward induction to establish global optimality over the full coarse-to-fine trajectory.Source: §E.4.2 · pp. 47–48
  • The optimal weak-learner sequence uniquely determines the optimal net decision tree and its corresponding canonical SDE, so the optimal policy is the optimal SDE.This identifies the discrete optimization policy directly with the resulting continuous-time model.Source: §E.4.2 · p. 48

F Implications of the GTSM Framework: A Unifying View of Score-Based Objectives

  • The GTSM framework unifies score-based training methods through one global objective, with discrete boosting summing local refinement errors and continuous CGTSM integrating score error over SDE time.Simple and weighted diffusion losses and consistency models are presented as approximations or special cases of this master objective.Source: F Implications of the GTSM Framework:… · p. 48

F.1 The Continuous GTSM Objective and Its Equivalence to Path Matching

  • CGTSM is defined as the integrated Fisher divergence between a model score and the ideal score across the process’s entire time horizon.The objective weights score discrepancies using the diffusion tensor’s induced Mahalanobis norm.Source: §F.1 · p. 48
  • Zero CGTSM loss with any strictly positive weighting w(t) > 0 holds if and only if the induced model path-space measure equals the ideal path-space measure.Girsanov’s theorem connects the path-space KL divergence to the same integrated squared score error minimized by CGTSM.Source: §F.1 · p. 49
  • The equivalence relies on Novikov’s condition, which holds for standard finite-dimensional diffusion models and for tree-derived flows through dyadic refinement that produces a continuous-path limit.Irreducible finite jumps fall outside the framework and would require a Lévy-process formalism.Source: §F.1 · p. 49

F.2 Deriving Standard Training Objectives as Special Cases of the CGTSM … G.4 Connection to CGTSM Framework

The CGTSM framework unifies standard diffusion objectives as principled approximations with distinct trajectory-level biases, and DSM-TREE instantiates the framework by distilling hierarchical tree decisions into neural networks. DSM-TREE is shown to be the uniformly weighted, discrete-time CGTSM objective over tree-depth levels.

F.2 Deriving Standard Training Objectives as Special Cases of the CGTSM

  • CGTSM provides a common language for deriving practical training objectives as approximations that prioritize tail-equivalence classes differently along the learning trajectory.The framework treats each objective as a distinct inductive bias rather than an ad-hoc heuristic.Source: §F.2 · p. 49
  • The simple diffusion loss uses uniform weighting and unbiased Monte Carlo sampling, while weighted diffusion losses set w(t)=λ(t) to prioritize selected trajectory regions.Larger weights at small t can counterbalance a learner’s tendency to learn coarse, large-t classes first and emphasize fine-grained details.Source: §F.2 · pp. 49–50
  • Consistency distillation approximates CGTSM by coupling adjacent tail-equivalence classes, prioritizing trajectory smoothness over pointwise score accuracy to enable larger inference jumps.The loss relates student and teacher predictions at adjacent points on probability-flow ODE trajectories.Source: §F.2 · p. 50

G DSM-TREE: Discretized Score Matching for Decision Trees

  • DSM-TREE distills a decision tree’s coarse-to-fine hierarchy into a continuous, differentiable neural classifier that preserves its partition structure and decision boundaries.It treats tree depth levels as a discrete trajectory for score matching.Source: G DSM-TREE: Discretized Score Matching… · pp. 50–51

G.1 The Tree Distillation Problem

  • Tree distillation must preserve nested partitions and discrete decision paths, so DSM-TREE trains a conditional network Mθ(x,j) to predict decisions at each tree level.This avoids learning only final outputs, which would discard the tree’s hierarchical structure and per-sample paths.Source: §G.1 · p. 51

G.2 Algorithm Overview

  • DSM-TREE builds a teacher tree, trains the conditional student on uniformly sampled depth levels, and recursively uses predicted decisions to mimic tree traversal at inference.The network learns intermediate decisions rather than directly mapping inputs to final outputs.Source: §G.2 · p. 52

G.3 Formal Problem Setup

  • Its formal objective applies cross-entropy to the optimal binary decision at uniformly sampled levels while masking samples that reached leaves before the sampled level.The masking indicator excludes undefined decisions after traversal terminates.Source: §G.3 · p. 52

G.4 Connection to CGTSM Framework

  • Theorem G.3 identifies DSM-TREE exactly as CGTSM with uniform weighting and time discretized to tree depth, because binary decision classification encodes the score’s directional information.Uniform sampling over levels is the Monte Carlo counterpart of summing the discrete trajectory, with leaf masking ensuring well-defined training.Source: §G.4 · p. 53

G.5 Convergence Analysis … H.1 Algorithm Overview

The paper establishes finite-sample convergence and path-wise consistency for DSM-TREE under realizability, bounded-tree, and Lipschitz-loss assumptions, while characterizing its training complexity. It then introduces TREEFLOW, which conditions flow matching on decision-tree paths to generate samples respecting learned partition structure.

G.5 Convergence Analysis

  • The convergence guarantee assumes realizability, finite tree depth D with L = O(2^D) leaves, and a G-Lipschitz cross-entropy loss.The analysis also uses the joint feature-and-level hypothesis class and uniform-convergence bounds over all tree levels.Source: §G.5 · p. 54
  • DSM-TREE converges under Assumption G.4, with a rate combining optimization and generalization errors whose depth dependence is linear and information-theoretically optimal.The bound is obtained after T gradient-descent steps with η = O(1/…), and the generalization analysis accounts for all D tree levels.Source: §G.5 · pp. 54–55
  • As T →∞ with BT/N →c > 0, DSM-TREE decision paths converge almost surely to the base tree’s ground-truth paths.The result follows because zero cross-entropy is equivalent to matching the true decision at every feature and level, which determines the full path.Source: §G.5 · p. 55

G.6 Computational Complexity

  • DSM-TREE training costs O(T · B · D · Cnet), comparable to standard tree training when T · B ≈N and the network hidden dimension satisfies h = O(log D).The complexity includes tree traversal and neural-network forward-backward passes, while retaining a flexible continuous representation of tree structure.Source: §G.6 · p. 56

H TREEFLOW: Conditional Flow Matching with Tree-Structured Paths

  • TREEFLOW uses decision-tree partitions as structured conditioning paths for conditional flow matching, unlike DSM-TREE’s distillation of tree boundaries into neural classifiers.The tree supplies ground-truth hierarchical paths for training generative flow models.Source: H TREEFLOW: Conditional Flow Matching… · p. 57

H.1 Algorithm Overview

  • TREEFLOW trains a decision tree, encodes each sample’s leaf path, and conditions a velocity field vθ(x, t, p, y) on that path and label.The model learns conditional velocities along linear interpolation paths using flow-matching regression.Source: §H.1 · p. 57
  • For generation, TREEFLOW selects a reference path from a target partition and integrates the tree-conditioned velocity field from Gaussian noise to produce synthetic samples matching the real partition structure.Generation uses the target label, desired partition, encoded reference path, and numerical integration over t ∈ [0, 1].Source: §H.1 · p. 57

H.2 Convergence Analysis for TREEFLOW … I.1 Details for Verifying the Equivalence (Experiments 1 & 2)

The appendix establishes TREEFLOW’s finite-sample and distributional convergence, characterizes its computational cost, and formalizes its equivalence to tree-weighted CGTSM. It also records the experimental implementation context used to verify these claims.

H.2 Convergence Analysis for TREEFLOW

  • The convergence analysis assumes bounded data support, balanced tree partitions, and deterministic bounded-norm path encodings, with optimization governed by standard smooth-loss SGD convergence.The hypothesis class decomposes into leaf-specific subclasses, making tree complexity and the number of leaves central to generalization.Source: §H.2 · pp. 58–59
  • TREEFLOW’s generated distribution converges in Wasserstein-2 distance to the training-data conditional distribution within each leaf partition as training increases.The guarantee preserves hierarchical structure rather than only matching the overall marginal distribution.Source: §H.2 · pp. 59–60

H.3 Computational Complexity

  • TREEFLOW training costs O(S · B · (L + D · Cnet)), combining path encoding with velocity-network forward and backward passes.Compared with standard diffusion training, path encoding adds O(L) per-batch overhead that is typically negligible for reasonably sized trees.Source: §H.3 · p. 60

H.4 Unified View: DSM-TREE and TREEFLOW as GTSM Instantiations

  • DSM-TREE and TREEFLOW are CGTSM solutions for discriminative and generative modeling, respectively, using hierarchical trajectories through decision-tree structure.DSM-TREE models decision-boundary trajectories, whereas TREEFLOW models distributional trajectories measured by Wasserstein distance.Source: §H.4 · pp. 60–61
  • The unified framework distinguishes supervision sources: DSM-TREE uses discrete tree decisions, while TREEFLOW uses conditional flow matching along path-conditioned interpolation trajectories.Both discretize the continuous CGTSM trajectory into tractable objectives using hierarchical decision-tree structure.Source: §H.4 · p. 61
  • TREEFLOW’s conditional flow-matching objective is equivalent to CGTSM with tree-structured, path-dependent weighting induced by leaf probabilities.Path conditioning decomposes the objective by leaf and assigns weight w_ℓ = P(X ∈ R_ℓ) to each partition.Source: §H.4 · pp. 61–62

I.1 Details for Verifying the Equivalence (Experiments 1 & 2)

  • The experimental appendix used Python, PyTorch, Scikit-learn, and a single NVIDIA A100 GPU for the reported experiments.The cited passage describes implementation and hardware context but does not provide separate results for Experiments 1 and 2.Source: I Experimental Results · p. 62

I.1.1 Implicit Tree Structure Discovery (Experiment 1) … Methodology

The experiments examine how diffusion dynamics reveal hierarchical structure, how information decays relative to decision trees, and whether tree logic can be transferred to neural models or used for conditional generation. Across these studies, learned dynamics support hierarchy discovery, tree entropy provides a benchmark for diffusion entropy, DSM-TREE learns differentiable tree decisions, and TREEFLOW conditions generation on tree paths.

I.1.1 Implicit Tree Structure Discovery (Experiment 1)

  • The study uses three standardized synthetic 2D datasets, each generated with 3200 samples, to provide clear, visualizable cluster structures comprising four-corner, 3 × 3 grid, and circular eight-cluster arrangements.The datasets contain 4 clusters centered at (±2, ±2) with standard deviation 0.3, 9 grid clusters with standard deviation 0.25, and 8 circular clusters of radius 2 with standard deviation 0.15.Source: §I.1.1 · p. 62
  • The hierarchy-discovery procedure simulates each ground-truth cluster under the model’s learned forward SDE and agglomeratively merges clusters when centroid distance falls below combined spread.Recorded merge times form a dendrogram, while the analytical forward process is not used for this structural analysis; the reverse PF-ODE is used only for visualization.Source: §I.1.1 · p. 63

I.1.2 Information Decay Analysis (Experiment 2)

  • The information-decay study treats a depth-weighted, normalized decision-tree entropy profile as a benchmark for comparing coarse-to-fine organization with diffusion’s entropy proxy.The diffusion proxy is 1/(1 + SNR(t)), where SNR(t) = αt/(1 − αt), and prototypes visualize tree-node averages alongside forward-diffused samples.Source: §I.1.2 · p. 64
  • The decision-tree benchmark achieves 77.05% accuracy on MNIST, 74.15% on Fashion-MNIST, and 85.35% on USPS, supporting its use as a meaningful hierarchy reference.The tree uses max depth=15, and the datasets include MNIST, Fashion-MNIST, and resized USPS images.Source: §I.1.2 · p. 64

Methodology

  • The algorithmic-instantiation methodology spans tabular datasets and standardized preprocessing, with DSM-TREE and TREEFLOW using neural architectures to emulate tree decisions or condition flow generation.DSM-TREE uses five UCI datasets, while TREEFLOW uses five standard tabular datasets framed as classification tasks.Source: §I.2.1 · p. 64, §I.2.2 · p. 64

I.2.1 DSM-TREE (Experiment 3)

  • DSM-TREE learns hierarchical decision logic with a fully differentiable neural network, achieving comparable and sometimes superior performance to its strong tree-based teacher.The experiment trains an oracle RandomForest, a Base Tree on its pseudo-labels, and a ConditionalSplitModel queried level by level until reaching a leaf.Source: §I.2.1 · p. 64

I.2.2 TREEFLOW (Experiment 4)

  • TREEFLOW uses decision-tree path encodings and class labels to condition a learned velocity field, generating samples by integrating from Gaussian noise.Its conditional MLP is trained with flow-matching MSE and generation uses 50 Euler steps; comparisons include GaussianCopulaSynthesizer, TVAE, CTGANSynthesizer, and TabDDPM.Source: Methodology · p. 65, §I.2.2 · p. 65

Evaluation Metrics

This section defines four metrics for utility, fidelity, structure, and efficiency, then reports TREEFLOW’s aggregated comparison with baselines. Across five runs, TREEFLOW often matches or exceeds leading utility while substantially reducing runtime.

Metrics

  • The evaluation measures utility with TSTR Accuracy, fidelity with Wasserstein Distance, structure with Correlation Error, and efficiency with total Runtime.TSTR uses a 100-estimator RandomForestClassifier trained on synthetic data and evaluated on the real test set; the other metrics compare real and synthetic test data.Source: Evaluation Metrics · p. 65

Results

Related papers

Loading 2605.00414v2…