PaperlayerView exact revision on arXiv

Source-linked AI summary

Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

Sai Niranjan Ramachandran, Suvrit Sra

arXiv:2605.00414v2cs.LGcond-mat.stat-mechcs.AI

TL;DR

Decision trees and diffusion models have lacked a unified account despite both hierarchically refining information. This work establishes their mathematical correspondence through GTSM and develops methods that achieve competitive tabular generation with 2X faster computation and neural distillation within 2% of teacher performance on 4/5 datasets.

  • Problem

    Decision trees and diffusion models appear conceptually separate despite both performing hierarchical refinement of information.

  • Method

    The paper formally maps hierarchical trees and diffusion flows under limiting refinement procedures and introduces GTSM as their shared optimization objective.

  • Results

    TREEFLOW achieves competitive tabular generation while being 2X faster, and DSM-TREE matches teacher performance within 2% on 4/5 datasets.

  • Takeaways & Limitations

    The unified framework supports hybrid models combining discrete hierarchical structures with continuous dynamics.

  • Takeaways & Limitations

    The theory relies on continuous-path refinement processes and smoothness assumptions and is evaluated primarily on continuous feature spaces.

Abstract

from arXiv · show

Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: \emph{Global Trajectory Score Matching (GTSM)}, for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: \treeflow, which achieves competitive generation quality on tabular data with higher fidelity and a 2\times computational speedup, and \dsmtree, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2\% on many benchmarks.

1 Introduction

The paper establishes a formal Tree↔Flow correspondence, showing that hierarchical decision trees and diffusion-associated deterministic flows are two views of the same underlying object. This unification yields Global Trajectory Score Matching (GTSM) as a common optimization principle and motivates algorithms combining tree-based structure with diffusion and neural models.

  • Tree↔Flow correspondence: Hierarchical decision trees and diffusion-associated deterministic flows are formally mathematically correspondent under suitable limiting refinement procedures.The correspondence identifies the two model classes as perspectives on the same underlying generative and discriminative object.
  • Tree↔Flow correspondence: A decision tree’s hierarchical coarse-graining defines a discrete-time Markov process whose continuous limit is a unique PF-ODE-described deterministic flow.Conversely, a suitable diffusion forward process induces a canonical hierarchy by progressively merging data-distribution modes.
  • GTSM: Global Trajectory Score Matching (GTSM) unifies greedy boosting and end-to-end score-based diffusion training as related solvers of a common objective.The paper characterizes gradient boosting and score-based diffusion training as optimal solvers in their respective discrete and continuous domains.
  • Algorithms: The unified perspective motivates TREEFLOW for competitive generation on tabular data and a method that distills complete tree hierarchies into neural networks.These algorithms leverage explicit, adaptive partitioning as a structural prior and combine strengths of tree-based and diffusion or neural frameworks.

2 The Tree–Flow Correspondence

The section establishes a bidirectional correspondence between decision trees and diffusion processes through continuous-time limits and moment-based clustering. Under dyadic refinement, trees induce deterministic PF-ODEs, while entropically homogeneous SDEs induce uniquely characterized hierarchical clusterings.

  • The Tree as a Discrete-Time Markov Process: A decision tree defines a discrete-time Markov chain whose conditional densities are piecewise constant on partition cells.The transition is p(x, k + 1) = M_k p(x, k) = E[p(x, k) | F_k+1].
  • The Tree as a Discrete-Time Markov Process: The chain traces monotonically decreasing entropy from the uniform root distribution toward the data distribution at the leaves.The endpoints are p(x, 0) = Unif(X) and p(x, D) ≈ p_data(x).
  • The Continuous-Time Limit: Under scale-consistent, locally refined dyadic interpolation, the tree density path admits a continuous-time generator, with a convergent subsequence yielding a time-invariant generator.The limiting generator is locally Lipschitz on compact intervals.
  • From Discrete Steps to Differential Equations: Higher-order jump moments vanish under refinement, and deterministic coarse-graining eliminates the diffusion term, so the limit is a first-order Liouville equation with characteristic PF-ODE ẋ = v(x, t).Theorem 2.5 identifies the continuous-time limit as deterministic rather than stochastic.
  • Reverse Mapping: SDEs to Trees: An entropically homogeneous SDE with well-separated initial modes induces a unique hierarchical clustering through moment-based merger times, fully characterized by its corresponding PF-ODE.The merger times satisfy an ultrametric inequality, and stationarity is necessary and sufficient for the tree to be rooted.

3 Global Trajectory Score Matching

Global Trajectory Score Matching (GTSM) decomposes intractable path-space matching into local score-matching objectives whose zero loss exactly characterizes full trajectory matching. In the continuous weak-learner limit, greedy boosting minimizes the resulting discrete objective globally optimally.

  • Global Trajectory Score Matching: GTSM replaces one intractable global path-space objective with an infinite sum of local score-matching losses over states, times, and infinitesimal trajectory steps.The local score specifies the optimal direction for the next infinitesimal reverse-process step.
  • Continuous GTSM: Zero CGTSM loss under any strictly positive weighting w(t) > 0 is necessary and sufficient for matching the full path-space measures, Pθ = P∗.The result follows by relating path-space KL divergence to the squared difference between process drifts, which reduces to the CGTSM integrand.
  • Discrete GTSM for Boosting: Net decision trees convert boosting’s additive learner sequence into structural refinements, with partitions formed by common refinement and leaf values given by summed predictions.Applying the Tree-to-Flow mapping yields processes progressing from coarse to fine approximations, guided by a discrete GTSM objective.
  • Discrete GTSM for Boosting: In the continuous limit with sufficiently rich weak learners, greedy boosting that minimizes each immediate stage cost is globally optimal for the discrete GTSM problem.The discrete objective is a sum of stage-wise score-matching costs, and the residual is an unbiased estimator of the optimal score update.

4 Algorithmic Instantiations

This section operationalizes the tree–diffusion unification through two algorithms for tabular data: TREEFLOW for generation and DSM-TREE for discriminative distillation. Both use hierarchical tree structure to provide neural networks with structural inductive biases while approximating Global Trajectory Score Matching.

  • Overview: Two proposed algorithms use hierarchical tree partitions to provide neural networks with structural inductive biases for tabular generation and discriminative distillation.The methods directly approximate the Global Trajectory Score Matching objective.
  • TREEFLOW: 2X faster generation is achieved by TREEFLOW, which conditions Conditional Flow Matching on decision-tree partitions while maintaining competitive generation.TREEFLOW addresses the computational cost of diffusion models for tabular data.
  • TREEFLOW: TREEFLOW assigns each data point a path encoding and trains a velocity field conditioned on these paths, enabling specialized flows and partition-targeted sampling.At generation time, a reference point supplies the conditioning path for moving noise toward synthetic data.
  • DSM-TREE: DSM-TREE distills an entire decision trajectory into a neural network, supervising complete hierarchical logic rather than only leaf predictions.This is presented as the first method to supervise networks on complete hierarchical logic rather than just leaf predictions.
  • DSM-TREE: DSM-TREE trains the network to predict each internal split, converting the tree’s coarse-graining flow into a continuous differentiable function that retains its bias.Every hierarchy level provides a directional guide during training.

5 Experiments

The experiments validate the Tree–Flow correspondence by showing analogous hierarchical structure and information decay in diffusion models and decision trees. TREEFLOW and DSM-TREE demonstrate practical utility for tabular generation and neural-network distillation.

  • Tree–Flow Correspondence: Diffusion models trained on synthetic 2D datasets reveal implicit hierarchical structures through time-domain agglomerative clustering of forward SDE trajectories.The method tracks cluster centroids and statistical spread under score-function-driven SDE evolution to identify merger times.
  • Information-Theoretic Analogy: Decision trees and diffusion processes exhibit near-identical sigmoidal information-decay trajectories, with progressively blurred tree prototypes paralleling noise-destroyed digit prototypes.The comparison uses weighted-average class entropy by tree depth and an information-theoretic proxy for forward diffusion.
  • DSM-TREE for Discriminative Modeling: DSM-TREE distills complete hierarchical decision logic into differentiable neural networks, matching teacher performance within 2% on 4/5 benchmarks.The baseline is a Base Tree trained on RandomForestClassifier oracle soft labels, while DSM-TREE replicates the teacher’s decision path at every level.
  • Summary of Experimental Findings: The experiments validate analogous hierarchical structure and information decay while demonstrating practical impact through competitive tabular generation and hierarchical neural-network distillation.The reported practical results include TREEFLOW’s 2X speedup and DSM-TREE’s matching of teachers within 2% on most benchmarks.
  • TREEFLOW for Generative Modeling: TREEFLOW achieves highest TSTR accuracy on 3/5 benchmarks, lowest Wasserstein distance on 4/5, lowest correlation error on 3/5, and is 2X faster than TabDDPM.Evaluation covers Utility, Fidelity, Structure, and Efficiency against GaussianCopula, TVAE, CTGAN, and TabDDPM.

6 Related Work

This work connects diffusion-model theory, tree-model interpretations, and hybrid architectures through a global architectural perspective. It bridges deterministic tree hierarchies and continuous diffusion flows while distilling complete tree structure into differentiable neural networks.

  • Theoretical Understanding of Diffusion Models: Diffusion-model theory commonly studies trajectory stability, dynamical regimes, or learned score functions, whereas this work provides a global architectural interpretation.The supplied passage contrasts existing statistical-physics and differential-geometric analyses with the paper’s architectural perspective.
  • Different Views of Trees: Existing tree theories treat boosting as functional gradient descent or model random tree structures probabilistically; this work bridges them through a fixed learned hierarchy reduced to a PF-ODE flow.The reduction holds in the limit and yields a continuous deterministic flow governed by a PF-ODE.
  • Hybrid and Generative Models for Tabular Data: +3.7% improvement on Heart Disease shows DSM-TREE can transfer hierarchical structure while matching or exceeding its teacher on 4/5 datasets.The comparison is against the Base Tree teacher.
  • Hybrid and Generative Models for Tabular Data: DSM-TREE distills complete tree hierarchies, rather than only final predictions, into differentiable neural networks while preserving interpretable structure.The approach enables end-to-end learning while retaining hierarchical interpretability.

7 Limitations, Future Work and Conclusion · A Notation

The paper unifies hierarchical decision trees and diffusion process flows through Global Trajectory Score Matching, while identifying smoothness-based limitations and introducing two theory-driven algorithms. The appendix separately summarizes the mathematical notation used throughout the theoretical appendices.

  • 7 Limitations, Future Work and Conclusion: The theoretical framework relies on continuous-path refinement processes and smoothness assumptions that may not always apply.Although the algorithmic approach is more general, evaluation focuses on continuous feature spaces to preserve theory–experiment alignment.
  • 7 Limitations, Future Work and Conclusion: Future work could extend the theory to intrinsic discontinuities using Lévy processes or rough-path theory.These approaches are proposed to formalize broader settings beyond the current smooth continuous-path assumptions.
  • 7 Limitations, Future Work and Conclusion: The framework establishes a formal correspondence between hierarchical decision trees and diffusion process flows within GTSM.It connects the two model classes through a shared optimization framework.
  • 7 Limitations, Future Work and Conclusion: Gradient boosting is proved to be a globally optimal greedy solver for a discrete GTSM formulation.Its stage-wise construction is connected to continuous score-based dynamics.
  • 7 Limitations, Future Work and Conclusion: The work introduces TREEFLOW and DSMTREE as theory-driven algorithms leveraging the tree–diffusion correspondence.The supplied conclusion identifies both as novel algorithms, though the passage truncates their detailed results.
  • A Notation: The theoretical appendices include a summary of the key mathematical notations used throughout.This notation overview is presented before the notation table.
  • A Notation: Table 1 provides the summary of notation and continues in a subsequent table.The supplied appendix contains entries labeled “Summary of Notation” and “(continued).”

B A Visual Roadmap of the Theoretical Derivations … C.3 The Tree Hierarchy as a Homogeneous Markov Process

The appendix first visually organizes the paper’s theoretical arguments, then formalizes the tree hierarchy as a probabilistic process whose coarse-graining transitions form a homogeneous Markov chain. The derivation proceeds from partitions and Markov foundations through increasing-entropy states to a uniquely defined conditional-expectation operator.

  • B A Visual Roadmap of the Theoretical Derivations: The roadmap flowcharts summarize the paper’s main derivations, including Tree-to-Flow, Flow-to-Tree, GTSM boosting, and the DSM-TREE and TREEFLOW constructions.They are intended as a conceptual scaffold before the full mathematical proofs.
  • C Detailed Derivation of the Continuous-Time Flow: The appendix supplies formal proofs and derivations for the results presented in Section 2, beginning with the probabilistic objects underlying the continuous-time flow.The setup defines partitions of a feature space and a joint density associated with each partition level.
  • C.1 Probabilistic Formulation and Markov Process Foundations: A partition is a collection of nonempty, mutually disjoint regions covering the feature space, while p(x, k) denotes the density associated with partition level k.The level index k labels the partition and is not itself a random variable.
  • C.1.1 Foundations of Markov Processes: A Markov process makes the future conditionally independent of the past given the present, and its transition densities obey the Chapman-Kolmogorov consistency condition.The Chapman-Kolmogorov equation provides the starting point for deriving the differential evolution of the process.
  • C.1.1 Foundations of Markov Processes: A homogeneous Markov process depends on elapsed time rather than absolute time, corresponding here to a transition operator M_k independent of step k.Continuous sample paths are required for the continuous-time limit to reduce to a drift-diffusion SDE.
  • C.2 Formalizing the Tree Hierarchy as a Sequence of Probabilistic States: Tree levels induce nested, progressively coarser partitions that map to a sequence of probabilistic states forming a trajectory of information loss.The process begins with a structured, low-entropy leaf model and ends at the uniform, maximum-entropy distribution over the feature space.
  • C.3 The Tree Hierarchy as a Homogeneous Markov Process: Each tree transition erases boundary information while conserving probability mass within regions of the coarser partition.These requirements are expressed as measurability with respect to the coarser sigma-algebra and partial-average conservation.
  • C.3 The Tree Hierarchy as a Homogeneous Markov Process: The transition operator is uniquely the conditional expectation with respect to the coarser sigma-algebra, so the density sequence forms a memoryless Markov chain.The current state is a deterministic function of the preceding state through the uniquely defined operator M_k.

C.3.1 Emergence of Homogeneity via Subsequential Limits … D.1.2 Discretizing the Flow via the Characteristic Function

The paper shows that refined decision-tree processes approach effective time-homogeneous continuous dynamics, yielding an exact drift-diffusion formulation. Conversely, homogeneous diffusions induce hierarchical structures through monotone entropy flow and statistically defined merger events.

  • C.3.1 Emergence of Homogeneity via Subsequential Limits: Finite-data decision trees are inherently inhomogeneous, but dyadic refinement yields subsequences whose generators converge locally to a time-invariant generator.The effective time-invariance is approximate, with propagator error O(2^-n); balanced trees improve the approximation.
  • C.4 The Continuous-Time Limit and its Governing Equations: The continuous-time limit is governed by the Kramers-Moyal expansion derived from the propagator density and Chapman-Kolmogorov equation.Taylor expansion, small-time expansion, and identification of propagator moments produce the governing evolution equation.
  • C.4.1 Truncation of the Kramers-Moyal Expansion: Continuous paths force all propagator moments D(n) for n > 2 to vanish, so Pawula’s theorem makes the Kramers-Moyal expansion terminate exactly at second order.The refined coarse-graining limit is proved to be a continuous-path Markov process, establishing the required condition.
  • C.4.2 The Fokker-Planck Equation and Equivalent SDE: The hierarchical coarse-graining limit is equivalent to both a Fokker-Planck equation and an Itô SDE with time-invariant drift and diffusion terms.The drift and diffusion correspond to the first and second propagator moments, while the construction also yields a probability-flow interpretation.
  • D The Implicit Hierarchical Structure of Continuous-Time Diffusion: The reverse direction maps continuous-time diffusion processes to canonical decision-tree-like hierarchies by tracking entropy flow and convergence behavior.The framework analyzes dendrogram construction, hierarchy depth and stability, and stationary-distribution conditions.
  • D.1.1 Coarse-Graining as a Flow of Entropy: For homogeneous diffusions with strictly positive-definite diffusion tensor, entropy is non-decreasing and strictly increases from any non-stationary initial distribution.The same marginal-entropy result applies to the SDE and its corresponding probability-flow ODE; reverse time reverses the entropy direction.
  • D.1.2 Discretizing the Flow via the Characteristic Function: Characteristic-function signatures and moment convergence define discrete merger events, producing well-separated initial clusters and distributionally meaningful, ultrametric merger times.Merger times coincide for an SDE and its probability-flow ODE and serve as proxies for convergence of the underlying conditional distributions.

D.1.3 Main Construction Theorem … E.3.3 The Boosting Update as an Optimal Score Matching Step

The paper establishes that suitable diffusion processes induce canonical dendrograms or rooted trees, while gradient boosting constructs a monotonic coarse-to-fine hierarchy whose updates are optimal meta-level score-matching steps. Stationarity determines whether the hierarchy is finite or continuously evolving, and the OU/VP setting yields a canonical rooted-tree correspondence.

  • D.1.3 Main Construction Theorem: Entropically homogeneous diffusion processes induce a unique canonical hierarchy equivalent to a dendrogram through ordered moment-based merger events.The construction uses temporal agglomerative clustering of merger times and yields an ultrametric isomorphic to a dendrogram.
  • D.2 The Role of Stationarity in Defining the Hierarchy’s Structure: Non-stationary processes induce infinitely deep, continuously evolving hierarchies, whereas stationary processes produce asymptotically stable hierarchies with mergers within a finite time horizon.Multiple stationary distributions can instead yield a forest of disconnected dendrograms, and finite-time depth remains threshold-dependent as distinguishability ϵ approaches zero.
  • D.3 The Condition for a Single, Rooted Decision Tree: A unique maximally entropic stationary distribution makes the induced hierarchy a single rooted tree; in high dimensions d ≥5, OU/VP diffusion satisfies this condition via N(0, I).The OU process has a unique Gaussian stationary distribution, which is treated as maximally entropic in high dimensions and acts as the geometric root.
  • D.3 The Condition for a Single, Rooted Decision Tree: The framework maps diffusion marginals and PF-ODEs to canonical hierarchies, establishing the bidirectional correspondence Tree ↔ PF-ODE and enabling empirical construction from reverse-time ODE endpoints.The terminal-noise partitioning procedure is presented as a practical algorithm for constructing the proven canonical decision-tree hierarchy.
  • E.1 Formalizing Gradient Boosting and the Net Decision Tree: Gradient boosting is formalized as stage-wise functional optimization whose weak learners approximate negative functional gradients, while the net decision tree represents the ensemble as one evolving partitioned function.The net tree uses the common refinement of constituent learner partitions and sums their predictions on each leaf, providing a monolithic geometric representation.
  • E.1.3 Monotonic Structural Refinement / E.1.4 The Canonical Hierarchy and the Net Decision Tree: Each boosting step strictly refines the feature-space partition, preserves the previous canonical hierarchy, and changes structure exclusively by refining the prior leaves.The historical sequence of nested partitions fixes a unique hierarchy for every ensemble, with the earlier tree embedded immutably in the next.
  • E.2 The Boosting Process as a Trajectory in SDE Space / E.2.1 Formalizing Temporal Coarseness via Tail Equivalence / E.2.2 The Boosting Process as a Coarse-to-Fine Trajectory: Mapping each net tree to an SDE produces a structured coarse-to-fine trajectory through SDE space, formalized by strictly nested tail-equivalence classes and decreasing associated times.Finer models approximate the ideal process from earlier times because they preserve more early-time information about the initial data distribution.
  • E.3 Score Matching in the Space of SDEs: The Local Update Rule / E.3.2 The Duality of Supervision in Trajectory Refinement / E.3.3 The Boosting Update as an Optimal Score Matching Step: For squared error, the residual rm+1 = y −Fm(x) is proportional to the optimal meta-score, so fitting the next weak learner implements a denoising score-matching update from Sm to Sm+1.The paper connects data-supervised residual fitting with structurally unsupervised trajectory refinement through functional equivalence of the learning paradigms.

E.3.4 Connecting the Meta-Score to the SDE Score Error · E.4 Global Optimality of the Greedy Trajectory · E.4.1 The Global Trajectory Score Matching (GTSM) Objective

The paper identifies the boosting residual with the integrated score error of the corresponding SDE, establishing asymptotic and conditional exact equivalence. It then formalizes the full boosting trajectory through the DGTSM objective and proves that greedy stage-wise refinement is globally optimal.

  • E.3.4 Connecting the Meta-Score to the SDE Score Error: The expected boosting residual is proportional to the integrated discrepancy between the ideal and current SDE score functions over the model-invalid temporal region.The model-score error is effectively supported only for t < t_m because the approximation error is negligible for t ≥ t_m.
  • E.3.4 Connecting the Meta-Score to the SDE Score Error: The expected residual is an asymptotically unbiased estimator of the integrated score error as N →∞ and the model-distribution mismatch vanishes.Approximation quality is controlled by sample size and residual variance.
  • E.4 Global Optimality of the Greedy Trajectory: Local optimality of each boosting step does not alone guarantee a globally optimal SDE trajectory, because greedy choices can produce a suboptimal overall path.The paper frames this gap as the distinction between myopic stage-wise decisions and trajectory-level optimality.
  • E.4 Global Optimality of the Greedy Trajectory: Dynamic programming and the Bellman principle establish that the greedy stage-wise algorithm is the provably optimal solution to a single global trajectory objective.The entire boosting process is formalized as a sequential decision problem.
  • E.4.1 The Global Trajectory Score Matching (GTSM) Objective: The DGTSM objective sums the individual Denoising Score Matching losses across the entire boosting trajectory.It measures the total squared error in following the optimal meta-score at every refinement stage.
  • E.4.1 The Global Trajectory Score Matching (GTSM) Objective: For squared error, DGTSM is equivalent to minimizing the sum of weak-learner squared errors against stepwise residuals, exactly matching the stage-wise boosting objective.The weak learner h_m+1 is the refinement operator selected at step m.

E.4.2 Provable Optimality via the Bellman Principle · F Implications of the GTSM Framework: A Unifying View of Score-Based Objectives · F.1 The Continuous GTSM Objective and Its Equivalence to Path Matching

The framework casts boosting as a finite-horizon deterministic sequential decision problem and proves that greedy gradient boosting is globally optimal for the DGTSM objective. It further defines CGTSM, establishes equivalence between zero score-matching loss and identical path-space measures, and identifies conditions supporting this equivalence for diffusion- and tree-derived flows.

  • E.4.2 Provable Optimality via the Bellman Principle: Boosting is formulated as a finite-horizon, deterministic sequential decision problem over states, weak-learner actions, deterministic transitions, and DSM stage costs.The state is a canonical-tree-representable SDE, and the stage cost is E[∥h_m+1 − r_m+1∥2].
  • E.4.2 Provable Optimality via the Bellman Principle: Bellman’s principle and backward induction show that the greedy policy minimizing each immediate residual loss is globally optimal over the full action space.The proof first analyzes finite ε-nets of weak learners and then takes ε_m →0.
  • E.4.2 Provable Optimality via the Bellman Principle: Gradient boosting directly implements this optimal greedy policy, while the resulting optimal weak-learner sequence uniquely defines the optimal net decision tree and corresponding SDE.The result characterizes the local update rule as a principled solution to a global score-matching problem over a coarse-to-fine trajectory.
  • F Implications of the GTSM Framework: A Unifying View of Score-Based Objectives: The GTSM framework proposes one global objective underlying multiple score-based training methods, including simple and weighted diffusion losses and consistency models.These methods are described as principled approximations or special cases of the master objective.
  • F.1 The Continuous GTSM Objective and Its Equivalence to Path Matching: CGTSM is defined as the integrated Fisher divergence between a model score and the ideal score across the process’s entire time horizon.Its connection to path-space KL divergence is established using Girsanov’s theorem.
  • F.1 The Continuous GTSM Objective and Its Equivalence to Path Matching: Zero CGTSM loss with any strictly positive weighting w(t) > 0 holds if and only if the induced model and ideal path-space measures are identical.The equivalence follows because the GTSM loss and reverse-time path-space KL divergence are both integrals of squared score error.
  • F.1 The Continuous GTSM Objective and Its Equivalence to Path Matching: The Novikov condition is a mild regularity assumption satisfied by standard diffusion models and tree-derived flows through finite-horizon regularity and dyadic refinement.For tree-derived flows, dyadic refinement ensures a continuous-path limit and well-behaved induced drift.

F.2 Deriving Standard Training Objectives as Special Cases of the CGTSM … G.4 Connection to CGTSM Framework

The paper presents CGTSM as a unifying objective from which standard diffusion losses arise as principled, differently biased approximations, and develops DSM-TREE as its discrete decision-tree instantiation. DSM-TREE distills hierarchical tree decisions into a differentiable neural classifier, with its masked, uniformly sampled training objective exactly corresponding to discrete-time CGTSM.

  • F.2 Deriving Standard Training Objectives as Special Cases of the CGTSM: CGTSM derives standard training objectives as principled approximations, each encoding a distinct inductive bias about the learning trajectory.The framework describes these biases through an objective integrating over tail-equivalence classes.
  • F.2 Deriving Standard Training Objectives as Special Cases of the CGTSM: The weighted diffusion loss is an unbiased Monte Carlo estimator of CGTSM with w(t) = λ(t), prioritizing selected tail-equivalence classes.Choosing larger λ(t) for small t can counterbalance a learner’s tendency to learn coarse classes at large t first.
  • G DSM-TREE: Discretized Score Matching for Decision Trees: DSM-TREE distills a decision tree into a neural network that preserves its decision boundaries while providing continuous, differentiable representations.Its central correspondence treats the tree’s coarse-to-fine hierarchy as score matching over discrete depth levels.
  • G.1 The Tree Distillation Problem: DSM-TREE addresses lost hierarchy and decision-path information by conditioning a network on each tree level and predicting the corresponding intermediate decision.This explicitly models the hierarchical trajectory rather than learning only final tree outputs.
  • G.2 Algorithm Overview: DSM-TREE trains a conditional model Mθ(x, j) and then traverses the teacher tree level by level, using predicted decisions to navigate from the root.The algorithm comprises base-tree generation, conditional score-model training, and neural-network inference that mimics tree traversal.
  • G.3 Formal Problem Setup: The DSM-TREE objective samples tree levels uniformly and applies cross-entropy only when a sample has not already reached a leaf.The indicator function masks leaf-reached samples during training.
  • G.4 Connection to CGTSM Framework: Theorem G.3 identifies DSM-TREE as the discrete-time CGTSM objective with uniform weighting w(t) = 1 and time discretized to tree depth levels.Tree depth induces a discrete sequence corresponding to the coarse-graining trajectory, with leaves at t0 = 0 and the root at tD = T.
  • G.4 Connection to CGTSM Framework: The binary tree decision is the discrete analogue of score directional information, while continuous squared score error corresponds to classification error in predicting that decision.With uniform level sampling and leaf masking, LDSM(θ) is exactly the discrete-time, uniformly weighted CGTSM objective.

G.5 Convergence Analysis … H.1 Algorithm Overview

The paper establishes finite-sample convergence and path-wise consistency for DSM-TREE under realizability, regularity, and bounded-tree assumptions, with training cost scaling linearly in tree depth. TREEFLOW then uses tree partitions as conditioning paths for conditional flow matching, enabling partition-aware generation.

  • G.5 Convergence Analysis: DSM-TREE’s convergence analysis assumes realizability of the true decision function, finite tree depth D, bounded leaves L = O(2^D), and G-Lipschitz cross-entropy loss.These conditions define the setting for the finite-sample guarantees.
  • G.5 Convergence Analysis: Under Assumption G.4, Theorem G.5 provides a finite-sample convergence guarantee for parameters obtained after T gradient-descent steps with learning rate η = O(1/
  • G.5 Convergence Analysis: The convergence proof decomposes excess risk into optimization and generalization errors and bounds generalization through the joint hypothesis class over features and tree levels.The analysis accounts for the discrete level variable and the tree’s structured hypothesis class.
  • G.5 Convergence Analysis: As T → ∞ with BT/N → c > 0, DSM-TREE’s decision paths converge to the true paths, because zero cross-entropy is equivalent to exact decision matching at every level.The path is determined by the sequence of level-wise decisions.
  • G.6 Computational Complexity: DSM-TREE training has complexity O(T · B · D · Cnet), where Cnet is the cost of a neural-network forward-backward pass.Each step samples data and levels, traverses the teacher tree, and performs neural-network computation.
  • H TREEFLOW: Conditional Flow Matching with Tree-Structured Paths: TREEFLOW uses decision-tree partitions as structured conditioning paths for conditional flow matching, unlike DSM-TREE’s distillation of tree boundaries into neural classifiers.
  • H.1 Algorithm Overview: TREEFLOW trains a decision tree to depth D, encodes each sample’s partition path p_i, and conditions a velocity field v_θ(x, t, p, y) on that path and label.The path encoding identifies the leaf partition and path to which each sample belongs.
  • H.1 Algorithm Overview: For generation, TREEFLOW selects a reference from a target partition, uses its encoded path to condition velocity-field integration from Gaussian noise, and outputs synthetic data matching real partition structure.The method extends conditional flow matching by using tree paths to guide partition-specific flows.

H.2 Convergence Analysis for TREEFLOW … I.1 Details for Verifying the Equivalence (Experiments 1 & 2)

The appendix establishes TREEFLOW’s finite-sample and distributional convergence, characterizes its training complexity, and unifies TREEFLOW with DSM-TREE through CGTSM. It also documents the experimental implementation used to verify the framework.

  • H.2 Convergence Analysis for TREEFLOW: The convergence analysis links optimization, generalization, and tree complexity, with larger leaf counts improving partition expressiveness but requiring more training data.The analysis uses bounded-support data, bounded path encodings, balanced trees, and standard SGD and Rademacher-complexity arguments.
  • H.2 Convergence Analysis for TREEFLOW: TREEFLOW converges in Wasserstein-2 distance to the conditional data distribution within each tree leaf as training steps increase.The guarantee is stated as lim S→∞ W2(pTREEFLOW(x|pℓ, y), pdata(x|X ∈Rℓ, y)) = 0.
  • H.3 Computational Complexity: TREEFLOW training costs O(S · B · (L + D · Cnet)), adding path-encoding overhead to standard diffusion-model network computation.The path-encoding overhead is described as O(L) per batch and is typically negligible relative to Cnet for reasonably sized trees.
  • H.4 Unified View: DSM-TREE and TREEFLOW as GTSM Instantiations: Both DSM-TREE and TREEFLOW solve CGTSM under different settings: discriminative boundary trajectories for classification and generative distribution trajectories for Wasserstein-error minimization.Both methods model decision-tree hierarchy as trajectories through tail-equivalence classes or path-conditioned flow trajectories.
  • H.4 Unified View: DSM-TREE and TREEFLOW as GTSM Instantiations: The two algorithms differ in supervision: DSM-TREE uses discrete tree decisions, whereas TREEFLOW uses conditional flow matching with linear interpolation paths and tree-path conditioning.This shared trajectory principle discretizes CGTSM into computationally tractable objectives using hierarchical tree structure.
  • H.4 Unified View: DSM-TREE and TREEFLOW as GTSM Instantiations: TREEFLOW’s conditional flow-matching objective is equivalent to CGTSM with tree-structured, path-dependent weighting over leaf partitions.Path conditioning decomposes the objective by leaf and assigns each partition weight wℓ= P(X ∈Rℓ).
  • I Experimental Results / I.1 Details for Verifying the Equivalence (Experiments 1 & 2): The experimental appendix reports Python implementations using PyTorch for neural networks and Scikit-learn for tree models and data processing, with all experiments run on one NVIDIA A100 GPU.These implementation details support the experimental setups, architectures, and training procedures for the reported results.

I.1.1 Implicit Tree Structure Discovery (Experiment 1) … Methodology

The experiments examine how diffusion dynamics reveal hierarchical structure and how tree logic can be transferred into differentiable models and generative systems. Across synthetic, image, and tabular settings, the methodology combines learned-SDE hierarchy discovery, entropy analysis, DSM-TREE distillation, and TREEFLOW modeling.

  • Methodology: The methodology specifies neural architectures, optimization schedules, and data transformations for diffusion, DSM-TREE, and TREEFLOW experiments.Examples include 400 diffusion epochs, 30,000 DSM-TREE training steps, and 1000 TREEFLOW steps, with standardized inputs where stated.
  • I.1.1 Implicit Tree Structure Discovery (Experiment 1): The hierarchy-discovery procedure performs agglomerative clustering in time by simulating each ground-truth cluster through the learned forward SDE.Euler-Maruyama integration advances clusters from t = 0 to t = T, with drift determined by the model’s score function rather than the analytical training process.
  • I.1.2 Information Decay Analysis (Experiment 2): Experiment 2 evaluates information decay on MNIST, Fashion-MNIST, and resized 28 × 28 USPS images using tree entropy and a diffusion entropy proxy.Tree entropy is computed from weighted node-level Shannon entropy, while the diffusion proxy is 1/(1 + SNR(t)).
  • I.1.2 Information Decay Analysis (Experiment 2): 77.05% MNIST, 74.15% Fashion-MNIST, and 85.35% USPS test accuracy are achieved by max-depth-15 decision trees.The reported accuracies substantially exceed the 10% random-chance reference, supporting the tree’s discriminative performance.
  • I.2 Details for Algorithmic Instantiations (Experiments 3 & 4): The algorithmic instantiations use teacher-tree supervision for DSM-TREE and tree-path conditioning for TREEFLOW across standardized tabular classification datasets.DSM-TREE is trained to reproduce hierarchical split decisions, whereas TREEFLOW uses encoded decision paths as model inputs.
  • I.2.1 DSM-TREE (Experiment 3): DSM-TREE trains a differentiable neural network to simulate traversal through a strong tree-based model and achieves comparable or sometimes superior performance.Inference queries the model iteratively from level j = 0 until a leaf is reached.
  • I.2.2 TREEFLOW (Experiment 4): TREEFLOW is evaluated against GaussianCopulaSynthesizer, TVAE, CTGANSynthesizer, and TabDDPM on five standardized tabular datasets.Its conditional MLP receives x, t, p, and y, uses a depth-10 decision-tree path encoder, and outputs a velocity.

Evaluation Metrics

The evaluation uses utility, fidelity, structure, and efficiency metrics, with aggregated means and standard deviations across five runs. TREEFLOW demonstrates a strong utility–efficiency trade-off, often matching or exceeding leading TSTR Accuracy while running more than twice as fast as other diffusion-based methods.

  • Evaluation Metrics: TSTR Accuracy measures utility by training a 100-estimator RandomForestClassifier on synthetic data and evaluating it on the real test set.Wasserstein Distance, Correlation Error, and Runtime measure fidelity, structure, and efficiency, respectively.
  • Evaluation Metrics: Wasserstein Distance is the average 1-D marginal distance between real and fake test data, while Correlation Error is the Frobenius norm between their correlation matrices.Runtime is total training time in seconds.
  • Evaluation Metrics: 5 runs provide the aggregated mean and standard deviation of all evaluation metrics in Table 3.TSTR Accuracy is maximized, while Wasserstein Distance, Correlation Error, and Runtime are minimized.
  • Evaluation Metrics: More than twice as fast, TREEFLOW often matches or exceeds the best-performing models in TSTR Accuracy against other diffusion-based methods.The reported comparison highlights a superior trade-off between utility and efficiency.
Loading 2605.00414v2…