Trees to Flows and Back: Unifying Decision Trees and Diffusion Models

arXiv cs.AI / 5/4/2026

💬 OpinionIdeas & Deep AnalysisTools & Practical UsageModels & Research

共有:

Key Points

The paper establishes a mathematical correspondence between hierarchical decision trees and diffusion processes under suitable limiting regimes, showing they can be unified despite different data/model structures.
It identifies a shared optimization principle called Global Trajectory Score Matching (GTSM), and argues that (idealized) gradient boosting is asymptotically optimal for it.
The authors demonstrate the ideas with two practical implementations: treeflow for faster, high-fidelity tabular data generation and dsmtree for distilling decision-tree logic into neural networks.
In experiments, treeflow reportedly delivers competitive generation quality with higher fidelity and about a 2× computational speedup, while dsmtree matches teacher performance within ~2% on many benchmarks.
Overall, the work provides both a theoretical bridge and concrete methods that connect discrete decision logic to modern diffusion-based generative modeling.

Abstract

Decision trees and diffusion models are ostensibly disparate model classes, one discrete and hierarchical, the other continuous and dynamic. This work unifies the two by establishing a crisp mathematical correspondence between hierarchical decision trees and diffusion processes in appropriate limiting regimes. Our unification reveals a shared optimization principle: \emph{Global Trajectory Score Matching (GTSM)}, for which gradient boosting (in an idealized version) is asymptotically optimal. We underscore the conceptual value of our work through two key practical instantiations: \treeflow, which achieves competitive generation quality on tabular data with higher fidelity and a 2\times computational speedup, and \dsmtree, a novel distillation method that transfers hierarchical decision logic into neural networks, matching teacher performance within 2\% on many benchmarks.