Kernel Density Machines

arXiv stat.ML / 3/27/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces Kernel Density Machines (KDM), a kernel-based framework designed to learn the Radon–Nikodym derivative (probability density between measures) under minimal assumptions.
KDM is formulated for general measurable spaces and avoids structural constraints typical of classical nonparametric density estimators.
The authors provide theoretical guarantees including consistency and a functional central limit theorem for a constructed sample estimator.
For scalability, they develop Nystrom-type low-rank approximations and prove optimal error rates, addressing a previously missing gap in density-learning guarantees.
Experiments and applications show KDM’s versatility for kernel two-sample testing and conditional distribution estimation, including dimension-free guarantees relative to locally smoothed approaches.

Abstract

We introduce kernel density machines (KDM), an agnostic kernel-based framework for learning the Radon-Nikodym derivative (density) between probability measures under minimal assumptions. KDM applies to general measurable spaces and avoids the structural requirements common in classical nonparametric density estimators. We construct a sample estimator and prove its consistency and a functional central limit theorem. To enable scalability, we develop Nystrom-type low-rank approximations and derive optimal error rates, filling a gap in the literature where such guarantees for density learning have been missing. We demonstrate the versatility of KDM through applications to kernel-based two-sample testing and conditional distribution estimation, the latter enjoying dimension-free guarantees beyond those of locally smoothed methods. Experiments on simulated and real data show that KDM is accurate, scalable, and competitive across a range of tasks.