Universality in Deep Neural Networks: An approach via the Lindeberg exchange principle

arXiv stat.ML / 5/5/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper studies fully connected deep neural networks in the infinite-width limit and compares them to an infinite-width Gaussian (random) limit model.
It provides quantitative bounds on the 2-Wasserstein distance between the finite-width network and its Gaussian limit, assuming regularity conditions on the activation function.
The key technical contribution is a “Lindeberg exchange principle” tailored to deep neural networks, enabling a controlled, layer-by-layer replacement of weights with Gaussian random variables.
The results aim to formalize how and how fast deep networks’ distributions converge to Gaussian behavior as width grows.

Abstract

We consider the infinite-width limit of a fully connected deep neural network with general weights, and we prove quantitative general bounds on the

2

-Wasserstein distance between the network and its infinite-width Gaussian limit, under appropriate regularity assumptions on the activation function. Our main tool is a Lindeberg principle for Deep Neural Networks, which we use to successively replace the weights on each layer by Gaussian random variables.