A Stable Neural Statistical Dependence Estimator for Autoencoder Feature Analysis

arXiv cs.LG / 3/13/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper proposes a stable neural dependence estimator for autoencoder feature analysis using a variational Gaussian formulation to quantify dependence among inputs, latent representations, and reconstructions.
It introduces an orthonormal density-ratio decomposition to improve stability and reduce computational cost relative to methods like MINE.
The method avoids input concatenation and product-of-marginals re-pairing, addressing key scalability and stability issues in dependence estimation.
An efficient NMF-like scalar objective is proposed, and assuming Gaussian noise as an auxiliary variable enables meaningful dependence measurements for feature analysis.
Empirical results show sequential convergence of singular values, supporting the method's effectiveness for quantitative autoencoder feature analysis.

Abstract

Statistical dependence measures like mutual information is ideal for analyzing autoencoders, but it can be ill-posed for deterministic, static, noise-free networks. We adopt the variational (Gaussian) formulation that makes dependence among inputs, latents, and reconstructions measurable, and we propose a stable neural dependence estimator based on an orthonormal density-ratio decomposition. Unlike MINE, our method avoids input concatenation and product-of-marginals re-pairing, reducing computational cost and improving stability. We introduce an efficient NMF-like scalar objective and demonstrate empirically that assuming Gaussian noise to form an auxiliary variable enables meaningful dependence measurements and supports quantitative feature analysis, with a sequential convergence of singular values.