Geometric regularization of autoencoders via observed stochastic dynamics

arXiv cs.LG / 4/20/2026

📰 NewsIdeas & Deep AnalysisModels & Research

Key Points

  • The paper tackles the challenge of learning a reduced simulator for stochastic dynamical systems evolving on an unknown low-dimensional manifold inside a high-dimensional space.
  • It proposes using the ambient covariance to recover coordinate-invariant tangent-space information, then builds two geometric regularizers (tangent-bundle and inverse-consistency) across a three-stage autoencoder/SDE pipeline.
  • The authors introduce a new function-space metric (“ρ-metric”) induced by the penalties, showing it is strictly weaker than the Sobolev H¹ norm while preserving comparable chart-quality generalization rates up to logarithmic factors.
  • A key theoretical contribution is an Ito-based encoder “pullback” target for drift, along with a bias decomposition that explains when and why standard decoder-side drift formulas incur systematic error for imperfect charts.
  • Experiments across multiple embedded surfaces (up to 201 dimensions) show large reductions in radial mean first-passage-time (MFPT) error (50–70%) and improved metastable Muller–Brown transition performance, with up to an order-of-magnitude reduction in ambient coefficient errors versus an unregularized autoencoder.

Abstract

Stochastic dynamical systems with slow or metastable behavior evolve, on long time scales, on an unknown low-dimensional manifold in high-dimensional ambient space. Building a reduced simulator from short-burst ambient ensembles is a long-standing problem: local-chart methods like ATLAS suffer from exponential landmark scaling and per-step reprojection, while autoencoder alternatives leave tangent-bundle geometry poorly constrained, and the errors propagate into the learned drift and diffusion. We observe that the ambient covariance~\Lambda already encodes coordinate-invariant tangent-space information, its range spanning the tangent bundle. Using this, we construct a tangent-bundle penalty and an inverse-consistency penalty for a three-stage pipeline (chart learning, latent drift, latent diffusion) that learns a single nonlinear chart and the latent SDE. The penalties induce a function-space metric, the \rho-metric, strictly weaker than the Sobolev H^1 norm yet achieving the same chart-quality generalization rate up to logarithmic factors. For the drift, we derive an encoder-pullback target via It\^o's formula on the learned encoder and prove a bias decomposition showing the standard decoder-side formula carries systematic error for any imperfect chart. Under a W^{2,\infty} chart-convergence assumption, chart-level error propagates controllably to weak convergence of the ambient dynamics and to convergence of radial mean first-passage times. Experiments on four surfaces embedded in up to 201 ambient dimensions reduce radial MFPT error by 50--70\% under rotation dynamics and achieve the lowest inter-well MFPT error on most surface--transition pairs under metastable M\"uller--Brown Langevin dynamics, while reducing end-to-end ambient coefficient errors by up to an order of magnitude relative to an unregularized autoencoder.