How unconstrained machine-learning models learn physical symmetries

arXiv cs.LG / 3/27/2026

💬 OpinionSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper studies how “unconstrained” machine-learning models (not hard-coded to obey physical symmetries) can still learn approximate equivariant behavior using simple data augmentation rather than strict architectural constraints.
It introduces rigorous metrics to quantify the symmetry content of learned representations and to measure how well model outputs satisfy the desired equivariant conditions.
The authors apply these diagnostics to two transformer/point-cloud-style architectures (one for atomistic simulations and one for particle physics) to analyze where and how symmetry information is acquired across network layers.
They propose a framework to diagnose spectral failure modes and show that injecting only the minimal necessary inductive biases can improve both stability and accuracy while retaining unconstrained models’ expressivity and scalability.
Overall, the work provides a methodology for evaluating physical-fidelity risks in ML systems and for guiding architecture/training choices to better preserve symmetry properties.

Abstract

The requirement of generating predictions that exactly fulfill the fundamental symmetry of the corresponding physical quantities has profoundly shaped the development of machine-learning models for physical simulations. In many cases, models are built using constrained mathematical forms that ensure that symmetries are enforced exactly. However, unconstrained models that do not obey rotational symmetries are often found to have competitive performance, and to be able to \emph{learn} to a high level of accuracy an approximate equivariant behavior with a simple data augmentation strategy. In this paper, we introduce rigorous metrics to measure the symmetry content of the learned representations in such models, and assess the accuracy by which the outputs fulfill the equivariant condition. We apply these metrics to two unconstrained, transformer-based models operating on decorated point clouds (a graph neural network for atomistic simulations and a PointNet-style architecture for particle physics) to investigate how symmetry information is processed across architectural layers and is learned during training. Based on these insights, we establish a rigorous framework for diagnosing spectral failure modes in ML models. Enabled by this analysis, we demonstrate that one can achieve superior stability and accuracy by strategically injecting the minimum required inductive biases, preserving the high expressivity and scalability of unconstrained architectures while guaranteeing physical fidelity.