Post-Training Augmentation Invariance

arXiv stat.ML / 4/24/2026

💬 OpinionTools & Practical UsageModels & Research

共有:

Key Points

The paper proposes a framework called post-training augmentation invariance that aims to add invariance to a pretrained network while keeping its behavior on the original (non-augmented) input distribution unchanged.
It introduces “augmented encoders” (probabilistic encoders that formalize augmentation-based encoding) as the core mechanism, and defines two training objectives: Markov-Wasserstein minimization and Wasserstein correlation maximization.
Experiments show that lightweight one-hidden-layer MLP adapter networks trained with these losses can be appended to a frozen pretrained feature extractor to achieve approximate augmentation invariance.
On STL10 using DINOv2 features, adding a trained adapter improves rotated-image classification accuracy to 94% versus 71% without the adapter, and noise-invariant performance rises from 58% to 86%.
The authors report the approach requires no fine-tuning of the pretrained backbone (frozen weights) and minimally perturbs original latent features, while adapters trained with alternative losses (SimCLR, HSIC maximization) fail and can corrupt the latent space.

Abstract

This work develops a framework for post-training augmentation invariance, in which our goal is to add invariance properties to a pretrained network without altering its behavior on the original, non-augmented input distribution. We define this notion precisely and additionally introduce augmented encoders, which are probabilistic encoders that formalize augmentation-based encoding processes and that serve as our fundamental object of study. We introduce two losses for augmented encoders, namely, Markov-Wasserstein minimization and Wasserstein correlation maximization, and we demonstrate empirically that both losses can be used to train lightweight, one-hidden-layer MLP adapter networks E_theta that, when appended to the latent space of a pretrained network F, do indeed lead to (approximate) post-training augmentation invariance. For example, on STL10 with F = DINOv2 features, the composite network C o E_theta o F, where C is a linear classifier and where E_theta is one of our proposed adapter networks, achieves 94% classification accuracy on arbitrarily rotated images, whereas a network of the form C o F without the adapter E_theta drops to 71% accuracy. Similarly, we can boost noise-invariant classification results from 58% up to 86%. Significantly, we obtain these results with no fine-tuning (the weights of F remain frozen throughout), and our methods introduce little corruption to the original features, since E_theta acts nearly isometrically on the non-augmented latent distribution. In contrast, we show that adapter networks trained with alternative candidate losses, specifically SimCLR and HSIC maximization, produce uncompetitive classification results and fundamentally corrupt the original latent space. Code available at: https://github.com/keenan-eikenberry/augmentation_invariance