CCAR: Intrinsic Robustness as an Emergent Geometric Property

arXiv cs.LG / 4/21/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that conventional supervised learning optimizes accuracy without controlling the geometry of learned features, which can lead to entangled and brittle representations.
It introduces Class-Conditional Activation Regularization (CCAR), a method that imposes a soft block-diagonal structure so class information is constrained to orthogonal latent subspaces.
The authors provide theoretical analysis showing that this geometric structural constraint relates to maximizing the Fisher Discriminant Ratio, connecting disentanglement to algorithmic stability.
Experiments indicate that CCAR yields robustness as an emergent property of the engineered feature space, outperforming baselines on benchmarks involving label noise and adversarial or corrupted inputs.

Abstract

Standard supervised learning optimizes for predictive accuracy but remains agnostic to the internal geometry of learned features, often yielding representations that are entangled and brittle. We propose Class-Conditional Activation Regularization (CCAR) to explicitly engineer the feature space, imposing a block-diagonal structure via a soft inductive bias. By shaping the latent representation to confine class energy to orthogonal subspaces, we create an intrinsic geometric scaffold that naturally filters noise and adversarial perturbations. We provide theoretical analysis linking this structural constraint to the maximization of the Fisher Discriminant Ratio, establishing a formal connection between geometric disentanglement and algorithmic stability. Empirically, this approach demonstrates that robustness is an emergent property of a well-engineered feature space, significantly outperforming baselines on label noise and input corruption benchmarks.