Meta Additive Model: Interpretable Sparse Learning With Auto Weighting

arXiv cs.LG / 4/23/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces a new Meta Additive Model (MAM) that improves sparse additive modeling by learning how to reweight individual loss terms based on data rather than relying on fixed choices.
Existing sparse additive models often optimize under mean-squared error and can degrade under complex, non-Gaussian noise such as outliers, noisy labels, and class imbalance; MAM targets these failure modes.
MAM uses a bilevel optimization setup where an MLP parameterizes the loss-weighting function using meta data, enabling robust learning across multiple task types.
The authors provide theoretical guarantees covering convergence, algorithmic generalization, and consistency of variable selection under mild assumptions.
Experiments show MAM outperforms several state-of-the-art additive models on both synthetic and real datasets across different data corruption scenarios.

Abstract

Sparse additive models have attracted much attention in high-dimensional data analysis due to their flexible representation and strong interpretability. However, most existing models are limited to single-level learning under the mean-squared error criterion, whose empirical performance can degrade significantly in the presence of complex noise, such as non-Gaussian perturbations, outliers, noisy labels, and imbalanced categories. The sample reweighting strategy is widely used to reduce the model's sensitivity to atypical data; however, it typically requires prespecifying the weighting functions and manually selecting additional hyperparameters. To address this issue, we propose a new meta additive model (MAM) based on the bilevel optimization framework, which learns data-driven weighting of individual losses by parameterizing the weighting function via an MLP trained on meta data. MAM is capable of a variety of learning tasks, including variable selection, robust regression estimation, and imbalanced classification. Theoretically, MAM provides guarantees on convergence in computation, algorithmic generalization, and variable selection consistency under mild conditions. Empirically, MAM outperforms several state-of-the-art additive models on both synthetic and real-world data under various data corruptions.