Efficient machine unlearning with minimax optimality

arXiv stat.ML / 4/8/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces a statistical framework for machine unlearning that targets removing specific data subsets without the expense of full model retraining, motivated by GDPR-style compliance and reducing bias/corruption.
It provides theoretical guarantees for generic loss functions and, for squared loss, develops an approach called Unlearning Least Squares (ULS).
The authors prove minimax optimality for parameter estimation of the remaining data under a setting that only allows access to the pre-trained estimator, forget samples, and a small subsample of remaining data.
They show the estimation error splits into an oracle term plus an “unlearning cost” driven by the proportion of data to forget and the bias of the forget model.
Experiments and real-data applications indicate the method can achieve performance close to full retraining while requiring substantially less data access.

Abstract

There is a growing demand for efficient data removal to comply with regulations like the GDPR and to mitigate the influence of biased or corrupted data. This has motivated the field of machine unlearning, which aims to eliminate the influence of specific data subsets without the cost of full retraining. In this work, we propose a statistical framework for machine unlearning with generic loss functions and establish theoretical guarantees. For squared loss, especially, we develop Unlearning Least Squares (ULS) and establish its minimax optimality for estimating the model parameter of remaining data when only the pre-trained estimator, forget samples, and a small subsample of the remaining data are available. Our results reveal that the estimation error decomposes into an oracle term and an unlearning cost determined by the forget proportion and the forget model bias. We further establish asymptotically valid inference procedures without requiring full retraining. Numerical experiments and real-data applications demonstrate that the proposed method achieves performance close to retraining while requiring substantially less data access.