A Representation-Level Assessment of Bias Mitigation in Foundation Models

arXiv cs.CL / 4/13/2026

📰 NewsSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper studies how bias-mitigation techniques alter the embedding space geometry of encoder-only and decoder-only foundation models by analyzing representational changes.
Using BERT and Llama2 as representative architectures, it compares baseline vs bias-mitigated variants to measure shifts in associations between gender and occupation terms.
Results indicate that bias mitigation reduces gender–occupation disparities, yielding more neutral and balanced internal representations across both model types.
The authors argue that these representational shifts are interpretable and can serve as an internal audit mechanism for validating debiasing effectiveness.
To enable broader evaluation of decoder-only models, the paper introduces and publicly releases WinoDec, a dataset of 4,000 sequences containing gender and occupation terms.

Abstract

We investigate how successful bias mitigation reshapes the embedding space of encoder-only and decoder-only foundation models, offering an internal audit of model behaviour through representational analysis. Using BERT and Llama2 as representative architectures, we assess the shifts in associations between gender and occupation terms by comparing baseline and bias-mitigated variants of the models. Our findings show that bias mitigation reduces gender-occupation disparities in the embedding space, leading to more neutral and balanced internal representations. These representational shifts are consistent across both model types, suggesting that fairness improvements can manifest as interpretable and geometric transformations. These results position embedding analysis as a valuable tool for understanding and validating the effectiveness of debiasing methods in foundation models. To further promote the assessment of decoder-only models, we introduce WinoDec, a dataset consisting of 4,000 sequences with gender and occupation terms, and release it to the general public. (https://github.com/winodec/wino-dec)