MOSAIC: Composable Safety Alignment with Modular Control Tokens

arXiv cs.AI / 3/18/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

MOSAIC proposes a modular safety alignment framework built on learnable control tokens that encode individual safety constraints and can be activated and composed at inference time on a frozen backbone model.
It addresses the limitations of static parameter-level safety policies and prompt-based methods by enabling context-dependent safety across users, regions, and applications.
The training uses order-based task sampling and a distribution-level alignment objective to improve efficiency and reduce over-refusal while preserving model utility.
Experiments indicate MOSAIC achieves strong defense performance with substantially lower over-refusal compared to traditional approaches.

Abstract

Safety alignment in large language models (LLMs) is commonly implemented as a single static policy embedded in model parameters. However, real-world deployments often require context-dependent safety rules that vary across users, regions, and applications. Existing approaches struggle to provide such conditional control: parameter-level alignment entangles safety behaviors with general capabilities, while prompt-based methods rely on natural language instructions that provide weak enforcement. We propose MOSAIC, a modular framework that enables compositional safety alignment through learnable control tokens optimized over a frozen backbone model. Each token represents a safety constraint and can be flexibly activated and composed at inference time. To train compositional tokens efficiently, we introduce order-based task sampling and a distribution-level alignment objective that mitigates over-refusal. Experiments show that MOSAIC achieves strong defense performance with substantially lower over-refusal while preserving model utility.