Unlocking Strong Supervision: A Data-Centric Study of General-Purpose Audio Pre-Training Methods

arXiv cs.AI / 3/30/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that audio pre-training is currently limited by weak, noisy, and scale-constrained labels, making progress depend heavily on better supervision than is commonly available.
It proposes a data-centric pipeline that uses a high-fidelity captioner to generate state-of-the-art captions and introduces a Unified Tag System (UTS) intended to connect speech, music, and environmental sounds.
The study runs a systematic comparison of multiple pre-training objectives using the newly created strong source data to understand how objectives affect what downstream tasks the model specializes in.
Results indicate that data quality and coverage are the dominant factors for performance, while the specific training objective mainly determines downstream task specialization.

Abstract

Current audio pre-training seeks to learn unified representations for broad audio understanding tasks, but it remains fragmented and is fundamentally bottlenecked by its reliance on weak, noisy, and scale-limited labels. Drawing lessons from vision's foundational pre-training blueprint, we argue that the audio field must first establish its own large-scale, strong supervision framework. We introduce a new data-centric pipeline that leverages a high-fidelity captioner to create SOTA-quality captions and the first Unified Tag System (UTS) that bridges speech, music, and environmental sounds. We then conduct a systematic comparative study of different pre-training objectives on these strong source data. Our experiments suggest that data quality and coverage are the primary drivers of performance, while the choice of objective dictates downstream task specialization.