FedUAF: Uncertainty-Aware Fusion with Reliability-Guided Aggregation for Multimodal Federated Sentiment Analysis

arXiv cs.LG / 3/17/2026

📰 NewsModels & Research

共有:

Key Points

FedUAF presents a unified multimodal federated learning framework that tackles missing modalities, non-IID data, and unreliable client updates through uncertainty-aware fusion and reliability-guided aggregation.
The method explicitly models modality-level uncertainty during local training to improve robustness when modalities are incomplete or noisy.
A server-side reliability-guided aggregation strategy down-weights unreliable clients to improve global performance over existing federated baselines.
Experimental results on CMU-MOSI and CMU-MOSEI demonstrate FedUAF outperforms state-of-the-art baselines across various missing-modality patterns and non-IID settings, and shows robustness to noisy clients.
By addressing practical data heterogeneity and client reliability, FedUAF advances real-world multimodal federated learning applications.

Abstract

Multimodal sentiment analysis in federated learning environments faces significant challenges due to missing modalities, heterogeneous data distributions, and unreliable client updates. Existing federated approaches often struggle to maintain robust performance under these practical conditions. In this paper, we propose FedUAF, a unified multimodal federated learning framework that addresses these challenges through uncertainty-aware fusion and reliability-guided aggregation. FedUAF explicitly models modality-level uncertainty during local training and leverages client reliability to guide global aggregation, enabling effective learning under incomplete and noisy multimodal data. Extensive experiments on CMU-MOSI and CMU-MOSEI demonstrate that FedUAF consistently outperforms state-of-the-art federated baselines across various missing-modality patterns and Non-IID settings. Moreover, FedUAF exhibits superior robustness against noisy clients, highlighting its potential for real-world multimodal federated applications.