Adaptive Learned State Estimation based on KalmanNet

arXiv cs.RO / 4/6/2026

📰 NewsSignals & Early TrendsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces Adaptive Multi-modal KalmanNet (AM-KNet), a learned hybrid state estimator designed for autonomous driving on real-world multi-sensor data.
AM-KNet adds sensor-specific measurement modules so the network can learn radar, lidar, and camera noise characteristics independently, improving robustness across sensors.
A hypernetwork with context modulation adapts filtering behavior based on target type, motion state, and relative pose to handle diverse traffic scenarios.
The method improves uncertainty handling by adding a covariance estimation branch using the Josephs form, trained with negative log-likelihood losses on estimation error and innovations.
Experiments on nuScenes and View-of-Delft show better estimation accuracy and tracking stability than the base KalmanNet, reducing the gap versus classical Bayesian filters.

Abstract

Hybrid state estimators that combine model-based Kalman filtering with learned components have shown promise on simulated data, yet their performance on real-world automotive data remains insufficient. In this work we present Adaptive Multi-modal KalmanNet (AM-KNet), an advancement of KalmanNet tailored to the multi-sensor autonomous driving setting. AM-KNet introduces sensor-specific measurement modules that enable the network to learn the distinct noise characteristics of radar, lidar, and camera independently. A hypernetwork with context modulation conditions the filter on target type, motion state, and relative pose, allowing adaptation to diverse traffic scenarios. We further incorporate a covariance estimation branch based on the Josephs form and supervise it through negative log-likelihood losses on both the estimation error and the innovation. A comprehensive, component-wise loss function encodes physical priors on sensor reliability, target class, motion state, and measurement flow consistency. AM-KNet is trained and evaluated on the nuScenes and View-of-Delft datasets. The results demonstrate improved estimation accuracy and tracking stability compared to the base KalmanNet, narrowing the performance gap with classical Bayesian filters on real-world automotive data.