LA-Pose: Latent Action Pretraining Meets Pose Estimation

arXiv cs.CV / 5/1/2026

📰 NewsIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper proposes LA-Pose, a pose estimation approach that leverages self-supervised inverse-dynamics pretraining to avoid reliance on large amounts of fully supervised 3D-labeled data.
LA-Pose learns latent action representations using inverse- and forward-dynamics models, then repurposes those latent features as inputs to a camera pose estimator that is fine-tuned with a small set of high-quality 3D annotations.
The method aims to keep pose prediction accurate and generalizable while preserving feed-forward efficiency during inference.
Experiments on driving benchmarks (including Waymo and PandaSet) show LA-Pose achieves competitive to superior results, with over 10% higher pose accuracy than recent feed-forward methods while using orders of magnitude less labeled data.
The authors claim this work is the first to specifically demonstrate the effectiveness of inverse-dynamics self-supervised learning for pose estimation.

Abstract

This paper revisits camera pose estimation through the lens of self-supervised pretraining, focusing on inverse-dynamics pretraining as a scalable alternative to the current trend of fully supervised training with 3D annotations. Concretely, we employ inverse- and forward-dynamics models to learn latent action representations, similar to Genie from large-scale driving videos. Our idea is simple yet effective. Existing methods use latent actions in their original capacity, that is, as action conditioning of world-models or as proxies of robot action parameters in policy networks. Our method, dubbed LA-Pose, repurposes the latent action features as inputs to a camera pose estimator, finetuned on a limited set of high-quality 3D annotations. This formulation enables accurate and generalizable pose prediction while maintaining feed-forward efficiency. Extensive experiments on driving benchmarks show that LA-Pose achieves competitive and even superior performance to state-of-the-art methods while using orders of magnitude less labeled data. Concretely, on the Waymo and PandaSet benchmarks, LA-Pose achieves over 10% higher pose accuracy than recent feed-forward methods. To our knowledge, this work is the first to demonstrate the power of inverse-dynamics self-supervised learning for pose estimation.