HAMLET: Switch your Vision-Language-Action Model into a History-Aware Policy

arXiv cs.RO / 4/16/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that robotic vision-language-action (VLA) policies often fail on history-dependent tasks because they ignore past observations and rely only on the current frame.
It introduces HAMLET, a framework that upgrades an existing VLA into a history-aware policy using per-timestep “moment tokens” plus a lightweight memory module that aggregates those tokens across time for action prediction.
The moment tokens are initialized via time-contrastive learning to better encode temporally distinctive perceptual information.
Experiments show large gains on long-horizon, history-dependent real-world tasks (e.g., 76.4% success on GR00T N1.5-based setup, up 47.2% from baseline).
HAMLET also improves prior-art results on RoboCasa Kitchen (64.1% → 66.4% in the 100-demo setup) and LIBERO (95.6% → 97.7%), demonstrating effectiveness across generic robot-manipulation benchmarks.

Abstract

Inherently, robotic manipulation tasks are history-dependent: leveraging past context could be beneficial. However, most existing Vision-Language-Action models (VLAs) have been designed without considering this aspect, i.e., they rely solely on the current observation, ignoring preceding context. In this paper, we propose HAMLET, a scalable framework to adapt VLAs to attend to the historical context during action prediction. Specifically, we introduce moment tokens that compactly encode perceptual information at each timestep. Their representations are initialized with time-contrastive learning, allowing them to better capture temporally distinctive aspects. Next, we employ a lightweight memory module that integrates the moment tokens across past timesteps into memory features, which are then leveraged for action prediction. Through empirical evaluation, we show that HAMLET successfully transforms a state-of-the-art VLA into a history-aware policy, especially demonstrating significant improvements on long-horizon tasks that require historical context. In particular, on top of GR00T N1.5, HAMLET achieves an average success rate of 76.4% on history-dependent real-world tasks, surpassing the baseline performance by 47.2%. Furthermore, HAMLET pushes prior art performance from 64.1% to 66.4% on RoboCasa Kitchen (100-demo setup) and from 95.6% to 97.7% on LIBERO, highlighting its effectiveness even under generic robot-manipulation benchmarks.