Task Tokens: A Flexible Approach to Adapting Behavior Foundation Models

arXiv cs.RO / 3/30/2026

💬 OpinionIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces “Task Tokens,” a method for adapting transformer-based behavior foundation models (BFMs) to specific control tasks without sacrificing their zero-shot flexibility.
Task Tokens learn a task-specific encoder via reinforcement learning while freezing the original BFM, injecting task-relevant information as additional tokens into the model’s input stream.
The approach is designed to balance reward design with prompt engineering by allowing user-defined priors to influence task adaptation more directly.
Experiments across multiple tasks—including out-of-distribution settings—show improved performance while maintaining generalization characteristics, and the method remains compatible with other prompting modalities.

Abstract

Recent advancements in imitation learning have led to transformer-based behavior foundation models (BFMs) that enable multi-modal, human-like control for humanoid agents. While excelling at zero-shot generation of robust behaviors, BFMs often require meticulous prompt engineering for specific tasks, potentially yielding suboptimal results. We introduce "Task Tokens", a method to effectively tailor BFMs to specific tasks while preserving their flexibility. Our approach leverages the transformer architecture of BFMs to learn a new task-specific encoder through reinforcement learning, keeping the original BFM frozen. This allows incorporation of user-defined priors, balancing reward design and prompt engineering. By training a task encoder to map observations to tokens, used as additional BFM inputs, we guide performance improvement while maintaining the model's diverse control characteristics. We demonstrate Task Tokens' efficacy across various tasks, including out-of-distribution scenarios, and show their compatibility with other prompting modalities. Our results suggest that Task Tokens offer a promising approach for adapting BFMs to specific control tasks while retaining their generalization capabilities.