Robot Planning and Situation Handling with Active Perception

arXiv cs.RO / 5/1/2026

📰 NewsDeveloper Stack & InfrastructureIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper argues that robots struggle with long-term autonomy because real-world environments are dynamic and can produce unforeseen execution-time problems like blocked doors or fallen objects.
It introduces VAP-TAMP, a planning and situation-handling framework that combines active perception with task and motion planning to respond when plans break down.
VAP-TAMP uses action knowledge to query vision-language models for strategic view selection and to assess the current situation during execution.
The framework builds and reasons over scene graphs to integrate task-level decisions with motion planning while handling both self-caused failures and external disturbances.
Experiments in simulation and on a mobile manipulation platform evaluate VAP-TAMP on service-oriented tasks.

Abstract

Current robots are capable of computing plans to accomplish complex tasks. However, real-world environments are inherently open and dynamic, and unforeseen situations frequently arise during plan execution, such as jamming doors and fallen objects on the floor. These situations may result from the robot's own action failures or from external disturbances, such as human activities. Detecting and handling such execution - time situations remains a significant challenge, limiting those robots' ability to achieve long-term autonomy. In this paper, we develop a planning and situation-handling framework, called VAP-TAMP, that enables robots to actively perceive and address unforeseen situations during plan execution. VAP-TAMP leverages action knowledge to strategically prompt vision-language models for active view selection and situation assessment, while constructing and reasoning over scene graphs for integrated task and motion planning. We evaluated VAP-TAMP using service tasks in simulation and on a mobile manipulation platform.