Lucid-XR: An Extended-Reality Data Engine for Robotic Manipulation

arXiv cs.CV / 5/4/2026

💬 OpinionDeveloper Stack & InfrastructureIdeas & Deep AnalysisModels & Research

共有:

Key Points

The paper introduces Lucid-XR, a generative, multimodal data engine designed to produce realistic training data for real-world robotic manipulation.
Lucid-XR’s key component, vuer, is a web-based physics simulator that runs on XR headsets to enable low-latency, internet-scale access to immersive virtual interactions.
The system combines on-device physics simulation with human-to-robot pose retargeting, and uses a physics-guided, language-steerable video generation pipeline to further expand the dataset.
The authors report zero-shot transfer, where robot visual policies trained solely on Lucid-XR synthetic data generalize to previously unseen, cluttered, and poorly lit environments.
Demonstrations cover multiple dexterous manipulation scenarios, including tasks involving soft materials, loosely bound particles, and rigid-body contacts.

Abstract

We introduce Lucid-XR, a generative data engine for creating diverse and realistic-looking multi-modal data to train real-world robotic systems. At the core of Lucid-XR is vuer, a web-based physics simulation environment that runs directly on the XR headset, enabling internet-scale access to immersive, latency-free virtual interactions without requiring specialized equipment. The complete system integrates on-device physics simulation with human-to-robot pose retargeting. Data collected is further amplified by a physics-guided video generation pipeline steerable via natural language specifications. We demonstrate zero-shot transfer of robot visual policies to unseen, cluttered, and badly lit evaluation environments, after training entirely on Lucid-XR's synthetic data. We include examples across dexterous manipulation tasks that involve soft materials, loosely bound particles, and rigid body contact. Project website: https://lucidxr.github.io