NTU, PKU and Partners Unveil ω-0 World Action Model to Transform Humanoid Home Operations
Home environments pose the ultimate test for humanoid robots. Unlike factories with structured and predictable pipelines, homes are dynamic and cluttered. A robot must navigate around furniture, adapt to moving targets, and maintain balance while performing tasks such as retrieving objects from the fridge or wiping a table. These seemingly mundane chores challenge the robot's perception, planning, and control capabilities.
To address these challenges, a research team from Nanyang Technological University, Peking University, HKUST (Guangzhou), and Beijing Academy of Artificial Intelligence introduced ω-0, a world action model based on latent-space prediction. This model takes language commands as input, simultaneously recognizes the environment and its own state, and directly outputs low-level action features that the control system can execute. This enables whole-body coordination for 'move-while-operate' tasks.
The team conducted extensive real-world evaluations on 11 highly difficult home operation scenarios, focusing on dynamic coordination, environmental adaptation, and autonomous decision-making. The tasks included cross-area object sorting, cleaning high and low positions, and picking up clothes from beds. The results were impressive: ω-0 Ego achieved a 79.1% success rate in first-person view, while ω-0 Omni reached 81.8% in full-scene mode. These figures significantly outperform established baselines such as π-0.5, EgoVLA, GR00T-N1.7, and ψ-0.
A key innovation of ω-0 is its rejection of the conventional video-to-action conversion, which is time-consuming and often inefficient. Instead, it adopts a unified query-based representation. In this architecture, future visual features provide high-level guidance about task progress and scene evolution, while action branches directly generate whole-body motor commands. This approach reduces latency and increases robustness, allowing the robot to adapt to unexpected changes during task execution.
To develop this model, the team designed a three-stage progressive training pipeline. The first stage involves dedicated pre-training for whole-body action VLM. The second stage is human-robot action latent fusion pre-training, which draws on advanced techniques from V-JEPA, integrating visual, linguistic, and proprioceptive data. By using video queries to predict future environmental features, the model embeds dynamic scene information into action representations. The third stage is fine-tuning with real-world data, where the model is exposed to a large corpus of home manipulation data, enabling it to adapt to diverse home environments.
Supporting this training effort, the researchers open-sourced ω-HOME, a large-scale dataset collected from real home settings. The dataset includes 40.3 hours of real environment data, 4,827 task trajectories, and 24 distinct tasks. Each trajectory contains synchronized language instructions, first-person and third-person camera views, proprioceptive states, and whole-body motion trajectories. This dataset significantly reduces the dependence on demonstration data for downstream tasks.
The 81.8% success rate is a remarkable achievement, but it is only a stepping stone. The journey toward a robot that can seamlessly integrate into daily life and operate autonomously over extended periods is still long. Nevertheless, ω-0 points the entire industry in a clear direction: treating the robot's body as an integrated whole rather than separate components. By learning to operate while moving and adjusting posture dynamically, robots can transition from laboratory demonstrations to practical home services, bringing us closer to the dream of embodied AI in everyday life.