AXIS Browser Data Engine Offers 207 Manipulation Tasks and 50,129 Trajectories

Robot manipulation datasets have expanded far slower than the models trained on them. Collection typically remains closed and centralized, with expert operators gathering demonstrations on lab hardware and shipping a static benchmark. A research team from Axis Robotics, UC Berkeley, Georgia Tech, and NTU proposes a different approach. Their system, AXIS, moves data collection into the browser and uses backend GPUs to enable continuous dataset expansion.

AXIS is partially deployable. The training code is public as a patch over OpenPI, and the teleoperation platform runs in any browser. However, the dataset, at 2.36 TB, is gated and restricted to non-commercial academic use; no policy checkpoints are released.

The core design is an asymmetry between the browser and backend. Contributors teleoperate a Franka Research 3 robot with a parallel-jaw gripper using a MuJoCo WebAssembly frontend, controlled via keyboard, mouse, virtual joystick, or gamepad. Physics and rendering run off the UI thread, ensuring state-action samples align with the simulator. Heavy tasks are handled on 8x RTX 4090 GPUs for rendering and 8x A100 GPUs for training.

Tasks are generated rather than manually designed. TaskGen breaks down language instructions into task, scene, and object configurations. It retrieves or generates meshes via an image-to-3D pipeline and proposes layouts, which are validated by a supervisor. Each task includes a structured success checker that the backend verifies.

The released dataset contains 207 tasks, over 50,000 episodes, and more than 60K variants across seven scene categories. Each trajectory includes extensive metadata, states, actions, success labels, and RGB-D observations. Over 70,000 community members contributed.

Data cleaning is a production stage. Samples with low joint variation are discarded, and motion is smoothed and resampled to a higher frequency. This reduces acceleration and jerk but also lowers replay success from 100% to 86.2%. Cleaned episodes are replayed in IsaacSim with physics disabled to randomize scenes and cameras.

In experiments, the team started from the released π0.5 checkpoint, a PaliGemma backbone with a Gemma action expert. They optionally pretrained on a simulation corpus and then fine-tuned on LIBERO. Using the full AXIS dataset, the model achieved 88.8 overall on LIBERO-Plus, compared to 83.9 for vanilla π0.5 and 57.5 for a volume-matched RoboCasa365 control. Gains are particularly large in sensor noise and camera perturbation.

However, per-axis improvements vary. While sensor noise and camera gains are significant, light and language perturbation regress compared to the baseline. Key takeaways include that AXIS offers 207 tasks and over 50K verified trajectories, continual pretraining lifts performance by 4.9 points, and scaling is generally positive but noisy. More information is available on the paper, project page, dataset, and platform.