Tsinghua-led Team Open-Sources RPent Embodied Agent Infrastructure, Boosting Task Speed 7x

As GPT-6 Astra begins real-robot operation tests, digital-world agent paradigms are moving into the physical world. This raises a concrete question: can general large models become a robot's 'brain' and enable it to complete complex real-world tasks?

Physical-world agents must not only 'think clearly' but also 'see, act quickly, and remember.'

Today, the embodied agent infrastructure RPent, jointly open-sourced by Tsinghua University, Infinigence, and Zhengxing Innovation, was released. RPent connects general LLM task understanding and planning, fine manipulation from expert models such as VLA, memory, tools, and robot interfaces into a closed loop from perception, decision, execution to feedback correction, enabling continuous adaptation and accumulation of verified experience.

On the LIBERO-PRO benchmark, it achieves 92.6% task success and over 7x end-to-end speedup.

RPent real-robot results: from 'moving' to 'doing tasks.' The release includes a series of real-robot demos showing several interesting open-ended tasks.

Notably, these tasks are not trained with a dedicated policy for each scenario. The robot's real change is moving from 'executing learned actions' to 'understanding tasks, invoking capabilities, and adjusting actions based on environmental changes.'

For example, when the task changes to 'put clean dishes into a cardboard box,' facing the same blue plate, a frozen VLA would follow its training and mistakenly place it into a metal basket. In RPent, the agent first observes the current environment and then chooses a placement based on the new goal.

If visual localization deviates during execution, it checks the result, excludes the wrong target, and re-localizes the plate in hand. When the task changes, the robot changes its actions accordingly.

In another task, the robot reads brand labels on bottles and bags and places different drinks into corresponding bags. After grasping a bottle, it does a small test lift to confirm a stable grip; when the original motion path is infeasible, it adjusts wrist posture and continues.

Facing a spoon occluded by bowls, the robot does not attempt a direct grasp but first moves two bowls away to make the target visible and reachable. This is no longer a pre-written action sequence but a complete 'observe-judge-execute-correct' loop.

In the last two tasks, the robot recombines a skill originally for 'pick and place container' to pour steel balls, and combines grasping with dual-arm coordination and sustained contact to hold a plate, wipe, and store it.

After successful exploration, RPent distills verified task logic into a Task Card. On re-execution, it can enable Flash Mode to directly reuse the flow via the Task Card, adjusting only as needed based on current visual state, avoiding re-calling the large model at every step and effectively reducing latency.

The key is not 'the robot learned a few more actions' but whether, as robots enter the open world with new tasks, objects, and obstacles, it can reorganize existing skills like a human to complete tasks outside the training distribution. RPent explores exactly this form of embodied agent: the robot no longer merely executes a fixed instruction but understands goals, invokes capabilities, responds to changes, and turns one success into reusable experience for the next.

Core design: letting models do what they are best at. RPent does not make one model omnipotent, handling everything from task understanding to manipulator control. Instead, it decomposes embodied intelligence into layers: general large models understand tasks and make plans; expert models such as VLA and WAM perform high-precision actions; a memory system accumulates experience and drives continuous optimization; the framework connects models, tools, and the real environment into a closed loop.

This forms an 'observe-plan-execute-feedback-replan' loop, enabling the agent to grow continuously in the real world. As agents move into the real world, what changes is not only the boundaries of model capabilities but also the infrastructure that supports them. RPent represents the next layer of infrastructure that must be rebuilt as large models truly enter the physical world.