Self-Contained Robotic Hand Walks on Fingertips and Manipulates Objects

A hand attached to an arm is typical for humans, but robots need not follow the same rules. Researchers at ETH Zurich have transformed an off-the-shelf robotic hand into a self-contained robot that walks on its fingertips while using those same fingers to manipulate its environment. The hand, reminiscent of Thing from The Addams Family, could enable operation of controls and handling of objects in places a traditional robot arm cannot reach.

Amirhossein Kazemipour, a Ph.D. candidate in ETH Zurich's Soft Robotics Lab who led the project, believes the hand could be useful for fiddly maintenance tasks in industrial settings or in search and rescue. In those scenarios, a robot could detach its hand to explore confined spaces too tight for the rest of its body. More broadly, Kazemipour says, the work advances a new approach to robotics in which body parts are not confined to a single role. He told IEEE Spectrum that he wanted to see how much a robotic hand could do without the rest of the robot, and asked: instead of the traditional view of robotic parts, what if those parts could perform tasks they were not designed for?

This is not the first attempt to get a robot hand to walk, but previous work relied on specially designed hands. The ETH team used off-the-shelf hardware from Wuji Technology, augmented with an 80-gram backpack containing a battery, an inertial measurement unit, and a Raspberry Pi Zero. Kazemipour says they chose a commercial hand because they did not want to compromise its manipulation capabilities. This introduced challenges, he adds, because a hand's shape is optimized for grasping rather than crawling.

To teach the hand to walk, the researchers trained a model in simulation using reinforcement learning, where models learn through trial and error with rewards for desired behavior and penalties for divergence. However, the unusual geometry of a hand meant they couldn't reuse approaches for legged robots. Standard quadrupeds or bipeds are symmetrical left and right, simplifying walking and balance. Hands are not symmetrical, with fingers of different lengths and an opposable thumb. The team introduced a novel method to keep the hand in a crawling stance during simulated training. To account for unequal digit lengths, they gave each finger its own target position relative to the palm based on where it naturally sits in the crawling posture. They then penalized fingers for straying too far, creating "virtual springs" that pull them back. Crucially, penalties for forward and backward movement are much less than for side-to-side, so stepping forward costs little relative to rewards.

The team also trained separate models for skills like righting the hand when it falls, pressing keyboard keys, and pushing objects to a target. All four models fit on the onboard computer, and the controller switches between them as needed. In real-world tests, the hand crawled across 14 different surfaces including smooth floors, metal grates, grass, and gravel. It made both right and left turns and reached an average speed of 9 centimeters per second. It righted itself in 21 of 25 trials. With a video feed from an overhead camera, it autonomously pushed a 41-gram cube to targets up to 40 cm away 15 times in a row.

Masahiko Inami, a professor at The University of Tokyo whose group has developed hand-shaped walking robots, says what interests him most is that the same fingers can move the hand, support its weight, and interact with its surroundings. He agrees a mobile hand could be useful for reaching confined spaces and operating controls an entire arm could not easily access, though practical deployment would require on-board perception and more robust navigation. Hideki Shimobayashi, a Ph.D. student in Inami's lab, notes that walking puts very different loads on a hand compared to grasping, so durability may be a challenge. Matei Ciocarlie, an associate professor of mechanical engineering at Columbia University, calls it a "cool result" suggesting robot hands don't have to be restricted to human-like capabilities. He says one can imagine an entirely new class of mobile manipulators that are simultaneously dexterous, able to traverse complex terrain, and able to manipulate payloads comparable in size to themselves.

Kazemipour sees his work as part of a longer-term vision for robots made of parts not tied to a single job or body plan, something he calls "autonomous modular embodiments." He says robotic body parts should not be permanently assigned to a single function; they can dynamically transition between roles, between being components of one embodiment system and autonomous agents, according to the task.