Pianists learn to play with robotic third thumb in just one hour

Pianists equipped with a third robotic thumb were able to adapt to using the extra digit after just one hour of practice, Imperial research has found.

Researchers from Imperial College London have been working to understand how well the human brain can cope with using extra limbs made possible through robotic technology.

To assess the effects of such devices on the brain, the research team, led by Professor Aldo Faisal of Imperial’s Department of Bioengineering, attached robotic extra thumbs to a group of piano players next to their little finger that could be controlled by the pianists using their feet.

The inspiration for the study came from humanity’s long-standing fascination with characters who have extra limbs, such as those found in Indian mythology and modern superhero comic books.

Professor Faisal said: “We wanted to see whether we can enhance people using extra limbs; specifically, an extra thumb that sits opposite - contralaterally, technically speaking - the thumb of your right hand.

“The basic question was: can we use 11 fingers in a task that requires actual skills? Are our brains capable of playing music with extra limbs? We answered that with a clear yes.”

Three thumbs up

The researchers found that the group of volunteers, which included six experienced piano players and six people who were new to the instrument, was able to learn how to play with all 11 digits within an hour of being shown how to use the extra thumb.

They also noted that the ability to quickly learn how to use the robotic digit was not limited to the experienced players, demonstrating that people are able to learn how to use an extra digit for unfamiliar tasks as well as for tasks they are used to.

Professor Faisal added: “To our surprise, it turns out having piano playing experience is not a strong predictor of how well you will play with 11 fingers.

“Instead, we discovered that your basic ability to move and control your body, your dexterity, and your agility is a much better predictor for being able to use this technology and play the piano with extra fingers.”

The study, which is published in the journal Scientific Reports, could be used to improve robotic devices that augment human activity, such as equipment used to improve efficiency in industrial settings. It could also help inform future research on prosthetic limbs to substitute missing limbs.

"To our surprise, it turns out having piano playing experience is not a strong predictor of how well you will play with 11 fingers." Professor Aldo Faisal Department of Bioengineering, Imperial College London

Dr Ali Shafti, of Imperial’s Department of Computing, who also worked on the research, said: “From a robotics point of view, whenever it comes to human interfacing and human-in-the-loop systems, it is important to understand the best way to integrate a human into a robotic device.

“This research helped us to see which factors of human motor coordination are important for supernumerary robotics and will speak to other types of wearable robotic devices, such as prosthetics or assistive robotics where human and robot bodies interface.

“Our results can help to identify what it is within the person you are augmenting that you should pay attention to in order to design a better system for them.”

Bionic manufacturing

The findings could be particularly useful for robotic equipment used in the aerospace and manufacturing industries, where people work in confined spaces and could need extra limbs or enhanced dexterity.

Professor Faisal added that the next step for the research team is to see if humans could use a whole robotic arm and to control the device using brain signals instead of foot controllers.

He said: “We have augmented the thumb and now the question is whether we can do that for a whole extra arm with fingers. The current interface by which people control the extra thumb is also pretty simple, but we are now looking into controlling that directly from brain signals. This could come from the spinal cord or other sources.

“Currently, the system relies on the foot controlling the thumb, but the question is whether we can still control the thumb flexibly and freely without giving up control of another limb.”

This research was funded by the European Union’s Horizon 2020 research and innovation programme and a UKRI Turing AI Fellowship Grant.

“Playing the piano with a robotic third thumb: assessing constraints of human augmentation” by Ali Shafti, Shlomi Haar, Renato Mio, Pierre Guilleminot and Aldo Faisal. Published 1 November 2021 in Scientific Reports.