Mind-controlled VR games could aid stroke rehabilitation

A virtual reality hand, controlled directly by the mind, could help stroke patients restore hand movement as part of VR-assisted physiotherapy.

Cogitat, a startup founded last year to commercialise Imperial research, is exploring medical and gaming applications of its core technology, a brain-computer interface that allows users to move in virtual reality (VR) environments by mentally willing themselves to move rather than physically moving. 

The company is forming partnerships with VR companies and developers to use the technology for a range of applications – beginning with games that could help stroke patients recover hand movement by training with a mind-controlled VR hand.

How a virtual hand could train real movements

Many stroke patients suffer from impaired upper limb movement due to disrupted connections between their brains and muscles, but thanks to the brain’s plasticity this can often be restored using regular physiotherapy exercises that help reconfigure the connections.

“If you can’t move your hand at all, repeated practice can be very frustrating. The games we are developing will offer a more rewarding experience,” explains Allan Ponniah, Cogitat’s CEO.

To help patients practice and improve their performances in a range of hand physiotherapy exercises, Cogitat and VR production studio Unit 9 with support from UK Research and Innovation (UKRI) are exploring the potential for VR physiotherapy games powered by a brain-computer interface that enables a virtual hand to perform the movements that the patient is attempting to make.

“If you can’t move your hand at all, repeated practice can be very frustrating, but the games we are developing will offer a more rewarding experience,” explains Allan Ponniah, Cogitat’s CEO. “Initially, users just imagine their movements and they see their hands moving in VR. As time goes on, it helps calibrate their hands. Once they move their hands a little bit, this is represented in the VR experience as a more successful performance. This pushes them towards a situation where they can, for example, fully open and close a fist.”

The concept for the games is motivated partly by research in neuroscience suggesting that the experience of seeing a motor task performed helps you develop the motor skills needed to perform that task yourself by stimulating so-called mirror neurons – brain cells that are activated both by witnessing and performing motor tasks.

Getting the best results in physiotherapy requires patients to carry out exercises frequently for an extended period of time. While the proposed VR rehabilitation games could be used as part of physiotherapy sessions at clinics, they also have the potential advantage of helping patients to carry out exercises at home. Game elements such as the opportunity to compete online with other players or to try to beat one’s own past performances could make the exercises more enjoyable.

“When we discussed with patients whether they would prefer to play games with their families or with other patients, predominantly they said other patients, so they can see within the group who is ahead of them, and who's behind, and understand which direction they're going in with their rehabilitation. They like spending time with other stroke patients and the competitive aspects of it,” says Mr Ponniah.

Advances in machine learning can help read minds

Advances in machine learning, which is adept at finding statistical regularities in noisy data, have made it possible to use EEG data more effectively. Last year, Cogitat came first in an international contest.

The Cogitat team say they are set apart by their prize-winning core technology, created using machine learning, that decodes a user’s motor intentions from brain wave data read by electroencephalogram (EEG) headsets.

Stefanos Zafeiriou, Cogitat’s co-founder and Chief Scientist, and Professor in Machine Learning & Computer Vision in Imperial's Department of Computing, says: “We’re very proud of our core technology for EEG decoding, which brings together cutting-edge deep learning advances from Imperial with the significant in-house expertise of Cogitat. "

Yates Buckley, Technical Partner at Unit 9, said: “What we dreamed to be able to do with EEG-BCIs is becoming a reality with Cogitat’s impressive detection accuracy. I think what we are working on will bring a step change to application of EEG to rehabilitation.”

EEG devices, which use electrical activity on the scalp as a proxy for brain activity, are orders of magnitude cheaper than other forms of brain imaging such as magnetic resonance imaging and portable enough to use outside research settings. However, their poor signal-to-noise ratio and spatial resolution historically has meant that they provide limited usable information about what is happening in the brain.

Recent advances in machine learning, which is adept at finding statistical regularities in noisy data, have made it possible to use EEG data more effectively, and last year Cogitat came first in an international contest to find the most effective technologies for reading motor cortex data from EEG, exceeding the performances of serious international competitors from academia, industry and government.

Cogitat also aims to distinguish its technology from others in the newly emerging field of VR-assisted physiotherapy by drawing on the knowledge of patients and medical experts at an early stage of the technology’s development. In partnership with Unit 9, the team has used a grant from UKRI's Audience of the Future Challenge to gain early input from physiotherapists, neurologists and potential users.

Andrew Chitty, Audience of the Future challenge director at UKRI said: “Combining Cogitat’s ground-breaking technology of brain-computer interfaces with the capabilities of VR and games technology offers the potential of making a real difference to the rehabilitation of stroke sufferers – one of the commonest causes of chronic disability in the UK. This is a fantastic example of the potential for deep tech innovation from the life sciences to cross fertilise with creative technologies, engage patients in their own care and build businesses that will revolutionise healthcare." 

Applications in entertainment and medicine

The Cogitat team is pursuing VR-assisted physiotherapy as one of several applications that their core brain-computer interface technology could help enable. Other applications include pure entertainment, for example VR games that provide a realistic sensation of walking or running despite being confined to a small space.

Dr Dimitrios Adamos, an Imperial research fellow and Cogitat's co-founder and CTO, says: “We are a deep-tech company that develops cutting-edge AI/ML-powered brainwave decoders trained on real-life EEG data. Our device-agnostic technology can interface with any EEG headset and sensor configuration, including devices that haven’t even hit the market.”

The technology could also help power the metaverse, a mooted future version of the internet under development by companies such as Facebook’s parent company Meta that replaces 2D text and images with 3D virtual worlds.

Top two images: Mock-ups of virtual reality physiotherapy experience by Unit 9. Third image: Getty.