A newly developed medical simulation using interactive holograms allows radiology trainees to learn new skills using augmented reality headsets.

The simulation was developed by a team of experts from Imperial’s Faculty of Medicine and Digital Learning Hub, and is a pioneering example of using the Microsoft HoloLens2 for medical training and to address the challenges in medical teaching and learning.

Computed tomography (CT) guided interventions are taught across the world mostly using a traditional mentored approach on real patients. However, as the number of procedures grows and complexity increases there is a need for new approaches to teach and train medical practitioners.

Professor Mohamad Hamady is part of the team that developed the simulation: “The field of interventional radiology is expanding in terms of the number of procedures and complexity, yet at the same time there is a significant shortage of skilled practitioners.

"This technology will provide trainees with the required highly technical skills in a safe and realistic environment. This digital learning technology can also shorten the training time and this will help trainees to perform medical procedures later on in life with much higher technical ability and safety.”

New approaches to radiology

The pioneering training session involved junior trainees from multiple centres in London. During the course, trainees practised simulations of image-guided needle biopsy (a medical test involving the extraction of sample tissues for examination to determine the presence of a disease).

This new simulation was delivered in mixed reality, a combination of ‘real’ and ‘virtual’ elements. Professor Hamady said: “By building the simulation in mixed reality it not only allows for accurate imaging placement, but also for crucial tactile feedback. It’s constructed from real patient X-rays and they are using real needles. They use the HoloLens2 to help them advance the needles towards the targets.”

The HoloLens2 is the second version of the Microsoft headset. It immerses the wearer in mixed reality, enabling them to interact with ‘holograms’ and real world objects simultaneously. 

The trial builds on the mixed reality breakthroughs by Dr Pratt and Dr Dimitri Amiras. The team have previously helped surgeons to 'see through' tissue to reconnect blood vessels during plastic and reconstructive surgery. Dr Amiras said: “This app really demonstrates the potential of mixed reality to provide high fidelity experience at the fraction of the cost of traditional methods. It is readily scalable and I am sure will form part of the training for doctors in the future. As technology improves this will not only change the way we train but also the way in which we perform procedures.”

Driving innovation in medicine

Dr. Philip Pratt, who developed the simulation software, said: “It’s unique that we’ve managed to create realistic images of the patient and biopsy needle together. The visualisations are almost identical to what a clinician would see when performing an actual biopsy on a real patient with a CT scanner.”

By teaming up with Imperial’s Digital Learning Hub this simulation technology has been created specifically for trainees and students. Thomas Hurkxkens, lead on using immersive technologies, said: “These technologies show great potential to redefine education and training fundamentally. By creating safe environments where students can test their skills, they now have even more opportunities to build their expertise and confidence.

"This simulation is a great example of how mixed reality can help create learning environments that are interactive and challenging.”

Alongside, the trial the team is carrying out a study into possible new applications for this technology. Professor Hamady said: “The initial feedback we are getting from our trainees and trainers is very encouraging. As the next step we want to translate this approach to other subjects, such as tumours. This technology will open the door to offer this learning method to trainees not only in London but beyond.”