Robot 'teacher' to help children with autism developed by scientists

Professor Pantic and the Zeno robot (right)

Scientists are investigating whether a cartoon-like robot can improve learning and emotional understanding in children with autism.

A team from Imperial College London and their partners in Europe have developed technology that enables an off-the-shelf robot called Zeno to interact in real-time with children who have autism. The researchers, who are currently trialling the technology in Serbia and the UK, believe that robots may provide an alternative approach for teaching children how to understand and convey their emotions.

(RoboKind ZENO R50 makes facial expressions. Video copyright: RoboKind Robotics)

Autism spectrum disorder (ASD) is a condition that affects social interaction, communication, interests and behaviour. It's estimated that about one in every 100 people in the UK has ASD. More boys are diagnosed with the condition than girls. Specialist teaching and therapy for children with autism focuses on helping them to learn to convey their emotions and understand other people’s emotions. This is done by getting them to understand the conventional ways that people express themselves through a combination of facial expression and gestures.

However, trying to read a combination of facial expressions and gestures can be confusing for the children and means they often prefer to avoid eye contact and contact in general. This has led scientists to wonder whether there are alternative methods for teaching children with autism, which are more aligned with the way they process information and see the world.

Now, the researchers on the four-year DeEnigma project are exploring whether robots can provide a consistent and fun way for children with autism to learn more about emotions. Robots, because of their programming, are precise, displaying the same facial expressions and gestures in every interaction. Researchers believe this consistency may be the key to helping young children with autism to learn the different facial expressions gestures people use to display their emotions. The robot also appeals to the way children with autism prefer things that are predictable.

Professor Maja Pantic, Imperial’s project lead from the Department of Computing, said: “Autism affects people in different ways. However, many struggle with understanding and conveying emotions, often preferring to shut out what they don’t understand. It is important to help them to understand how people convey their emotions so that they don’t find human interactions so confusing and that is why we think our project is so important.”

'Zeno says': About the game

The DeEnigma project, which began in 2016, is a collaboration between five European universities including Imperial, plus autism charities and industrial partners.

The researchers have combined a range of audio, visual, facial recognition and artificial intelligence technologies, which they have been developing over the years, into one platform. The platform was then programmed to be able to detect and decipher sound and visual cues from the children they are interacting with.

When the platform is connected to Zeno, it enables it to spontaneously interact in real-time with a child and conduct a simple lesson.

Professor Pantic added: “It is amazing to see our technology bringing Zeno to life for children with autism. They absolutely love the robot. Their faces really light up when they see it. For example, we’ve had feedback from a parent who said their non-verbal child of six spoke for the first time in excited anticipation about his next lesson with Zeno, which is amazing to hear.”

The lesson involves a child playing with the robot in a game called ‘Zeno says’. The first part of the game involves Zeno asking the child to describe what they see in a set of pictures, such as images of people being happy or angry. 

The child must explain what the faces in the picture mean and how he or she would display those emotions. Zeno responds by showing the correct expressions to the child and adding useful comments. Lastly, Zeno tells a simple story that involves a friend accidentally stepping on its favourite toy and breaking it. Zeno asks the child to explain how the friend felt in that situation.

The first research of its kind

This interaction between autistic children and a robot, where the robot can ‘see’ and ‘hear’ the child and recognise their expressions, is the first research of its kind, says Professor Pantic. A previous study had programmed a robot to deliver a monologue to children with autism as a method of teaching them about feelings, but the technology did not allow for there to be any spontaneous sensor-based interaction, making it less realistic. The DeEnigma team believe spontaneous interactions may make learning exciting, fun and easier for children with autism.

The ultimate aim of the project is to roll Zeno-like robots out to specialist schools to help teach children with autism. The DeEnigma team also have ethical approval for sharing the data they collect with other research groups in Europe. The team say this should mean faster progress can be made to understand more about the condition in children and how it affects their development.

Professor Pantic adds that sharing information across Europe gives the project a unique edge. She predicts that it will have a long-lasting impact in research on autism.