Scientists have developed a novel application of deep learning to monitor the quality of mental health text support services for better quality care.

There is increasing demand for accessible mental health support, particularly through digital services such as anonymous text messaging.

A team of scientists from Imperial’s Department of Computing, the Institute of Global Health Innovation and the Data Science Institute, along with Mental Health Innovations collaborated to better understand those in distress and to improve mental health text support services such as ‘Shout’ - a free, confidential text line that offers emotional support and resources to individuals struggling with their mental health.

In their new paper, published in Natural Language Processing Journal the researchers used a large dataset of anonymised Shout conversations to create the first, state-of-the-art deep learning model that analyses anonymous conversations from a text messaging service for people in mental distress.

Their model helps to understand what makes conversations between texters and mental health support volunteers effective and makes accurate predictions about texter demographics across the aggregated conversations, suicide risk level and conversation helpfulness.

The research also emphasises the importance of ethical considerations in deploying AI technologies in sensitive areas, ensuring that models are transparent, unbiased, and maintain human oversight and ensure data privacy as the top priority.

Predicting suicide risk and texter demographics

The researchers used a combination of deep learning models to analyse the anonymous conversations. This involved training models on anonymised conversation data to identify factors like suicide risk and texter demographics such as age group.

Conversations with Shout often involve individuals at significant risk of suicide or self-harm and having AI support in the future to review and flag conversations with texters who are potentially at high risk could aid the highly trained clinical staff who provide expert safeguarding support.

Additionally, texter demographics can help volunteers to understand more about the texter base, to allow for tailored training and better-quality support services. Currently, this information relies on voluntary surveys which around 20% of texters complete – this is where AI can help to get an even fuller picture of the texter demographics.

Machine learning can help address these concerns by highlighting conversations with high-risk individuals, either post-conversation - for review purposes, or in real time - for pre-emptive action. Machine learning models can also extrapolate survey answers onto conversations without individuals completing a survey. 

This work shows that language models based on deep learning significantly improve our understanding of this highly subjective dataset.

A first-of-its-kind deep learning model

According to the researchers: “As one of the first research projects on the Shout dataset and one of the first significant attempts to apply advanced deep learning to a text messaging service, this project is a proof-of-concept demonstrating the potential of using deep learning to text messages.”

Ariele Noble from Mental Health Innovations said: "Mental Health Innovations embraces the most powerful new developments in data science, machine learning and generative AI in order to bring scalable digital solutions to support the mental health of the UK. This fascinating research makes a significant contribution to the field, and exemplifies the importance of research collaborations and partnerships."

Through this work, researchers successfully met their objectives to enhance the understanding of mental health needs and the impact of services such as Shout. The best-performing model, ToBERT, made accurate predictions about texter demographics, suicide risk level and conversation helpfulness. 

Moving forward, the team is seeking further funding to refine these models, and to support the implementation of these advanced AI tools in real-world services, helping to improve support for individuals experiencing mental health distress.

-

‘Novel application of deep learning to evaluate conversations from a mental health text support service’ by Cahn et al., published on 18 November 2024 in Natural Language Processing Journal.

Image sourced from Kendel Media, Pexels.