Predicting mental health crises… will it work for MY patients?

In our recent study, we investigated how to transfer machine learning algorithms for prediction of mental health crises across healthcare contexts.
Predicting mental health crises… will it work for MY patients?
Like

Share this post

Choose a social network to share with, or copy the URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

“Okay, but will it work for MY patients?” This is one of the burning questions on every medical practitioner’s mind when they hear about new Machine Learning (ML) algorithms promising to improve their field. While ML has shown transformative potential, its real-world adoption remains slow. The key to broader implementation lies in demonstrating that these algorithms can perform consistently across diverse populations without losing their edge. Our research showcases an ML model that breaks boundaries – clinical, demographic and geographical.  We have proven that our predictive algorithm for mental health crises can be successfully transferred and replicated across diverse healthcare settings, from a public healthcare provider in the UK to a private one in the US.

A mental health crisis refers to any situation in which a person’s actions, feelings, or behaviors present a risk to themselves or others, or prevent them from functioning within their community (e.g., suicide attempts, mental breakdowns, or self-harm events). In recent decades there has been a steady increase in mental health related visits to emergency departments. Most of these visits are from individuals suffering mental health crises. Accurate prediction of these crises would enable healthcare providers to manage their limited resources and intervene sooner, proactively reaching out to individuals at risk. This proactive approach can mitigate the effects of a mental health crisis or even prevent it leading to a healthier population while simultaneously reducing the burden on the healthcare system.

As with any tool used in clinical settings, it is vital that ML algorithms for mental health crisis prediction are rigorously tested across multiple settings and target populations before being adopted in practice. The success of these algorithms depends on their transferability—being trained on one dataset and effectively applied in another setting, and their replicability. Model development methods must be faithfully reproduced under new conditions. Consistent performance across these variations is crucial for successful implementations in the fields of medicine and psychiatry. The path to building such prediction algorithms presents unique challenges, including disparate data collection processes, data architectures, operational protocols, and patient demographics that vary considerably across systems.

In this study, we transferred our ML models developed on a dataset of about 17,000 patients from a National Health Service trust in England to a private hospital in Chicago, Illinois (US). The study cohort comprised about 3,000 patients hospitalized between 2018 and 2020 and the dataset contained electronic health records for those patients. The patients, aged 16 and older, had a history of mental health crises, with at least one crisis episode during the study period. We started with models that were originally trained on UK data, then adapted and retrained some of them using data from a private hospital in Chicago. We also developed new models tailored specifically to the US dataset, using the same methods that worked in the UK.

The results were remarkable — our models consistently performed well, aligning with what we observed in previous research. But what does this mean in practical terms? To put it in perspective, in our study, slightly less than 1% of patients each week, roughly 30 people out of the entire cohort, are at very high risk for a mental health crisis in the following four weeks. Our models accurately identified 7 or 8 of these patients from the top 100 patients flagged each week, ranked by riskiness score. Yearly, that adds up to over 360 mental health crises potentially mitigated or even prevented, just by implementing this system with  a single healthcare provider!

The high performance achieved by our ML models makes them suitable for implementation in a number of different contexts. They can be used both in a setting where historical medical data is unavailable, by relying on a pretrained model and in a context where it’s not possible to transfer the model as is (e.g. due to contractual reasons). If both historical data and a pretrained model are available, they can be leveraged together to further fine-tune the model. Finally, our methods show that these algorithms can be used to create a fully custom solution for a particular healthcare provider, to further improve  model performance, especially in those situations where clinical notes are available.

The findings from this work have significant clinical implications. Imagine a scenario where a patient is becoming more depressed, increasing their risk of suicide. The algorithm alerts the care management team, enabling them to reach out and intervene before a crisis takes place. A crisis prediction based tool can help flag such risks, thereby improving patient outcomes while simultaneously reducing healthcare costs due to better caseload management.

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Follow the Topic

Mental Health
Humanities and Social Sciences > Behavioral Sciences and Psychology > Clinical Psychology > Mental Health
Machine Learning
Mathematics and Computing > Computer Science > Artificial Intelligence > Machine Learning
Medical and Health Technologies
Life Sciences > Health Sciences > Clinical Medicine > Medical and Health Technologies
Artificial Intelligence
Mathematics and Computing > Computer Science > Artificial Intelligence
  • npj Digital Medicine npj Digital Medicine

    An online open-access journal dedicated to publishing research in all aspects of digital medicine, including the clinical application and implementation of digital and mobile technologies, virtual healthcare, and novel applications of artificial intelligence and informatics.

Related Collections

With collections, you can get published faster and increase your visibility.

Applications of Artificial Intelligence in Cancer

In this cross-journal collection between Nature Communications, npj Digital Medicine, npj Precision Oncology, Communications Medicine, Communications Biology, and Scientific Reports, we invite submissions with a focus on artificial intelligence in cancer.

Publishing Model: Open Access

Deadline: Dec 30, 2024

Progress towards the Sustainable Development Goals

The year 2023 marks the mid-point of the 15-year period envisaged to achieve the Sustainable Development Goals, targets for global development adopted in September 2015 by all United Nations Member States.

Publishing Model: Hybrid

Deadline: Ongoing