Why is anticipating a mental health crisis so hard?
A mental health crisis is a serious episode in which a person's mental health sharply worsens, sometimes leading to emergency care or admission to hospital. The authors recall that spotting at-risk patients in time can improve their care and reduce the burden and costs for the health system.
The problem is that mental health conditions are very common. Reading through the complex records of every patient by hand to decide who to contact first is not feasible in practice.
What did the researchers do?
They built a machine learning model, a program that learns from a lot of past data. It uses electronic health records to monitor each patient continuously and estimate the risk of a mental health crisis over a 28-day period.
They then ran a 6-month prospective study to see how these estimates are used in the real work of care teams, that is, in managing caseloads or in reducing the risk of a crisis.
What were the results?
The model reaches an area under the ROC curve of 0.797 (1 would be perfect). At the chosen threshold, it spots 58% of crises, with a specificity of 85%, meaning it correctly clears 85% of patients who do not have one.
The area under the precision-recall curve is 0.159. This low figure shows that, among the patients flagged, a large share do not end up in crisis: there are false alarms. During the 6 months of follow-up, clinicians found the flags useful, either for managing their caseload or for reducing the risk of crisis, in 64% of cases.
Key figures
- 58%Share of mental health crises spotted by the model over 28 days (sensitivity), at 85% specificity.
- 64%Share of cases where clinicians judged the flags useful in the 6-month prospective study.
- 0.797The model's area under the ROC curve (1 = perfect performance).
What could this mean for patients?
A tool like this could help teams reach out earlier to people whose situation is getting worse, before a crisis breaks out. The authors see it as a way to organise care better when teams cannot read every record.
It is a signal to be watchful, not a diagnosis. The decision to call a patient back, change a treatment or offer an appointment always belongs to health professionals.
What are the limits of this study?
The model gets things wrong: it misses 42% of crises and raises alerts that are not followed by a crisis. The authors do not show that these flags actually reduce the number of crises or improve patients' health. That remains to be proven.
The prospective follow-up lasted 6 months and relies on clinicians' opinion of how useful the flags were, which is a judgement and not a patient outcome. The authors describe their work as a first exploration of this question.
When things feel heavy: a word about Mima
This study looks at spotting earlier when someone's situation is getting worse. It is still research, and nothing replaces the link with your care team.
Mima is a listening app: you can talk to her at any time, she does not judge, and she keeps a record of what you are going through so you can share it with your team. She is neither treatment nor therapy. In an emergency, call an emergency number or a professional.
Frequently asked questions
Can AI spot a mental health crisis in advance?
In this 2022 study, a model flagged 58% of crises occurring within 28 days, at 85% specificity. It also gets things wrong: some crises slip past it and some alerts lead to no crisis.
What are electronic health records?
They are patients' digital hospital files, with their appointments, treatments and notes. The model analyses them continuously.
Did clinicians find the tool useful?
In a 6-month follow-up, flags were judged useful for managing caseloads or reducing the risk of crisis in 64% of cases.
Does the tool show that crises go down?
No. The study assesses how useful clinicians found the flags, not a drop in the number of crises. That remains to be shown.
The original study
Garriga R, Mas J, Abraha S, et al. “Machine learning model to predict mental health crises from electronic health records.” Nature Medicine, 16 May 2022. DOI : 10.1038/s41591-022-01811-5
Read the full study in Nature MedicineThis article summarises a scientific study and is not medical advice.