Using artificial intelligence (AI) in mental health services and research has potential, but a new study finds significant shortcomings that may indicate overly accelerated promotion of new AI models that have yet to be evaluated as viable in the real world.
How AI can support mental health services
In 2021, over 150 million people in the WHO European Region were living with a mental health condition. Over the last few years, the COVID-19 pandemic has made matters worse. People have been less able to access services, and increases in stress, adverse economic conditions, conflict and violence have shown how vulnerable mental health can be.
In parallel, AI has been giving rise to a revolution in medicine and health care. AI is seen as a novel tool in the planning of mental health services, as well as in identifying and monitoring mental health problems in individuals and populations. AI-driven tools can use digitized health-care data – available in a range of formats including electronic health records, medical images and hand-written clinical notes – to automate tasks, support clinicians and deepen understanding of the causes of complex disorders.
WHO/Europe’s “Regional digital health action plan for the WHO European Region 2023–2030”, launched in September 2022, also recognizes the need for innovation in predictive analytics for better health through big data and AI.
“Given the increasing use of AI in health care, it is relevant to assess the current status of the application of AI for mental health research to inform about trends, gaps, opportunities and challenges,” says Dr David Novillo-Ortiz, Regional Adviser on Data and Digital Health at WHO/Europe, and co-author of the study.
Challenges
“Methodological and quality flaws in the use of artificial intelligence in mental health research: a systematic review”, authored by experts from the Polytechnic University of Valencia, Spain, and WHO/Europe, looked at the use of AI for mental health disorder studies between 2016 and 2021.
“We found that AI application use in mental health research is unbalanced and is mostly used to study depressive disorders, schizophrenia and other psychotic disorders. This indicates a significant gap in our understanding of how they can be used to study other mental health conditions,” adds Dr Ledia Lazeri, Regional Advisor for Mental Health at WHO/Europe.
Because of the possibilities AI offers, policy-makers may gain insight into more efficient strategies to promote health and the current state of mental disorders. However, AI often involves complex use of statistics, mathematical approaches and high-dimensional data that could lead to bias, inaccurate interpretation of results and over-optimism of AI performance if it is not adequately handled. The study found significant flaws in how the AI applications process statistics, infrequent data validation and little evaluation of the risk of bias.
In addition, several other areas cause concern, such as the lack of transparent reporting on AI models, which undermines their replicability. The study found that data and models mostly remain private, and there is little collaboration between researchers.
“The lack of transparency and methodological flaws are concerning, as they delay AI’s safe, practical implementation. Also, data engineering for AI models seems to be overlooked or misunderstood, and data is often not adequately managed. These significant shortcomings may indicate overly accelerated promotion of new AI models without pausing to assess their real-world viability,” explains Dr Novillo-Ortiz.
“Artificial intelligence stands as a cornerstone of the upcoming digital revolution. In this study, we had a glimpse of what is to come in the next few years and will drive health-care systems to adapt their structures and procedures to advance in the provision of mental health services,” adds Antonio Martinez-Millana, Assistant Professor at the Polytechnic University of Valencia, and co-author of the study.
Select study results were presented at an event organized by WHO/Europe on 7 December 2022. Entitled “Big data analytics and AI in mental health,” the event brought together experts from across the European Region to discuss how to realistically use AI models in planning mental health services, as well as safety and success factors, such as involving people with mental health conditions in the development process.