EMR-based prediction models developed and deployed in the HIV care continuum: a systematic review

EMR-based prediction models are now used in clinical decision support for HIV care. However, there is limited information about the methodological issues and clinical implications in the HIV care settings.
Published in Healthcare & Nursing
Like

Share this post

Choose a social network to share with, or copy the shortened URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

Read the paper

SpringerLink
SpringerLink SpringerLink

Electronic medical record-based prediction models developed and deployed in the HIV care continuum: a systematic review - Discover Health Systems

Objective To assess the methodological issues in prediction models developed using electronic medical records (EMR) and their early-stage clinical impact on the HIV care continuum. Methods A systematic search of entries in PubMed and Google Scholar was conducted between January 1, 2010, and January 17, 2022, to identify studies developing and deploying EMR-based prediction models. We used the CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies), PROBAST (Prediction Model Risk of Bias Assessment Tool), and TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) statements to assess the methodological issues. In addition, we consulted reporting guidelines for early-stage clinical evaluation of decision support systems to assess the clinical impact of the models. Results The systematic search yielded 35 eligible articles: 24 (68.6%) aimed at model development and 11 (31.4%) for model deployment. The majority of these studies predicted an individual’s risk of carrying HIV (n = 12/35, 34.3%), the risk of interrupting HIV care (n = 9/35), and the risk of virological failure (n = 7/35). The methodological assessment for those 24 studies found that they were rated as high risk (n = 6/24), some concerns (n = 14/24), and a low risk of bias (n = 4/24). Several studies didn’t report the number of events (n = 14/24), missing data management (n = 12/24), inadequate reporting of statistical performance (n = 18/24), or lack of external validation (n = 21/24) in their model development processes. The early-stage clinical impact assessment for those 9/11-deployed models showed improved care outcomes, such as HIV screening, engagement in care, and viral load suppression. Conclusions EMR-based prediction models have been developed, and some are practically deployed as clinical decision support tools in the HIV care continuum. Overall, while early-stage clinical impact is observed with those deployed models, it is important to address methodological concerns and assess their potential clinical impact before widespread implementation. Systematic review registration: PROSPERO CRD42023454765.

EMR-Based Prediction Models in HIV Care

Introduction

The integration of electronic medical record (EMR)-based prediction models into HIV care is an exciting development poised to enhance clinical decision-making and patient outcomes. However, many studies in this area struggle with issues such as bias from missing data and a lack of external validation. Despite these challenges, the use of EMR-based models has shown promising improvements in the HIV care continuum. This review aims to evaluate the methodological robustness, applicability, and clinical implications of these models in HIV care.

Methods

We conducted a systematic review of studies published between January 1, 2010, and January 17, 2022, using PubMed and Google Scholar. The review adhered to CHARMS, PROBAST, and TRIPOD guidelines to assess methodological quality and consulted early-stage clinical evaluation guidelines for decision support systems to determine clinical impact.

Results and discussion

Over the past decade, 35 studies on EMR-based modeling in HIV care have been identified, with 24 focused on model development and 11 on deployment. Most studies (88.6%) originated from high-income countries, predominantly the United States.

Trends in Research

The number of studies employing EMR-based prediction models has steadily increased, reflecting the growing interest in their potential to support clinical decision-making. Between 2014 and 2017, 13 studies were published, compared to 10 between 2018 and 2021 and only 4 between 2010 and 2013. This could be due to the increased interest in prediction model development and its potential applications as clinical decision support, followed by the rise of EMRs in healthcare settings. The other argument could be narrated as the need for risk stratification. Since resources are getting scarce and a classical, “one-size-fits-all” intervention strategy is clinically inefficient, researchers are looking for a solution like predicting and stratifying patients into risk levels and tailoring intervention for those who may benefit most.

Outcomes in HIV Care

The reviewed studies focused on various outcomes within the HIV care continuum:

  • 34.3% predicted individual risk of carrying HIV
  • 25.7% predicted the risk of care interruption.
  • 20% predicted virological failure.
  • Other outcomes included phenotype prediction, new HIV diagnoses, clinical status, CD4 count changes, comorbidity burden, and risk of readmission and death.

Model Development
We evaluated 24 studies focused on model development and validation. Among these, 15 (62.5%) were diagnostic models designed to predict the current risk of event outcomes, and 9 (37.5%) were prognostic models aimed at predicting future event risks within the HIV care continuum. Regarding data sources, 14 (58.3%) studies used EMRs from multiple centers, while 10 (41.7%) relied on EMR data from a single site.
In terms of methodologies, 11 (45.8%) of the studies utilized machine learning algorithms, 9 (37.5%) employed generalized linear models like logistic regression and Cox regression, and 5 (20.8%) used chart review-based algorithm classification [Table 1]. The median sample size for model development was 2633 (interquartile range: 795, 138, 806), with 7 studies (29.2%) having a sample size of less than 1000. The median number of predictors was 20 (interquartile range: 12, 94), with 7 studies using 50 or more predictors and 4 studies using fewer than 10 predictors.

Model Validation and Performance Metrics
Model validation is a critical stage in predictive modeling. Most included studies performed internal validation: 12 (50%) used cross-validation techniques, 4 (16.7%) employed sample splitting, and 3 (12.5%) used bootstrapping. Notably, 4 studies (16.7%) used more than one validation technique [Table 1].
For performance evaluation, 14 (58.3%) studies used the c-statistic or area under the receiver operating characteristic curve (AUC) to measure discrimination. Additional measures included sensitivity (reported in 11 studies, 45.8%) and positive predictive value (PPV) (reported in 8 studies, 33.3%). Model calibration was assessed in 6 (25%) studies, with one study reporting both an F1 score and an AUC. However, none of the studies included decision curve analysis (DCA) to evaluate the clinical value and implications of the models.

Table 1|Model development methods, and model validation techniques (n=24).

Items

Number of studies (%)

Clinical context of  models 

 

Diagnostic

15(62.5)

 

Prognostic

9(37.5)

Model development methods

 

 

Traditional regression models

9(37.5)

Machine learning  models

10(41.7)

Chart review-based algorithm/classification

5(20.8)

Model selection strategy

 

 

Predictor selection before modeling

3(12.5)

Predictor selection during modeling

14(58.3)

Full model approach

7(29.2)

Model validation techniques

 

 

Cross-validation

12(50)

Split sample

4(16.7)

Bootstrapping Validation

3(12.5)

Multiple forms of validation

4(16.7)

Other forms (e.g. Z-score)

1(4.2)

 

 Methodological Pitfalls

Common methodological issues included insufficient reporting of outcome events, handling of missing data, and inadequate statistical performance metrics such as discrimination and calibration. These pitfalls hinder the generalizability and performance of prediction models in new settings or populations.

Early-Stage Clinical Impact

Despite methodological concerns, deployed EMR-based models have shown positive early-stage impacts. These include increased HIV screening rates, improved linkage to care, reduced care interruption, and enhanced patient engagement. For instance, one study reported a rise in monthly HIV screenings from 7 to 550 and increased linkage to care from 15% to 100%. Another noted a 10% reduction in loss to follow-up, a 3.8% improvement in clinical appointments, and a 15% increase in achieving viral suppression.

Conclusions

EMR-based prediction models are being developed and deployed to support clinical decision-making in HIV care. While early-stage impacts are promising, most studies face significant methodological challenges that need addressing. External validation in different settings is crucial to ensuring the models' robustness and reliability.

Recommendations and Policy Implications

To harness the full potential of EMR-based prediction models in HIV care:

  1. Address methodological concerns during model development to enhance predictive accuracy.
  2. Conduct external validations to test model performance in varied settings and populations.
  3. Evaluate the clinical consequences comprehensively before widespread implementation to ensure improved patient outcomes and informed clinical decisions.

By overcoming these challenges, EMR-based prediction models can become a cornerstone of personalized and efficient HIV care.

 

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Subscribe to the Topic

Health Informatics
Life Sciences > Health Sciences > Health Care > Health Informatics

Related Collections

With collections, you can get published faster and increase your visibility.

Health Communication in the Middle East and North Africa (MENA)

The main aim of the “Health Communication in the Middle East and North Africa (MENA)” Collection is to unite the communication scholars of the MENA region working in health communications field. This area has gained more prominence and visibility with the COVID-19 pandemic across the globe. MENA region health communication scholars need to be connected to each other and with the world. This topical collection which will be tied to a symposium hosted by University of Sharjah in December 2023 will provide spaces for discussing the current state, contributions and advances in the health communication by the MENA scholars. The collection will establish an overview and recent works of the MENA academics in health communication. The symposium also welcomes contributions from other country scholars studying the health communication issues of MENA region.

In this collection, we would like to explore health communication from different perspectives, all tied to MENA region countries and comparative studies. Since this will be the first initiative, the collection is not having a theme other than a regional boundary. Potential topics include, but are not limited to:

• health in the media

• digital health

• health inequalities

• fake news and misinformation

• influencers

• health tourism

• health communication education

• communication with health-care providers and patients

• children as patients

• the impact of COVID-19 on health care communication

Keywords: Health Communication; Middle East and North Africa; Health Care; Behavior Change; Social and Behavioral Sciences

Publishing Model: Open Access

Deadline: Aug 01, 2024

AI Augmented Healthcare Systems and Services

Overview

Interest and advances in AI applications in healthcare have surged in recent years. There already are several AI applications in medicine that are used in a variety of ways, such as clinical, diagnostic, rehabilitative, surgical, and predictive practices. AI-augmented technologies can devour and analyze and help detect disease and guide clinical decisions. This is mostly due the amount of “big” data generated, especially from imaging. AI applications can handle the vast amount of data produced and find new information that would otherwise remain hidden in the mass of medical big data. Apart from disease prevention, where AI can facilitate lifestyle changes and mitigate future disease risks, identification of new drugs for health services management and patient care treatments are additional areas that AI can support. The latest advances are in wearable antennas that can be employed on people of all ages, athletes, and patients for a continuous monitoring of vital signs as well as a cellular activity.

The curation of this topical collection is motivated by the growing number of articles published in literature on AI & Healthcare and the challenges that are discussed therein. It is hoped that its contents will illuminate the existing issues in the current AI applications in and advance investigations that will ultimately translate to real-world clinical use.

Topics of Interest

The scope of the collection will cover but not be limited to:

- ML and DL applications in healthcare and health services

- Big Data and Data Analytics in clinical imaging

- AI-Augmented medical image analysis

- Uses of immersive (VR, AR, MR and ER) technologies in healthcare

- Individualized patient care

- Wearable antennas for in-body, on-body and off-body communication

- AI-enabled precision medicine

- AI-enabled drug discovery

- Use of AI for personalisation of healthcare services

- Patient-centered innovation of healthcare AI as well as challenges and opportunities to this type of innovation

- AI solutions for health promotion and disease prevention

- AI solutions for healthcare management

Keywords: Artificial Intelligence in Healthcare, Immersive Technologies in Healthcare, Big Data and Data Analytics in Healthcare and Services, AI-Augmented Prognosis, Diagnosis and Treatment, Artificial Intelligence for Healthcare Management, Personalisation of Healthcare Services, Patient-Centered Innovation.

Publishing Model: Open Access

Deadline: Jun 29, 2024