Background

Recent growth in studies has shown the effectiveness of digital health technologies in recruiting and remotely monitoring the health and behavior of large and diverse populations of interest in real-world settings. However, retaining and engaging participants to monitor their long-term health trajectories has remained a significant challenge. Uneven participant engagement combined with attrition over the course of the study could lead to an imbalanced study cohort and data collection, which may severely impact the generalizability of findings. Therefore, there is an urgent need to understand key risk factors that impact long-term participant engagement (months to years) in remote digital studies, including the feasibility of collecting active (surveys) and passive (smartphones and wearables) data streams.

In this paper, we assessed the following 3 key research questions:

(1) Is participant retention associated with real-world factors (e.g., sociodemographics, app usage, response latency, and disease severity)? 

(2) Are there potential patterns in participants’ long-term engagement, including differences between active and passive data streams? 

(3) Are there significant differences in participants’ characteristics across different long-term engagement patterns? 

Dataset

This paper evaluated the potential factors impacting long-term participant retention and engagement in a large, multinational cohort - Remote Assessment of Disease and Relapse-Major Depressive Disorder (RADAR-MDD) study^1,2. The RADAR-MDD study recruited over 600 participants from 3 Europe countries (United Kingdom, Spain, Netherlands) and monitored participants’ Phone-Active (e.g., PHQ8 surveys), Phone-Passive (e.g., GPS and Bluetooth data), and Fitbit-Passive (sleep, step, and heart rate) data for up to 2 years via RADAR-base platform (https://radar-base.org/).

We considered a variety of potential factors including participants' socio-demographics, the study site, the smartphone brand, baseline depression symptom severity, comorbidity, depression medication, as well as app usage behavior (response latency: notification response time and survey completion time).

Results

The participant retention rates at the end of 43-week as quantified using Phone-Active, Phone-Passive, and Fitbit-Passive data streams were 54.6% (N=335), 47.7% (N=293), and 67.6% (N=415), respectively (Fig. 1a). Similarly, the participant retention rates in the 94 weeks measured by the three data streams were 48.2% (N=151), 39.3% (N=123), and 54.0% (N=169), respectively (Fig. 1b).

Patterns in participants’ day-to-day data sharing were assessed using an unsupervised K-means method across the three data streams separately (Fig. 2a). Three subgroups showing distinct participant engagement patterns (C1: most engaged, C2: medium engaged, and C3: least engaged), emerged across each data stream (Fig. 2b). Figure 2c shows a marked transition where 65.4% of participants (N=151) from the C2 cluster and 44.6% of participants (N=91) from the C3 cluster, based on the Phone-Active data stream, transitioned to the most engaged C1 cluster of the Fitbit-Passive data stream.

Across the three engagement clusters in each data stream, we found notable differences in participants’ behavior (survey response and completion times), baseline depression symptom severity, and age (Fig 3).

Key Take-Aways

(1) Participants with higher PHQ8 scores contributed less data both actively and passively, indicating that participants with higher depression symptom severity may be at a higher risk of not engaging in fully remote studies.

(2) Participants with shorter survey response and completion times tend to complete more surveys and wear Fitbit for a significantly longer period. Such objective metrics on participants’ app-usage behavior may be potentially useful for predicting long-term engagement early.

(3) Age is a significant indicator of participant retention and engagement. Older participants have a lower risk of disengaging from the study app and tend to contribute more surveys and phone passive data than the younger participants.

(4) Wearable devices with minimal participant burden could help researchers collect high-density data over the longer term in real-world settings.

Reference:

1 Matcham, F. et al. Remote assessment of disease and relapse in major depressive disorder (RADAR-MDD): a multi-centre prospective cohort study protocol. BMC Psychiat. 19, 1–11 (2019).

2 Matcham, F. et al. Remote Assessment of Disease and Relapse in Major Depressive Disorder (RADAR-MDD): recruitment, retention, and data availability in a longitudinal remote measurement study. BMC Psychiat. 22, 1–19 (2022)