By Brinnae Bent and Jessilyn Dunn

BIG IDEAs Lab, Duke University Departments of Biomedical Engineering and Biostatistics & Bioinformatics

Wearable technology has the potential to transform healthcare and research through accessible, continuous, and longitudinal health monitoring. The coming ubiquity of wearable technology will particularly provide a unique opportunity to revolutionize health care in communities with traditionally limited healthcare access. However, the accuracy of wearable technologies has been a hotly debated topic both in research and in popular culture. As wearable technologies are increasingly being used for clinical research and healthcare, it is critical to understand their accuracy and determine how measurement errors may affect research conclusions and impact healthcare decision-making. 

This summer STAT News published a wide-reaching article suggesting that “Fitbit and other wearables may not accurately track heart rates in people of color.” At this point, our study analyzing sources of inaccuracies in wearable technologies was already well underway, and we were eager to gather the final data and perform our analysis to uncover the truth. 

In this study, we tested optical heart rate sensors on six of the most popular devices on the market for both consumers (Apple Watch, Fitbit, Garmin, XiaoMi Miband) and for research (Empatica E4, Biovotion Everion) against the clinical reference standard for monitoring heart rate, the electrocardiogram (ECG). We tested the devices on a population that was roughly evenly distributed across all skin tones according to the Fitzpatrick skin tone scale. Study participants performed a variety of activities of daily living, including rest, walking, and typing on a keyboard, as shown in the top of Figure 1. 

Figure 1. Visual Abstract. We tested 6 wearable devices on 53 individuals approximately equally distributed on the Fitzpatrick skin tone scale during 4 different activities. We tested devices against the clinical gold standard, electrocardiogram (ECG).

We utilized robust statistical analysis and modeling and demonstrated that the type of device plays a key role in the accuracy of heart rate measurements. Further, the type of physical activity going on during a measurement also plays an important role in the accuracy of the heart rate measurement: we saw an approximate 30% drop in accuracy when a person is walking versus when they are seated at rest. Debunking speculations, at least for the wearable devices that we tested in this study, we showed that skin tone does not in fact significantly affect the accuracy of optical heart rate measurements (Figure 2). 

Figure 2. Error in Heart Rate across Skin Tones and Devices at Rest and During Activity. Mean Absolute Error in heart rate (bpm) across skin tones and devices at (a) rest and (b) during physical activity. Error is calculated as the difference between the ECG and wearable reported heart rate at every simultaneous measurement. Fitzpatrick skin tones 1-6 are represented with an approximately equal number of participants in each skin tone. Error bars represent the 95% confidence interval

By 2021, it is expected that 121 million Americans will use wearable devices. With the number of chronically ill patients and health system utilization in the US at an all-time high, the integration of low-cost, convenient, and accurate wearable sensors to support healthcare is increasingly sought after. It is imperative that researchers, clinicians, and health consumers recognize that the information derived from different wearables should not be weighted equally for drawing study conclusions, performing meta-analyses, and making health-related decisions. Algorithms that are used to calculate digital biomarkers should consider error and measurement quality under various circumstances, including those in this study. Digital biomarker interpretation must account for data quality and accuracy for health decision-making.