Journal List > Korean Circ J > v.49(5) > 1122104

Hwang, Kim, Choi, Cho, Nam, and Kim: Assessing Accuracy of Wrist-Worn Wearable Devices in Measurement of Paroxysmal Supraventricular Tachycardia Heart Rate

Abstract

Background and Objectives

Wrist-worn wearable devices provide heart rate (HR) monitoring function via photoplethysmography technology. Recently, these devices have been used by patients to measure the HR when palpitation occurs, but few validation studies of these instruments have been conducted. We assessed the accuracy of these devices for measuring a HR.

Methods

This study enrolled 51 consecutive patients with a history of paroxysmal supraventricular tachyarrhythmia (SVT) or paroxysmal palpitations who were scheduled to undergo an electrophysiological study (EPS). Three devices were assessed: Apple Watch Series 2 (Apple), Samsung Galaxy Gear S3 (Galaxy), and Fitbit Charge 2 (Fitbit). Patients were randomly assigned to wear 2 different devices. The HR at baseline and induced SVT were measured during the EPS. After successful ablation of SVT, HR measurements was also done during atrial and ventricular pacing study.

Results

The mean patient age was 44.4±16.6 years and 27 patients were male (53%). The accuracy (within ±5 beats per minute [bpm] of an electrocardiogram [ECG] measurement) of the baseline HR measurements was 100%, 100%, and 94%, for Apple, Galaxy, and Fitbit, respectively. The HR during induced SVT ranged from 108 bpm to 228 bpm and the accuracy (within ±10 bpm of an ECG) was 100%, 90%, and 87% for the Apple, Galaxy, and Fitbit, respectively. During pacing study, accuracy of these devices was also acceptable but tended to decrease as the HR increased, and showed differences between the devices.

Conclusions

Wrist-worn wearable devices accurately measure baseline and induced SVT HR.

Trial Registration

Clinical Research Information Service Identifier: KCT0002282

INTRODUCTION

The wrist-worn wearable devices including so-called smart-watches and activity trackers are now very popular and offer heart rate (HR) monitoring technology via photoplethysmography (PPG) technique.1) By measuring the pulsatile signal, each pulse ‘peak’ can be interpreted as an R wave, i.e. one heartbeat, and then translated into the HR.2) Previous studies have shown that the PPG technique using a smartphone camera with specialized smartphone applications allows accurate HR measurements and even can be used to diagnose atrial fibrillation (AF).2)3)4)5) With the emergence of the concept of ‘wearable technology,’ wrist-worn wearable devices are now widely used for HR measurements instead of smartphone applications. However, there are still few published studies evaluating the accuracy of these devices, especially in an arrhythmic situation.6) The objective of our study was to assess the accuracy of wrist-worn wearable devices in the measurement of HR, particularly during tachycardia.

METHODS

Study population

Patients aged from 18 to 80 years with a history of paroxysmal palpitations or documented paroxysmal supraventricular tachyarrhythmia (SVT) and who were scheduled to undergo an electrophysiological study (EPS) were eligible. The detailed exclusion criteria were as follows: 1) a history or suspicion of arterial disease in the upper extremities; 2) a prior coronary artery bypass using a radial artery; 3) receiving dialysis through the arteriovenous fistula of the upper extremity; 4) previous upper limb amputation; 5) skin disorders or ample hair around the wrist; 6) pregnancy; and 7) refusal to participate. The study protocol complied with the Declaration of Helsinki and was approved by the Institutional Review Board of Asan Medical Center. This study was registered with International Clinical Trials Registry Platform (Unique identifier: KCT0002282, registration date: 2017.3.28). All patients gave written informed consent.

Study design

All procedures were done under conditions of a fasting state with light sedation after withdrawal of all antiarrhythmic drugs for at least a period of 5 half-lives. The EPS was performed in accordance with our institutional clinical practice. A 12-lead electrocardiogram (ECG) recording by Prucka Electrophysiology lab (Prucka, GE Medical Systems Information Technology, Milwaukee, WI, USA) was used to provide criterion measures of HR. Three of the most widely used wrist-worn wearable devices were assessed: Apple Watch Series 2 (Apple; Apple Inc, Cupertino, CA, USA), Samsung Galaxy Gear S3 (Galaxy; Samsung Electronics Co., Suwon, Korea), and Fitbit Charge 2 (Fitbit; Fitbit Inc., San Francisco, CA, USA). The patients were randomly assigned to wear two different devices, one on each wrist. The devices were tightly placed above their ulnar styloid process during the study. The HR was measured in the following situations:
  • 1) Baseline HR prior to the EPS

  • 2) Induced SVT HR

  • 3) Atrial pacing study after successful ablation:

    • A. pacing cycle lengths (PCLs): 600 ms, 500 ms, 430 ms, 380 ms, 330 ms, 300 ms

    • B. performed only 1: 1 atrioventricular (AV) conduction was maintained

    • C. at least a 30-second resting period between pacing studies

  • 4) Ventricular pacing study after successful ablation:

    • A. PCLs: 600 ms, 500 ms, 430 ms, 380 ms, 330 ms, 300 ms

    • B. at least a 30-second resting period between pacing studies

In each situation, the HR measured by each device was compared with a simultaneous ECG HR. If the devices failed to provide a measurement in initial 30 seconds, a second and third attempt to obtain HR was made. If there was still no result after third attempts or within 3 minutes, then the event was recorded as a failure of the devices.

Definition of accuracy

An accurate measurement was defined when a difference between the device-measured HR and an ECG-recorded HR is 5 beats per minute (bpm) or less. The proportion of measurements which showed a difference of HR within 10 bpm was also evaluated because we thought it was clinically acceptable range. Blood pressures were measured during baseline and induced SVT.

Statistical analysis

Continuous variables were described as a mean±standard deviation and categorical variables were described as number (percentage). The degrees of agreement between HR measurements were assessed by the intraclass correlation coefficient with 95% confidence intervals and the Bland-Altman plot. The comparison between the three devices was tested using the Fisher's exact test. All statistical analyses were performed using the MedCalc software package, version 18.2.1 (MedCalc Software, Mariakerke, Belgium). A p value <0.05 was considered statistically significant.

RESULTS

Patient characteristics

A total of 51 patients were enrolled and 27 patients (53%) were male. The mean age was 44.4±16.6 years. The patient characteristics are summarized in Table 1. All patients had structurally normal hearts. Patients were randomly assigned to wear 2 different devices, one on each wrist. Hence, our study was designed to take 34 measurements per device.
Table 1

Baseline characteristics

kcj-49-437-i001
Characteristics Values
Male 27 (53)
Age (years) 44.4 ± 16.6
Height (cm) 163.1 ± 16.4
Weight (kg) 64.5 ± 9.1
Body mass index (kg/m2) 23.7 ± 3.0
Baseline SBP (mmHg) 118.4 ± 19.3
Baseline DBP (mmHg) 73.1 ± 11.7
Baseline HR (bpm) 71.6 ± 13.5
SVT SBP (mmHg) 103.3 ± 21.0
SVT DBP (mmHg) 70.6 ± 18.5
SVT HR (bpm) 169.3 ± 34.4
Values are presented as number (%) or mean±standard deviation.
DBP = diastolic blood pressure; bpm = beats per minute; SBP = systolic blood pressure; SVT = supraventricular tachycardia.
The induced SVTs were as follows: atrioventricular nodal reentrant tachycardia (AVNRT) in 23 patients (45%), atrioventricular reentrant tachycardia (AVRT) in 23 patients (45%), atrial tachycardia (AT) in 3 cases (6%), and typical atrial flutter (AFL) in 1 case (2%). Tachycardia was not induced in one patient. All induced SVTs were successfully ablated without complication.

Measurement of heart rate

The time required to measure the HR was about 20 seconds for the Apple and Galaxy, and 30 to 40 seconds for the Fitbit. In the case of Apple and Fitbit, HR was continuously measured and displayed. With Galaxy, fixed measured HR was displayed only once after the measurement period. A total of 1,063 HR measurements were attempted across all the three devices. Among these measurements, 876 of the HR measurements were within ±5 bpm difference and 902 were within ±10 bpm difference as compared to the ECG measurements. Therefore, the overall accuracy of the wrist-worn wearable devices for the measurement of HR was 82.4% for ±5 bpm difference and 84.9% for ±10 bpm difference.

Measurement of baseline heart rate

The baseline HR in our study patient ranged from 50 to 101 bpm. In one patient, Fitbit showed an error from the beginning. The baseline HR was therefore measured in 34 patients with the Apple and Galaxy, and in 33 patients with Fitbit.
The accuracy of the baseline HR measurement, i.e. within ±5 bpm of the ECG value, was 100%, 100%, and 94% for the Apple, Galaxy, and Fitbit, respectively. There was no statistically significant difference between these devices in terms of accuracy.

Measurement of supraventricular tachyarrhythmia heart rate

Tachycardia was not induced in 1 patient and was not sustained in 4 patients (2 AT, 1 typical AFL, 1 AVRT). Hence, we could not measure the induced SVT HR in these patients. In three patients (2 AVNRT, 1 AVRT), we also could not measure the HR because these patients complained of severe symptoms. As described above, Fitbit showed an error from the beginning in one patient. In addition, one Galaxy measurement and two Fitbit measurements failed. Therefore, an induced SVT HR was measured in 28 patients with Apple, 29 patients with Galaxy, and 30 patients with Fitbit.
The induced SVT HR ranged from 108 to 228 bpm. The accuracy of the HR measurements to within ±5 bpm of the ECG value was 89.3%, 89.7%, and 83.3%, respectively, for the Apple, Galaxy, and Fitbit, which improved to 100%, 89.7%, and 86.7%, respectively, when the criterion was changed to ±10 bpm difference with the ECG measurement. The intraclass correlation coefficients of each device with a Bland-Altman plot of the induced SVT HR measurements are shown in Figure 1. A statistically significant difference was not observed between the three devices (within ±5 bpm difference, p=0.780). The degree of HR did not influence the device accuracy. A summary of the baseline and SVT HR measurements is provided in Table 2.
Figure 1

Bland-Altman plots. This figure shows Bland-Altman plots of wrist-worn wearable device measured versus electrocardiography measured supraventricular tachycardia heart rate. The 95% limits of agreement and ICC were also shown. In all 3 devices, the mean difference was about 0 and most of the measurements were within 95% degree of agreements. The ICCs of devices were also high.

CI = confidence interval; ICC = intraclass correlation coefficient.
kcj-49-437-g001
Table 2

Accuracy of wearable devices in measuring baseline and induced SVT heart rate

kcj-49-437-i002
Clinical Situation Device
Apple Galaxy Fitbit p value
No. of patients 34 34 33
Baseline (within ±5 bpm) 34 (100) 34 (100) 31 (94) *
No. of patients with SVT 28 29 30
SVT (within ±5 bpm) 25 (89) 26 (90) 25 (83) 0.780
SVT (within ±10 bpm) 28 (100) 26 (90) 26 (87) 0.568
Values are presented as number of patients (%).
bpm = beats per minute; SVT = supraventricular tachycardia.
*Statistical analysis not applicable.

Measurement of heart rate during atrial and ventricular pacing

During atrial and ventricular pacing study after SVT ablation for inducibility assessments, we attempted to measure the HR at each PCL using the three wearable devices. Although not a real arrhythmia, we hypothesized that the hemodynamic status of the pacing study would be similar to spontaneously occurred tachyarrhythmia. Four patients declined to participate in the pacing study.
In the atrial pacing study, Apple and Galaxy showed more than 90% accuracy up to PCL 380 ms (same as HR 160 bpm). The Fitbit showed a similar accuracy to the other devices at a PCL of 600 ms (same as 100 bpm of HR) but showed a statistically significant lower accuracy than the other devices at higher pacing rates. At a PCL of 330 and 300 ms, HR measurements were attempted in a small number of patients due to a loss of 1:1 antegrade AV conduction and they were excluded from the analysis.
In the ventricular pacing study, the Apple showed a greater than 90% accuracy up to a PCL of 330 ms (same as 180 bpm of HR). The Galaxy showed more than 90% accuracy up to a PCL 430 ms (same as 140 bpm of HR), but only an accuracy of 80% at a PCL of 380 ms (same as 160 bpm of HR) and 70% at 330 and 300 ms (same as 180 and 200 bpm of HR). Similar to the atrial pacing study, the Fitbit showed similar accuracy to other devices at PCL of 600 ms (HR 100 bpm), and statistically significant lower accuracy at higher pacing rate. In summary, the accuracy of the Apple and Galaxy was high and did not significantly change with the pacing rate. However, the Fitbit showed modest accuracy at lower pacing rates which decreased dramatically for rapid pacing rates. These differences between devices in accurate HR measurements during pacing study were statistically significant. The results of the pacing study are summarized in Table 3.
Table 3

Results of the atrial and ventricular pacing study

kcj-49-437-i003
Pacing cycle length Pacing site and device
Atrial pacing Ventricular pacing
Apple Galaxy Fitbit p value Apple Galaxy Fitbit p value
600 ms (100 bpm)
Attempted 33 30 29 29 25 25
Within ±5 bpm 33 (100) 29 (97) 27 (93) 0.198 29 (100) 25 (100) 20 (80) 0.007
Within ±10 bpm 33 (100) 29 (97) 27 (93) 0.198 29 (100) 25 (100) 22 (88) 0.010
500 ms (120 bpm)
Attempted 32 29 28 32 29 27
Within ±5 bpm 31 (97) 26 (90) 18 (64) 0.000 32 (100) 27 (93) 20 (74) 0.002
Within ±10 bpm 31 (97) 28 (97) 20 (71) 0.062 32 (100) 27 (93) 23 (85) 0.025
430 ms (140 bpm)
Attempted 29 22 24 33 29 29
Within ±5 bpm 26 (90) 20 (91) 19 (79) 0.005 33 (100) 28 (97) 18 (62) <0.001
Within ±10 bpm 26 (90) 20 (91) 21 (88) 0.032 33 (100) 28 (97) 20 (69) 0.005
380 ms (160 bpm)
Attempted 19 16 17 31 29 27
Within ±5 bpm 18 (95) 14 (88) 12 (71) 0.016 31 (100) 24 (83) 10 (37) <0.001
Within ±10 bpm 18 (95) 14 (88) 12 (71) 0.045 31 (100) 24 (83) 13 (48) <0.001
330 ms (180 bpm)
Attempted 9 10 13 31 28 26
Within ±5 bpm 7 (78) 8 (80) 6 (46) * 28 (90) 20 (71) 7 (27) <0.001
Within ±10 bpm 8 (89) 8 (80) 6 (46) 28 (90) 20 (71) 8 (31) <0.001
300 ms (200 bpm)
Attempted 7 7 8 31 27 24
Within ±5 bpm 5 (71) 3 (43) 3 (38) * 23 (74) 20 (74) 6 (25) 0.002
Within ±10 bpm 5 (71) 3 (43) 3 (38) 23 (74) 20 (74) 7 (29) 0.005
Values are presented as number of patients (%).
*Statistical analysis not applicable.

DISCUSSION

In this study, we evaluated the accuracy of three widely used commercial wrist-worn wearable devices, Apple, Galaxy, and Fitbit, in the measurement of HR during SVT and during atrial and ventricular rapid pacing. The results showed that these three devices can accurately measure a baseline and SVT HR and that this accuracy is not affected by the degree of HR. Further validation was attempted during rapid atrial and ventricular pacing to simulate tachycardia. The devices again showed an acceptable accuracy, but this tended to decrease as the HR increased. In the pacing study, the Apple showed highest accuracy and Galaxy was comparable. The Fitbit showed a more than 70% accuracy below PCL 430 ms (same as 140 bpm of HR), but this was less than 50% at higher PCL.
Palpitation is one of the most common symptoms in patients visiting cardiology clinics. However, differentiating benign condition from clinically significant cardiac arrhythmia and making an accurate diagnosis are challenging due to its episodic nature. Holter monitoring and event recorders are currently in use but have availability and accessibility limitations. Accurate HR measurement during palpitations with smart devices that are used by most people may aid in this situation.
Currently, HR measurement technology using smart devices can be divided into 2 groups. The first of these uses PPG technology, an optical method that measures changes in tissue blood volume caused by the pressure pulse. The PPG method has been widely used and is well validated in the medical settings.7) The other group uses a single lead ECG which is achieved using a pair of external electrodes, either attached to the smartphone-case or as an independent additional device that is connected with specialized apps. Cardiio rhythm (Cardiio Inc., Cambridge, MA, USA) and AliveCor Kardia Mobile (AliveCor Inc., Mountain View, CA, USA) are a representative PPG-based application and single lead ECG system, respectively, that can be used with a smartphone. Both systems have been proven to be capable of accurate HR measurement and even to screen for AF in large-scale studies.4)8) Smart wearable devices adopt PPG-based HR monitoring system and prior studies have reported good accuracy of wearable devices in measurement of resting HR. However, some studies also have reported that variable accuracy among devices in activity tracking and exercise HR measurement.6)9)
When assessing HR using photoplethysmography technology, additional factors that affect accuracy of HR measurement should be considered such as body temperature, serum hemoglobin, skin status and blood pressure. For example, blood pressure can decrease during induced SVT.
As described above, HR and ECG monitoring are both possible with smartphone applications using either a PPG technique or single lead ECG system. This is finally evolving into the identification of arrhythmia. Recent study reported that twice-weekly single-lead ECG using an AliveCor device with remote interpretation in ambulatory patients ≥65 years of age at increased risk of stroke was significantly more likely to identify incident AF than routine care over a 12-month period.8) McManus et al.5) reported that pulse-based discrimination of AF, premature atrial contraction, and premature ventricular contraction were possible using a novel app that they developed. Notably, however, Wackel et al.10) attempted to measure HRs during SVT in pediatric patients using smartphone applications. But they found that the accuracy and reliability of smartphone applications was below that needed to confidently detect SVT in children, especially for a HR >200 bpm.
Many patients have now used wrist-worn wearable devices to measure their HR when palpitation occurs instead of smartphone applications but studies to validate these devices especially in arrhythmic situations are scarce. To the best of our knowledge, this is the first study to validate the accuracy of wrist-worn wearable devices in measuring induced SVT and rapidly paced HR.
Despite the accuracy of wearable devices, there is an inherent limitation of arrhythmia detection with HR based monitoring device. Sudden increment of HR might be a clue of SVT in some patients and provide reasonable suspicion of SVT for clinician especially in a situation lacking evidence in other exams. Currently, it is impossible to diagnose arrhythmia and to distinguish arrhythmia type using only pulse-based monitoring system. The accuracy of the device can be predicted, but there is always concern about whether it can improve the accuracy of the diagnosis. To increase the diagnostic value, prospective study is needed to increase the diagnostic rate in patients with symptoms of palpitation, not in the laboratory.
In our pacing study, the accuracy of the wrist-worn wearable devices ranged from 70% to 100% and differences between devices were observed. These results were different from those of actual SVT HR measurement study. Although the wrist-worn devices showed consistent accuracy of HR measurement during induced SVT, the accuracy of the devices was influenced by PCL during atrial stimulation. This result may suggest the accuracy of these devices could be affected by cycle length of tachycardia. And there may be a difference in technology between the devices if the accuracy is different. However, the number of patients who were attempted to measure at higher atrial pacing rates were small (mainly due to loss of 1:1 antegrade atrioventricular conduction). In addition, pacing study is not a real arrhythmic situation. Therefore, to interpret these discrepancies, further studies with a larger number of patients are needed.
On the other hand, we found that wearable devices show acceptable accuracy in HR measurement during rapid ventricular pacing. This suggests that they could possibly be used to measure HR in cases of hemodynamically stable ventricular tachycardia. But differences between devices were still observed.
There are several limitations in our study. First, we enrolled only a small number of patients. Second, as described above, we measured regular SVT HR in very stable patients in motionless state.6)10) Third, we did not consider the skin color or the skin photosensitivity that could affect the accuracy of the PPG technique.11) And all of our patients were Asian. Fourth, blood pressure could not be taken together with all HR measurements. Those measurements which were classified as “failed” or “low accuracy” may have been affected by a low blood pressure. Invasive arterial blood pressure monitoring was also not performed for our current study patients because it is not routine practice for SVT ablation in our hospital.
In conclusion, the accuracy of wrist-worn wearable devices in the measurement of baseline and SVT HR during an EP study is high. These devices also show an acceptable accuracy during atrial and ventricular pacing studies which simulate spontaneous tachycardia. Further validation of these devices using a well-designed, adequately powered study is needed.

Notes

Conflict of Interest The authors have no financial conflicts of interest.

Author Contributions

  • Conceptualization: Hwang J, Kim J, Choi KJ, Nam GB.

  • Data curation: Hwang J, Kim J, Choi KJ, Cho MS, Nam GB.

  • Formal analysis: Hwang J, Kim J, Choi KJ, Cho MS, Nam GB.

  • Investigation: Hwang J, Kim J, Choi KJ, Cho MS, Nam GB, Kim YH.

  • Methodology: Hwang J, Kim J, Choi KJ, Cho MS, Nam GB, Kim YH.

  • Project administration: Kim J, Choi KJ, Nam GB, Kim YH.

  • Resources: Kim J.

  • Supervision: Kim J, Choi KJ, Nam GB.

  • Validation: Hwang J, Kim J, Cho MS, Nam GB.

  • Visualization: Hwang J, Cho MS, Kim YH.

  • Writing - original draft: Hwang J.

  • Writing - review & editing: Kim J, Choi KJ, Cho MS, Nam GB, Kim YH.

References

1. Carpenter A, Frontera A. Smart-watches: a potential challenger to the implantable loop recorder? Europace. 2016; 18:791–793.
crossref pmid
2. Scully CG, Lee J, Meyer J, et al. Physiological parameter monitoring from optical recordings with a mobile phone. IEEE Trans Biomed Eng. 2012; 59:303–306.
crossref
3. McManus DD, Lee J, Maitas O, et al. A novel application for the detection of an irregular pulse using an iPhone 4S in patients with atrial fibrillation. Heart Rhythm. 2013; 10:315–319.
crossref pmid
4. Chan PH, Wong CK, Poh YC, et al. Diagnostic performance of a smartphone-based photoplethysmographic application for atrial fibrillation screening in a primary care setting. J Am Heart Assoc. 2016; 5:e003428.
crossref
5. McMANUS DD, Chong JW, Soni A, et al. PULSE-SMART: pulse-based arrhythmia discrimination using a novel smartphone application. J Cardiovasc Electrophysiol. 2016; 27:51–57.
crossref pmid
6. Wang R, Blackburn G, Desai M, et al. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol. 2017; 2:104–106.
crossref pmid
7. Allen J. Photoplethysmography and its application in clinical physiological measurement. Physiol Meas. 2007; 28:R1–R39.
crossref
8. Halcox JP, Wareham K, Cardew A, et al. Assessment of remote heart rhythm sampling using the AliveCor heart monitor to screen for atrial fibrillation: the REHEARSE-AF study. Circulation. 2017; 136:1784–1794.
pmid
9. Case MA, Burwick HA, Volpp KG, Patel MS. Accuracy of smartphone applications and wearable devices for tracking physical activity data. JAMA. 2015; 313:625–626.
crossref pmid
10. Wackel P, Beerman L, West L, Arora G. Tachycardia detection using smartphone applications in pediatric patients. J Pediatr. 2014; 164:1133–1135.
crossref pmid
11. Spierer DK, Rosen Z, Litman LL, Fujii K. Validation of photoplethysmography as a method to detect heart rate during rest and exercise. J Med Eng Technol. 2015; 39:264–271.
crossref pmid
TOOLS
ORCID iDs

Jongmin Hwang
https://orcid.org/0000-0001-9710-0945

Jun Kim
https://orcid.org/0000-0002-3573-638X

Kee-Joon Choi
https://orcid.org/0000-0002-9055-049X

Min Soo Cho
https://orcid.org/0000-0001-9867-4991

Gi-Byoung Nam
https://orcid.org/0000-0003-4391-5406

You-Ho Kim
https://orcid.org/0000-0003-0063-6300

Similar articles