Accuracy of methods for detecting an irregular pulse and suspected atrial fibrillation: A systematic review and meta-analysis

Abstract

Background

Pulse palpation has been recommended as the first step of screening to detect atrial fibrillation. We aimed to determine and compare the accuracy of different methods for detecting pulse irregularities caused by atrial fibrillation.

Methods

We systematically searched MEDLINE, EMBASE, CINAHL and LILACS until 16 March 2015. Two reviewers identified eligible studies, extracted data and appraised quality using the QUADAS-2 instrument. Meta-analysis, using the bivariate hierarchical random effects method, determined average operating points for sensitivities, specificities, positive and negative likelihood ratios (PLR, NLR); we constructed summary receiver operating characteristic plots.

Results

Twenty-one studies investigated 39 interventions (n = 15,129 pulse assessments) for detecting atrial fibrillation. Compared to 12-lead electrocardiography (ECG) diagnosed atrial fibrillation, blood pressure monitors (BPMs; seven interventions) and non-12-lead ECGs (20 interventions) had the greatest accuracy for detecting pulse irregularities attributable to atrial fibrillation (BPM: sensitivity 0.98 (95% confidence interval (CI) 0.92–1.00), specificity 0.92 (95% CI 0.88–0.95), PLR 12.1 (95% CI 8.2–17.8) and NLR 0.02 (95% CI 0.00–0.09); non-12-lead ECG: sensitivity 0.91 (95% CI 0.86–0.94), specificity 0.95 (95% CI 0.92–0.97), PLR 20.1 (95% CI 12–33.7), NLR 0.09 (95% CI 0.06–0.14)). There were similar findings for smartphone applications (six interventions) although these studies were small in size. The sensitivity and specificity of pulse palpation (six interventions) were 0.92 (95% CI 0.85–0.96) and 0.82 (95% CI 0.76–0.88), respectively (PLR 5.2 (95% CI 3.8–7.2), NLR 0.1 (95% CI 0.05–0.18)).

Conclusions

BPMs and non-12-lead ECG were most accurate for detecting pulse irregularities caused by atrial fibrillation; other technologies may therefore be pragmatic alternatives to pulse palpation for the first step of atrial fibrillation screening.

Introduction

Atrial fibrillation (AF) has a prevalence that increases with age.^1,2 AF is associated with significant morbidity and mortality, most notably from its associated four to fivefold increased risk of ischaemic stroke, ³ and poses a significant public health burden. ⁴ The SAFE trial was the largest randomised study of AF screening in primary care and found this to be an effective method for increasing AF detection when compared to routine practice.^5,6 Combined with the subsequent provision of antithrombotic therapy,^7,8 screening is likely to reduce thromboembolic complications from AF ⁹ and, consequently, this has been proposed to improve AF.^10,11

AF screening is a two-stage process. Firstly, asymptomatic patients with irregular pulses are identified and then AF is confirmed or excluded using 12-lead electrocardiography (ECG).^8,10 The accuracy with which irregular pulses are caused by AF is important; a high false positive rate would result in many patients having unnecessary ECG examinations, whereas a high false negative rate could mean excessive AF diagnoses are missed.

The SAFE trial used opportunistic pulse palpation to identify patients over 65 years of age with an irregular pulse. Although this approach was found to be effective, no other methods for identifying irregular pulses have been tested in trials of AF screening. Pulse palpation may not be the optimal ‘first step’ in identifying patients with suspected AF and using modified blood pressure (BP) monitors, pulse oximetry or non-12-lead ECG devices, for example, could improve the process of screening. ¹² Despite pulse palpation being a cheap method for detecting AF, this results in high false positive cases of suspected AF and it may be possible to improve the accuracy of identifying patients with suspected AF using alternative methods. This systematic review aimed to provide evidence to underpin the first stage of AF screening by describing and comparing the diagnostic accuracies of methods for identifying patients with irregular pulses caused by AF.

Methods

Search strategy and selection criteria

This study was conducted in accordance with guidelines and methods for systematic reviews and meta-analyses of diagnostic tests.^13–16 We searched MEDLINE, EMBASE, Cumulative Index to Nursing and Allied Health (CINAHL) and Latin American and Caribbean Health Sciences Information System (LILACS) in all languages published until 16 March 2015 (Appendix 1). In addition, the reference lists of national guidelines, review articles and included studies were hand-searched to identify eligible studies. All randomised trials and observational studies, with the exclusion of case reports and case series, which recruited participants ≥18 years of age, investigated any method of identifying patients with an irregular pulse or suspected AF (the index test) and compared the index test with any ECG interpreted by a competent professional (the reference standard), involved healthcare professionals identifying patients with an irregular pulse, and reported sufficient data to enable calculation of diagnostic accuracy were included. Studies that investigated invasive or echocardiographic methods of identifying AF were excluded. After the removal of duplicate records, two reviewers (JT and MJ) independently screened citations for relevance and reviewed full-text articles using predetermined eligibility criteria. Any disagreements were resolved by consensus with a third reviewer (TC).

Data extraction

Data were independently extracted using a prespecified data extraction form by two reviewers (JT and MJ). Any disagreements were resolved by consensus with a third reviewer (TC). When studies reported findings using multiple thresholds for the same intervention, only data in which thresholds maximised the sensitivity of the index test were extracted in order to avoid duplicate inclusion of the same index test. The lead authors of studies for which there were insufficient data for calculation of diagnostic accuracy were contacted to ascertain missing data; studies were only excluded if no additional data were identified or if the authors failed to respond.

Study quality was appraised using the quality assessment of diagnostic accuracy studies-2 (QUADAS-2) instrument.^15,17 In addition, the studies were graded using the quality scale reported by Van den Bruel et al.; ¹⁸ studies were rated as grade A if they fulfilled all QUADAS-2 criteria. Studies were graded D if there was no or unclear verification of the index test findings with the reference standard, or if the index test results were interpreted unblinded to the results of the reference test. Studies in which there was an unduly long time delay between index and reference tests, or the reference test was not independent of the index test, or the reference test was interpreted unblinded to the results of the index test were graded C. The remaining studies that did not fall into these categories were graded B.

Statistical analysis

We constructed 2 × 2 contingency tables to enable calculation of the sensitivity and specificity of each method for identifying patients with suspected AF. Meta-analysis, using the bivariate hierarchical random effects method, was used to determine the average operating points for sensitivity and specificity, and enabled the construction of summary receiver operating characteristic (SROC) plots with 95% prediction regions. We also calculated positive and negative likelihood ratios (PLR and NLR) for each detection method. Unlike sensitivity and specificity, likelihood ratios make explicit the impact of the test result on the probability of disease. To minimise heterogeneity we analysed the results a priori in groups of each method for identifying an irregular pulse. Subgroup analyses were planned according to study quality and studies conducted in primary care providing there were four or more studies within subgroups. Heterogeneity is presumed in meta-analyses of diagnostic test studies and the I² statistic cannot be reliably used for its assessment. ¹⁶ We therefore described any variation in the outcomes from included studies and our pooled estimates by visual inspection of the SROC plots and how close individual studies lie to the predicted receiver operating characteristic curve. ¹⁶ Publication bias was assessed within categories of method for detecting AF using Deeks’ funnel plot asymmetry test; a P value less than 0.10 was used to signify the presence of publication bias. Analyses were conducted using Stata version 11.0 and Review Manager 5.2 for quality assessments.

Results

After the removal of duplicate records we identified 5418 potential citations. From these, 69 were identified for detailed evaluation (Figure 1). After full-text review, 21 studies were included in the final analyses (Appendix 2).^9,19–38 Five studies met the selection criteria, but reported insufficient outcome data and were excluded (Appendix 3).^39–43

Figure 1.

Study selection and stratification.

Study characteristics

Of the 21 studies included (Appendix 2), there were two randomised trials,^9,30 seven case–control,^{20,21,26,32–35} two cohort^24,38 and 10 cross-sectional^{19,22,23,25,27–29, 31,36,37} studies. The 21 studies investigated 39 interventions (n = 15,129 pulse assessments), which were categorised as blood pressure monitors (BPMs) (six studies; seven interventions),^{25,28,33,36–38} non-12-lead ECG (10 studies; 20 interventions),^{9,19–22,24,25,31,32,35} pulse palpation (six studies; six interventions)^{9,25,27,30,32,34} and smartphone applications (three studies; six interventions).^23,26,29 The five studies that were excluded due to insufficient reporting of outcome data investigated pulse palpation, pulse oximetry, smartphone applications and single-lead ECG as methods for detecting AF. Although the majority of studies avoided a case–controlled design, only four were prospective and seven studies were conducted in a primary care setting.^{9,25,27,30,32,34,35} AF prevalence ranged from 5.7% to 25.4% in studies with a prospective design.^9,24,30,38 There was substantial variation in the proportion and/or prevalence of AF in studies within each category of detection method.

Five studies excluded participants if they were less than 65 years of age and for two studies age of inclusion was 75 years.^{9,25,27,32–34,38} Six studies included participants who were 18 years or older.^{24,28–31,36} Ten studies excluded patients that had been fitted with pacemakers and/or implantable defibrillators.^{22,25,27,28,33,35–38}

For the majority of studies, the reference standard was 12-lead ECG interpreted by at least one trained physician/cardiologist. One study did not specify the training of the clinician interpreting reference ECGs. ¹⁹ Five studies reported other reference standards;^{27,29,30,34,38} one study used either 12-lead ECG or ECGs derived from cardiac telemetry; ²⁹ three studies used single or limb-lead ECG;^27,30,34 one study used ECGs derived from Holter monitors. ³⁸

Quality assessment

The methodological quality of included studies using QUADAS-2 criteria is presented in Figure 2. Study quality was generally low. Using the additional quality grading system, we classified one study as A grade having met all QUADAS-2 criteria. ³⁰ Eleven studies were graded category C or D.^{9,19,21–23,27,29,32–34,36} Studies with the lowest methodological quality (D grade) were classified as this due to either the interpretation of the reference standard being unclear or at high risk of bias, or due to the index test being interpreted unblinded to the results of the reference standard. Category C studies were graded as such because it was unclear whether there was an appropriate time interval between the index test and reference standard. The remaining nine studies were categorised as grade B in methodological quality.^{20,24–26,28,31,35,37,38}

Figure 2.

Study quality according to QUADAS-2 criteria.

Data synthesis

Forest plots for diagnostic accuracies of the four methods for detecting AF are presented (Appendices 4–7). BPMs had a pooled sensitivity of 0.98 (95% confidence interval (CI) 0.92–1) and specificity of 0.92 (95% CI 0.88–0.95); PLR of 12.1 (95% CI 8.2–17.8) and NLR of 0.02 (95% CI 0.00–0.09). There were similar diagnostic accuracies for studies that investigated smartphone applications, sensitivity 0.97 (95% CI 0.95–0.99), specificity 0.95 (95% CI 0.88–0.98), PLR 19 (95% CI 8–45), NLR 0.03 (95% CI 0.01–0.05); and non-12-lead ECGs, sensitivity 0.91 (95% CI 0.86–0.94), specificity 0.95 (95% CI 0.92–0.97), PLR 20.1 (95% CI 12–33.7), NLR 0.09 (95% CI 0.06–0.14). Although pulse palpation had a sensitivity that was comparable to the other methods for detecting suspected AF (sensitivity of 0.92; 95% CI 0.85–0.96), there was a substantially lower specificity for this method (specificity 0.82; 95% CI 0.76–0.88); PLR and NLR for pulse palpation were 5.2 (95% CI 3.8–7.2) and 0.1 (95% CI 0.05–0.18), respectively.

SROC plots for the methods of detecting AF are presented in Figure 3. Visual inspection of the plots confirms the accuracy of pulse palpation was lower than other methods for detecting AF. There was substantial variation in outcomes of the studies investigating non-12-lead ECG from the predicted receiver operating characteristic curve and suggests the heterogeneity among these studies was greatest. In contrast, the heterogeneity was lowest among studies that investigated smartphone applications for detecting AF.

Figure 3.

Summary receiver operating characteristic plots of interventions for the detection of atrial fibrillation.

There was no evidence of publication bias; Deeks’ funnel test P = 0.34, P = 0.11, P = 0.14 and P = 0.27 for studies investigating BPMs, non-12-lead ECG, smartphone applications and pulse palpation, respectively.

There were only sufficient studies to perform bivariate subgroup analyses according to study quality for BPM, non-12-lead ECG and pulse palpation interventions. After the exclusion of studies with the lowest (D grade) quality, there were no substantial differences to the primary findings. (BPMs: sensitivity 0.96 (95% CI 0.91–0.98), specificity 0.93 (95% CI 0.89–0.96); non-12-lead ECG: sensitivity 0.92 (95% CI 0.86–0.95), specificity 0.94 (95% CI 0.91–0.97); pulse palpation: sensitivity 0.93 (95% CI 0.86–0.97), specificity 0.81 (95% CI 0.76–0.85).)

Sufficient studies to perform bivariate subgroup analyses for primary care studies were available for pulse palpation and non-12-lead ECG interventions. The findings were similar to our primary analyses, although the specificity of non-12-lead ECGs was slightly lower (non-12-lead ECGs: sensitivity 0.91 (95% CI 0.83–0.95), specificity 0.89 (95% CI 0.85–0.92); pulse palpation: all studies were conducted in primary care and findings already presented above).

Discussion

This review of 21 studies for methods of detecting irregular pulses caused by AF found modified BPMs and non-12-lead ECG devices had the greatest diagnostic accuracy. Although the sensitivities of all methods for identifying those with AF were similar, the specificity of pulse palpation was substantially lower, which gives rise to more false positive test results.

Strengths and limitations

This study is the first systematic review and meta-analysis of interventions for detecting suspected AF. A strength was the use of a protocol, consistent with guidelines for systematic reviews of diagnostic test studies. We used a comprehensive search strategy and our findings supported the lack of publication bias. It is therefore likely that we included small studies with less significant findings. Five studies were excluded from meta-analyses due to insufficient reporting of outcome data; outcomes that were reported from these studies were consistent with our pooled findings and the effect of their exclusion is likely to be minimal. Only four of the 21 included studies were prospective and there were a number of methodological weaknesses in most studies, as reflected by our assessments of study quality; only one study was judged to fulfil all QUADAS-2 criteria. Most studies were conducted in a secondary care setting and there was substantial variation in the proportion of patients with AF. This limits the generalisability of our findings to primary care populations that AF screening is intended for. As expected, there was heterogeneity among studies within all intervention categories and this is likely to be attributable to differences in study populations and design. Variation was greatest for studies investigating non-12-lead ECG for detecting AF. This may be due to differences in detection methods within this category as non-12-lead ECG interventions included single-lead, three-lead and reconstructed ECG. Furthermore, difference in healthcare professionals used to interpret ECGs is also likely to contribute to the heterogeneity within this intervention category.

Findings in context of previous research

Guidance recommends pulse palpation as the first step for AF screening.^10,11 This method of identifying those with an irregular pulse was evaluated as part of two randomised trials in primary care; combined with confirmatory 12-lead ECG for diagnosing AF; these studies suggest this is an efficacious and cost-effective approach to screening.^9,30 More recently, studies have tended to evaluate alternative screening technologies. Our review identified four methods (non-12-lead ECG, BPMs, smartphone applications and pulse oximetry) as alternative methods for detecting pulse irregularities, although the latter method was not eligible for inclusion in our analyses. Of all interventions, pulse palpation had the lowest diagnostic accuracy for detecting pulse irregularities caused by AF, as reflected by its lower specificity. This could be due to differences in the threshold of each method to rule in or out AF; electronic methods, such as BPMs, use software algorithms to determine the severity of pulse irregularity and only those patients meeting a predetermined cut-off point are classified as having AF. In contrast, studies investigating pulse palpation required clinicians to classify the pulse as being regular or irregular. It is therefore conceivable that pulse palpation could result in greater false positive cases of suspected AF due to patients with transient pulse irregularities (e.g. ventricular extra-systoles) being referred for ECG testing, which software algorithms would have excluded.

Although pulse palpation is considered a cheap and feasible method for detecting patients with an irregular pulse, ¹² our review highlights the potential for alternative technologies to improve the identification of those with suspected AF. Any method of detecting pulse irregularities caused by AF should, in addition to being cheap, be quick, simple and accurate for it to be a cost-effective intervention in primary care. Recent guidance from the National Institute for Health and Care Excellence (NICE) advocates the use of automated BPMs for the detection of suspected AF in patients being screened or monitored for hypertension. ⁴⁴ We found BPMs to have a substantially greater accuracy for detecting pulse irregularities caused by AF than pulse palpation; such devices are likely to be pragmatic alternatives to pulse palpation as blood pressure checks are an integral component of existing cardiovascular screening programmes in primary care. ⁴⁵ Furthermore, automated devices would enable screening to be conducted by all healthcare professionals without the need for additional training. However, to date there have been no economic analyses comparing BPMs to pulse palpation for detecting suspected AF and this would help to inform optimal planning and service configurations of any future AF screening programme.

Our review highlights the potential of smartphone applications for detecting suspected AF. This method had a similar diagnostic accuracy as BPMs. However, only three studies investigated this detection modality; two of these investigated multiple software algorithms in the same cohort of patients and all studies were small in sample size. This is likely to inflate our estimates of diagnostic accuracy for smartphone applications, and our findings should be interpreted with caution. However, if these findings are replicated in larger studies that represent those targeted by screening, this raises the possibility of using such technologies within both the clinic and home settings.

The second stage of AF screening is diagnosing AF in patients with an irregular pulse. The gold standard test for diagnosing AF is 12-lead ECG interpreted by a competent professional.^8,11 The range and accuracies of methods for diagnosing AF using 12-lead ECG is not known. Indeed, optimising the accuracy of both stages to AF screening will help ensure the effectiveness of this intervention and planning of appropriate service configuration for delivering screening.

Conclusion

BPMs and non-12-lead ECG devices had a greater accuracy for detecting pulse irregularities caused by AF than pulse palpation. Newer technologies may therefore be a pragmatic alternative to pulse palpation for identifying patients with suspected AF as part of national screening programmes.