Sage Journals: Discover world-class research

Abstract

Background:

Lyme disease, caused by infection with Borrelia burgdorferi, is the most common vector-borne disease in the United States. The standard two-tier testing (STTT) algorithm suffers from low sensitivity, misinterpretation, and long turnaround time, preventing timely detection and treatment. To address these challenges, we hypothesized that the canine point-of-care (PoC) SNAP 4Dx Plus test used to detect Borrelia burgdorferi antibodies could be employed for human diagnosis.

Materials and Methods:

The SNAP 4Dx Plus testing was conducted in accordance with the manufacturer’s instructions, with results read by manual inspection. All analyses were conducted using R version 4.3.1, and agreement between the PoC assay and the STTT was assessed using kappa statistics with GraphPad software.

Results:

We included 102 previously-tested human serum samples, of which 19 samples (18.6%) were STTT positive. Compared to the STTT, the SNAP 4Dx Plus test demonstrated a low sensitivity of 0.16 (95% CI 0.03 to 0.40).

Conclusion:

Overall, our results do not support the use of the SNAP 4Dx Plus LD assay for the diagnosis of human Lyme disease. Differences in antibody concentrations between human and canine samples may partly explain our findings.

Background

Lyme disease (LD), caused by infection with Borrelia burgdorferi (Bb) and transmitted through the bite of Ixodes species ticks, is the most common vector-borne disease in the United States (Mead, 2015; Rosenberg et al., 2018). Approximately 40,000 cases of LD are reported annually, although, estimates using alternative surveillance approaches suggest that upward of ten times this many cases are diagnosed each year (Kugeler et al., 2021). Conventional serological testing methods require a venous blood draw followed by multiple assays performed in a stepwise manner (i.e., two-tier testing) in a clinical laboratory. The standard two-tier testing (STTT) algorithm utilizes an enzyme immunoassay (EIA) for immunoglobulin M (IgM) and immunoglobulin G (IgG) as the first tier, followed by a western blot (WB) for confirmatory testing (i.e., second tier), if first tier results are positive or equivocal. The IgM component of the STTT algorithm suffers from a high false-positive rate (Lantos et al., 2016; Seriburi et al., 2012), particularly early in the course of the disease, while during the same period, the IgG component is less sensitive. Immunoblots are also relatively labor-intensive and misinterpretation of the results is common (Boyce et al., 2020). While the Centers for Disease Control and Prevention recently endorsed a modified two-tier testing (MTTT) algorithm that replaces the immunoblot with a second EIA (Mead et al., 2019), this approach still requires laboratory infrastructure with results available only after 48–72 h. Therefore, improving access to testing and early detection through the development of rapid, point-of-care (PoC) tests could simplify LD diagnosis and expedite treatment, while also yielding improved surveillance data.

The SNAP 4Dx Plus test (IDEXX Reference Laboratories, Inc., Westbrook, ME, USA) is widely used in veterinary settings for annual heartworm and tick-borne disease screening in canines. The test utilizes a C6 peptide for the detection of canine antibodies against B. burgdorferi (Beall et al., 2022). The SNAP 4Dx Plus test is an enzyme-linked immunosorbent assay (ELISA) that can be completed at the PoC providing results in under 10 min (O’Connor, 2015). When compared with WB results, the SNAP 4Dx Plus test demonstrated a high level of sensitivity (95.5%) and specificity (99.4%) when testing canine samples from the northeastern United States, where Lyme is endemic (Beall et al., 2022). Notably, the SNAP 4Dx Plus uses species “agnostic” reagents, meaning that—at least in principle—the test should identify anti-Borrelia antibodies in human specimens. Interestingly, a single-tier C6 peptide ELISA has demonstrated somewhat higher sensitivity and slightly lower specificity compared with the STTT for the diagnosis of LD in humans, although uptake is limited (Wormser et al., 2013).

In contrast to the SNAP 4Dx Plus, the Sofia^® 2 Lyme+ fluorescent immunoassay (FIA) (QuidelOrtho. San Diego, CA, USA) is a commercially available, the US Food and Drug Administration (FDA)-approved lateral-flow assay intended to be used as a first-tier test for LD in humans at PoC (QuidelOrtho, 2024). The Sofia^® 2 Lyme+ FIA provides a rapid turnaround to detect IgM and IgG Bb antibodies with a reported sensitivity of 92.3% and specificity of 90.1% for Lyme IgM, and sensitivity of 96.4% and specificity of 95.2% for IgG as the first test of the STTT, which must then be followed by a second-tier WB (QuidelOrtho, 2024). However, previous studies have evaluated the performance of the Sofia^® Lyme+ 2 FIA against high-incidence settings (Lewandrowski et al., 2023), emphasizing the need for studies in low-incidence settings.

We sought to evaluate the potential utility of the SNAP 4Dx Plus test for the serodiagnosis of LD in human subjects. We hypothesized that the SNAP 4Dx Plus test in humans would be performed comparably with the STTT. To compare our approach with existing PoC options in lower transmission settings, we also assessed the performance of the Sofia^® 2 Lyme+ FIA in parallel.

Methods

We assembled a test panel of 105 human serum samples that had previously undergone STTT for LD between August 2021 and June 2023. Among those, 101 samples were collected as part of routine care based on the determination of the clinical provider across the University of North Carolina Health System. Given the low number of LD seropositive samples available, which was only 15, 4 additional STTT-positive samples were provided by Galaxy Diagnostics (Research Triangle Park, NC). Relevant clinical information was abstracted from electronic medical records to classify the onset of symptoms as either (i) <30 days, in which case the IgM WB result was considered in the result determination, or (ii) >30 days, in which case the IgM WB result was not considered. Accordingly, 19 samples (18.1%) were identified as STTT positive and 86 samples (81.9%) as STTT negative (Table 1).

Table 1.

Summary of Standard Two-Tier Testing Results for Samples Included in Panel (n = 105)

Test result	N	%	Interpretation
First-tier EIA (−)	54 (51.4%)		Negative
First-tier EIA (+)^a	51 (48.6%)		—
WB IgM (−), IgG (−)	24 (22.9%)		Negative
WB IgM (−), IgG (+)	4 (3.8%)		Positive
WB IgM (+), IgG (+)	7 (6.7%)		Positive
WB IgM (+), IgG (−)	—	—
Symptoms ≤30 days	13 (12.4%)	8 (7.6%)	Positive
		5^b (4.8%)	Negative
Symptoms >30 days	3 (2.9%)	—	Negative

Western blot is subsequently conducted if the first-tier EIA is positive or equivocal, following the STTT algorithm.

Subsequently retested as part of routine care, but failed to develop IgG response suggestive of false-positive IgM (Seriburi et al., 2012).

EIA, enzyme immunoassay; IgG, immunoglobulin G; IgM, immunoglobulin M; WB, western blot.

The SNAP 4Dx Plus testing was conducted in accordance with the manufacturer’s instructions with results read by manual inspection (IDEXX, 2021). Similarly, the Sofia^® 2 Lyme+ FIA yielded the IgM and IgG results using a Sofia^® 2 analyzer and cassette following the instructions (QuidelOrtho, 2024). Study staff, who were blinded to two-tier results, recorded the results of the PoC assays. Two independent readers interpreted test results with disagreements resolved by a third reviewer. Insufficient sera were available for six (5.7%) samples, in which cases we were not able to perform one of the assays.

The SNAP 4Dx Plus test utilizes a C6-peptide-based assay, the test performance of which was previously found to be similar to the two-tier testing algorithm (Wormser et al., 2013). Therefore, we considered a positive test equivalent to the combined (i.e., first and second tier) results of the STTT, which was used as the reference standard for the determination of test performance (e.g., sensitivity, specificity). In contrast, Sofia^® utilizes a total IgM/IgG immunoassay so results were compared only with the first-tier EIA results of the STTT, in accordance with the FDA approval. As a post hoc secondary analysis, we explored the test performance of a hypothetical “all PoC” algorithm employing the Sofia^® 2 FIA results as the first-tier test and the SNAP 4Dx Plus test as the second tier when the Sofia^® was positive. The results obtained through our newly developed algorithm were compared with the STTT results. All analyses were conducted using the “epiR” package in R version 4.3.1 (Stevenson et al., 2023). Additionally, agreement between the PoC assays and the STTT was obtained using the kappa statistics, with the level of agreement using publicly available GraphPad software (GraphPad, 2024). We classified the strength of agreement quantifying the kappa statistics as no agreement (<0), slight (0–0.20), fair (0.21–0.40), moderate (0.41–0.60), substantial (0.61–0.80), and almost perfect (0.81–1.00) (Landis and Koch, 1977).

The collection of the remnant samples and associated clinical data utilized in the study was approved by the institutional review board of the University of North Carolina at Chapel Hill (IRB 21-0356).

Results

We successfully performed the SNAP 4Dx Plus testing on 102 of the 105 (97.1%) samples after excluding 3 samples owing to insufficient volume. Compared with the STTT, the sensitivity of the SNAP 4Dx Plus test was 15.8% (95% confidence interval [CI] 3.4–39.6%), while the specificity was 98.8% (95% CI: 93.5–100%) (Table 2). The SNAP 4Dx Plus test had a lower sensitivity in comparison with the previously reported sensitivity (64%) obtained with a single C6-peptide EIA (Branda et al., 2011). It displayed a fair agreement by the kappa statistics between the SNAP test and the STTT (κ = 0.21, 95% CI: −0.01 to 0.43).

Table 2.

IDEXX SNAP 4Dx Plus Versus Standard Two-Tier Testing (n = 102)

SNAP 4Dx Plus	STTT
SNAP 4Dx Plus	Positive	Negative	Total
Positive	3	1	4
Negative	16	82	98
Total	19	83	102
Sensitivity, n/N (%) [exact 95% CI]	3/16 (15.8%) [3.4–39.6%]
Specificity, n/N (%) [exact 95% CI]	82/83 (98.8%) [93.5–100%]

CI, confidence interval; STTT, standard two-tier testing.

A total of 97 samples were tested with the Sofia^® 2 Lyme+ FIA as 8 samples had insufficient volume remaining to perform testing. The sensitivity of the Sofia^® 2 Lyme+ FIA was 58.1% (95% CI: 42.1–73.0%) and the specificity was 85.2% (95% CI: 72.9–93.4%) when compared with the STTT EIA result (Table 3). The kappa statistics between the Sofia^® FIA and the first tier of the STTT indicated a moderate agreement (κ = 0.44, 95% CI: 0.27–0.62).

Table 3.

QuidelOrtho Sofia^® 2 Lyme+ FIA Versus First-Tier Enzyme Immunoassay of the Standard Two-Tier Testing (n = 97)

Sofia^® 2 Lyme+ FIA	First-tier EIA
Sofia^® 2 Lyme+ FIA	Positive	Negative	Total
Positive	25	8	33
Negative	18	46	64
Total	43	54	97
Sensitivity, n/N (%) [exact 95% CI]	25/43 (58.1%) [42.1–73.0%]
Specificity, n/N (%) [exact 95% CI]	46/54 (85.2%) [72.9–93.4%]

CI, confidence interval; FIA, fluorescent immunoassay; EIA, enzyme immunoassay.

When combing these results in our hypothetical all-PoC algorithm, the two-tier algorithm employing the Sofia^® assay as the first tier followed by confirmation with the SNAP 4Dx (second tier) did not improve diagnostic performance in terms of sensitivity (21.4%, 95% CI: 4.7–50.8%) and specificity (100%, 95% CI: 96.6–100%) (Table 4). The kappa between the two algorithms also demonstrated a fair agreement (κ = 0.32, 95% CI: 0.04–0.60).

Table 4.

QuidelOrtho Sofia^® 2 Lyme+ FIA Followed by IDEXX SNAP 4Dx Plus Versus Standard Two-Tier Testing, SNAP 4Dx Plus as the Confirmatory Test of the Sofia^® 2 Lyme+ FIA Positives (n = 97)

Sofia^® 2 Lyme+ FIA and SNAP 4Dx Plus	STTT
Sofia^® 2 Lyme+ FIA and SNAP 4Dx Plus	Positive	Negative	Total
Positive	3	0	3
Negative	11	83	94
Total	14	83	97
Sensitivity, n/N (%) [exact 95% CI]	3/14 (21.4%) [4.7–50.8%]
Specificity, n/N (%) [exact 95% CI]	83/83 (100%) [96.6–100%]

Discussion

Our study explored the performance of a veterinary C6-peptide-based lateral flow assay designed to test dogs for B. burgdorferi exposure for the serological diagnosis of LD in humans. Compared with the results of STTT, the SNAP 4Dx Plus test demonstrated low sensitivity (15.8%) against our panel of stored human sera. Similarly, we observed a lower sensitivity with the Sofia^® FIA than previously reported, when compared with the STTT first-tier EIA result. Both tests showed high specificity. When we attempted to combine these assays, the low sensitivity of the SNAP 4Dx Plus test substantially limited the performance. Overall, these findings suggest that, despite the demonstrated accuracy in dogs and potential logistical benefits, the SNAP 4Dx Plus test cannot be recommended as a diagnostic tool for human disease, at least when using sera as the sample source.

There are multiple factors that may explain the relatively poor performance of the SNAP 4Dx Plus test. Because the test was not developed for human use, there are no standard references regarding the C6 antibody levels nor direct comparisons between human and canine levels. In human LD diagnosis, the C6 peptide EIA considers a C6 index of 2.4 as the appropriate positive cut-off point in the first tier and 3.0 in the second tier to ensure high sensitivity (Rouhiainen et al., 2021). However, in canine LD, the Lyme Quant C6 Antibody Test uses a C6 antibody level of >30 U/mL as significant (IDEXX, 2021). Any difference in antibody levels between humans and canines, which likely determines the different diagnostic thresholds, may largely explain the SNAP 4Dx’s low sensitivity derived in our study.

Second, we employed frozen human sera instead of fresh whole blood specimens obtained by fingerstick. This was largely done out of convenience as whole blood is not always collected as part of the routine clinical evaluation of patients suspected of having LD and even when it is, it is generally discarded relatively quickly after testing is complete (e.g., blood counts). Even though the Sofia^® 2 FIA package insert recommends fresh capillary blood, various serum panels were used to validate the assay as described in the product inserts (QuidelOrtho, 2024).

To our knowledge, our study is the first to compare the veterinary SNAP 4Dx Plus test with standard diagnostic assays for LD in humans. While the low sensitivity argues against human use, the study does have important limitations that may impact the validity and generalizability of this conclusion. First, our reference standard, the STTT, has its own limitations including low sensitivity in the early LD stage (Bobe et al., 2021). However, if the low sensitivity of the STTT was a concern, we would have expected higher sensitivity and lower specificity with the SNAP test owing to false negatives from the STTT (i.e., SNAP+/STTT−), but this was not what we observed. Furthermore, the information on symptoms related to LD had been collected only from patients with IgM WB positive and IgG WB negative, so it is possible that there were individuals with typical symptoms early in the course of the disease who were still without a detectable EIA. Given the relatively low LD incidence rate in central North Carolina, we suspect this is an uncommon situation. From the veterinary perspective, we are unable to compare the diagnoses of each tier as the stepwise algorithm is not employed for canines. Additionally, we evaluated the sensitivity of Sofia^® FIA in comparison with the STTT first tier, without conducting confirmatory WB for positive EIAs. The absence of this second tier may have contributed to the lower sensitivity observed in Sofia^® FIA.

Further research, including prospective studies leveraging whole blood samples, is necessary to confirm our findings. Anecdotally, we have heard of veterinarians using the tests for screening and self-diagnosis. At present, however, our findings suggest that the SNAP 4Dx Plus test should not be used for human testing.

Conclusions

Our results demonstrate that the SNAP 4Dx Plus test, effective for canine use, displayed low sensitivity for the diagnoses of LD compared with the STTT when applied to human sera. Differences in the relative levels of Bb antibodies between humans and canines may partly explain our findings but require confirmation.

Footnotes

Acknowledgment

We wish to thank Galaxy Diagnostics for providing additional positive serum samples to augment the test panel.

Authors’ Contributions

Study conception and design: R.M.B. Funding: R.M.B. Study implementation: P.P., L.U., D.G., H.A.A., and L.E. Data analysis: G.L. and R.M.B. First draft of article: G.L. and R.M.B. Revisions: All authors.

Data Availability Statement

Deidentified individual data that support the results will be shared beginning 9 to 36 months following publication, provided the investigator who proposes to use the data has approval from an Institutional Review Board (IRB), Independent Ethics Committee (IEC), or Research Ethics Board (REB), as applicable, and executes a data use/sharing agreement with UNC.

Disclaimer

The content is solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services.

Author Disclosure Statement

With financial support from IDEXX Laboratories, Edward B. Breitschwerdt has lectured to veterinarians on the topic of vector-borne infectious diseases and has been a paid consultant during the development and validation of rapid assays developed by IDEXX Laboratories for veterinary medical applications. Also, in conjunction with Dr. S. Sontakke and North Carolina State University, Edward B. Breitschwerdt holds US Patent No. 7,115,385 Media and Methods for Cultivation of Microorganisms, which was issued on October 3, 2006. He is a cofounder, shareholder, and Chief Scientific Officer for Galaxy Diagnostics, a company that provides advanced diagnostic testing for the detection of Bartonella spp. infections. Barbara A. Qurollo receives partial salary support from IDEXX Laboratories and has been a paid consultant during the development and validation of rapid assays developed by IDEXX Laboratories for veterinary medical applications. IDEXX Laboratories did not have any role in the design, conduct, or analysis of the study.

All other authors declare: no financial relationships with any organizations that might have an interest in the submitted work in the previous three years except that noted in the funding section; no other relationships or activities that could appear to have influenced the submitted work.

Funding Information

Funding was provided by a “Creativity Hub” award from the UNC Office of the Vice Chancellor for Research. This publication was supported in part by Cooperative Agreement Number U01CK0006 from the Centers for Disease Control and Prevention.

References

Beall

, Mainville

, Arguello-Marin

, et al. An improved point-of-care ELISA for the diagnosis of anaplasmosis and ehrlichiosis during the acute phase of tick-borne infections in dogs. Top Companion Anim Med, 2022; 51:100735; doi: 10.1016/j.tcam.2022.100735

Bobe

, Jutras

, Horn

, et al. Recent progress in Lyme disease and remaining challenges. Front Med, 2021; 8:666554; doi: 10.3389/fmed.2021.666554

Boyce

, Speight

, Lin

, et al. Errors in diagnostic test use and interpretation contribute to the high number of Lyme disease referrals in a low-incidence state. Open Forum Infect Dis, 2020; 7(1):ofaa009; doi: 10.1093/ofid/ofaa009

Branda

, Linskey

, Kim

, et al. Two-tiered antibody testing for Lyme disease with use of 2 enzyme immunoassays, a whole-cell sonicate enzyme immunoassay followed by a VlsE C6 peptide enzyme immunoassay. Clin Infect Dis, 2011; 53(6):541–547; doi: 10.1093/cid/cir464

GraphPad. Quantify Agreement with Kappa. 2024. Available from: https://www.graphpad.com/quickcalcs/kappa1/ [Last accessed December 11, 2023 ].

IDEXX. SNAP^® Test Quick Reference Guide. 2021.

Kugeler

, Schwartz

, Delorey

, et al. Estimating the frequency of Lyme disease diagnoses, United States, 2010–2018. Emerg Infect Dis, 2021; 27(2):616–619; doi: 10.3201/eid2702.202731

Landis

, Koch

. The measurement of observer agreement for categorical data. Biometrics, 1977; 33(1):159–174; doi: 10.2307/2529310

Lantos

, Lipsett

, Nigrovic

. False positive Lyme disease IgM immunoblots in children. J Pediatr, 2016; 174:267–269.e1; doi: 10.1016/j.jpeds.2016.04.004

10.

Lewandrowski

, Branda

, Lewandrowski

, et al. Evaluation of the rapid quidel sofia 2 lyme immunoassay as a first-tier test in a two-tier testing algorithm for Lyme disease: Comparison to the zeus ELISA Borrelia VlsE1/pepC10 IgG/IgM assay followed by immunoblot. Am J Clin Pathol, 2023; 160(1):58–61; doi: 10.1093/ajcp/aqad007

11.

Mead

, Petersen

, Hinckley

. Updated CDC recommendation for serologic diagnosis of Lyme disease. MMWR Morb Mortal Wkly Rep, 2019; 68(32):703; doi: 10.1558/5/mmwr.mm6832a4

12.

Mead

. Epidemiology of Lyme disease. Infect Dis Clin North Am, 2015; 29(2):187–210; doi: 10.1016/j.idc.2015.02.010

13.

O’Connor

. SNAP assay technology. Top Companion Anim Med, 2015; 30(4):132–138; doi: 10.1053/j.tcam.2015.12.002

14.

QuidelOrtho. For Use with Sofia 2 Lyme+ FIA. 2024. Available from: https://techdocs.quidelortho.com/7978cfe2-c5fe-4b6b-9e1f-6074be9e08f0 [Last accessed March 11, 2024 ].

15.

Rosenberg

, Lindsey

, Fischer

, et al. Vital signs: Trends in reported vectorborne disease cases—United States and Territories, 2004–2016. MMWR Morb Mortal Wkly Rep, 2018; 67(17):496–501; doi: 10.1558/5/mmwr.mm6717e1

16.

Rouhiainen

, Pietikäinen

, Kortela

, et al. C6 peptide enzyme immunoassay in Lyme borreliosis serology. J Microbiol Methods, 2021; 180:106122; doi: 10.1016/j.mimet.2020.106122

17.

Seriburi

, Ndukwe

, Chang

, et al. High frequency of false positive IgM immunoblots for Borrelia burgdorferi in clinical practice. Clin Microbiol Infect, 2012; 18(12):1236–1240; doi: 10.1111/j.1469-0691.2011.03749.x

18.

Stevenson

, Sergeant

, Heuer

, et al. epiR: Tools for the analysis of epidemiological data. 2023.

19.

Wormser

, Schriefer

, Aguero-Rosenfeld

, et al. Single-tier testing with the C6 peptide ELISA kit compared with two-tier testing for Lyme disease. Diagn Microbiol Infect Dis, 2013; 75(1):9–15; doi: 10.1016/j.diagmicrobio.2012.09.003