Abstract
Background
Accurate and rapid testing for SARS-COV-2 antibodies could improve the diagnosis and management of COVID-19. In this study, we aim to evaluate the diagnostic accuracy of a commercially available point-of-care lateral flow kit independently and in comparison to an established platform-based system.
Method
Samples from 144 PCR-confirmed COVID-19 cases and 130 pre-pandemic negative controls were tested in parallel by MP Rapid 2019-NCOV IgM/IgG Combo test and Roche Elecsys. Comparison of results based on serum and capillary blood testing was undertaken.
Results
Sensitivity at day 15 onwards was 100% for both methods. Between days 1 and 7 post admission, the IgM/IgG Combo test and Roche Elecsys shown sensitivity of 74% (95%CI: 62%-85%) vs. 67% (95% CI: 55%–79%, P = 0.3947). Combo test specificities were 100% for IgG, 98.5% for IgM vs. Roche Elecsys specificity of 100%. Concordance analysis showed 98.5% agreement to the Roche Elecsys method (Cohen’s Kappa 0.96 95% CI [0.92–0.99]). Capillary blood results showed complete agreement with serum samples using the Combo test.
Conclusion
In comparison to Roche Elecsys, our data show that the MP Rapid 2019-NCOV IgM/IgG Combo test provides a high-confidence assay system for the detection of previous exposure to SARS-COV-2 infection with advantage of affording near-patient testing.
Introduction
Severe acute respiratory distress syndrome SARS-CoV-2 is an infection that has affected millions of patients worldwide with implications for both societal and economic stability. Laboratory diagnosis and management of COVID-19 has been instrumental in combating the spread of SARS-CoV-2. Real-time reverse transcription chain reaction (RT-PCR) tests that detect the presence of the virus are relevant in the acute phase only. SARS-CoV-2 antibody assays can detect antibodies to the SARS-CoV-2 virus during the course of the disease and post recovery.
SARS-CoV-2 antibody testing is vital for understanding the scale of the pandemic, assessing the immune response to SARS-CoV-2 infection, monitoring of convalescent patients and assessment of vaccine efficacy. Reports from Wuhan presented early evidence for the potential diagnostic role of serological testing. Accurate and rapid serological testing for SARS-CoV-2 infection could improve the diagnosis of disease and may help control the spread of the pandemic, in both the hospitalized and community setting. 1
Several different serological testing methods for SARS-CoV-2 antibodies, including some marketed for rapid use at the point of care (POCT), are commercially available. In the UK, to enable large-scale testing for healthcare and social care staff, Public Health England (PHE) supported the evaluation and the release of several of these assays systems, including a number of POCT tests and self-testing kits. 2 However, the performance characteristics of these kits were described as variable, and further evaluation may be required depending on the intended use and the clinical setting. 2 Evidence supporting the use of rapid Lateral Flow immunoassay (LFIA)-based POCT kits diagnostically is lacking. 3
The Roche Elecsys, a platform based, high-throughput serology test, was one of the first evaluated assays for SARS-CoV-2 Antibodies. PHE evaluation data showed a specificity and sensitivity at day >14 of 100%. 4 The method has since been routinely adopted in a number of testing centres, including that of the author. This study was conducted to determine the clinical and analytical performance of Rapid 2019-nCOV-IgM/IgG Combo card test independently and in comparison to Roche Elecsys® Anti-SARS-CoV-2.
Material and methods
This study was approved by the HRA and Health and Care Research Wales (HCRW) (Reference No. 20/YH/0190).
Clinical sensitivity
A total of 144 serum samples from recently diagnosed 49 symptomatic hospitalized patients and 15 non-hospitalized patients, were diagnosed with SARS-CoV-2 infection using RT-PCR test, with a strong low cycle threshold value of ≤35 (Altona RealStar SARA-CoV-2 RT-PCR, Altona Diagnostics GmbH, Germany). 5
Samples were collected from 49 patients admitted to hospital during March and April 2020 with laboratory-confirmed COVID-19, whose clinical severity ranged from moderate to critical. Samples were collected after initial PCR confirmation on admission and at various time points thereafter (day 1 to day 44) and stored at –80°C until the day of analysis. Non-hospitalized subjects were recruited from PCR-positive NHS workers at a median of 43 days from symptoms onset. Day 1 represents initial positive PCR.
Clinical and analytical specificity
Clinical specificity
A total of 130 archived serum samples collected from healthy healthcare volunteers from West Yorkshire were tested. These prepandemic samples were collected in 2014 and stored at –80°C until the day of analysis.
Analytical specificity
Interference was assessed with nine samples collected from nine participants with PCR (BIOFIRE® Respiratory Panel 2, bioMerieux, France)-confirmed seasonal coronavirus infection. Samples were collected 12–16 weeks post the infection onset date.
Matrix effect
Thirty-six paired capillary blood samples (finger prick) and serum samples were measured on 2019-NCOV-IgM/IgG Combo test. This group comprised of samples from PCR-positive individuals, individuals with COVID-19 symptoms, but not PCR tested and asymptomatic individuals.
Analytical platform
All sets of samples were analysed by the following methods:
The Roche Elecsys® Anti-SARS-CoV-2, on e602 immunoassay platform (Roche Diagnostics Ltd, West Sussex, UK). Roche Elecsys is a sandwich immunoassay for the qualitative detection of combined IgM/IgG antibodies against a recombinant nucleocapsid protein.
The 2019-NCOV-IgM/IgG Combo test (MP Biomedicals Germany GmbH, Eschwege, Germany). This study was performed using LOT number 2005E0046. The analysis was conducted in accordance with the manufacturer’s instruction. The IgM/IgG Combo card is a lateral flow immunoassay which detects SARS-CoV-2 IgM and IgG antibodies by binding to immobilized RBD antigen (S Protein) and nucleocapsid (N protein) on the test strip. The test is initiated by applying 2 µL specimen plus two drops of buffer, in separate wells. The presence of a coloured (purple) band, at 10 min after the specimen application, in the IgM and/or IgG test window, plus a control band indicates a positive test. A negative result is indicated by the presence of a control band only.
The results of both tests are interpreted as positive, not detected or invalid. For purposes of this study, IgM/IgG Combo card results have been interpreted by two independent reviewers.
Statistical analysis
The minimum sample size of 100 was based on the minimum sample size to detect a sensitivity of at least 90%, based on 80% power. 6 Diagnostic sensitivity and specificity were calculated under the following assumption: all samples obtained prior to the pandemic were considered as true negative, and all PCR-positive results are considered as a true positive. Sensitivity was calculated as TP/TP+FN, and specificity was calculated as TN/FP+TN. Cohen’s Kappa analysis was performed to calculate per cent of agreement (overall, positive and negative) of results from each assay. Analysis was performed using Analyse-it software for Microsoft Excel 2 (Analyse-it Software, Ltd, Leeds-UK).
Results
Clinical sensitivity was determined using samples from PCR positive for SARS-CoV-2 patients at different time points. A sensitivity of 100% at >14 days was achieved by both assays (Table 1). Patients were also grouped depending on the time between admission and testing; the rate of seropositivity at days 1–7 for IgM/IgG Combo test and for Roche Elecsys were 74% (95%CI: 62%–85%) vs. 67% (95% CI: 55%–79%, P = 0.3947), respectively.
Clinical specificity and sensitivity of Roche Elecsys and IgM/IgG Combo card.
In days 1–7; 46/62 (74.0%) of PCR-positive patients tested positive; 5/62 (8%) for IgM alone, 30/62 (48%) for IgM/ IgG combined and 11/62 (18%) for IgG alone. After day 21, there were no samples positive for only IgM alone, 29/41 (71%) were positive for IgM and IgG and 12/41 (29%) IgG positive only. Table 2 shows the development of antibodies of four PCR-positive patients who were antibody negative on admission. IgG in these patients preceded IgM or appeared simultaneously over the course of hospitalization.
Antibody type for four hospitalized patients who were initially seronegative during the observation period.
Clinical specificity was 100% for Roche Elecsys. The overall calculated specificity for 2019-nCOV IgM/IgG Combo card test was 98.5% for IgM and IgG combined; when IgM and IgG were considered individually, IgG and IgM specificities were 100% and 98.5%, respectively (Table1).
Comparing qualitative results of SARS-CoV-2 antibody assays, the 2019-nCOV IgM/IgG Combo card test showed an overall agreement of 98.5.% (Cohen’s Kappa 0.96, 95% CI interval of 0.92–0.99) with Roche Elecsys (Table 3).
Concordance of SARS-CoV-2 antibody assays.
We examined the cross-reactivity of each assay with nine convalescent human serum samples in patients with recent PCR-positive NL63, OC43 and HKU1 infection. The Roche Elecsys had no cross-reactivity to any of the common coronavirus family members tested. 2019-nCOV IgM/IgG Combo also showed no IgG cross-reactivity; however, a faint IgM band to seasonal coronavirus strain OC43 was observed with one sample (Table 4).
Analytical specificity of IgM/IgG Combo card test.
To assess the effect of sample matrix, the performance of IgM/IgG Combo card was evaluated using 36 pairs of capillary blood and serum samples, prospectively collected from non-hospitalized symptomatic participants. The kit instruction sheet advices that Combo Card is suitable for use with finger-prick blood and serum. The manufacturer’s claim proved correct. In all 36 individuals, the results of the capillary blood were in complete agreement with those obtained with the serum sample (24 IgG positive, 2 IgM and IgG positive and 10 not detected).
Discussion
Serology testing offers an important tool to test for past infection and immune response to SARS-CoV-2 infection. In this study, we conducted a comparative laboratory evaluation of a LFIA POCT test against a PHE evaluated laboratory-based assay for SARS-CoV-2 serology. Early versions of rapid LFIA for antibody detection were subject of criticism, mainly due to poor specificity and sensitivity. 3 A UK-based recent study showed variable performances of 11 POCT serology test kits. 2 The clinical specificity of Roche Elecsys assay was reported by PHE and other as between 98 and 100%.4,7,8 In this study, we demonstrated that performance of the IgM/IgG Combo card is comparable to the Roche Elecsys method with 98.5% agreement (Cohen’s Kappa 0.96 95% CI [0.92–0.99]). We also demonstrated that IgM/IgG Combo card had high clinical specificity of 98.5%. While this is slightly lower than Roche specificity of 100%, IgM/IgG Combo card specificity of 98.5% satisfies the MHRA-recommended minimum standard performance for POCT serology tests. 6
The IgM/IgG Combo card test showed no IgG cross-reactivity with human serum samples positive for other seasonal coronaviruses or with the 130 pre-pandemic samples. However, a weak IgM cross-reactivity was observed with serum samples positive for OC43 and also in 2 out of the 130 pre-pandemic samples. Up-to-date, no strong evidence shows that the presence of antibodies offer immunity; however, a false positive may still have detrimental effect if this test has been used clinically or post vaccination; hence, strategies to confirm positive results should be employed.
For screening, PHE recommend clinical sensitivity of 98% of serology assays on specimens collected ≥20 days from symptoms onset. 4 IgM/IgG Combo card test and Roche Elecsys both achieved a sensitivity of 100% from day 15 onward.
The clinical and diagnostic utility of SARS-CoV-2 antibody testing is not yet fully established. Serologic assays are unlikely to be useful for the early detection of SARS-CoV-2 infection because patients become seropositive five to seven days post symptom onset. However, an increasing number of studies suggest that serology testing has a role beyond screening. Antibody detection methods have the potential to increase overall SARS CoV-2 PCR testing sensitivity and guide stratification of patients with infection when the PCR result is negative. 9 For this purpose, we specifically compared the sensitivity of each assay between days 1 and 7, in the cohort of 62 SARS-CoV-2-positive patients. Combo Card IgM/IgG and Roche Elecsys seropositivity in days 1–7 were 74% (95%CI: 62%–85%) and 67% (95% CI: 55%–79%, P = 0.3947) respectively. According to the published literature, admission to hospital generally occurs around five to seven days after symptom onset.10,11 The precise dates of symptom onset were unfortunately not available, so we took day 1 as the date of hospital admission. Although this does not change the comparison between the two assays, potentially, this means the sensitivities are even lower than reported here.
Interestingly, only five (8%) of all positives were IgM seropositive, all other days 1–7 positives were both IgM and IgG positive. This may reflect late presentation, as positivity rate in this study was assessed against admission day, rather than the date of symptom onset. Multiple studies reported that IgM and IgG production SARS-CoV-2 occurs at around the same time.12–14 Our finding of early IgG and simultaneous appearance of IgM/IgG (Table 2) agrees with another study where the onset of disease was used rather than day of admission. 12 Researchers observed three types of seroconversion; synchronous seroconversion of IgM and IgG, IgM seroconversion earlier than that of IgG, and IgM seroconversion later than that of IgG. 12 This finding challenges the use of IgM as an early marker of SARS-CoV-2 infection and suggests careful strategies with clear clinical aims are required before the roll out of serology testing in hospitalized patients.
One limitation of this study is that it was conducted at a single centre with a relatively small number of COVID-19-positive cases. Another limitation was the small size of the interference panel, e.g. none of the serum samples tested contained 299E strain. Furthermore, we were not able to analytically confirm the presence of antibodies to seasonal coronaviruses; hence, we restricted the use of convalescent sera to those patients within 12–16 weeks from the infection date.15,16 Further work is needed to evaluate any cross-reactivity with other respiratory viruses and winter illnesses, to increase confidence in the kit’s analytical specificity.17,18
In conclusion, IgM/IgG Combo card antibody test is easy to perform using venous or capillary blood samples. The analytical performance of Combo card for SARS-CoV-2 antibodies was comparable to Roche Elecsys, with benefits of rapidity and simplicity of test performance; identifies potential utility as a supportive test in SARS-CoV-2 infection.
Footnotes
Acknowledgements
The authors thank Ms Sybille Sullivan and Mrs Victoria Mercer for their help and support with this study.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by Sterilab Services, Harrogate, UK.
Ethical approval
Approved by the National Research Ethics Committee (Reference No. 20/YH/0190).
Guarantor
BC.
Contributorship
NJ designed the study and drafted the paper. NL, FS, ZH, VH collected samples and interpreted data. VA advised on statistics. JC and BC advised on microbiology and immunology aspects. All authors contributed to this paper and all read and agreed the final version of the article.
