Abstract
Background
Anti-SARS-CoV-2 antibodies have been used in the study of the immune response in infected patients. However, differences in sensitivity and specificity have been reported, depending on the method of analysis. The aim of the present study was to evaluate the diagnostic accuracy of an algorithm in which a high-throughput automated assay for total antibodies was used for screening and two semi-automated IgG-specific methods were used to confirm the results, and also to correlate the analytical results with the clinical data and the time elapsed since infection.
Methods
We studied 306 patients, some hospitalized and some outpatients, belonging to a population with a high prevalence of COVID-19. One-hundred and ten patients were classified as SARS-CoV-2 negative and 196 as positive by polymerase chain reaction.
Results
The algorithm and automated assay alone had a specificity and a positive predictive value of 100%, although the sensitivity and negative predictive value of the algorithm was higher. Both methods showed a good sensitivity from day 11 of the onset of symptoms in asymptomatic and symptomatic patients. The absorbance of the total antibodies was significantly higher in severely symptomatic than in asymptomatic or mildly symptomatic patients, which suggests the antibody level was higher. We found 15 patients who did not present seroconversion at 12 days from the onset of symptoms or the first polymerase chain reaction test.
Conclusion
This study highlights the proper functioning of algorithms in the diagnosis of the immune response to COVID-19, which can help to define testing strategies against this disease.
Introduction
Since the appearance and global expansion of the epidemic caused by SARS-CoV-2, the international medical community has been in need of the rapid development and commercialization of reagents for the diagnosis, prognosis and monitoring of the disease, as well as for epidemiological studies. The gold standard method for the diagnosis of COVID-19 is the detection of the viral messenger RNA by polymerase chain reaction (PCR), although some antigen tests have demonstrated their efficacy in confirming infection in symptomatic patients. 1 The qualitative or quantitative determination of anti-SARS-CoV-2 antibodies has been suggested as useful for the study of the immune response in infected patients, identifying infection hot spots, assessing group immunity and evaluating vaccination status. 2
However, the practical application of the new methods for the determination of anti-SARS-CoV-2 antibodies has had to face two main problems: firstly, important differences have been detected in the positive predictive value (PPV) in scenarios of low prevalence of infection. This is an important issue, since PPV depends on the prevalence of the disease, which varies greatly from country to country. It has been suggested that the assay of choice should be determined by each country independently based on the local prevalence of SARS-CoV-2 infection. 2 At the time, we recruited our patients, Spain had a relatively high prevalence of infection (around 6%) similar to other areas of Western Europe. 3 Secondly, there is an urgent need to bring rapid and reliable tests to market that can be carried out in the vast majority of clinical laboratories. That has forced government agencies and biodiagnostic companies to balance the high demand with little data available on assay performance, leading to some relaxation of the assessment criteria. 2 , 4 , 5
The Elecsys® Anti-SARS-CoV-2 nucleocapsid electrochemiluminescent immunoassay is a high-throughput fully automated method that determines the level of total antibodies and, due to its speed and feasibility, is being used extensively by laboratories around the world.6–9 However, the recommendations of the Spanish health authorities and scientific societies require IgG concentrations to be determined specifically in order to make decisions such as taking patients out of isolation from the quarantine zone and reinstating health professionals who had suffered the disease. 10 , 11 To reduce the likelihood of a false-positive result and maximize the PPV of testing, the CDC Interim Guidelines for COVID-19 Antibody Testing have suggested an orthogonal testing algorithm so that individuals who are positive by a total antibody test are retested with a second IgG-specific antibody test. 12 Our laboratory has developed an algorithm in which the Elecsys® anti-SARS-CoV-2 was used for screening, and two semi-automated IgG-specific methods were used to confirm the results. In the present study, we sought to evaluate the diagnostic accuracy of this algorithm in both hospitalized and in outpatients belonging to a population with a high prevalence of COVID-19, to compare the algorithm with the determination of total antibodies alone and to correlate the analytical results with the clinical data and the time elapsed since infection.
Methods
Participants
We conducted a retrospective study on 306 consecutive COVID-19 patients selected from all those who attended the Hospital Universitari de Sant Joan between 18 November 2020 and 2 January 2021. The only inclusion criterion was age over 18 years. We excluded patients with a life expectancy of less than 24 h, a severely compromised liver function or incomplete medical records. Patients were classified according to the recommendations of the Health Department of the regional government of Catalonia, 13 as follows: Uninfected patients were those without any previous positive PCR and who were not reported to have had any previous COVID-19-related symptoms. These were patients who had undergone a SARS-CoV-2 infection study and a PCR analysis as per protocol before a routine medical intervention. Asymptomatic infected patients were those who had a positive PCR with no symptoms of COVID-19. Mildly symptomatic were patients with positive PCR and COVID-19-compatible symptoms who did not require hospitalization. Severely symptomatic were patients with positive PCR and COVID-19-compatible symptoms who required hospitalization. PCR analyses were carried out using the VIASURE SARS-CoV-2 Real Time PCR Detection Kit (CerTest Biotec, Zaragoza, Spain), or with the Procleix® method in a Panther automated extractor and amplifier (Grifols Laboratories, Barcelona, Spain). We considered the date of onset of infection as the date the symptoms first appeared or the date of the first positive PCR. The clinical characteristics of patients are given in Table 1. This study was approved by the Comitè d’Ètica i Investigació en Medicaments (Institutional Review Board) of Institut d’Investigació Sanitària Pere Virgili (Resolution CEIM 040/2018, amended on 16 April 2020).
Descriptive characteristics of the patients enrolled in the study.
aDays elapsed between the onset of symptoms or the first positive PCR and a positive total antibody result. Data are shown as cases and percentages (in parentheses), except for age, that is shown as mean and 95% confidence interval.
Analytical measurements
The determination of total antibodies using the Elecsys® Anti-SARS-CoV-2 assay was used as a screening method. This method detects total antibodies against the nucleocapsid antigen in an electrochemiluminescence immunoassay (ECLIA) on a Cobas® e801 analyser (Roche Diagnostics, Rotkreuz, Switzerland). According to the manufacturer, results with a cut-off index (COI) >1.000 are considered positive. With the aim of trying to increase the sensitivity, when using the algorithm, we decreased the COI to 0.4, and we measured IgG concentrations in all samples with positive results. This cut-off was established from our previous studies with receiver operating characteristics plots that showed a sensitivity of 89.5% and a specificity of 99.7% at COI = 0.4 (Figure 1). Subsequently, samples with COI > 0.4 with the Elecsys® anti-SARS-CoV-2 total antibodies assay were analysed using a semi-automated anti-IgG ELISA based on the native antigen of SARS-CoV-2-infected Vero E6 cells, strain 2019-nCoV/Italy-INMI1 in a Chorus/TRIO analyser (Enzywell SARS-CoV-2 IgG, Diesse Diagnostica Senese, Siena, Italy). The manufacturer reports sensitivity of this assay at 98% and specificity at 100%. Values above the upper limit of the assay (>1.1 optical density, OD) were considered IgG positive without additional confirmatory tests, while negative values were confirmed using another semi-automated ELISA that detects IgG SARS-CoV-2 antibodies against N and S proteins (Vircell Spain SLU, Granada, Spain) in a Triturus® System (Grifols Laboratories, Barcelona, Spain). Sensitivity of this assay is reported at 70% and specificity at 100%. Values above the upper limit (>6 OD) were considered positive (Figure 2). In the design of the algorithm and in the calculation of diagnostic precision, ambiguous results were considered negative.

Receiver operating characteristics plot of the results obtained with the Elecsys® anti-SARS-CoV-2 immunoassay in COVID-19-positive and -negative patients confirmed by polymerase chain reaction.

Algorithm for the determination of the serological response in patients with COVID-19.
Statistical analyses
Data are given as numbers and percentages or medians and 95% confidence intervals. Statistical comparisons between two groups were made using the Mann-Whitney U test. Statistical significance was set at P ≤ 0.05. All calculations were made using the SPSS 25.0 statistical package (SPSS Inc., Chicago, IL, USA).
Results
The algorithm and the Elecsys® anti-SARS-CoV-2 assay alone had a specificity and a PPV of 100%, although the sensitivity and negative predictive value (NPV) of the algorithm were slightly higher than that of the Elecsys® alone. Indeed, the algorithm classified as positive four patients who were negative in the Elecsys® anti-SARS-CoV-2 assay (Table 2). When comparing the percentage of negative and positive patients using both methods as a function of the severity of the patient's symptoms and the time elapsed since infection, we found that they performed similarly, and presented good sensitivity from day 11 of the appearance of symptoms. However, when evaluating the performance of the Diesse IgG ELISA alone, we found that this method failed to detect antibody levels in asymptomatic and mildly symptomatic patients (Table 3). An interesting finding is that the COI of the total antibodies was significantly higher in severely symptomatic patients than in the others, which suggests that the antibody level was higher (Table 4).
Diagnostic accuracy of the algorithm and the total antibodies Elecsys® anti-SARS-CoV-2 assay.
Note: Results are shown as number of cases or as percentages and 95% Confidence Intervals.
COI: cut-off index; NPV: negative predictive value; PPV: positive predictive value.
Performance of the algorithm, the measurement of total antibodies with the Elecsys® anti-SARS-CoV-2 assay and the measurement of IgG antibodies with the Diesse ELISA in COVID-19 patients segregated according to the severity of symptoms and the days after the infection onset.
Note: Results are shown in number of patients.
COI: cut-off index.
Median optical density values obtained by the Elecsys® anti-SARS-CoV-2 assay in in COVID-19 patients segregated according to the severity of symptoms and the days after the infection onset.
Note: Results are shown as medians and 95% confidence intervals.
aP < 0.05, with respect to asymptomatic and mildly asymptomatic patients, by the Mann-Whitney’s U test.
bP < 0.01, with respect to asymptomatic and mildly asymptomatic patients, by the Mann-Whitney’s U test.
COI: cut-off index.
Next, we wanted to study those patients who did not present seroconversion, defined as a negative serological study with the appearance of symptoms or first PCR ≥ 12 days. (Table 5). We had 15 patients (7.7% of total infected patients), the majority of whom were asymptomatic or mildly symptomatic. Of these cases, six had positive or equivocal IgG between 20 and 40 days of symptoms appearing, and one of them showed seroconversion at 102 days, which we might consider as a possible initial false positive or a probable reinfection. The only two cases of symptomatic hospitalization were a patient whose primary diagnosis was pneumonia by multidrug-resistant Pseudomonas aeruginosa and another with Streptococcus gallollyticus endocarditis. In the former, SARS-CoV-2 was detected after several days in hospital (nosocomial COVID-19).
Individual characteristics of patients who did not present seroconversion at 12 days from the onset of symptoms or first PCR.
bOptical densities between 4 and 6 are considered equivocal, according to the manufacturer (Vircell).
Discussion
Specificity and sensitivity are important characteristics of analytical tests. However, the PPV and the NPV are more relevant to clinical application, and in deciding how tests can best be used and interpreted. The main parameters that determine the PPV are disease prevalence and specificity, while the NPV is mainly determined by prevalence and sensitivity. Thus, prevalence is a decisive factor in assessing the diagnostic efficacy of a serological test, but it can vary significantly depending on location, a point which is crucial in the context of a global pandemic. 2 , 6 In the present study, we report that the Elecsys® anti-SARS-CoV-2 assay had an excellent PPV in Spain, with a relatively high prevalence of COVID-19. Our PPV is higher than that reported in Israel, 2 for example, and similar to that found in Great Britain 4 and Austria. 14 However, this method measures total antibodies; therefore, an additional determination of IgG antibodies would be required to confirm seroconversion. In this regard, we applied an algorithm where the cut-off point of the Elecsys® Anti-SARS-CoV-2 assay was changed. Some studies recommend an optimized use of cut-offs, in order to increase sensitivity without affecting specificity.15–18 We applied a lower cut-off than the manufacturer's recommendation, together with an algorithm in order to optimize a positive IgG result. This is the recommendation of the health authorities of some other European countries, and also of the CDC of the United States.10–12 The performance of the algorithm is very similar to that of the determination of total antibodies. However, without a confirmation of anti-IgG, crucial decisions cannot be made, such as taking patients out of the quarantine zone or reinstating health professionals who suffer the infection. We agree with previous reports recommending sequential studies in order to take clinical decisions in populations with a low prevalence and to ensure a positive IgG response.19–21
Both the Elecsys® anti-SARS-CoV-2 assay and the algorithm performed well in asymptomatic, mildly symptomatic and severe symptomatic patients from day 11 following the onset infection. However, we found that the Diesse semi-automated IgG ELISA, if used alone, is unable to distinguish between positive and negative patients when they were asymptomatic or mildly symptomatic, working well only in the severe cases. The nature of the antigen used to generate the antibodies can play a decisive role. The Diesse IgG ELISA uses the inactivated whole virion, while the Elecsys® anti-SARS-CoV-2 assay employs the nucleoprotein antigen, and the Vircell IgG ELISA uses the nucleoprotein and the spike protein. It has been pointed out that antibodies to the nucleoprotein are more sensitive in early infection, 19 , 22 while the combination of antibodies to the nucleoprotein and spike has been shown to enhance the assay diagnostic performance. 23 A previous study reported good specificity and sensitivity of the Diesse kit, 24 but it was mostly carried out on hospitalized patients, which might explain the apparent discrepancy with our results.
We found the optical densities obtained with the Elecsys® anti-SARS-CoV-2 assay increased in relation to the severity of the infection, from day 11 following the onset of symptoms. These results would suggest a higher antibody titre in the most severe patients. Although this has already been reported by some researchers, others have not found this relationship, probably because they did not have a large proportion of asymptomatic/mildly symptomatic cases. 4 , 25 Finally, we conducted a detailed study of patients without seroconversion at 12 days of the onset of symptoms, with slower kinetics in many cases of asymptomatic and mild symptoms, and we observed a similar proportion to that reported in a study from Israel. 2 A possible explanation for the existence of this subgroup, made up of asymptomatic and mildly symptomatic patients, is that T lymphocytes perhaps play a more important role in them than the production of IgG by B lymphocytes, as previously suggested. 26 It might also be possible that active immune responses are less strong in these patients, suggesting that uncontrolled adaptive responses are involved in secondary phase immunopathology.26,27 In our case, none of the patients was immunocompromised. However, this reasoning is merely speculative, and we believe that more clinical data are needed to understand the response, or lack thereof, of these patients.
A limitation of our study is that we cannot be entirely sure that patients classified as uninfected have not experienced an asymptomatic infection at some point. Other studies evaluating methods for the determination of anti-SARS-CoV-2 antibodies have used samples collected before the pandemic as negative. In our case, this has not been possible, but we think that our approach is valid, since it reflects current clinical practice. In addition, this is a unicentre study in a hospital that covers a relatively small geographical area. However, we believe that the results obtained are relevant since they might be representative of many similar centres in Western Europe and in the Mediterranean area, although little information is yet available on this issue.
Footnotes
Acknowledgements
Editorial assistance was provided by Phil Hoddy at the Service of Linguistic Resources of the Universitat Rovira i Virgili.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by a grant from the Fundació la Marató de TV3 (201807–10), Barcelona, Spain and by the CERCA Programme, Generalitat de Catalunya, Barcelona, Spain.
Ethical approval
Ethics approval was given by Comitè d’Ètica i Investigació en Medicaments (Institutional Review Board) of Institut d’Investigació Sanitària Pere Virgili (Resolution CEIM 040/2018, amended on 16 April 2020).
Guarantor
JC.
Contributorship
XGB, SI and JC contributed to the study planning. XGB, AHA, LF, SC and JMS performed the experiments. SI, AC and JJ were involved in patient recruitment. XGB, SI, IP, FB and JC contributed to analysis and interpretation of the data. XGB and JC wrote the first draft of the article. All authors contributed to further writing the paper and approved the final version for publication.
