Abstract
Background
The usefulness of procalcitonin (PCT) and C-reactive protein (CRP) as individual biomarkers, and in combination, for the identification of infections in a critically ill patient cohort was evaluated retrospectively.
Methods
Best cut-off values for PCT and CRP for a diagnosis of sepsis in critically ill patients were determined using receiver operator characteristic (ROC) curve analysis. Both combined tests and individual tests were performed for PCT and CRP, with positive and negative results recorded, and accuracy evaluated using odds ratios (OR).
Results
In the 55 critically ill patients studied, the best cut-off value for PCT for a diagnosis of sepsis was 1.1 ng/ml (sensitivity, specificity and negative predictive values [NPV] were 82%, 68% and 71%, respectively). In addition, the best cut-off value for CRP was 50.7 mg/l ( sensitivity, specificity and NPV of 90%, 68% and 83%, respectively). Measuring PCT and CRP in combination provided a sensitivity of 79%, a specificity of 86%, and a positive predictive value (PPV) of 90%. Diagnostic OR for the combination of biomarkers versus CRP alone (19 and 18, respectively) were greater than that for PCT (9).
Conclusion
For critically ill patients, CRP and CRP in combination with PCT were found to be suitable biomarkers for diagnosing sepsis, based on their NPV and PPV.
Introduction
Manifestations of sepsis can be serious, although they do not always represent specific indicators of infection. In addition, noninfectious clinical syndromes can exhibit sepsis-like symptoms, thereby making a diagnosis of sepsis more difficult. Generally, a diagnosis of sepsis is suspected, yet not necessarily confirmed, in the absence of a positive culture. However, given the time required to complete a culture, antibiotics are often prescribed prior to obtaining a definitive diagnosis in order to avoid missing an infection. Thus, the ability to recognize bacterial infections in their early stages could guide the treatment needed, reduce unnecessary administration of antibiotics and control the development of drug-resistant strains. 1
Procalcitonin (PCT) and C-reactive protein (CRP) are biomarkers that have been studied for their ability to identify bacterial infections. PCT is a prehormone form of calcitonin, which is normally secreted by C cells of the thyroid in response to hypercalcaemia. Therefore, under normal conditions, serum PCT levels are typically negligible. 2 However, bacterial endotoxins are potent stimuli for the synthesis of PCT, and the kinetics of this response is more rapid than that involving CRP. PCT is released into the circulation ∼3–4 h after exposure to an endotoxin, and reaches peak levels 8–24 h later. 3 As a result, PCT has been proposed as a marker of bacterial infection in critically ill patients. 4
C-reactive protein is synthesized by the liver in response to a stimulus that mainly includes interleukin (IL)-6. Between 4 and 6 h after stimulation, a considerable increase in CRP levels is observed. These levels double every 8 h, then peak between 36 and 50 h after an infection or acute inflammation occurs.5,6 In the clinic, CRP is one of the most frequently assayed inflammatory markers, and has been shown to be useful for diagnosing an infection, for evaluating a patient’s response to antimicrobial therapy 7 and for screening critically ill patients. 8
To improve the accuracy of diagnosing sepsis, monitoring of both novel biomarkers and traditional markers of sepsis together has been proposed. 9 For PCT and CRP, limited data are available regarding the use of both biomarkers for evaluating infection among critically ill adult patients. 10 Therefore, the objective of the present study was to evaluate the usefulness of assaying PCT and CRP, in combination and individually, for the identification of infections in critically ill adult patients.
Patients and methods
Study setting and population
This retrospective study was conducted at the First Hospital of Jilin University (Changchun, China) using data obtained from critically ill patients treated between January 1 and December 21, 2012. The study was conducted to assess the usefulness of serial PCT and CRP measurements in evaluating infections in an intensive care unit (ICU) setting. Study approval was obtained from the Hospital Ethics Committee. Informed consent was waived because of the retrospective and observational nature of the study. This is a retrospective study evaluating patient outcomes after discharge. No data will be disclosed to other institutions and all identifiers linking to the patient will be destroyed after data collection is complete. There is no prospective participant involvement or observation involved with this research activity.
Patients aged 18 years or older, diagnosed with sepsis, with initial PCT and CRP levels recorded in their medical histories, were included in this study. Patients with unclear diagnoses, and patients who were hospitalized for <72 h, were excluded.
Data collection
Standardized electronic case report forms from the database of the clinical laboratory of a 25-bed general ICU at the First Hospital of Jilin University (Changchun, China) were reviewed. Infection was defined as the presence of a pathogenic micro-organism in a sterile milieu (such as blood or cerebrospinal fluid) and/or a clinically suspected infection that justified the administration of antibiotics. 11 Sepsis was defined according to current definitions. 12
Quantification of PCT and CRP levels
The assay for PCT was semiquantitative and employed immunochromatography (BRAHMS PCT-Q, Diagnostica GmbH, Hennigsdorf, Germany). The reference range was <0.05 ng/ml. Plasma CRP was measured using a Vitros 250 Dry Chemistry System (Ortho-Clinical Diagnostics, Rochester, NY, USA). The reference range was <10 mg/l. Initial PCT and CRP levels were defined within 6 h of receiving intensive care, with samples for bacterial culture obtained at the same time. The following variables were also recorded: age; sex; initial body temperature; initial white blood cell (WBC) count; sepsis-related organ failure assessment (SOFA) score; 13 Acute Physiology and Chronic Health Evaluation (APACHE II) score. 14
Statistical analyses
Data were presented as median (interquartile range [IQR]) or
For the diagnostic evaluation of biological markers, the sensitivity and specificity for each cut-off point for each marker was recorded. A comparison of the diagnostic accuracy of these markers was made using receiver operator characteristic (ROC) curve analyses, by calculating the area under the curve (AUC), and by calculating standard errors using the nonparametric method described by Hanley and McNeil. 15 Suggested cut-off values for PCT and CRP in the diagnosis of infection in critically ill patients were determined according to the Youden index. 16
The combined test was defined as testing with a single primary CRP test (or PCT test), plus PCT (or CRP) used as an additional test. The combined test was recorded as positive when PCT and CRP were simultaneously positive, while a negative value was recorded when both, or either, of the values were negative. The sensitivity, specificity, and positive and negative predictive values (PPV and NPV, respectively) for PCT and CRP were evaluated individually and in combination for a diagnosis of sepsis using a two-by-two table. Positive and negative likelihood ratios (LR+ and LR−, respectively) and odds ratios (OR) were also calculated, with their 95% confidence intervals (CI) determined using the exact binomial method. The accuracy of the combined test compared with the single test was evaluated using OR. The diagnostic OR of a test was considered to be the ratio of the odds of positivity for a disease relative to the odds of positivity for no disease and equal to LR+/LR−. 17
All reported
Results
Characteristics of a critically ill adult patient population studied to determine whether C-reactive protein (CRP) and procalcitonin (PCT) analyses could be used in the diagnosis of sepsis.
Values expressed as median (interquartile range, 25–75%),
IQR, interquartile range; APACHE II, Acute Physiology and Chronic Health Evaluation score; 12 SOFA, sepsis-related organ failure assessment; 11 WBC, white blood cell count; MV, mechanical ventilation.
Statistical tests used: Mann–Whitney
In the ROC curve analyses (Figure 1), the area under the ROC curve for PCT was 0.81 ± 0.06 (95% CI 0.70, 0.93), and for CRP was 0.82 ± 0.07 (95% CI 0.67, 0.96). As a result, the best cut-off value for PCT for a diagnosis of sepsis was 1.1 ng/ml; this threshold value was associated with sensitivity, specificity and NPV of 82%, 68% and 71%, respectively (Table 2). The best cut-off value for CRP to diagnose sepsis was 50.7 mg/l, with sensitivity, specificity and NPV of 90%, 68% and 83% respectively (Table 2).
Receiver operator characteristic curves for C-reactive protein (CRP) (solid line) and procalcitonin (PCT) (dot-dash line), measured in a critically ill patient cohort ( Sensitivity and specificity of C-reactive protein (CRP) and procalcitonin (PCT) measurements, when evaluated as potential biomarkers for a diagnosis of sepsis in a critically ill adult patient population ( CI, confidence interval; PPV, positive predictive value; NPV, negative predictive value; LR+, positive likelihood; LR−, negative likelihood; OR, odds ratio. CRP versus PCT. PCT versus combination assay (PCT + CRP). CRP versus combination assay (CRP + PCT). Statistical tests used: Mann–Whitney 
The combination assay of PCT and CRP levels was associated with a sensitivity of 79%, a specificity of 86%, and a PPV of 90% (Table 2). Diagnostic OR for the combination assay of PCT and CRP levels (19), and for levels of CRP alone (18), were also greater than that for PCT (9). However, the difference between the diagnostic OR for the combination and for CRP alone was not statistically significant (Table 2).
Discussion
In the present study, two markers of inflammation (PCT and CRP) were evaluated individually and in combination to evaluate their diagnostic value for sepsis. Based on the best cut-off values identified, the diagnostic accuracies for CRP (and for both PCT and CRP) were found to be better than that for PCT alone. In addition, CRP was associated with better sensitivity and NPV, while the combination of PCT and CRP exhibited better specificity and PPV.
In an ICU setting, body temperature and WBC are the most frequently measured parameters, although body temperature is a poor indicator of sepsis. Moreover, in a critical care setting, particularly during the postoperative period, a fever is usually not caused by sepsis. 18 Regarding an increase in WBC, this is typically associated with infection, and leucopenia can occur during severe infections. 12 However, several studies have found that WBC has a low diagnostic performance for evaluating infections.19,20 PCT and CRP are better markers of infection than temperature and WBC, 7 and are the most widely used biomarkers for sepsis. However, they are limited by their inability to distinguish sepsis from other inflammatory conditions. In the present study, the cut-off values for PCT and CRP (1.1 ng/ml and 50.7 mg/l, respectively) were associated with a high sensitivity, yet both had a low specificity for diagnosing sepsis.
Measuring CRP levels in critically ill patients has been found to be of great value. For example, this parameter facilitates a diagnosis of sepsis and can be used to guide the appropriate use of antibiotics.21,22 CRP may also be a more convenient marker to assay, compared with PCT, due to its better sensitivity in predicting bacterial infections and its lower cost.23,24 In the present study, CRP was found to have a higher sensitivity than PCT for diagnosing sepsis in critically ill patients. This observation may be due to the presence of a local infection, which is more readily detected by measuring CRP rather than PCT, as previously proposed. 25 However, CRP has drawbacks for diagnosing sepsis. For example, in adults, elevated levels of CRP are associated with burns, trauma, surgery, pancreatitis, and rheumatic disorders, in addition to bacterial and fungal infections. 26 Therefore, the specificity of CRP for an infection is low. In the present study, detection of CRP was associated with a high sensitivity (90%) and NPV (83%) for a diagnosis of sepsis in critically ill patients. Furthermore, of the patients in this cohort with negative CRP values (i.e. levels <50.7 mg/l), 15/18 (83%) were not diagnosed with sepsis. However, when the test is applied to all of the patients in an ICU, the prevalence of sepsis on the NPV should be considered. For example, the prevalence of sepsis was 60% for the present cohort, and NPV will generally increase with decreasing prevalence (and vice versa). 27 Correspondingly, the NPV of CRP would be improved when the prevalence of sepsis is <60%, and the incidence of sepsis in other ICU has been reported to be much lower than this.28,29 Therefore, due to the greater sensitivity and NPV of CRP, it is a factor well suited for eliminating a diagnosis of sepsis. 27 Moreover, when elevated levels of CRP return to normal or decrease below normal in patients with sepsis, this can indicate either the disappearance of sepsis or an improvement in a patient’s condition. Correspondingly, a change in CRP levels may be used to determine when antibiotics can be discontinued.30–32 However, a prospective study is needed to confirm these recommendations.
Studies have found that PCT could be a more accurate diagnostic parameter than CRP for differentiating between nonsepsis and sepsis in critically ill patients;33,34 Luzzani et al. 35 proposed that PCT should replace CRP as a marker of sepsis in the ICU setting. PCT has become a well-known biomarker that may help physicians working in the emergency department (ED) to identify sepsis promptly and guide antibiotic administration. 36 Travaglino et al. 37 demonstrated that PCT may be useful for assisting ED physicians, in terms of detecting infectious disease during early risk stratification in those presenting with acute dyspnoea (with an AUC of 0.65 and a cut-off of 0.09 ng/ml). Wang and colleagues found that the serum PCT level, and not the CRP level, could be used as a negative predictor for excluding bloodstream infections, with a best cut-off value of 0.80 ng/ml (83% sensitivity; 65% specificity) in critically ill patients. 38 In addition, the PCT level may be related to the severity of sepsis, 39 and may have value as a prognostic marker of sepsis. 40
Nevertheless, although PCT may play an important role in the management of patients with sepsis, it has its own insufficiencies. The value of PCT for critically ill patients has been questioned, due to its clinical limitations. 41 It was hypothesized that PCT could not reliably differentiate sepsis from other noninfectious causes of systemic inflammatory response syndrome in critically ill adult patients.42–44
Magrini et al. 40 found that both PCT and CRP were useful as diagnostic and prognostic markers of infectious disease, but that PCT was a better (and earlier) marker than CRP. However, whereas CRP had a sensitivity of 100% for a diagnosis of sepsis, its specificity was only 9%, compared with values of 76% and 69%, respectively, for PCT. 40 In the present study, we found that CRP and PCT had the same specificity (68%), but CRP had a higher sensitivity (90%) than PCT (82%). Our findings indicate that PCT may be a diagnostic marker of sepsis, but did not show superiority to CRP. Nevertheless, PCT consistently has a high specificity, and it is produced by a different mechanism than CRP.3,5 A weak but significant correlation has been found between PCT and CRP when sepsis occurs. 40 For these reasons, we suggest that PCT and CRP could be used together, with the high sensitivity of CRP compensating for the low sensitivity of PCT. Considering the distinct advantages and disadvantages of each method, the combined use of PCT and CRP may overcome these insufficiencies.
In practice, the combination of two diagnostic tests involves two outcomes: the first is that the two tests are different: either one of the two tests is positive. The second is that both tests are the same (either positive or negative). When both tests are positive (or negative), the specificity of the combined test will be the higher specificity value of the component tests, whereas the sensitivity will be no greater than the lower sensitivity of the component tests. The opposite occurs with an ‘either positive’ test. When choosing which rule to administer for an adjunct test, it must be determined whether the goal is to improve test performance in either (or both) of the affected and unaffected populations. 45
In the present study, the specificity of diagnosis was improved using a ‘both positive’ combination of PCT and CRP. However, while a comparison of a biomarker’s overall performance can usually be achieved using ROC analysis, the combination of best cut-offs for CRP and PCT cannot be compared using ROC. Therefore, the methods of a diagnostic OR were used, which is a single indicator of test performance and is unaffected by disease prevalence. 17
When PCT and CRP were combined in a ‘both positive’ test format, a higher specificity (86%) and higher PPV (90%) for a diagnosis of infection was observed in critically ill patients in this cohort. For example, 29 patients tested positive for both factors, 26 of whom (89.7%) had a confirmed diagnosis of sepsis. When proposing the combination approach, we considered whether a decrease in the overall performance of diagnosis of infection in the critically ill patients would be observed. However, by comparing OR, we observed that the accuracy of diagnosis for the combination of factors (OR 19) was better than that of PCT (OR 9) alone. Furthermore, although the combination of factors did not improve the overall performance of CRP (OR 18), neither did it reduce the accuracy, and it improved the specificity in critically ill patients. In addition, the use of PCT and CRP was associated with a good PPV. Therefore, based on the high specificity and PPV for the ‘both positive’ test format applied in the present study, a deterministic diagnosis of sepsis can be achieved. However, if the test was applied to all patients in an ICU, the influence of the prevalence of sepsis on the PPV should be considered. In general, PPV will decrease with a decrease in prevalence. 27 Correspondingly, the PPV of the combined factors will be <90% if the prevalence of sepsis is <60%. Therefore, since the low sensitivity and PPV associated with this format can be influenced by the prevalence of sepsis, a combination of factors should only be used to provide additional support for a positive diagnosis, and not as a routine assay.
There were limitations associated with the present study. First, based on its retrospective nature and small patient sample, further analyses could not be performed. For example, a comparison of likelihood ratios was not performed due to an insufficient number of patients. Secondly, there was no gold standard by which to diagnose sepsis. Only a variety of clinical manifestations and bacterial culture results could be used. Thirdly, PCT and CRP were evaluated in relation to a diagnosis of infection, and were not evaluated with respect to the severity and mortality of infections experienced by the cohort of critically ill patients examined. Finally, the cost of evaluating both PCT and CRP levels was not considered. In conclusion, for the data from critically ill patients analysed retrospectively in this cohort, CRP was found to be suitable for eliminating a diagnosis of sepsis due to its greater NPV. In contrast, PCT was not found to be a better marker for diagnosing sepsis than CRP. However, the use of a ‘both positive’ format for the assays of PCT and CRP levels was found to be suitable for obtaining a deterministic diagnosis of sepsis, due to the specificity and PPV associated with this combination of biomarkers. Therefore, the results of this study demonstrate that assays of sepsis biomarkers can be effective in obtaining a diagnosis of sepsis, and further studies are needed to evaluate additional combinatorial assays.
Footnotes
Declaration of conflicting interest
The authors declare that there are no conflicts of interest.
Funding
This study was supported by a grant from National Health and Family Planning Commission of the People's Republic of China (Special Fund for Health Scientific Research in the Public Interest) Program: No. 201202011.
