Abstract
Background:
Pleural fluid (PF) carcinoembryonic antigen (CEA) is a widely used diagnostic marker for malignant pleural effusion (MPE). Recent studies revealed that PF to serum CEA was also a promising diagnostic parameter for MPE.
Objective:
We aimed to investigate whether PF to serum CEA ratio and delta CEA (PF minus serum CEA) provided added value to PF CEA in diagnosing MPE.
Methods:
Patients with pleural effusion in a retrospective cohort (BUFF) and a prospective cohort (SIMPLE) were included. The clinical characteristics of the patients were extracted from their medical records. The diagnostic value of CEA ratio and delta CEA was estimated by a receiver operating characteristics (ROC) curve, net reclassification improvement (NRI), and integrated discrimination improvement (IDI).
Results:
A total of 148 patients in the BUFF cohort and 164 patients in the SIMPLE cohort were enrolled. The BUFF cohort had 46 MPE patients and 102 benign pleural effusion (BPE) patients, and the SIMPLE cohort had 85 MPE patients and 79 BPE patients. In both cohorts, MPE patients had significantly higher PF CEA, serum CEA, CEA ratio, and delta CEA. The area under ROC curves (AUCs) of PF CEA, CEA ratio, and delta CEA were 0.78 (95% CI: 0.67–0.88), 0.80 (95% CI: 0.72–0.89) and 0.83 (95% CI: 0.75–0.91) in the BUFF cohort, and 0.89 (95% CI: 0.83–0.94), 0.86 (95% CI: 0.80–0.92), and 0.84 (95% CI: 0.78–0.91) in the SIMPLE cohort. The differences between the AUCs of PF CEA, CEA ratio, and delta CEA did not reach statistical significance. The continuous NRI and IDI of CEA ratio and delta CEA were <0.
Conclusion:
CEA ratio and delta value cannot provide added diagnostic value to PF CEA. The simultaneous determination of serum and PF CEA should not be adopted in clinical practice.
Introduction
Pleural effusion (PE) is a common sign caused by various disorders. 1 Malignant pleural effusion (MPE) is caused by primary and metastatic pleural malignancies. 2 The diagnosis of MPE is a challenge for clinicians because the sign and symptoms of MPE are not specific, and the currently available diagnostic tools have some limitations. 3 Pleural fluid (PF) cytology is an inexpensive diagnostic tool with high specificity for MPE, but its sensitivity is only 0.46. 4 In addition, the diagnostic accuracy of PF cytology depends on the experience of observers. Thoracic ultrasound-guided, computed tomography (CT)-guided biopsies and thoracoscopy have been widely used for detecting MPE; however, they are invasive and need special training. 5 Serum or PF tumor biomarkers are usually used to detect MPE because they are easy to perform, objective, and inexpensive.6,7 However, their diagnostic accuracy was unsatisfactory when used alone.6,8
PF carcinoembryonic antigen (CEA) has been proposed to be a diagnostic marker for MPE for a long time. It has high specificity (>0.90), but its sensitivity is approximately 0.50.8–10 Some studies reported that PF to serum CEA ratio had high diagnostic accuracy for MPE.11–13 However, whether the CEA ratio provides added diagnostic value to serum and PF CEA remains unknown. In addition, the diagnostic accuracy of delta CEA (PF CEA minus serum CEA) remains largely unknown. This study aimed to investigate whether CEA ratio or delta CEA provides added diagnostic value to serum or PF CEA. This study was reported following the Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement. 14
Materials and methods
Participant selection
The participants in this study were from the BUFF (Biomarkers for patients with Undiagnosed pleural eFFusion) study and the SIMPLE (a Study Investigating Markers in PLeural Effusion) study. These two studies aimed to investigate the diagnostic accuracy of laboratory tests in PE patients. The BUFF study is a retrospective study in which the patients who visited the Affiliated Hospital of Inner Mongolia Medical University (AHIMMU) with PE and received diagnostic thoracocentesis and PF biochemical analyses between July 2017 and July 2018 were included. Patients were excluded if any of the following criteria was met: (1) <18 years; (2) with a large portion of missing value, (3) without serum or PF CEA determination; (4) discharged with no definitive diagnosis.
In the SIMPLE study, the patients with PE who visited the AHIMMU between September 2018 and July 2021 were prospectively enrolled. The inclusion and exclusion criteria of the SIMPLE study are identical to that of the BUFF study. The design details of the SIMPLE study have been described previously. 15 In addition, with the inclusion and exclusion criteria, we enrolled the patients who visited the Affiliated Changshu Hospital of Xuzhou Medical University (ACHXZMU) between June 2020 to July 2021.
Data collection
In both studies, we extracted the clinical data on admission from the participants’ electronic medical records. The extracted data include demographic characteristics, PF, and serum biochemistries. In the case of patients in whom more than one test was performed, only the first one before making the final diagnosis was used for analysis. Because caregivers ordered both PF and serum CEA levels, their results were not blinded to the caregivers who made the definitive diagnosis. Serum and PF CEA levels were determined by the Architect i2000SR immunoassay analyzer (Abbott Laboratories, USA) in both cohorts. The diagnosis of MPE was made based on PF cytology or pleural biopsy. 3 In some patients, MPE was diagnosed in the presence of primary cancer and BPE can be excluded by treatment response and follow up. Tuberculosis PE (TPE) was made with positive PF Z-N staining, culture, or biopsy. 16 Some TPE patients were diagnosed by the response to anti-tuberculosis treatment. Parapneumonic pleural effusion (PPE) and heart failure (HF) were diagnosed based on clinical presentations, imaging, and response to treatment.17,18
Statistical analysis
We used the Kolmogorov–Smirnov method to test the normality of continuous data. The Student’s t-test or Mann–Whitney U-test was used to compare continuous data. The Chi-square test was used to compare categorical data. A receiver operating characteristics (ROC) curve was used to estimate the diagnostic accuracy of biomarkers or parameters investigated. The area under the ROC curve (AUC) was used to estimate the global diagnostic accuracy of the index test. The net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were used to test the added value of CEA ratio or delta CEA to PF CEA. 19 All analyses were performed with R (version 4.0.5), and p < 0.05 was considered statistically significant.
Results
Characteristics of the participants
Figure 1 is a flowchart of patient selection. A total of 148 patients (46 MPEs and 102 BPEs) in the BUFF study and 164 patients (85 MPEs and 79 BPEs) in the SIMPLE study were included. The characteristics of these patients are listed in Table 1. In both cohorts, BPE patients had lower serum CEA, PF CEA, and CEA ratio than MPE patients (p < 0.001 for all). While the delta CEA (PF CEA minus serum CEA) was significantly higher in BPE patients (p < 0.001).

A flowchart of patient selection.
Characteristics of the participants.
ADA, adenosine deaminase; BPE, benign pleural effusion; BUFF, biomarkers for patients with undiagnosed pleural effusion; CEA, carcinoembryonic antigen; Delta CEA, PF minus serum CEA; LDH, lactate dehydrogenase; MPE, malignant pleural effusion; PF, pleural fluid; SIMPLE, a study investigating markers in pleural effusion; WBC, white blood cell.
Diagnostic accuracy of serum and PF CEA, as well as their ratio and delta value for MPE
Figure 2 is the ROC curve of serum and PF CEA, CEA ratio, and delta CEA. Because the data-driven selection of optimal cut-off value may overestimate the diagnostic accuracy of the index test, 20 we prespecified the cutoff values of PF CEA, serum CEA and delta CEA at different cutoff values (e.g., 5 ng/mL, 20 ng/mL, 50 ng/mL and 500 ng/mL). We used the cut-off of 5 ng/mL because the reference interval of serum CEA is 0–5 ng/mL. This cut-off was also widely used by previous studies. 10 We also used the cut-off of 50 ng/mL because 1.00 specificity can be obtained under this cut-off, as indicated by earlier studies with large sample sizes.21,22

ROC curves of serum CEA, PF CEA, CEA ratio, and delta CEA for MPE.
The AUCs, sensitivities, and specificities of serum CEA, PF CEA, CEA ratio, and delta CEA at the fixed cut-offs were summarized in Table 2. In both the BUFF and the SIMPLE study, PF CEA had a higher AUC than serum CEA. However, the difference between the AUCs of serum and PF CEA did not reach statistical significance in the BUFF study (p = 0.22). In the SIMPLE study, the AUC of PF CEA was significantly higher than that of serum CEA (p = 0.01).
Diagnostic accuracy of serum and pleural fluid CEA for malignant pleural effusion.
AUC, area under the curve; BUFF, biomarkers for patients with undiagnosed pleural effusion; CEA, carcinoembryonic antigen; CI, confidence interval; Delta CEA, PF minus serum CEA; PF, pleural fluid; SIMPLE, a study investigating markers in pleural effusion.
Similar to previous studies with the cut-off of 5 ng/mL,8,10 the sensitivity and specificity of PF CEA in both cohorts were approximately 0.60 and 0.90, respectively. At the cut-off of 50 ng/mL, PF CEA failed to reach 1.00 specificity in both cohorts. This is because two BPE patients (one in the SIMPLE cohort and the other in the BUFF cohort) with empyema and purulent PF had extremely high CEA levels (BUFF cohort: 443.10 ng/mL, SIMPLE cohort 67.81 ng/mL). To reach a specificity of 1.00, the cut-off should be 500 ng/mL. The corresponding sensitivities under this cut-off were 0.22 in the SIMPLE cohort and 0.26 in the BUFF cohort. Given that the AUC of PF CEA is higher than that of serum CEA, we compared the AUCs of CEA ratio and delta CEA with PF CEA in the next step. However, no significant differences were observed (p > 0.05 for all comparisons). Notably, the point estimate values of AUCs for CEA ratio and delta CEA in the SIMPLE cohort were even lower than that of PF CEA.
The added value of CEA ratio, delta CEA to PF CEA
The AUC has limitations in estimating the overall diagnostic accuracy of a given index test. 23 Therefore, we used NRI and IDI to determine whether the CEA ratio and delta CEA provide added diagnostic value to PF CEA. These two statistical methods are widely used in estimating the added diagnostic value. The results of NRI and IDI are listed in Table 3. Interestingly, the point estimate values of NRI and IDI in both the BUFF cohort and the SIMPLE cohort were <0, demonstrating that the CEA ratio and delta CEA do not provide added diagnostic value to PF CEA.
The added value of CEA ratio and delta CEA to pleural fluid CEA.
BUFF, biomarkers for patients with undiagnosed pleural effusion; CEA, carcinoembryonic antigen; CI, confidence interval; Delta CEA, PF minus serum CEA; IDI, integrated discrimination improvement; NRI, net reclassification improvement; PF, pleural fluid; SIMPLE, a study investigating markers in pleural effusion.
Table 4 lists the studies that investigated the diagnostic accuracy of PF CEA, serum CEA and CEA ratio in a head-to-head manner. Five studies were searched in the PubMed database, and the AUCs of serum CEA, PF CEA, and the ratio were summarized.24–28 All early studies and this study indicated that the AUC of PF CEA was higher than that of serum CEA. Interestingly, three studies found that the AUC of CEA ratio was lower than that of PF CEA,24–26 while two previous studies found that the AUC of CEA ratio was higher than that of PF CEA.27,28 However, no study compared the diagnostic accuracy of CEA ratio and PF CEA with rigorous statistical methods, such as AUC comparison (C-statistics), NRI and IDI.
Summary of the studies investigating the diagnostic accuracy of CEA ratio for MPE.
AUC, area under the curve; BPE, benign pleural effusion; BUFF, biomarkers for patients with undiagnosed pleural effusion; CEA, carcinoembryonic antigen; MPE, malignant pleural effusion; NR, not reported; PF, pleural fluid; SIMPLE, a study investigating markers in pleural effusion.
Discussion
The major findings of this study are as follows: First, both serum and PF CEA had moderate diagnostic accuracy for MPE, and the diagnostic accuracy of serum CEA seemed inferior to that of PF CEA. Second, the CEA ratio and delta CEA did not provide added diagnostic value to PF CEA.
The diagnostic accuracy of serum and PF CEA for MPE remains an open issue. Although some studies have investigated the diagnostic accuracy of serum and PF CEA for MPE, only a few of them compared the diagnostic accuracy of serum and PF CEA in a head-to-head manner,24–28 and none of these studies have compared the diagnostic accuracy of serum and PF CEA levels with rigorous statistical methods. Some researchers investigated the diagnostic accuracy of CEA ratio (PF/serum) for MPE. However, no previous study has investigated the added diagnostic value of CEA ratio to PF or serum CEA levels. Determining the added diagnostic value of the CEA ratio is important because the simultaneous determination of serum and PF CEA increases the medical cost and makes patients uncomfortable. One strength of this study is that we used two rigorous statistical methods, namely NRI and IDI, to determine the added diagnostic value of the CEA ratio. These two statistical methods have been widely used to determine the added value of a diagnostic marker or model. 23 Another strength of this study is that we investigated the diagnostic accuracy and added diagnostic value of delta CEA. To our knowledge, only one study has investigated the diagnostic accuracy of delta CEA for MPE. 26 Although the AUCs of delta CEA and PF CEA were not compared with a statistical method, the estimated point value of delta CEA is lower than that of PF CEA. Our study, along with the previous study, 26 indicates that delta CEA cannot improve the diagnostic accuracy of PF CEA.
Previous studies indicated that the specificity of PF CEA was 1.00 under the cut-off of 50 ng/mL.21,22 However, the specificity under this cut-off failed to reach 1.00 in our cohorts because of two empyema patients with purulent PF. Compared with other types of BPE, empyema patients have significantly higher PF CEA.29,30 In a cohort with 70 empyema patients, the mean PF CEA level is 108.6 ng/mL (range: 5.2–1001.3 ng/mL). 29 In addition, most empyema patients have purulent PF. 31 Therefore, we conclude that PF CEA >50 ng/mL can only be used to confirm MPE in patients with non-purulent PF.
This study has some limitations. The major limitation of this study is the small sample size and the retrospective design of the BUFF study. In addition, the ratio or delta value of other tumor markers such as carbohydrate antigen (CA) 125, CA 15-3, CA 19-9, and neuron-specific enolase (NSE) were not investigated. This is because these tumor markers in PF were usually not ordered by the clinicians in our hospital.
In conclusion, this study indicated that the CEA ratio does not provide added diagnostic value to PF CEA. Therefore, the simultaneous determination of serum and PF CEA should not be encouraged in clinical practice.
