Abstract
Background
Vitamin B12 deficiency occurs frequently, especially among the elderly. However, screening for vitamin B12 deficiency is hampered by poor sensitivity of the existing total vitamin B12 assay. Methylmalonic acid (MMA) is considered as the most representative indicator of metabolic vitamin B12 deficiency and is used as such in this study. The aim of this study was to validate the clinical usefulness of holotranscobalamin (holoTC) as an initial screening assay for metabolic vitamin B12 deficiency in a mixed patient population.
Methods
Three hundred and sixty blood samples were collected by five Dutch hospitals. Vitamin B12 and holoTC in serum were measured (AxSYM; Abbott). MMA in serum was measured by tandem mass spectrometry (LC-MS/MS).
Results
Receiver operating curve (ROC) analysis demonstrated a greater area under the curve (AUC) for holoTC than for vitamin B12 in detecting vitamin B12 deficiency characterized by three predefined cut-off levels of MMA. A cut-off value of 32 pmol/L of holoTC resulted in the highest sensitivity (83%) with acceptable specificity (60%) in detecting MMA concentrations above 0.45 μmol/L. The combination of vitamin B12 and holoTC did not improve diagnostic accuracy at this cut-off level.
Conclusions
HoloTC has a better diagnostic accuracy than vitamin B12 and can replace the existing vitamin B12 assay as a primary screening test in patients suspected of vitamin B12 deficiency. Critical evaluation of cut-off values of holoTC indicated that a cut-off value of 32 pmol/L can be considered in screening for metabolic vitamin B12 deficiency (defined by MMA > 0.45μmol/L) in a mixed patient population.
Introduction
Vitamin B12 deficiency occurs frequently in the elderly and can be caused by several factors like impaired absorption or inadequate dietary intake. 1 Humans cannot synthesize vitamin B12 (i.e. cobalamin) and therefore it has to be obtained through the diet. Vitamin B12 deficiency is commonly asymptomatic in the early stages, but can later on be associated with macrocytic anaemia and typical neurological symptoms, either alone or in combination. 2,3 These symptoms are, however, far from specific and are variable in their presence and severity. Establishing vitamin B12 deficiency on objective grounds is therefore challenging. In most studies, the measurement of the metabolic consequence of vitamin B12 deficiency, an increase in methylmalonic acid (MMA), has been applied. 4–7 A shortage of vitamin B12 leads to insufficient activity of the enzyme methylmalonyl-CoA mutase, and hence to accumulation of its substrate MMA.
MMA is considered as the gold standard in the diagnosis of metabolic vitamin B12 deficiency. 8 However, MMA can only be measured by complex, specialized methods which are relatively expensive. Vitamin B12 measurement in serum is frequently used as a first-line assay in screening for vitamin B12 deficiency. Diagnostic accuracy of this assay is hampered by a low sensitivity, which results in a relatively large amount of false negatives. 5 Recently, the holotranscobalamin (holoTC) assay was implemented on an automated analyser (AxSYM Active B12 assay; Abbott, Hoofddorp, The Netherlands) facilitating high-throughput holoTC measurements in standard clinical chemistry laboratories. 9
Several studies have assessed the diagnostic accuracy of holoTC in relation to metabolic vitamin B12 deficiency. 4–7 Most studies reported comparable or modestly superior diagnostic accuracy of holoTC based on receiver operating curve (ROC) analysis. However, large discrepancies exist with regard to the choice of cut-off value for holoTC (range: 20–45 pmol/L), which makes data interpretation difficult. In addition, different populations were studied, like the elderly, vegetarians and healthy controls, which might have influenced the choice of cut-off value. 4,5,7 In this study, we validated the diagnostic accuracy of holoTC as a screening test for metabolic vitamin B12 deficiency in patients suspected of vitamin B12 deficiency and compared this with vitamin B12 in a Dutch multicenter study using clinical samples for which vitamin B12 was requested.
Material and methods
Study population
Patient samples were collected between November 2006 and July 2007 from clinical chemistry laboratories in the Netherlands. Originally, each centre aimed to collect samples of 250 patients of whom vitamin B12 was requested. Each centre was asked to select 20 patients with a vitamin B12 concentration below 100 pmol/L, 80 patients with vitamin B12 between 100 and 200 pmol/L, 100 patients with vitamin B12 between 200 and 300 pmol/L and 50 patients with vitamin B12 above 300 pmol/L. After determination of vitamin B12 in serum, the samples were stored at −80°C. When sample collection was completed, frozen samples were sent on dry ice to the Erasmus MC and were stored at −80°C until further analysis.
Individuals were selected on the basis of age (≥18 years), normal renal function (estimated glomerular filtration rate [eGFR] ≥60 mL/min/1.73 m2) and availability of vitamin B12, holoTC and MMA measurements in serum. In total, 360 individuals could be included in ROC analysis, which were collected at Erasmus MC (n = 148), Meander MC (n = 4), UMC Groningen (n = 61), Gelre Hospitals (n = 62) and Star-MDC (n = 85). Informed consent was not obtained as leftover material from routine analysis was used for this study which was processed anonymously.
Laboratory methods
Serum vitamin B12 was determined in each sample by the same type of analyser (AxSYM; Abbott,) at the Erasmus MC (n = 299 samples) and at the UMC Groningen (n = 61 samples). HoloTC was determined in each serum sample at the Erasmus MC by the AxSYM analyser (Abbott). Analytical imprecision (i.e. within-run and total imprecision were below 8% for three pools with different concentrations of holoTC [i.e. 20, 40 and 80 pmol/L]). The assay was linear in the range from 5 to 86 pmol/L. Samples with holoTC values above 86 pmol/L were diluted with saline to obtain values within the linear range. Serum MMA was measured by tandem mass spectrometry (LC-MS/MS) at the Erasmus MC. 10 The laboratory participates in the European Research Network for evaluation and improvement of screening, Diagnosis and treatment of Inherited disorders of Metabolism (ERNDIM; 0 to +0.5SD from consensus MMA concentration) for external quality control.
The 95% reference range of holoTC, vitamin B12 and MMA was determined in 100 healthy blood bank donors (i.e. 50 men and 50 women) at the Erasmus MC and was 21–117 pmol/L for holoTC, 145–637 pmol/L for vitamin B12 and 0.09–0.45 μmol/L for MMA. Three predefined cut-off values of MMA were applied to define metabolic vitamin B12 deficiency: MMA > 0.32μmol/L (i.e. 90th percentile); serum MMA > 0.45 μmol/L (i.e. 97.5th percentile); and MMA > 0.77 μmol/L (i.e. 99th percentile).
Haemoglobin (Hb), mean corpuscular volume (MCV), platelets, leukocytes (WBC) and creatinine were determined in all samples by standard automated laboratory techniques. Reference values were established at the Department of Clinical Chemistry of the Erasmus MC in 100 healthy blood bank donors. eGFR was calculated using the original ‘Modification of Diet in Renal Disease’ (MDRD) study equation. 11 Ethnicity was not known to us and therefore no correction was performed as described for African-Americans. Renal dysfunction was characterized as GFR < 60 mL/min/1.73 m2 according to MDRD guidelines.
Statistical analysis
Creatinine, GFR, MMA, vitamin B12 and holoTC were positively skewed and therefore were 10log-transformed. Mean and range were calculated for age, Hb, MCV, platelets and WBC. Geometric means with corresponding range were calculated for creatinine, GFR, MMA, vitamin B12 and holoTC. MMA was dichotomized prior to ROC analysis. Test performance and ROC analysis were performed by Analyze-it (Version 2.21; Analyze-it Software Ltd, West Yorkshire, UK). Differences between areas of holoTC and vitamin B12 were calculated by Analyze-it. ROC decision plots depicting sensitivity and specificity for different cut-off values were generated with Analyze-it. P < 0.05 was considered statistically significant. All P values were two-tailed.
Results
Characteristics of study population
MCV, mean corpuscular volume; HoloTC, holotranscobalamin; eGFR, estimated glomerular filtration rate; MMA, methylmalonic acid
*Reference values depicting either the 2.5% lower value or 2.5% higher value depending on the biomarker
†Based at reference values established at the Department of Clinical Chemistry of the Erasmus MC
‡Geometric mean
¶According to MDRD guidelines
#Only individuals with normal renal function (=eGFR ≥60) were included
**1 mmol Hb/L ∼ 1.7 g/dL
ROC analysis using MMA > 0.32 μmol/L to characterize vitamin B12 deficiency demonstrated an area under the curve (AUC) of 0.63 (95% CI 0.56–0.70) for vitamin B12 and AUC of 0.70 (95% CI 0.64–0.77) for holoTC (Figure 1a, P = 0.01). When a cut-off value of MMA > 0.45 μmol/L was applied, the AUC for vitamin B12 and holoTC were, respectively, 0.70 (95% CI 0.61–0.79) and 0.78 (95%CI 0.69–0.87) (Figure 1b, P = 0.06). Applying a cut-off value of MMA > 0.77 μmol/L resulted in an AUC of 0.73 (95% CI 0.60–0.87) for vitamin B12 and AUC 0.92 (95% CI 0.85–0.98) for holoTC (Figure 1c, P = 0.01).
Receiver operating curve (ROC) analysis of holotranscobalamin (holoTC) and vitamin B12 in diagnosing metabolic vitamin B12 deficiency characterized by different cut-off values for methylmalonic acid (MMA) based at 90th, 97.5th and 99th percentiles in 360 patients with normal renal function. (a) Sensitivity is plotted against 1-specificity for holoTC (black solid line) and vitamin B12 (black dotted line) in 102 patients with metabolic vitamin B12 deficiency and 258 patients with normal MMA concentrations as characterized by MMA > 0.32 μmol/L (i.e. 90th percentile), P = 0.01. (b) ROC analysis of holoTC (black solid line) and vitamin B12 (black dotted line) in 47 patients with metabolic vitamin B12 deficiency and 313 patients with normal MMA concentrations as characterized by MMA > 0.45 μmol/L (i.e. 97.5th percentile), P = 0.06. (c) ROC analysis of holoTC (black solid line) and vitamin B12 (black dotted line) in 13 patients with metabolic vitamin B12 deficiency and 347 patients with normal MMA concentrations as characterized by MMA > 0.77 μmol/L (i.e. 99th percentile), P = 0.01. A diagonal reference line is depicted (grey solid line), which indicates no discrimination. AUC, area under the curve
ROC decision plots were used to evaluate different holoTC and vitamin B12 cut-off values in relation to screening for metabolic vitamin B12 deficiency in our mixed population. Metabolic vitamin B12 deficiency was, for this analysis, characterized by MMA > 0.45 μmol/L as this was the upper limit of the 95% reference range. The ROC decision plot based on sensitivity and specificity showed optimal cut-off values of vitamin B12 between 136 and 216 pmol/L; a lower cut-off resulted in an undesirable sensitivity of less than 50% and a higher cut-off resulted in specificity below 50% (Figure 2a). Within this range, optimal sensitivity was obtained at a cut-off value of 180 pmol/L (e.g. above this level, the amount of true positives remained stable, whereas the amount of false positives increased rapidly). The clinical decision point of 180 pmol/L for vitamin B12 was compared with the 2.5% lower reference value of 145 pmol/L. By increasing the cut-off value from 145 to 180 pmol/L for vitamin B12, the amount of true positives increased from 25 to 30, resulting in an increase in sensitivity from 53% to 64%. Specificity decreased from 81% to 64% (Figure 2a).
Receiver operating curve (ROC) decision plots in 360 patients with normal renal function. (a) ROC decision plot based on sensitivity (solid line) and specificity (dotted line) for different cut-off values of vitamin B12 in relation to vitamin B12 deficiency characterized by methylmalonic acid (MMA) > 0.45 μmol/L. Cut-off values between 136 and 216 pmol/L resulted in a sensitivity and specificity higher than 50% (grey box). The optimal cut-off value is depicted by the arrow ( = 180 pmol/L). (b) ROC decision plot based on sensitivity (solid line) and specificity (dotted line) for different cut-off values of holoTC. Cut-off values between 19 and 36 pmol/L resulted in a sensitivity and specificity higher than 50% (grey box). The optimal cut-off level is depicted by the arrow (=32 pmol/L)
The same analysis was performed for holoTC for which optimal cut-off values were obtained between 19 and 36 pmol/L (Figure 2b). At a cut-off level of 36 pmol/L, the highest sensitivity was obtained; however, above a cut-off level of 32 pmol/L, no gain in true positives was reached despite a steep decrease in specificity (Figure 2b). Comparison of the clinical decision point of 32 pmol/L with the lower 2.5% reference value of 21 pmol/L showed that 39 true positives were identified instead of 30 when the cut-off level of 32 pmol/L was applied for holoTC, which resulted in an increase in sensitivity from 64% to 83%. Specificity decreased accordingly from 88% till 60%.
Studies have suggested that the combination of holoTC and vitamin B12 would have a better diagnostic accuracy in explaining MMA concentrations than holoTC or vitamin B12 alone. 4 We evaluated diagnostic accuracy of different combinations of vitamin B12 and holoTC and compared this with a single test result to screen for metabolic vitamin B12 deficiency defined by MMA > 0.45 μmol/L. 12 First, we compared the gain of true positives by adding holoTC after being negative for the vitamin B12 test (i.e. holoTC as an add-on test); this resulted in a gain of 10 true positives and seven false positives compared with the test results of the single vitamin B12 assay. A total number of 40 true positives were identified, which is comparable with the total number of true positives identified by the single holoTC test (39 true positives). Second, we evaluated whether holoTC can be used as primary test after which all negatives are subjected to a secondary vitamin B12 test (i.e. vitamin B12 as an add-on test). This design resulted in a gain of one true positive and seven false positives compared with the test results of the single holoTC test.
Discussion
In this study, we validated the diagnostic accuracy of vitamin B12 and holoTC in screening for metabolic vitamin B12 deficiency using clinical samples from five Dutch hospitals. We confirmed that holoTC can replace vitamin B12 in screening for metabolic vitamin B12 deficiency in this population and determined the clinical decision point for holoTC at 32 pmol/L.
Studies comparing diagnostic accuracy of vitamin B12 and holoTC in detecting metabolic vitamin B12 deficiency
Hcy, homocysteine; ROC, receiver operating curve; HoloTC, holotranscobalamin; MMA, methylmalonic acid; AUC, area under the curve; eGFR, estimated glomerular filtration rate; GC-MS, gas chromatography-mass spectrometry; LC-MS/MS, liquid chromatography tandem mass spectrometry
*Results of present study
†Mean
‡Median
¶Study included more groups but only the German controls and vegetarians are depicted in the table; first number corresponds to German controls and second number corresponds to vegetarians
#First number corresponds to women and second number corresponds to men
**Based at ROC analysis
††Based at reference values
‡‡Cut-off based on literature findings
¶¶Not used in diagnosis of vitamin B12 deficiency
##First number corresponds to first mentioned cut-off value of MMA and the other numbers correspond to the other mentioned cut-off values for MMA
***Vitamin B12 was used to diagnose vitamin B12 deficiency
Results from our study showed that an absolute improvement of 10–20% in sensitivity can be reached by replacing vitamin B12 by holoTC in screening for metabolic vitamin B12 deficiency depending on the chosen cut-off value. Cut-off values of holoTC differ depending on the study population and criteria used for metabolic vitamin B12 deficiency. 4–7,14 Sometimes the cut-off is chosen identical to the 2.5% lower reference value of a healthy population and otherwise at a specifically chosen clinical decision point. We showed that application of the lower 2.5% reference value for holoTC (<21 pmol/L) resulted in a relatively low sensitivity of 64% with a specificity of 88%. At this cut-off level, 17 out of the 48 patients with elevated MMA would have been missed (i.e. 36% false negatives). We consider vitamin B12 deficiency a serious condition which might result in severe irreversible neurological complications. Hence, as little patients as possible should therefore be missed (e.g. low amount of false negatives) in screening for vitamin B12 deficiency, which requires a test with a high sensitivity. Therefore, we evaluated by ROC analysis, the highest achievable sensitivity with acceptable specificity and showed that a cut-off value of holoTC between 19 and 36 pmol/L is desirable, as this resulted in sensitivity and specificity both higher than 50%. The highest sensitivity was obtained at a cut-off level of 36 pmol/L; however, results from our study showed that a cut-off value above 32 pmol/L resulted in a plateau at which the gain in sensitivity was stabilized despite a steep decrease in specificity. Others used cut-off values ranging from 20 to as high as 45 pmol/L depending on the population studied. 4–7,14–16 Our study demonstrates that cut-off values above 36 pmol/L are undesirable, as this resulted in a specificity of less than 50%. We have chosen a cut-off value of 32 pmol/L for holoTC based upon the results of our study for screening for metabolic vitamin B12 deficiency but we appreciate that other choices are equally tenable.
In a clinical perspective, a positive test result for holoTC (<32 pmol/L) is indicative for vitamin B12 deficiency; however, with a specificity of 60% and a prevalence of 13%, the chance of being a false positive is considerable. Therefore, in our opinion, three scenarios can be followed in this case: (i) all patients with holoTC <32 pmol/L are treated with cobalamin; or (ii) MMA is measured as a secondary test to confirm diagnosis in all patients with holoTC < 32 pmol/L; or (iii) treatment is started in patients with holoTC < 21 pmol/L as the presence of vitamin B12 deficiency is relatively certain at this cut-off level (e.g. specificity is 88%) and MMA is measured in patients whose holoTC is between 21 and 32 pmol/L to confirm diagnosis.
It was suggested by Miller et al. 4 that diagnostic accuracy can be increased by combining test results of vitamin B12 and holoTC. In our study, we evaluated whether combined testing with vitamin B12 and holoTC would result in identification of a higher number of true positives and showed that the combination of holoTC and vitamin B12 does not increase the number of true positives that were already identified with the single holoTC assay. These results demonstrated that the combination of holoTC and vitamin B12 does not have a additive diagnostic value, and confirms that holoTC can replace vitamin B12 in screening for metabolic vitamin B12 deficiency.
In summary, we confirmed that holoTC has a better test performance than vitamin B12 and therefore can replace vitamin B12 in detecting metabolic vitamin B12 deficiency. Based on results from this multicentre study, we suggest applying a cut-off value of 32 pmol/L for holoTC in screening for metabolic vitamin B12 deficiency.
DECLARATIONS
