Abstract
Abstract
Background
Between-method differences in external quality assurance schemes (EQAS) are assumed to be indicative of between-method differences in results for fresh diagnostic serum samples. However, this assumption is only valid if the EQAS fluid is shown to be commutable between methods with the fresh human samples. Using fresh human serum samples, we studied the commutability of six UK National EQAS (UKNEQAS) samples and two reference serum preparations using five methods for the measurement of albumin.
Methods
The VITROS® Albumin Slide was compared with two liquid reagent bromocresol green-based methods and two immunoturbidimetric methods. Thirty fresh human serum samples, six UKNEQAS samples and two reference materials were analysed over a period of five days by all methods. The reference materials were: a certified protein reference material, CRM470 (protein reference preparation) and the Scandinavian NFKK Reference Serum X.
Results
A concentration-dependent tendency towards non-commutability was seen for the six UKNEQAS samples. Two of the methods recovered the CRM470 above the upper limit of the expected range and the remaining three were within the expected range.
Conclusions
Assessment of commutability is important in the investigation of between-method differences in EQAS. External quality assurance and reference materials should have been demonstrated to be commutable and be available at multiple concentrations, before they may be considered appropriate for the assessment of accuracy.
Introduction
Commutability is the ability of a material to perform identically to fresh human serum specimens in a measurement system with respect to variations in the measurement conditions such as those encountered when different procedures are used to measure the same measurand. 1,2 Control and calibrator materials often exhibit characteristics, commonly termed ‘matrix’ effects that result in them behaving differently from patients' specimens, despite being prepared from human materials. 3–5 Although not always clearly understood, matrix effects, and hence non-commutability can arise from several sources such as the material chosen as the base or starting matrix, constituents being added to the base or starting matrix to achieve the desired level of measurand, the addition of stabilizing agents and lyophilization. There is evidence that matrix effects are relatively common and that all routine analytical methods may demonstrate them. 6 In addition, it has been shown that the propensity to non-commutability of processed materials is greater where the analytical method principles of the methods being used in the commutability study differed significantly. 3
Currently available methods for the measurement of albumin in serum or plasma vary in their principles of measurement, ranging from dye-binding methods to immunoturbidimetric and immunonephelometric methods. All these methods have been shown to demonstrate significant metrological differences. Thus, bromocresol purple is less precise due to a molar extinction coefficient with bound human albumin half that of bromocresol green (BCG). 7 BCG shows a pronounced reactivity to various globulins, 8 although measuring the absorbance after a short reaction time can substantially reduce the effect of non-specific binding. 8 Immunonephelometric methods, although more specific than dye-binding methods, have been shown in one case to have a between-day imprecision 2.7–4.6 times that of a BCG method. 9
The analysis of samples distributed by external quality control schemes is an important aspect in the assessment of the performance of routine laboratory methods. Such samples should be shown to be commutable with fresh human serum samples if they are to be used for the assessment of the accuracy between different methods.
The aim of this study was to evaluate the commutability with six fresh human serum samples distributed by the UK National External Quality Assessment Scheme (UKNEQAS), a certified protein reference material (CRM470) and a reference material supplied by the Danish Institute for External Quality Assurance for Laboratories in Healthcare (NFKK Reference Serum X), between the VITROS Chemistry Products Slide (Ortho-Clinical Diagnostics, Rochester, NY, USA) for the measurement of albumin in serum and four commercially available albumin methods that utilize reagents in liquid format.
Materials and methods
Equipment and reagents
A VITROS 250 analyser (Ortho-Clinical Diagnostics) was used for the VITROS MicroSlide (Ortho-Clinical Diagnostics) method for the analysis of albumin.
Two analysers were used to obtain results from liquid format reagents: Cobas Mira (Roche Diagnostics, Mannheim, Germany) and Kone30 (Thermo Electron Corporation, Finland).
Four commercially available methods for the measurement of albumin in human serum using liquid format reagents were used in this study. Two of the methods were based on the BCG dye-binding principle from different manufacturers: ABX Diagnostics, Montpellier, France (Reference A11A01664) and Thermo Electron Corporation (Reference 981767). The remaining two methods were based on the immunoturbidimetric principle from different manufacturers: ABX Diagnostics (Reference A11A01623) and Thermo Electron Corporation (981660). The two reagents obtained from ABX were used on a Cobas Mira analyser. The two reagents from Thermo Electron were used on a Kone30 analyser.
The ABX BCG albumin method was calibrated using the ABX Calibrator (Reference A11A00108) and the calibration validated using ABX N-Control quality control fluid (Reference A11A01653) and ABX P-Control quality control fluid (Reference A11A01654), all of which were obtained from ABX Diagnostics. The ABX immunoturbidimetric albumin method was calibrated using the ABX Protein Cal (Reference A11A01698) and the calibration validated using ABX Protein Control L/H quality control fluid (Reference A11A01700), all of which were obtained from ABX Diagnostics.
The VITROS Chemistry Products Albumin Slide was calibrated using the VITROS Chemistry Products Calibrator Kit 4 (Reference 1204668) and the calibration validated using VITROS Chemistry Products Performance Verifier I quality control fluid (Reference 8067324) and VITROS Chemistry Products Performance Verifier II quality control fluid (Reference 8231474), all of which were obtained from Ortho-Clinical Diagnostics.
The Kone BCG albumin method was calibrated using the Konelab Calibrator I (Reference 980501) and the calibration validated using Konelab Nortrol quality control fluid (Reference 981043) and Konelab Abtrol quality control fluid (Reference 981044), all of which were obtained from Thermo Electron Corporation. The Kone immunoturbidimetric albumin method was calibrated using the Konelab Microalbuminuria Calibration set (Reference 981239) and the calibration validated using Konelab Microalbuminuria Control quality control fluid (Reference 981253), all of which were obtained from Thermo Electron Corporation.
Specimens and procedures
All reagents, calibrators and controls were prepared according to the manufacturer's recommendations. All methods were calibrated using the appropriate calibrators and using the protocol recommended by the manufacturer. Calibrations were verified with quality controls at two different concentrations.
Six lyophilized UKNEQAS (Birmingham, UK) samples were kindly supplied by the scheme organizers for the purpose of this study. All the UKNEQAS samples were lyophilized with a human serum base matrix.
The CRM470 Reference Protein Preparation (Reference BCR-470) was obtained from the Joint Research Centre, Institute of Reference Materials and Measurements, Geel, Belgium. This fluid is a lyophilized sample with a human serum base matrix.
The NFKK Reference Serum X was obtained from the Danish Institute for External Quality Assurance for Laboratories in Health Care (DEKS), University Hospital Herlev, Herlev Ringvej 75, Denmark. The six UKNEQAS samples (EQA-1 to EQA-6) and the CRM470 were all reconstituted as recommended by the suppliers. The CRM470 protocol requires 1.0 g of water to be added by weight on a precision balance. The exact weight of water added must be noted and correction made for volumes less than or greater than 1.0 g. The weight of water added was 0.99963 g. Thus, the target value was adjusted from 39.7 to 39.69 g/L with the consequent acceptable range adjusted to 38.89 to 40.39 g/L. The ‘NFKK Reference Serum X’ was supplied as a frozen liquid (−70°C) and hence did not need reconstitution. Prior to analysis, the thawing process recommended by the supplier was followed.
Testing was run over five days with six different fresh human serum samples being analysed on each day across all methods. In accordance with the instructions for use, the CRM470, NFKK Reference Serum X and EQA samples once reconstituted or thawed, were only run on the day of reconstitution. All samples were measured in duplicates.
Data processing
The means of all duplicate pairs were calculated. From these, the commutability of the EQA samples, the CRM470 and the NFKK Reference Serum X were assessed according to the criteria laid down in the Clinical and Laboratory Standards Institute (CLSI) guideline EP14-2A. 10 Briefly, for each method combination being investigated, the means of the duplicate pairs were fitted by both a first-order and second-order polynomial by least squares regression analysis. The most appropriate regression fit was chosen based on the two-tailed t-test for the second-order squared-term coefficient being significantly different from zero at the 95% confidence interval (95% CI). Once the most appropriate regression line had been selected, 95% CI were calculated for each sample and a continuum constructed above and below the line of regression from these. This continuum was then used as the decision criteria for the samples under assessment for commutability. Samples falling outside the 95% CI continuum were classified as non-commutable, whereas those falling within the 95% CI continuum were considered commutable. It should be noted that the span of the 95% CI continuum either side of the line of regression is dependent upon the magnitude of the dispersion of the data around the line of regression. The smaller the dispersion, the more narrow will the prediction interval continuum become and the more stringent will be the test of commutability, and vice versa. Altman-Bland difference plots were used to assess the relationship between the VITROS Chemistry Products Albumin Slide and each of the comparative methods for human samples. The differences for the reference preparations and the EQA samples were overlaid onto the difference plots.
Results
For the Vitros method with respect to the ABX and the Kone immunoturbidimetric methods, the squared-term coefficient of the second-order polynomial regression equation was not significantly different from zero at the 95% CI (P = 0.496 and 0.326, respectively). Similarly, for the Vitros method with respect to the ABX BCG method, the squared-term coefficient was not significantly different from zero (P = 0.407). Hence, the assessment of commutability for these three method combinations was made using a linear regression model to fit the human serum sample sets.
In contrast, the squared-term coefficient of the second-order polynomial fit between the Vitros method and the Kone BCG method was significantly different from zero at the 95% CI (P = 0.012). Thus, the assessment of commutability for this method combination was made using a second-order polynomial to fit the human serum sample set.
Commutability
The EQA samples showed a general trend towards non-commutability as the albumin concentration decreased for all method combinations except that between the Vitros method and the Kone BCG method. This is evidenced by a general trend to an increasing magnitude of y-residual with decreasing albumin concentration (Figure 1). In the latter case, all EQA samples lay close to the best-fit line of regression, with the exception of the highest concentration sample, EQA-6, which lay close to the decision limit beneath the regression line (Figure 1d). Notwithstanding, for all method combinations, all UKNEQAS samples were found to be commutable with the exception of EQA-1, which had the lowest albumin concentration, the highest concentration samples being fully commutable with fresh human samples (Figures 1a–d). Sample EQA-1 lay close to the boundary of the upper 95% CI for both the two Vitros, ABX method combinations (Figures 1a and c), whereas it lay just outside this boundary for the Vitros, Kone immunoturbidimetric method combination, and hence was considered non-commutable between these two methods (Figure 1b).
(a) Linear regression plot for Vitros Albumin (x) vs. ABX immuno (y). (b) Linear regression plot for Vitros Albumin (x) vs. Kone immuno (y). (c) Linear regression plot for Vitros Albumin (x) vs. ABX BCG (y). (d) Second-order polynomial regression for Vitros Albumin (x) vs. Kone BCG (y). (+) denotes fresh human serum; (▪) denotes ‘X’ fluid; (◊) CRM470; (▴) EQA samples EQA-1 to EQA-6. NB: EQA and reference preparation data are overlaid onto the regression plots created from human serum data
Both the CRM470 and the NFKK Reference Serum X were found to be commutable for all method combinations.
Recovery of reference preparations
Values obtained for the two reference materials for all methods are shown in Table 1. The VITROS method, the ABX BCG method and the ABX immunoturbidimetric methods all recovered a value for the CRM470 that was within the expected range, whereas the Kone BCG method and the Kone immunoturbidimetric method both over-recovered the CRM470 (Table 1). Only the VITROS method recovered the NFKK Reference Serum X within the expected range. All the other methods over-recovered the albumin in this sample (Table 1).
Values obtained for the CRM470 and ‘X’ fluid reference preparations
*Range adjusted for the weight of water used to reconstitute the CRM470 †Concentration outside expected range
Method comparison
Visual inspection of the Altman-Bland difference plots 11 for the immunoturbidimetric methods vs. the VITROS method showed a proportional element such that, generally, there is a tendency for samples with low-albumin concentrations to show positive biases, whereas samples with albumin concentrations above 30 g/L are more evenly distributed around the zero bias line (Figures 2a and b). Visual inspection of the difference plot for the ABX BCG method showed the biases to be systematic across the concentration range tested (Figure 2c). Visual inspection of the difference plot for the Kone BCG method showed the bias to have a pronounced proportionality with concentration (Figure 2d).
(a) Comparative method ABX immunoturbidimetric assay Altman-Bland difference plot for fresh human serum. (b) Comparative method Kone immunoturbidimetric assay Altman-Bland difference plot for fresh human serum. (c) Comparative method ABX bromocresol green (BCG) assay Altman-Bland difference plot for fresh human serum. (d) Comparative method Kone BCG assay Altman-Bland difference plot for fresh human serum. (+) Fresh human serum; (▪) ‘X’ fluid; (◊) CRM470; (▴) EQA samples EQA-1 to EQA-6. NB: EQA and reference preparation data points are overlaid onto difference plot created from human serum data
Discussion
The dispersion of the fresh human sample data around the line of regression was greater between the Vitros BCG method and the two immunoturbidimetric methods than between the Vitros BCG method and the other BCG methods. This is expected as the measurement principles differ significantly in the first case and are very similar in the second. 3 The wide range of dispersions found may reflect the variations in concentrations of different proteins with their individual interferences in different assays and point to the fact that measurement of albumin in plasma or serum is complicated and may need improvements.
The wider span of the 95% CI continuum encountered for the comparison between the Vitros method and the two immunoturbidimetric methods meant that the test for commutability was less stringent than for the comparison with the two BCG methods. This could, to some extent, be ameliorated by significantly increasing the number of fresh human samples used in the study for these methods.
The concentration-dependent progression towards non-commutability seen in the EQA samples may suggest a change in the composition of these samples that is related to the way in which decreasing concentrations of analyte are achieved. Samples with low concentrations of albumin supplied by a Welsh External Quality Assurance Scheme are produced by diluting a normal human serum with water. A previous study 9 has demonstrated that these samples showed a tendency to under-predict when analysed with the VITROS Chemistry Products Albumin Slide compared with an immunoturbidimetric assay. In addition, the same study showed that diluting the CRM470 reference preparation with phosphate buffer also produced an under-recovery of albumin when assayed with the VITROS method. There are few clinical conditions that result in a high albumin concentration in serum, the most common being dehydration. Even in these cases, the albumin concentration rarely reaches levels that would require sample dilution. Thus, samples with albumin concentrations manipulated by dilution with water present an unrealistic model of samples that will be encountered in the clinical laboratory.
Given the relatively poor commutability of the lowest concentration UKNEQAS sample and the general structured nature of the relationship between commutability and concentration for these samples, interpretation of results by participating laboratories in terms of the accuracy in the prediction of natural human serum samples should be made with caution. At the lower concentrations of albumin in the UKNEQAS samples, such judgements would be inappropriate. Unless commutability with human serum samples has been demonstrated between methods, any conclusions relating to the accuracy of an analytical method based on the results acquired when measuring EQA samples might be erroneous and misleading.
The CRM470 is a certified reference material that has assigned values for a number of proteins commonly measured in serum, including albumin. It is a processed fluid with a human serum base matrix. This base matrix has been filtered, stabilized by addition of sodium azide, aliquoted and lyophilized. It is designed to be a primary calibrator against which working calibrators for use with commercially available reagent kits may claim traceability. At the same time, CRM470 is available to the routine laboratory and thus could have a role in the arbitration between routine methods. It is therefore of interest to know whether the CRM470 is commutable with fresh human serum samples between commercially available methods. We have shown that the CRM470 is commutable with fresh human serum samples between the VITROS Chemistry Products Albumin Slide and four alternative commercially available methods. Given the apparent commutability of the CRM470, particularly with respect to the specific immunoturbidimetric assays, it is also possible to assess the accuracy of the various methods directly with this preparation. In our hands, the CRM470 showed a 99% recovery of the target albumin value when measured with the VITROS method and the ABX BCG, and the ABX immunoturbidimetric methods both recovered a value for the CRM470 within its expected range, whereas the Kone BCG method and the Kone immunoturbidimetric methods recovered a value above the upper limit of the expected range (Table 1). Nonetheless, caution needs to be exercised, as the CRM470 is a single point in the measuring range of serum albumin. The difference plot for VITROS vs. the Kone BCG method showed a pronounced proportional component to the relationship between the two methods. Thus, an increasing error was seen for patients' samples at concentrations either side of the CRM470; biases becoming increasingly negative at lower concentrations and more positive at higher concentrations. Simply adjusting the accuracy of the Kone BCG method with respect to the CRM470 would not necessarily be expected to guarantee accuracy over the entire range of concentrations likely to be encountered in the clinical environment.
The NFKK Reference Serum X is a natural human serum pool that has undergone minimal processing. The pooled serum has been filtered, aliquoted and frozen at −70°C. The proposed application for the NFKK Reference Serum X is perhaps best described as a tertiary calibrator. It has values assigned for 26 constituents including albumin. The value assignment process used to establish the reference value for the NFKK Reference Serum X and its proposed application has been described elsewhere. 12 The accuracy base of the NORIP reference range study is closely linked to Reference Serum X. 13,14 Briefly, it is proposed that the clinical laboratory analyses the NFKK Reference Serum X using their current routine method and if the assigned value is recovered within a goal based upon a specified fraction of the biological variation for each analyte, then the laboratory may report results for patients' samples using reference ranges established in the NORIP reference range study. The validity of this accuracy link must crucially depend upon the NFKK Reference Serum X being commutable between methods, but we are not aware of any formal commutability studies for this fluid for albumin methods. Based on the chosen criteria, the NFKK Reference Serum X was found to be commutable between the VITROS method and the other four methods. For this fluid, only the VITROS method recovered a value within the proposed expected interval. All the other methods predicted a value for the NFKK Reference Serum X above the upper limit of the proposed range (Table 1). However, the same caution needs to be exercised in the interpretation of this finding as was earlier described for the CRM470 and for the same reasons. As the value assignment for the albumin concentration of Serum X is traceable to CRM470, 12 it seems paradoxical that both the methods are in agreement with CRM470 and not with Serum X concentrations. However, the difference is slight as the two ABX methods recovered concentrations in the high part of the CRM470 acceptance interval. This difference might be explained by different matrix effects within the confidence intervals of the commutability tests; CRM470 being a modified lyophilized material.
For three of the method combinations, the Altman-Bland difference plots showed an element of proportionality in bias across the albumin concentration range tested. The estimate of average bias calculated from the Altman-Bland bias calculation 11 is only valid where the bias between methods is shown to be constant over the concentration range and not proportional. This constraint thus renders the estimate of average bias between the Vitros method and either of the two ABX methods or between the Vitros method and the Kone BCG method in this study inappropriate, and thus an estimate is not cited here.
Conclusion
Judgements about the absolute accuracy of any given routine analytical method based on the recovery of albumin in EQA samples must be made with caution and require that the commutability of EQA samples with respect to patients' samples be assessed.
Both reference preparations were found to be commutable between the VITROS Chemistry Products Albumin Slide and the four other comparative methods despite one being highly processed and the other minimally processed. Conversely, the EQA samples showed a concentration-dependent tendency towards non-commutability, probably as a result of a modified matrix. The same effect would be expected for any EQA sample produced the same way. We conclude that both the CRM470 and the NFKK Reference Serum X are expected to behave in the same way as patients' serum for the methods tested. Using a single point reference material may give a misleading impression of the overall accuracy of a routine method when applied to patients' samples. Thus, if they are to be used as an arbiter of accuracy between methods, patients' samples covering the measurable range should be included in any assessment.