Abstract
The de novo establishment of reference intervals (RIs) for all variables is beyond the capabilities of many small laboratories. Thus, recent international recommendations propose procedures to adopt RIs established by “donor” laboratories after validation in “receiving” laboratories. The objective of the current study was to use recently published RIs of canine hemostasis tests as possible donor values and evaluate the validation procedure with randomized sets of values obtained in another study of canine RI determination of prothrombin time (PT), activated partial thromboplastin time (APTT), fibrinogen, and antithrombin (AT). The preanalytical, analytical, and demographic conditions of the donor and receiving laboratories were first compared. To represent new reference individuals, 25 validation sample sets of 20 results of the receiving laboratory were randomly selected for each variable and compared with the RI of the donor laboratory. Validation was rejected in all cases for APTT and AT. Donor RI could be validated in 14 of 25 cases for fibrinogen and in 4 of 25 cases for PT. When preanalytical and analytical differences existed between donor and receiving laboratories, validation procedures consistently rejected preexisting RI. When the differences are smaller, the variability of the results obtained in the validation sample sets tested may be responsible for validations or rejections, which can lead to further misinterpretations of results from patients. Validation of a preexisting reference interval is certainly an interesting option for small laboratories, but progressive determination of the laboratory's own reference interval is probably a better long-term solution.
The International Federation of Clinical Chemistry and Laboratory Medicine–Clinical and Laboratory Standards Institute (IFCC-CLSI) recommends that each laboratory should establish its own reference intervals to ensure that the values correctly reflect the analytical characteristics of the methods used and the demographic characteristics of the individuals tested. However, the de novo determination of a reference interval is a long, difficult, and expensive process, and beyond the capabilities of most laboratories. Therefore, the latest issue 2 of the IFCC-CLSI recommendations proposes a procedure to validate preexisting, properly determined reference intervals: “every laboratory is more than capable of verifying the applicability of reference intervals in its own population.”
The proposed validation procedure can be based on “subjective […] judgment of the laboratorian” or, preferably, on a procedure “using small numbers of reference individuals.” 2 This latter procedure consists of measuring the variable in specimens from 20 reference individuals from the receiving laboratory. After eliminating possible outliers, if all but 2 values are within the reference limits, the reference interval can be validated. If 3–4 values are outside the reference interval, another validation sample set of 20 reference individuals must be tested; if all but 2 values in this new validation sample set are within the reference limits, then the reference interval can be validated. When more than 4 values are outside the reference limits, the reference interval must be determined de novo according to the recommendations. 2,4
To the authors' knowledge, the validation procedure has only been used once in veterinary clinical pathology for the determination and/or validation of reference intervals in Bernese Mountain dogs. 6 However, effects of interindividual variability within the small reference sample group used in the validation procedure have not been evaluated. The current study was therefore designed to test the soundness of transferring hemostasis reference intervals, and more specifically, the effects of the composition of the small reference sample group used by the receiving laboratory. It is based on previously published reference intervals of canine hemostasis in 56 dogs 1 (used herein as donor laboratory data) and individual results obtained in a previous study 5 (used herein to represent the procedure in the receiving laboratory). The variables studied were antithrombin (AT), prothrombin time (PT), activated partial thromboplastin time (APTT), and fibrinogen.
The subjective validation was based on a comparison of the preanalytical, analytical, and demographic conditions of the 2 laboratories as summarized from the 2 publications in Table 1. The 139 results obtained for each variable in the receiving laboratory were randomized 25 times using the RAND function of Excel. a The first 20 results obtained from each series were chosen to represent the small reference sample group hereafter called the validation sample set. The 25 validation sample sets were examined for possible outliers according to Tukey test at the >3 interquartile range criterion. When an outlier was thus detected, the next value in the series was used as a substitute, as stated in the recommendations. 2 Results were then ordered and compared with the donor reference limits to be validated, based on the following 3 criteria: 1) if 2 or less values were outside the limits, the reference interval was considered validated; 2) if 5 or more values were outside the limits, the procedure was stopped and the validation was considered impossible; and 3) if 3 or 4 values were outside the limits, the next 20 results in the corresponding series of randomized values were tested as previously, to mimic a new validation sample set of 20 reference individuals.
Comparison of the donor and receiving laboratory conditions for the test of validation of canine hemostasis reference intervals.
The reference limits of the donor laboratory with their 90% confidence intervals (when available) are indicated on Table 1, which shows that the different equipment and reagents were from the same manufacturer. Preanalytical conditions also differed, but in both cases the stability of the specimens had been validated. 1,5 Comparison of demographic conditions showed that a variety of breeds was used in both studies, and that there was a larger age range and a higher proportion of females in the receiving laboratory.
When evaluating the validation sample sets, there was no significant effect between draws when evaluating values of APTT, fibrinogen, and AT (analysis of variance [ANOVA], P > 0.05), but a significant effect was observed for PT (ANOVA, P = 0.035). The dispersion of values in the validation sample sets also differed according to the draw with coefficients of variation (CV) of 4.5–9.9% for PT, 5.5–9.1% for APTT, 19.4–45.7% for fibrinogen, and 8.9–18.6% for AT.
Detailed results obtained for the first 20 fibrinogen values from the 25 validation sample sets are presented in Figure 1. An outlier was identified in 2 validation sample sets and was replaced by the next value in the randomized nonordered series of values (the outlier in validation sample set number 7 was replaced by 1.38 g/l, and the outlier in validation sample set number 17 by 1.42 g/l). According to the reference limits of the donor laboratory, 10 validation sample sets had values of 2 or less outside the limits tested; 6 validation sample sets had values of 5 or more outside the limits tested; and 9 validation sample sets had 3 or 4 values outside the limits tested.
In a second step, the following 20 nonordered results of the 9 latter corresponding series were examined (Fig. 2). After this replacement, there were 4 validation sample sets, in which values of 2 or less were within the donor laboratory reference interval and 5 in which values more than 2 were outside. Finally, in this test of the IFCC-CLSI validation procedure for the fibrinogen reference interval, there were 14 of 25 cases in which the criteria for validation were met and 11 cases in which the criteria were not met.
For APTT and AT (Fig. 3), there were 5 or more values outside the reference limits of the donor laboratory in each validation sample set. For PT, there were 4 validation sample sets, where 2 or less values were outside the limits, 19 where there were 5 or more values outside the limits, and 2 where there were only 4. For the latter, there were 3 or 4 values outside the limits in the next validation sample sets of 20 values (results not shown).
Test of the validation of a previously published reference interval for canine plasma fibrinogen concentration (dotted lines). Each vertical set of dots represents 20 randomly selected results obtained in a study of canine hemostasis reference intervals. * = outliers according to Tukey >3 IQR criterion; R = rejected; V = validated; ? = needs further investigation, according to International Federation of Clinical Chemistry and Laboratory Medicine–Clinical and Laboratory Standards Institute recommendations.
Results obtained in the second step of the test of validation of canine plasma fibrinogen reference intervals. Each black vertical set of dots represents the second series of 20 randomly selected results obtained in a study of canine hemostasis reference intervals. * = outliers according to Tukey >3 IQR criterion; R = rejected; V = validated; ? = needs further investigation, according to International Federation of Clinical Chemistry and Laboratory Medicine–Clinical and Laboratory Standards Institute recommendations.
Test of the validation of previously published reference intervals for canine plasma prothrombin time (PT), activated partial thromboplastin time (APTT), and antithrombin (dotted lines). Each vertical set of dots represents 20 randomly selected results obtained in a study of canine hemostasis reference intervals. * = outliers according to Tukey >3 IQR criterion; V = validated; ? = needs further investigation, according to International Federation of Clinical Chemistry and Laboratory Medicine–Clinical and Laboratory Standards Institute recommendations.
The validation of preexisting reference intervals by receiving laboratories is a very interesting option for all small- or medium-sized laboratories, as this may avoid the long, difficult, and expensive selection of a large number of well-characterized reference individuals for the establishment of de novo reference interval. In the current study, hemostasis tests were chosen because of the availability of the data and also because they are reported to be strongly instrument- and reagent-specific. 7 The first option for validation is “a subjective assessment” based on “careful inspection of the pertinent factors of the original appropriate reference value study.” 2 This step is necessary but cannot be sufficient. Most often, the full details of the demographic variables are not extensively reported, and the complete set of reference values is not available. However, this first step should be used to exclude possible validation, when conditions are too different in the receiving laboratory or are not reported at all.
In the present case, the study used as basis for the donor laboratory was recent, and strictly based on IFCC-CLSI recommendations, except for the number of reference individuals (n = 56). Demographic conditions did not seem to differ notably. The analyzers used belonged to the same manufacturer and were based on the same technology, but the reagents were different. It was the subjective opinion of the investigators that the proofs of identity were not reliable enough to transfer the reference interval without a validation study.
In the test of validation, the random selection of 20-value validation sample sets represented the recommended procedure 2 to select 20 reference individuals representative of the demographics of the receiving laboratory. It was surprising to observe that there was such variability in the validation sample sets of randomly selected values. In the example of fibrinogen, the interindividual variability has recently been reported to have a CV of 28%. 8 In the present study, interindividual fibrinogen variability in the validation sample sets ranged from 19.4% to 45.7%, but this did not result in statistically significant differences between the validation sample sets. However, this variability increased the possibility of obtaining values outside a preexisting reference interval. Validation was estimated to be acceptable in slightly more than 1 of 2 cases (56%) by use of the small reference sample groups, which is not really better than coin tossing. The reference interval that had been established from the whole sample group of 139 dogs (Table 1) has a notably higher upper limit than the corresponding limit of the donor laboratory. As a consequence 8.6% of healthy dogs (12/139) would have been considered to have false-positive results by use of the upper limit of the donor laboratory (in healthy individuals, there are only 2.5% of false positives at this limit).
Less variability within the validation sample sets was observed for the other variables, as previously reported for PT and AT, whereas much higher between-dog variability (69.3%) had been reported for APTT. 8 Validation for APTT and AT was impossible in all cases. This is consistent with the notably different reference intervals of the 2 laboratories, and is likely due principally to the reagents used. For PT, validation would have been accepted in 4 of 25 cases, whereas the upper limit of the donor laboratory was 0.7 sec lower than in the current study, 5 which might cause clinical misinterpretations.
These results suggest that the validation of preexisting reference intervals can be an interesting option for a receiving laboratory, and that it seems especially valuable in demonstrating when a preexisting reference interval is inappropriate for a receiving laboratory. Results also suggest that there can be unexpected variability, and that users should be very cautious when granting validation with a first validation sample set of 20 values. This should entice laboratories wishing to use this procedure to progressively collect results obtained in reference animals to compute their own reference intervals, possibly using small reference sample groups with relevant statistical methods. 3 This is especially important when the analytical methods can notably impact the results, which is the case in hemostasis testing, or when reference intervals are only available from textbooks in which analytical and demographic details are rarely reported.
Footnotes
a.
Microsoft Corp., Redmond, WA.