Critical difference calculations revised: inclusion of variation in standard deviation with analyte concentration

Abstract

Background

The critical difference (CD), the smallest difference between sequential laboratory results which is associated with a true change in the patient, is commonly calculated by assuming the same standard deviation (SD) for the initial and subsequent measurements. The calculation of the CD is re-examined without making this assumption.

Methods

A formula for CD is developed, which specifies that even with the assumption of constant coefficient of variations (CV) at the two measurement concentrations used in the calculation, there will be different SDs due to different concentrations.

Results

The effect of removing the assumption of constant SD is to increase the CD for rises in analyte concentration and to decrease the CD for falls in concentration. These effects are caused by increased SD for the second measurement compared with the first when the second measurement is higher, and the reverse when the second is lower.

Conclusions

Replacing the usual assumption of similar total result SD for both measurements included in the CD calculation with a calculation of the SD at both analyte concentrations leads to an increase in the magnitude of the CD for rises in analyte concentration and a decrease for falls in analyte concentration. This change is proposed for all forms of CD calculations.

Background

The critical difference (CD), also known as the reference change value, is a parameter used to assist with interpretation of laboratory results. The CD is defined as the smallest difference between sequential laboratory results in a patient which is likely to indicate a true change in the patient. The CD can be expressed as a percent (CD^%) or an absolute value in the measurement units of the analyte (CD ^u ). These are related by CD ^u = CD^% × A units, where A is the analyte concentration from which the CD will be determined. The CD can be calculated for different probabilities (CD _p ) that the difference is not due to random variability in the results although it is commonly set to 95% (CD₉₅). The most commonly quoted formula for calculating the critical difference at the 95% confidence limit is CD₉₅ ^% = 2.77 × √(CV _{a ²} + CV _{i ²} ) where CV _a = analytical coefficient of variation and CV _i = CV of within-subject biological variation. ^{1, 2} The coefficient of 2.77 is derived from √2 × 1.96 where 1.96 is the probability factor (Z) for 95% probability of a true change in either directions, multiplied by √2 to account for the fact that the CD includes variation in two measurements. For the purposes of the following discussion, the analytical and within-subject biological variation (including pre-analytical variation) are combined to make a total result variation as follows: CV_tot = √(CV _{a ²} + CV _{i ²} ).

The usual formula for CD assumes that the SD_tot (total result standard deviation) of the first result measurement and the measurement that would be made at the CD₉₅ level are the same. This is a simplification that becomes progressively less true as the CV_tot increases, and thus the difference between the initial concentration and the concentration at the critical difference increases.

Methods

The usual formula can be modified to include the fact that although the first result and the result at the CD were assumed to have the same CV_tot, they would have different SD_tot due to their different values. If the second result was higher than the first, then for the same CV_tot, the SD_tot would be larger than the SD_tot for the first result, and the converse would apply if the second result was smaller. The expanded formula is as follows: CD^% _p = Z × √[(CV_tot)² + ((1 + CD^% _p ) × CV_tot)²]. In this formula, the CV_tot at the level of the CD was calculated by multiplying the CV_tot at the first measurement concentration by 1 + CD. Although this formula was derived using a different CV at the second measurement, as this was applied to the concentration at the first measurement it was numerically equivalent to the same CV at different concentrations of the second measurement. This formula can be solved for CD^% _p after re-expression as a quadratic with two solutions. Solution 1 was the CD^% _p for an increase in concentration and solution 2 for a decrease in concentration. CD^% _p (1) = (−b − √(b ² − 4 × a × c))/2a; CD^% _p (2) = (−b + √(b ² − 4 × a × c))/2a where a = CV_tot ² − 1/Z ²; b = 2 × CV_tot ²; and c = 2 × CV_tot ². A graphical example of the CD calculation using the usual equation and the revised approach is shown in Figure 1.

Figure 1

Graphical example of the current (a) and revised (b) calculations of critical difference (CD). A simulation of a test with a CV_tot of 20% and a first result of 10 units. According to current CD calculations (a) the first result (vertical solid black line), the first result plus the CD₉₅ (vertical solid grey line) and the first result minus the CD₉₅ (vertical dashed black line) have the same standard deviation (SD) (distributions shown) and the values are equally spaced at ±5.54 units. With the revised CD calculations (b), the SD of each result varies with result concentration with a CD for an increase of 8.04 units and for a decrease of −4.45 units

Results

Using these formulae, the value of the coefficient by which CV_tot must be multiplied to determine the CD^% for different values of CV and Z can be calculated. These are shown in Table 1 for probabilities of 80, 90, 95 and 99%. In each case with increasing CV_tot, there was a marked increase in the coefficient for increases in analyte concentration and a more gradual reduction with increasing values of CV_tot for falls in analyte concentration. For example, for increases in analyte concentration at the 95% probability level, at a CV_tot of 1% the CD₉₅ is 2.8 × CV_tot, at 10% is 3.3 × CV_tot, at 20% is 4.1 × CV_tot and at 30% is 5.6 × CV_tot. For falls in analyte concentration the formula at a CV_tot of 10% is 2.4 × CV_tot, at 20% is 2.2 × CV_tot and at 30% is 2.1 × CV_tot. The ‘standard’ coefficient of 2.77 for 95% probability was approached as the CV_tot became very small. Note also that for high values of CV_tot, the calculation failed to produce a reasonable answer. For example at the 99% probability level, with CV_tot greater than 40%, and for CV_tot > 50% at the 95% probability level, the calculation for an increase in concentration produced a negative number. This occured when it was not possible to reach this level of significance due to the very wide scatter of the higher value.

Table 1

Coefficients for critical difference (CD) calculations

	Probability of true change
	Decreasing concentration				Increasing concentration
CVtot	80%	90%	95%	99%	80%	90%	95%	99%
2%	1.78	2.28	2.70	3.52	1.84	2.39	2.85	3.78
5%	1.73	2.21	2.60	3.35	1.90	2.48	2.99	4.03
10%	1.67	2.10	2.46	3.13	2.00	2.66	3.25	4.55
20%	1.55	1.94	2.24	2.81	2.26	3.16	4.05	6.43
30%	1.47	1.81	2.09	2.63	2.62	3.98	5.61	12.52
40%	1.40	1.73	2.00	2.58	3.18	5.59	9.97	*

CV, coefficient of variation

The table shows the coefficient by which the CV_tot must be multiplied to determine the CD^% for decreasing and increasing changes in the analyte concentration for different probabilities of a true change in the patient

*Unable to calculate CD for these values of probability and CV_tot (see the text)

Conclusions

It can be seen that taking the variation in SD with analyte concentration into account has a number of consequences. Firstly, there are observed changes in the calculated CD from currently accepted values as shown in the table. Additionally, there is addition of an asymmetry to the CD values. Unless the CV_tot is small, a following result must increase further to reach statistical significance than it must fall to reach the same significance. Another consequence is that the ‘bedside’ calculation becomes a little more complex. For example, with an analyte with a CV_tot of 10%, the previous calculation for a difference with a 95% chance of being real would be a rise or fall of approximately 28%. Using the current theory, the difference would be a rise of 33% or a fall of 25%. For an analyte with a CV_tot of 30%, a CD₉₅ of ±84% becomes +170% or −63%. These values are calculated from Table 1 by multiplying the value for CV_tot by the relevant coefficient. The approach described here is also relevant to skewed data which are log-transformed for CD calculations with the CD converted back to untransformed data for application. ³ In this case, the revised calculation should be applied to the transformed data prior to back-conversion.

The theory of CD is based on providing a theoretical understanding of the factors that contribute to result variability so that medical decisions can be improved. At this time it may be considered a theory in development with many areas in need of further research and understanding. Such areas may include variation in analytical CV at different analyte concentrations, ² consideration of sensitivity as well as specificity in CD calculations, ⁴ variation in CVi in different states of health, ⁵ at different ages or over different time periods and defining the required level of significance for clinical decisions. ¹ The effect of the reporting interval size (RIS) on CD calculations has already been described and a factor of approximately 1.5 × RIS should be added to CD ^u to allow for this effect. ⁶

In conclusion, the usual method for calculating the CD includes an assumption of similar total result SD for both measurements used in the calculation. If this assumption is replaced by calculation that includes differences in the total SD for the two measurements, an increasing CD is seen with increasing values of CV_tot for rises in analyte concentration and a decrease in the magnitude of the CD is seen for falls in analyte concentration.

DECLARATIONS

Competing interests: No competing interests to declare.

Funding: This research was conducted by GJ within his employment at St Vincent's Hospital, Sydney. There was no external funding.

Ethical approval: No ethics committee approval was sought for this study as no patient data were used and no patient interventions were conducted.

Guarantor: GJ.

Contributorship: All work was done by GJ.