Differing uncertainties and Bland-Altman plots: An observation

A recent paper ¹ evaluated the effects of low-end censoring and differing method uncertainties on Bland-Altman plots. ² In particular, differing uncertainties were shown to produce an unexpected but systematic diagonal tilt in plots of differences versus means (i.e. a parallelogram shape), thereby producing bias additional to that generated by data censoring alone. However, in a recent personal communication, Prof. DM Hawkins (School of Statistics, University of Minnesota) pointed out that the optimum estimate of the mean is not the simple average (i.e. (A + B)/2 in the case of paired A and B values), but a weighted average

$$\left(\text{A}\times {\text{V}}_{\text{B}}+\text{B}\times {\text{V}}_{\text{A}}\right)/\left({\text{V}}_{\text{A}}+{\text{V}}_{\text{B}}\right)$$

(1)

where V_A and V_B denote measurement variances of A and B, respectively. A simple simulation experiment, at a single target mean, demonstrated that plotting differences against weighted averages eliminates the diagonal effect previously reported. Three steps are involved: (i) randomly draw 10,000 paired A, B results from two Gaussian distributions each with Mean = 10 and SD = 1.0, and plot (A–B) differences against simple averages; (ii) repeat the experiment using the same sequence of pseudorandom numbers with the alteration that the Gaussian distribution generating A values has SD = 1.5; (iii) plot differences against simple averages and also against weighted averages, i.e. (A + 2.25B)/3.25. Figure 1 shows the results.

Figure 1.

Bland-Altman plots of 10,000 paired results. (a) Equal uncertainties, (b) and (c) differing uncertainties.

In all cases, Bland-Altman summary results were completely in accordance with expectation and were identical in the differing uncertainty cases (Figure 1(b) and (c)), as calculations depend solely on observed (A–B) differences. 95% limits of agreement enclosed 95.02% of the data points in Figure 1(a) and 95.12% in Figure 1(b) and (c). Figure 1(b) and (c) data patterns represent further information on the ‘diagonal’ effect. The underlying cause is a matter for theoreticians.

In practice, it is feasible that, with duplicate measurements, variance functions could be estimated for both sets of paired results, thereby allowing predicted V_A and V_B values to be used in conjunction with equation (1), but that would add another level of complexity to Bland-Altman analyses. Is it worth the effort? Probably not. When methods have differing uncertainties, low-end censoring will always generate spurious biases regardless of how averages are calculated. The key factor is retrieval and use of actual rather than censored results. ¹