Uncertainty of sweat chloride testing: does the right hand know what the left hand is doing?

Abstract

Although analytical variation in sweat electrolyte testing can be easily estimated, there is limited data on total variation. This study aims to evaluate the total variation of the sweat test by measuring the difference between sweat electrolyte values in specimens obtained simultaneously from two sites. Chloride is recommended in published guidelines as the only discriminant for the diagnosis of cystic fibrosis, and sodium may be measured as a guide to the adequacy of collection and analysis. Both are reported here. Sweat was collected in patients by the Gibson Cooke method from two sites simultaneously. Coefficient of variation in this laboratory is 4.1 and 5% for chloride and sodium, respectively. 295 patients had sufficient sweat collected from both sites for analysis. The values for chloride and sodium were compared between the two sites. The total coefficient of variation (CV_t) calculated for the whole group between the two sites was 20.2% for chloride and 16.9% for sodium, and the standard deviations 4.3 mmol/L and 4.8 mmol/L, respectively. In patients with intermediate chloride concentrations; in different age groups; and when those tests with a difference between sodium and chloride concentration of more than 15 were excluded, minimal differences in these figures were observed. Use of strictly defined cut-off points to discriminate between normal and intermediate electrolyte values, and between intermediate and raised electrolyte values, does not reflect the variation in sweat electrolyte content found within an individual patient. This has important implications for reporting.

Introduction

The sweat test remains the primary test for the diagnosis of cystic fibrosis (CF). It is unlikely to be replaced in the near future by mutation analysis given the fact that over 1500 variations ¹ in the CFTR gene have been documented to date, making it an inappropriate first-line testing strategy for clinical laboratories. ² Likewise, the availability of nasal mucosa potentiometry is restricted to a few laboratories.

Because of the importance of the test in diagnostic decision-making, guidelines for the performance of the sweat test have recently been published in order to attempt to standardize the performance of the test. ^3–5

In these guidelines, a cut-off of 40 mmol/L for chloride has been recommended in order to discriminate between healthy patients and those with an increased chance of CF. Likewise, a cut-off of 60 mmol/L has been proposed to separate the patients highly likely to have CF from those with increased chance.

However, cut-off values which are interpreted rigidly could potentially be misleading, especially if the uncertainty of the measurement is significant. Such uncertainties could occur, for example, from the procedures of sweat collection, analytical imprecision, and within-subject and temporal variations. These uncertainties of measurement may have led to significant misclassification, with false positives up to 15% and false negatives up to 12%. ⁴

Recently, many clinical laboratories have been expressing the degree of analytical uncertainty in a way that can be understood by clinicians, ⁶ as a consequence of undertaking accreditation to ISO 15189 standard. ⁷ Generally, this is to establish an ‘uncertainty of measurement’ figure for each assay, which is twice the standard deviation (SD), that is, approximately the 5th and 95th percentile values, of the mean value of the quality control at each level. While this establishes the analytical uncertainty, it does not take into account other sources of variation. It is therefore useful to be able to offer clinicians information about the size and nature of variations arising from other sources.

It has been an established practise in this laboratory over many years to analyse sweat collected from two sites simultaneously, in order to minimize any re-collections necessary. This has created a unique opportunity to explore our data, in order to evaluate the differences in electrolyte values recorded between the two sites as a way of measuring the total coefficient of variation (CV_t) of the test. This figure is a sum of variation due to analysis, within-patient variations, and collection variations. It does not take into account variation which occurs over time.

Method

The values from 298 successful paired sweat tests performed over a period of eight years, from October 1996 to September 2004, were available for analysis. It is not possible to be certain how many were performed in response to increased risk detected in the national newborn screening programme, but 52 were three months old or less and therefore most likely to be screened patients, leaving 246 patients who were tested for clinical reasons. During this period, there were no changes in the method of collection and analysis of the sweat. Sweat was collected by pilocarpine iontophoresis using the Gibson Cook method ⁸ onto Whatman paper from two sites simultaneously, nearly always the inner aspect of the right and left arms. Analysis for chloride content was by a mercuric thiocyanate colourimetric method and for sodium by flame photometry (Instrument Laboratory 643, Lexington, Massachusetts). Internal analytical imprecision at 53 mmol/L is 5% for sodium, and for chloride at 8.9 mmol/L is 4.1%. This laboratory has participated in the Royal College of Pathologists of Australasia external quality assurance programme since it was introduced in 1999 with satisfactory results. The mean six-monthly %CV for both sodium and chloride have been less than 10%. This scheme covers sodium and chloride concentrations from 10 to 120 mmol/L, with the allowable limits of performance for both sodium and chloride at ±2 mmol/L up to a concentration of 20 mmol/L and ±10% above 20 mmol/L. This laboratory has followed the NCCLS (now Clinical and Laboratory Standards Institute, CLSI) recommendation of spotting these external quality assessment aqueous solutions onto pre-weighed filter paper and then processing in parallel with the unknowns.

Tests were reported clinically as values for both chloride and sodium for right and left arms separately, together with an interpretive comment. If the test on one side did not yield sufficient weight, only the single test was reported. A sweat rate of >1 mg/m²/minute was required, which given the size of our collecting surface area was 50 mg weight.

Data were analysed using the MedCalc for Windows (MedCalc Software, Belgium) software version 8.2. The SD between paired values was calculated and variance derived. The mean of the sum of variances is the mean variance, and the square root is the mean SD, from which the coefficient of variation can be derived. This was done for the group as a whole and separately for those with one or more values in the 30–70 mmol/L range, that is, between the two cut-off values recommended by the guidelines ^3–5 plus or minus about two SD.

This analysis was repeated for the age groups 0–6 months; 6–12 months; 1–5 years; and over 5 years separately, and again for those with a difference of chloride and sodium values of 15 mmol/L or less. We also evaluated the relationship of sweat weight and age, and sweat chloride and age.

Lastly, the number of patients for whom the classification (normal, intermediate, raised) would change as a result of the differences between the paired values was obtained, both as a fraction of the whole, and as a fraction of those with intermediate values.

Results

Between October 1996 and September 2004, 497 sweat tests were performed of which 298 gave results for two sites. Of those who failed both sites, 17 (35%) were aged less than three months, compared with 51 (17%) aged less than three months with successful bilateral tests. Three further tests were excluded as the chloride values on one of the results was unmeasurable, which was thought likely to be analytical error. Twenty were repeat tests on nine patients.

Mean and median population sweat weight and sweat chloride and sodium values and confidence intervals for each site are shown in Table 1. This indicates that there is no important systematic difference between the two sites not accounted for by random error and the size of the population. Similar values were recorded in a previous publication ⁹ ; however, the latter underestimate the population distribution, because they appear to have been compiled by pooling the values from the two sites, which are not independent values.

Table 1

Comparison of mean and median values recorded on left and right sides (CI = 95% confidence interval)

	Left	Right
Mean sweat weight (CI)	145 (137–153)	147 (139–154)
Median sweat weight (CI)	145 (140–149)	148 (144–152)
Mean sweat Cl− (CI)	21 (18.9–23.7)	21 (18.6–23.4)
Median sweat Cl− (CI)	14 (13.0–15.8)	14 (13.0–15.7)
Mean sweat Na+ (CI)	28 (25.7–30.3)	28 (25.6–30.7)
Median sweat Na+ (CI)	21 (19.0–23.0)	20 (18.0–22.0)

For the group as a whole (n = 295), the overall between-pair SD was 4.3 mmol/L and 4.8 mmol/L for chloride and sodium, respectively. The mean chloride value is 21 mmol/L and the CV_t therefore 20.2%, whereas the mean sodium is 28.1 mmol/L with CV_t = 16.9%. Comparable values for CV_t persisted across the age ranges less than six months, 7–12 months, 1–5 years and over five years, and also in those tests where the difference between chloride and sodium was less than 15 mmol/L. (Table 2).

Table 2

Mean SD between pairs of values and the CV_t for different groups of sweat test patients

Patients	SD Cl−	Chloride %CVt	SD Na+	Sodium %CVt
Total, n = 295	4.3	20.2	4.8	16.9
Cl− between 30 and 70	7.2	17.2 (n = 47)	7.0	15.9 (n = 95)
Na+–Cl−>15 excluded, n = 245	3.5	19.4	4.1	17.6
Age < 6 months, n = 72	3.4	18.2	4.0	17.7
Age 7–12 months, n = 36	2.5	18.3	2.9	15.2
Age 1–5 years, n = 111	4.3	23.3	4.7	18.9
Age > 5 years, n = 74	5.3	17.9	5.7	13.7

SD, standard deviation; CV_t, coefficients of variation; Cl⁻, chloride; Na⁺, sodium

When the analysis is restricted to the patients with at least one value between 30 and 70 mmol/L, i.e. an intermediate range defined as the cut-off levels established by the guidelines ± ∼2 SD, the SD becomes 7.2 mmol/L and 7.0 mmol/L for chloride and sodium, respectively and the CV_t becomes 17.2 and 15.9%.

The variation between the chloride concentrations obtained from the two different sites would suggest, had only one site been tested, that up to 13 patients (4.4% of the total group, 48% of those with at least one intermediate values) could be classified differently.

There was no correlation of sweat weight and age (R ² = <0.0002 for both left and right tests), nor was there any correlation of sweat weight and chloride value (R ² = <0.035 for both left and right tests). Although it was not possible to directly compare CV_t for different sweat volumes, there was no relation between variance and sweat volume (R ² = 0.006).

Discussion

In this study, we have sought to quantify additional components of the test variability by comparing simultaneous sweat electrolyte values from two sites. There would appear to be no systematic difference between the values recorded from the two sites (Table 1). Calculated in this way, overall values of CV_t = 20.2 and 16.9% for chloride and sodium, respectively result in a significant variation to be expected from any one test. The majority of patients however are clearly normal or clearly abnormal, even allowing for the variation we have found, with a smaller proportion in the intermediate range (n = 47 in this sample) in whom this variation is important in their classification.

In a previous study aimed at developing a method for sweat test quality assurance ⁹ utilising simultaneous bilateral sweat electrolyte values, the author calculated the population SDs for normal (<40), intermediate (40–60) and raised (>60) results. Regression analysis was used to demonstrate the agreement between two testing sites. It was suggested that comparison of current year parameters with previous year parameters might help to provide some quality assurance. However, the evolution of statistical techniques since then allows a fresh view of data acquired in a similar way.

More recently, in a study of adult volunteers ¹⁰ over two years, sweat was stimulated by pilocarpine iontophoresis and collected by Wescor Macroduct (Wescor Inc, Utah, USA) on eight occasions in three, and 12 occasions in one subject. Total %CV was calculated as 14.9–32.9% in different individuals. This is even higher than our own figures, probably because it includes temporal variation as well.

In a study, ¹¹ of sweat conductivity values, (which in themselves are not directly comparable with sweat electrolyte values), sweat conductivity was assessed weekly for five weeks in 15 healthy adult volunteers, 20 healthy infants and 20 known CF patients. Although their group is not entirely comparable with ours, the within-subject CV_t of their healthy infant group was 18%, a value consistent with our CV_t of 20.2% (for the whole group).

The NCCLS (CLSI) guidelines ³ suggest, as an index of internal quality control, that sweat sodium and chloride concentrations agree within 15 mmol/L. The evidence for adopting this measure appears to be expert opinion and it is hard to perceive how much scientific merit is in this, particularly as the chloride:sodium ratio varies significantly with total concentration. In our study, use of this exclusionary criterion did not improve the CV_t. Paired testing however, coupled with knowledge of the variation to be expected, is a more precise way of confirming test validity.

Calculated from our data, the total variation for the whole group means that a chloride value of 40 (i.e. at the proposed lower cut-off), has a 95% probability of lying between 31 and 49 (40 ± 2 × SD for overall values), and for a value of 60 (the upper proposed cut-off), a 95% probability of lying between 51 and 69. Should the CV_t value for the intermediate sweat test group be used, the situation might be worse. These are rather wide limits and call into question the usefulness of adhering to fixed cut-off values for the purposes of reporting. This is especially so, as these values quantify the difference between simultaneous sweat tests, but do not take into account differences within an individual over time, nor of inter-assay variability.

Defined cut-off values, in addition, fail to take account of the fact that the CF spectrum is likely to be continuously distributed ¹² between those who have no disease at all, through mild symptoms and signs, to severe manifestations, with a corresponding range of increasingly severe electrolyte changes. Furthermore, the diagnosis of CF must take into account not only the sweat test and mutation results, but a critical evaluation of the clinical presentation of the patients. ¹³

An evidence-based laboratory medicine method of expressing the result of a sweat test might therefore reflect this degree of uncertainty in a way that is understood and useful for clinicians. An option might be to quote sweat chloride values together with a figure for analytical uncertainty of measurement and for individual variables, derived from this and similar studies.

We conclude that when intra-individual variation is taken into account, application of prescribed cut-off values to classify tests fails to reflect the reality of variation. We submit that cut-off values which could potentially misclassify up to 33% of patients with intermediate results are inappropriate, and may obscure the continuous distribution of the manifestations of disease in the CF spectrum. In the light of both biological and analytical considerations, CF should be regarded as a ‘spectrum disorder’ and a dichotomous assignment of disease or no disease may not be appropriate. Further, comparable results from sweat collected from two sites simultaneously permits improved confidence and is a useful internal quality control procedure. The uncertainty inherent in the collection and measurement of sweat chloride could be recognized in the reporting of these values. It is suggested that bodies publishing future guidelines recognize variability arising from the collection as well as analysis of sweat electrolytes, and that this is reflected in recommendations for reporting.