Comparison of in-clinic point-of-care and reference laboratory total thyroxine immunoassays for diagnosis and post-treatment monitoring of hyperthyroid cats

Abstract

Objectives

The Catalyst One Chemistry Analyzer (IDEXX Laboratories) is a point-of-care instrument that can measure total thyroxine (TT4) by immunoassay. The aims of this study were to evaluate the analytic performance of the Catalyst TT4 assay in feline sera and to examine agreement of the Catalyst TT4 results with those measured by immunoassay at a veterinary reference laboratory.

Methods

Assay precision, reproducibility and linearity were evaluated for the Catalyst TT4 assay. For method comparison, TT4 concentrations in serum samples from 157 cats (127 hyperthyroid, 30 radioiodine-treated cats) were analyzed by both in-clinic and reference laboratory methods.

Results

The Catalyst TT4 demonstrated good precision and reproducibility (coefficients of variation ⩽8.5%) and excellent linearity in the diagnostic range of 6–150 nmol/l. Differences between the two TT4 methods showed no proportional or fixed bias (Bland–Altman plots) but did demonstrate greater spread of values at higher TT4 concentrations. Statistical analysis of percent differences between methods indicated 95% limits of agreement of ± 30%. When serum TT4 concentrations were classified as low, high or within the reference interval (12–50 nmol/l) for each assay, there was strong agreement (96.8%) in classification between methods.

Conclusions and relevance

The Catalyst TT4 assay provided precise serum TT4 concentrations in the 157 samples analyzed, which agreed well with results provided by a reference laboratory. Cats with Catalyst TT4 concentrations near decision thresholds (eg, normal vs high) should either have TT4 concentration repeated a few weeks later and/or undergo further testing (eg, free T4, serum thyroid-stimulating hormone, thyroid scintigraphy) to determine thyroid status.

Introduction

Veterinary practitioners commonly measure total thyroxine (TT4) concentrations for both diagnosis and monitoring of hyper- and hypothyroidism.^1–3 In-clinic analyzers allow practitioners to measure TT4 concentrations without sending samples to reference laboratories. However, the accuracy of some analyzers for TT4 quantification has been questioned.⁴ Recently, IDEXX Laboratories introduced an immunoassay method to measure TT4 concentrations using the point-of-care Catalyst One Chemistry Analyzer. This test uses ELISA technology in a new dry-slide format, providing TT4 test results within 15 mins. No independent assessments of assay performance for this assay exist.

Therefore, we sought to evaluate the Catalyst Total T4 Test (hereafter referred to as Catalyst TT4) for measurement in feline serum by sampling cats with untreated and treated hyperthyroidism and comparing results obtained by the Catalyst method with those obtained by a veterinary reference laboratory.

Materials and methods

Study design and selection of cats

This study was conducted in two parts. The first was designed to evaluate the analytic performance of the Catalyst TT4 assay and validate the test for use in feline sera. The second was a prospective cross-sectional study that included untreated and radioiodine-treated hyperthyroid cats examined at the Animal Endocrine Clinic.

Validation of Catalyst TT4 assay and evaluation of analytic performance in cat sera

In order to validate the analytic performance of the Catalyst TT4 immunoassay for use in cats, a number of studies were performed at IDEXX Laboratories in Westbrook, ME, USA.

Total precision was calculated according to Clinical and Laboratory Standards Institute (CLSI) EP05-A method guidelines.⁵ To determine within-run precision (intra-assay coefficients of variation [CV]), assays of five replicates of three serum pools (with concentrations of 11 nmol/l, 40 nmol/l and 182 nmol/l) were run on the same day. To determine inter-assay CV, assays of 20 replicates of four serum pools (with concentrations of 18 nmol/l, 32 nmol/l, 64 nmol/l and 129 nmol/l) were performed on five separate days.^5–7

The linearity of the Catalyst TT4 assay was validated according to CLSI guidelines.⁸ Two feline serum pools were created, containing a low and high TT4 concentration, by gravimetric addition of L-thyroxine (Sigma-Aldrich) to pooled feline serum at the specified levels. Seven intermediate concentration pools were prepared by mixing various amounts of the low and high concentrations to produce nine levels of equally spaced TT4 concentrations. The concentration of each serum pool was determined using the reference laboratories’ immunoassay. Each sample was measured three times on two Catalyst One Chemistry Analyzers (IDEXX Laboratories), and the mean values were calculated. Linearity was determined by comparing the observed TT4 concentrations following dilution with the expected (calculated) TT4 concentrations.⁹

Cross-sectional study of cats to examine agreement of Catalyst TT4 results with those measured by a veterinary reference laboratory

We conducted this prospective study over a 5 month period, from August 2015 to January 2016, and included 127 cats referred to the Animal Endocrine Clinic for treatment of hyperthyroidism with radioiodine. During the same period, we examined 30 cats that were re-evaluated 3–6 months after successful ¹³¹I treatment. In all 157 cats, our evaluation included a complete history and physical examination, routine laboratory testing (complete blood count and serum biochemical profile) and determination of serum TT4 concentration. In addition, all cats underwent thyroid scintigraphy,^10,11 which was used as the reference (gold) standard to classify cats as hyperthyroid or euthyroid.

Blood for TT4 determination was collected into glass serum tubes. Within 1 h of collection, blood samples were centrifuged, with serum then removed and divided into two aliquots. One aliquot was immediately used to measure each cat’s serum TT4 concentration using a Catalyst TT4 slide on a Catalyst One Chemistry Analyzer.¹² The second aliquot of each cat’s serum was stored at 4^°C, submitted to the veterinary reference laboratory (IDEXX Reference Laboratories, Totowa, NJ, USA), and analyzed on the same day; the laboratory was blinded to the Catalyst TT4 results.

Assays for TT4

The veterinary reference laboratory used a previously reported automated enzyme immunoassay (EIA) method to measure serum TT4 concentrations (DRI T4 assay; Microgenics).^13,14 This EIA TT4 assay was validated for use in cats and a reference interval (RI; 12–50 nmol/l) established (by use of data from 165 clinically normal cats, aged 7–19 years), as previously described.¹⁴ The assay was linear in the range of 6–150 nmol/l for feline serum, but linearity was poor with TT4 concentrations >150 nmol/l, with flattening of the slope and high variation at TT4 concentrations >150 nmol/l (upper limit of dynamic range of assay). Dilution of feline samples containing TT4 concentrations >150 nmol/l (Immulite 2000 Multi-Diluent 1; Siemens Healthcare Diagnostics) and subsequent re-assay was therefore routinely performed in order to provide results with acceptable accuracy at these higher TT4 concentrations. The accuracy of this method for serum dilution was confirmed by evaluating feline serum pools containing >150 nmol/l and comparing the serum TT4 concentrations with those measured by liquid chromatography–tandem mass spectrometry.

The Catalyst TT4 immunoassay was performed using the designated single-test TT4 slide on a Catalyst One Chemistry Analyzer, according to the operator’s guide.¹⁵ Dilution of serum samples was not required or performed for the Catalyst TT4 immunoassays.

Data and statistical analysis

Data were assessed for normality by the D’Agostino–Pearson test and by visual inspection of graphical plots.¹⁶ Because the data were not normally distributed, we used non-parametric statistical tests for all comparisons,¹⁷ which were performed using proprietary statistical software (GraphPad Prism, version 7.0 [GraphPad Software] and MedCalc, version 14.12.0 [MedCalc Software]). Results are reported as median (interquartile range [IQR] 25th–75th percentile) and are represented graphically as box plots. For all analyses, statistical significance was defined as P <0.05.

To determine the clinical utility of the Catalyst One Chemistry Analyzer, serum TT4 concentration from each cat was classified as ‘low’ (⩽11 nmol/l), ‘high’ (⩾51 nmol/l) or ‘within RI’ (12–50 nmol/l)¹⁴ for both methods. Each pair of measurements was then assessed as being ‘in agreement’ if both methods placed a cat into the same category or ‘not in agreement’ if they failed to place a cat into the same category.

Continuous variables were compared between groups by the Mann–Whitney U-test. Agreement was assessed by limits of agreement plots to compare overall agreement of the serum concentrations of TT4 measured by the reference laboratory and the in-house Catalyst One Chemistry Analyzer.^18–20

Results

Animals

The 127 cats with untreated hyperthyroidism ranged in age from 8–18 years (median 13 years; IQR 12–15 years). Breeds included 106 domestic longhair and shorthair cats, Maine Coon (n = 7), Siamese (n = 5), Russian Blue (n = 3), Persian (n = 2), Norwegian Forest Cat (n = 2) and one cat each of the Ragdoll and Burmese breeds. Of the cats with hyperthyroidism, 65 (51%) were male and 62 were female; all had been neutered.

The 30 ¹³¹I-treated cats ranged in age from 8–17 years (median 13 years; IQR 12.8–16 years). Breeds included 27 domestic longhair and shorthair cats, and one cat each of the Siamese, American Curl and Devon Rex breeds. Of these cats, 16 (53%) were male and 14 were female; all had been neutered.

Validation of TT4 concentrations measured by Catalyst TT4 immunoassay

The intra-assay CV of the Catalyst TT4 assay, calculated for three feline serum pools containing low to high concentrations of TT4, was 3.4%, 3.0% and 2.0%, respectively (mean 2.8%). The inter-assay CV, calculated for four feline serum pools containing low to high concentrations of TT4, was 5.7%, 6.1%, 5.9% and 8.5%, respectively (mean 6.6%). The Catalyst TT4 assay was linear from 0–150 nmol/l and within 9% from 150–250 nmol/l (r² = 0.99; Figure 1).

Figure 1

Linearity of the Catalyst Total T4 assay method for serum total thyroxine (TT4) concentration using pooled feline serum containing low to high TT4 concentrations. The line of equality is shown (dashed line), as well as the line of best fit (solid line). Each marker represents the mean of six measurements (two Catalyst One Chemistry Analyzers; three measurements each)

Comparison of TT4 concentrations reported by two methods

The serum TT4 concentrations measured by the veterinary reference laboratory and the Catalyst TT4 assay did not differ in all the 157 cats of this study (median 103 nmol/l, IQR 63–136 nmol/l, range 8–290 nmol/l vs median 106 nmol/l, IQR 64–147 nmol/l, range 10–238 nmol/l; P = 0.68).

When examined by limits of agreement plots, the differences between the two TT4 methods showed no proportional or fixed bias (Figure 2a). However, the data demonstrated marked heteroscedasticity (ie, scatter of values for differences increased progressively as the average values increased),²⁰ especially at higher average TT4 concentrations (Figure 2a). Subsequent analysis of normalized differences of the data (by use of a percent difference plot)^20,21 showed 95% limits of agreement of ± 30% (Figure 2b). When actual TT4 data from within or near the RI (0–60 nmol/l) were compared, the scatter of values for differences was much tighter (Figure 2c).

Figure 2

Limits of agreement plots for total thyroxine (TT4) concentrations measured by the in-clinic Catalyst One Chemistry Analyzer and the veterinary reference laboratory. (a) Actual data; (b) normalized differences (difference divided by average values); and (c) actual data for cats within or near the reference interval. In all plots, the thick dashed line represents the line of perfect agreement, the thin dashed lines represent the 95% limits of agreement and the thin solid line represents the regression line

When each cat’s serum TT4 concentration was classified as low (⩽11 nmol/l), high (⩾51 nmol/l) or within RI (12–50 nmol/l) for each of the assay methods, there was strong agreement (152/157 samples [96.8%]) based on the classification of results into these three groups (Table 1). Of the 157 cats, low, within RI and high TT4 concentrations were measured in five, 28 and 124 cats by the veterinary reference laboratory, and in four, 27 and 126 cats by the Catalyst TT4 assay (Table 1). Compared with the TT4 results obtained from the veterinary reference laboratory, the Catalyst TT4 results agreed in 4/5 cats with low values, 27/28 cats with RI values and 126 (rather than 124) cats with high values. Of the cats with confirmed hyperthyroidism, three cats would have been missed (normal TT4 values) with the EIA assay and only one missed with the Catalyst TT4 assay.

Table 1

Comparison of number of cats with low serum total thyroxine (TT4) concentrations, within reference interval (RI) TT4 concentrations and high TT4 concentrations as measured by the veterinary reference laboratory and the in-clinic Catalyst One Chemistry Analyzer

		Veterinary reference laboratory (EIA) TT4 assay
		Low TT4 concentration	TT4 within RI	High TT4 concentration
Catalyst	Low TT4 concentration	4	0	0
TT4 assay	TT4 within RI	1	25	1
	High TT4 concentration	0	3	123

EIA = enzyme immunoassay

Twelve of the 157 cats had measured TT4 concentrations near the low (n = 5) or high (n = 7) decision threshold values (10–15 nmol/l and 45–55 nmol/l, respectively; Figure 2c). All five of the cats with discrepant results between the veterinary reference laboratory and the Catalyst TT4 assay had TT4 concentrations near these decision threshold values. Of these five cats, one had TT4 concentrations near the low decision threshold and four had concentrations near the high decision threshold. In 3/4 cats with discrepant results near the high decision threshold, determination of free T4 concentration and thyroid scintigraphy confirmed that the Catalyst TT4 method provided results that most accurately classified the cats’ thyroid status.

The TT4 results disagreed markedly in only one cat (uppermost outlying data point on both Figures 1a and 1b). This cat, which was clinically hyperthyroid and had ‘hot’ bilateral nodules when imaged with thyroid scintigraphy (thyroid:salivary ratio, 3.3%; RI 0.5–1.5),^10,11 had a clearly high serum TT4 concentration (116 nmol/l) from the reference laboratory but a high–normal TT4 concentration (45 nmol/l) when measured in-house with the Catalyst TT4 assay.

Discussion

Our study demonstrates that the Catalyst TT4 assay performs well and provides valid diagnostic results in most hyperthyroid cats evaluated before and after treatment, based upon the clinical thyroid status of the cat and results of thyroid scintigraphy. However, the Catalyst TT4 concentrations do not agree perfectly with those measured by a veterinary reference laboratory. The degree of disagreement between test methods increases with higher TT4 concentrations and can result in a 30% difference in measured TT4 concentrations.

The two most common methodologies used for the measurement of TT4 today are a non-radioactive chemiluminescent EIA or a homogeneous EIA that can be run on automated biochemistry analyzers, both of which require submitting samples to reference laboratories.^1,13,22 Therefore, an inexpensive, in-clinic analyzer could provide veterinary clinicians with a more rapid diagnosis. In our hands, the Catalyst TT4 assay demonstrated excellent analytical precision and reproducibility at low, median and high serum concentrations of TT4 in feline serum. The critical decision limit for the diagnosis of hyperthyroidism is a serum TT4 concentration of between 40 and 60 nmol/l, and at these concentrations the assay performance was excellent. Linearity of the assay was also demonstrated with feline serum, especially in the critical diagnostic range and up to 150 nmol/l.

However, assay validation generally requires comparison of a new method against a reference standard. Our study shows that the Catalyst TT4 concentrations do not agree completely with those measured by a veterinary reference laboratory, and that this imprecision increases with increasing TT4 concentrations (a phenomenon known as heteroscedasticity, visualized in Figure 2a as a fan-shaped distribution of data). When differences between the methods were normalized to the mean measurements (Figure 2b), we observed a difference of approximately 30% across the range of TT4 concentrations. At least part of this poor agreement with high values may be due to issues with the TT4 EIA assay used at the veterinary reference laboratory rather than problems with the Catalyst TT4 assay itself. In feline sera, the EIA assay is linear up to TT4 concentrations of approximately 150 nmol/l, but the assay is unable to detect accurately concentrations higher than 160 nmol/l. In order to correct for this lack of linearity at higher TT4 concentrations with the EIA assay, feline serum samples that provided TT4 results ⩾150 nmol/l were routinely diluted and re-assayed in order to more reliably provide a more accurate result. It is certainly possible that inaccuracy introduced by pipetting of the sample or diluent negatively influenced the final results,^23,24 as the magnitude of such an error would be multiplied by the dilution factor when determining the final TT4 concentration. In any case, marked disagreement in measured TT4 concentrations (>20 nmol/l) between the veterinary reference laboratory and the Catalyst TT4 assay only occurred at high concentrations (>100 nmol/l), well above the threshold value needed to diagnose hyperthyroidism.

Because a high degree of imprecision between assay methods might be problematic (a difference of approximately 30% across the range of TT4 concentrations is not insignificant), we examined the likelihood of misclassifying cats with the Catalyst TT4 assay, by categorizing the cats (based on their reference laboratory TT4 concentrations) as hypothyroid, euthyroid and hyperthyroid. We found very strong agreement (96.8%) in classification between the two methods in our sample population. In most cats, the TT4 concentrations near low or high threshold values that did not agree were not dramatically different – the difference was small enough in absolute terms not to misclassify the cat’s thyroid status. However, in 3/4 untreated hyperthyroid cats that showed discrepant results near the high decision threshold (euthyroid vs hyperthyroid), the Catalyst TT4 method, rather than the reference laboratory method, provided results that most accurately classified the cats’ thyroid status, when compared with the scintigraphic diagnosis. Of all of the comparisons, TT4 concentration varied markedly in only one hyperthyroid cat that had a clearly high serum TT4 concentration (116 nmol/l) when measured by the reference laboratory, but a high-to-normal TT4 concentration (45 nmol/l) when measured by the Catalyst TT4 assay, which would have resulted in a misdiagnosis if a clinician relied on the Catalyst TT4 assay. The cause of this discrepancy between the two assays was not apparent.

Only 12 of our 157 cats had ‘borderline’ TT4 concentrations; that is, near the upper and lower decision threshold values (10–15 nmol/l and 45–55 nmol/l, respectively). Given the imprecision that we identified between testing methods, we would expect clinical misclassification of cats to be greatest at these thresholds, where a 30% difference in measured TT4 concentrations could easily misclassify a cat’s thyroid status. Indeed, 4/5 of our misclassified cats had reference laboratory TT4 concentrations within 5 nmol/l of a decision threshold. Interestingly, in all four of these cats (one with lower and three with upper decision threshold values), the TT4 concentration measured by the Catalyst One Chemistry Analyzer was higher than that measured by the reference laboratory. In 3/4 untreated hyperthyroid cats that showed discrepant results near the high decision threshold, the Catalyst TT4 results most accurately classified their correct thyroid status. Based on the potential difference in measured TT4 concentrations between testing methods, clinicians using the Catalyst One Chemistry Analyzer should verify TT4 results with an additional thyroid testing (eg, free T4 concentration) if they fall within 30% of the disease threshold (eg, ± 15 nmol/l of the 50 nmol/l threshold). Results outside of this range are unlikely to misclassify the thyroid status of the cat.

There are a number of limitations to this study. First of all, as we looked at only hyperthyroid cats before and after treatment in this study, we did not investigate the effect of other disease states on the Catalyst TT4 assay. Furthermore, this assay method needs further analysis with higher numbers of euthyroid or hypothyroid cats of various ages, cats with low and high serum protein concentrations, hemolysis or icterus to more broadly assess the test performance across the entire feline population. We chose to compare the Catalyst TT4 assay with the EIA assay, a method that might have as-yet undetermined limitations (ie, more studies are needed to evaluate the influence of protein concentrations, pH and storage on T4 results derived by EIA). Comparing the Catalyst T4 method with another T4 method, such as chemiluminescent EIA or radioimmunoassay, should be carried out (these studies are in progress), as other methods have been used more widely in veterinary medicine.²²

Conclusions

Our results show that the Catalyst TT4 assay provides relatively accurate and reliable serum TT4 concentrations over a wide, clinically useful range in hyperthyroid cats, providing in-clinic results equivalent to those obtained by a veterinary reference laboratory. Cats with Catalyst TT4 concentrations near decision thresholds (eg, normal vs high) should have a follow-up history and physical examination carried out a few weeks later, with either the TT4 concentration repeated and/or further testing (eg, free T4, serum TSH, thyroid scintigraphy) performed to determine thyroid status.

Footnotes

Conflict of interest

GE Bilbrough and K Cote are employees of IDEXX Laboratories, Westbrook, ME, USA. Neither of the other authors (ME Peterson or M Rishniw) have any potential conflicts of interest to declare.

Funding

IDEXX provided the Catalyst One Chemistry Analyzer to the Animal Endocrine Clinic for the 5 month period of this study, provided the Catalyst TT4 slides for the in-clinic testing and measured the serum total thyroxine concentrations (by EIA) at the IDEXX reference laboratory.

Accepted: 12 April 2017

References

Kemppainen

Birchfield

. Measurement of total thyroxine concentration in serum from dogs and cats by use of various methods. Am J Vet Res 2006; 67: 259–265.

Lien

Huang

. Evaluation of a point-of-care enzyme-linked immunosorbent assay for determination of basal serum total thyroxine concentration in cats. J Vet Clin Sci 2008; 1: 86–89.

Higgs

Costa

Freke

, et al. Measurement of thyroxine and cortisol in canine and feline blood samples using two immunoassay analysers. J Small Anim Pract 2014; 55: 153–159.

Lurye

Behrend

Kemppainen

. Evaluation of an in-house enzyme-linked immunosorbent assay for quantitative measurement of serum total thyroxine concentration in dogs and cats. J Am Vet Med Assoc 2002; 221: 243–249.

McEnroe

Durham

Goldford

, et al. Evaluation of precision of quantitative measurement procedures; approved guideline. In: CLSI document EP05-A3. 3rd ed. Wayne, PA: Clinical and Laboratory Standards Institute, 2014.

Refsal

Nachreiner

. Hormone assays and collection of samples. In: Mooney

Peterson

(eds). BSAVA manual of canine and feline endocrinology. 4th ed. Quedgeley: British Small Animal Veterinary Association, 2012, pp 1–7.

Tiwari

. Bioanalytical method validation: an updated review. Pharm Methods 2010; 1: 25–38.

Tholen

Kroll

Astles

, et al. Evaluation of the linearity of quantitative measurement procedures: a statistical approach; approved guideline. Wayne, PA: Clinical and Laboratory Standards Institute, 2003.

Jhang

Chang

Fink

, et al. Evaluation of linearity in the clinical laboratory. Arch Pathol Lab Med 2004; 128: 44–48.

10.

Peterson

Broome

. Thyroid scintigraphy findings in 2,096 cats with hyperthyroidism. Vet Radiol Ultrasound 2015; 56: 84–95.

11.

Peterson

Guterl

Rishniw

, et al. Evaluation of quantitative thyroid scintigraphy for diagnosis and staging of disease severity in cats with hyperthyroidism: comparison of the percent thyroidal uptake of pertechnetate to the thyroid-to-salivary ratio and thyroid-to-background ratios. Vet Radiol Ultrasound 2016; 57: 427–440.

12.

IDEXX Laboratories. Catalyst One Veterinary Chemistry Analyzer. https://www.idexx.com/small-animal-health/products-and-services/catalyst-one-analyzer.html (accessed March 2, 2017).

13.

Williams

Archer

. Validation of an automated enzyme immunoassay for the measurement of serum total thyroxine in cats. Vet Clin Pathol 2016; 45: 148–153.

14.

Lucy

Peterson

Randolph

, et al. Efficacy of low-dose (2 millicurie) versus standard-dose (4 millicurie) radioiodine treatment for cats with mild-to-moderate hyperthyroidism. J Vet Intern Med 2017; 31: 326–334.

15.

IDEXX Laboratories. Idexx Catalyst One Chemistry Analyzer. Operator’s guide. https://www.idexx.com/resource-library/smallanimal/catalyst-one-operators-guide-en.pdf (accessed March 2, 2017).

16.

D’Agostino

. Tests for normal distribution. In: D’Agostino

Stephens

(eds). Goodness-of-fit techniques. New York: Marcel Dekker, 1986, pp 367–420.

17.

Conover

. Practical nonparametric statistics. 3rd ed. New York: Wiley, 1999.

18.

Bland

Altman

. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986; 1: 307–310.

19.

Bland

Altman

. Measuring agreement in method comparison studies. Stat Methods Med Res 1999; 8: 135–160.

20.

Ludbrook

. Confidence in Altman–Bland plots: a critical review of the method of differences. Clin Exp Pharmacol Physiol 2010; 37: 143–149.

21.

Dewitte

Fierens

Stockl

, et al. Application of the Bland-Altman plot for interpretation of method-comparison studies: a critical investigation of its practice. Clin Chem 2002; 48: 799–801.

22.

Peterson

. More than just T4: diagnostic testing for hyperthyroidism in cats. J Feline Med Surg 2013; 15: 765–777.

23.

Rodrigues

Curtis

. Instrument performance verification: micropipettes. In: Chan

Lam

Zhang

(eds). Practical approaches to method validation and essential instrument qualification. New York, NY: John Wiley & Sons, 2010, pp 327–346.

24.

Bertermann

. Pipet quality control: a microliter of prevention. Am Biotechnol Lab 2004; 22: 18–25.