Evaluation of Four Commercial Enzyme-Linked Immunosorbent Assays for the Diagnosis of Bovine Paratuberculosis in Chilean Dairy Herds

Abstract

The accuracy of 4 commercial enzyme-linked immunosorbent assays (ELISAs) for diagnosis of bovine paratuberculosis was compared using sera from 53 Mycobacterium avium subsp. paratuberculosis (MAP) fecal culture–positive dairy cows (cases) and sera from 345 dairy cattle resident in 11 fecal culture–negative herds on 2 consecutive occasions 1 year apart (controls). The specificity of all 4 ELISA kits was >99%, and their diagnostic sensitivity ranged from 30.2% to 41.5%. Pairwise comparison of ELISAs found no significant differences (McNemar's chi-square test > 0.05), and assay agreement for categorical assay interpretation (positive or negative) was high (>98%) with κ values ranging from 0.84 to 0.95. Receiver operating characteristic (ROC) curve analysis and the corresponding area under the ROC curves indicate that kit B had the highest overall accuracy. Thus, all 4 ELISA kits for bovine paratuberculosis had comparable accuracy when tested on Chilean dairy cattle, with kit B having a slight statistical advantage based on ROC area under the curve analysis. This suggests that any of the 4 kits could be appropriate for herd certification and for paratuberculosis control programs on Chilean dairy cattle.

Johne's disease, or paratuberculosis, is a chronic, infectious disease caused by Mycobacterium avium subsp. paratuberculosis (MAP). The disease occurs worldwide, affecting many domestic and wild animal species, especially ruminants. Clinical paratuberculosis is characterized by chronic granulomatous enteritis with clinical signs of diarrhea, weight loss, decreased milk production, and mortality; however, most infected cattle show no clinical signs during the prolonged incubation stage of infection. Although cattle generally become infected as young calves, fecal shedding of MAP starts after 2 years and clinical symptoms appear after an incubation period of 2–10 years. 20 The economic impact and possible link to Crohn's disease have motivated development of control programs at the herd, regional, and national levels. 3,4,8,11 These programs are based on avoiding or limiting cow–calf transmission by changes in herd management and identifying and removing infected, or at least the most infectious, adult cattle. Diagnosis of MAP infection in these animals can be made by methods based on either detection of the organism in fecal samples or detection of antibodies in serum or milk samples. 2,11 Fecal culture for MAP is considered the most commonly used reference test for evaluation of serological tests because it is widely available and the most sensitive and specific test for paratuberculosis. However, it is expensive and slow, requiring 8–16 weeks for completion of testing. 2 Among tests for serum antibody to MAP, enzyme-linked immunosorbent assay (ELISA)–based methods are the most widely used. Several commercial ELISA kits for bovine paratuberculosis are currently available, and multiple studies have compared their accuracy. 5,12,14,19 The purpose of the present study was to evaluate the accuracy of 4 commercial ELISA kits using bovine sera from Chilean dairy cattle herds.

Serum samples from lactating cows >2 years old from dairy herds in southern Chile collected during 2003–2005 for a previous paratuberculosis test validation study were used for test comparison (Kruze J: 2003, Validación de una prueba de ELISA para detectar infección por Mycobacterium avium subsp paratuberculosis en rebaños lecheros de la Décima Región, Chile [Validation of a commercial ELISA kit to detect Mycobacterium avium subsp. paratuberculosis infection in dairy herds of the Tenth Region, Chile]; Fondo de Mejoramiento del Patrimonio Sanitario, Servicio Agrícola y Ganadero, Ministerio de Agricultura, Chile; Proyecto C3–70–10–21). Fifty-three cows from 9 herds were fecal culture–positive, and the remaining 345 cows from 11 dairy herds were fecal culture–negative on 2 consecutive occasions 2 years apart. Fecal samples were collected from the rectum of each animal using individual, disposable, polyethylene sleeves, and cultured on homemade Herrold egg yolk (HEY) medium with mycobactin J a , following the processing protocol developed at Cornell University. 18 Decontamination before culture with hexadecylpyridinium chloride (HPC) b and an antibiotic brew (nalidixic acid, vancomycin, amphotericin B) b was carried out following the procedure recommended by the National Animal Disease Center as described elsewhere. 18 An aliquot (0.15 ml) of the final suspension of each fecal sample was used to inoculate 3 HEY tubes with mycobactin and 1 HEY tube without mycobactin. All tubes were incubated at 37°C for up to 16 weeks and examined weekly for bacterial growth. Colonies resembling MAP and showing mycobactin-dependence were tested by IS900 polymerase chain reaction (PCR). 10 Blood samples were collected in 10-ml Vacutainer Tubes c from the coccygeal vein, and the sera was harvested and frozen at −80°C until retested for ELISA kit comparison. All sera were tested in duplicate wells using 4 commercial ELISA kits designated as kits A, d B, e C, f and D, g and performed according to manufacturer's recommendations. All 4 kits have a preabsorption step with Mycobacterium phlei antigens to enhance assay specificity by limiting cross-reactions with mycobacteria other than MAP. Cutoff values recommended for each kit were used according to kit instructions. Positive and negative control sera provided by the manufacturers were included on each plate. Test results were interpreted as negative or positive. Results classified as suspect or inconclusive were considered negative. Assays with a between-well coefficient of variation of ≥10% were repeated, and the second result was used for data analysis. The case definition for a MAP-infected herd was any herd with 1 or more fecal culture–positive cows. A noninfected herd was a herd in which all cattle were fecal culture–negative on 2 whole-herd tests at least 1 year apart. Sensitivity was defined as the percentage of fecal culture–positive cows testing ELISA-positive. Specificity was defined as the percentage of non-MAP–infected cows testing ELISA-negative. A z-statistic for comparison of independent proportions was used to determine sensitivity and specificity differences among ELISA kits. McNemar's chi-square analysis and κ calculation were used to evaluate agreement between pairs of assays. 13 Differences were considered significant at P < 0.05. Receiver operating characteristic (ROC) curves were constructed by plotting the true positive rate (sensitivity) versus 1-specificity (false-positive rate) using GraphPad Prism version 4.00 for Windows. h The area under the ROC curve (AUC) ranges between 0.5 and 1.0, where an AUC of 1.0 represents a test with perfect accuracy (sensitivity = 100%, and specificity = 100%). Low, moderate, and high overall assay accuracy was defined as AUCs of 0.5–0.7, 0.7–0.9, and 0.9–1.0, repectively. 10

Figure 1.

Receiver operating characteristic curves for 4 commercial enzyme-linked immunosorbent assay kits for the diagnosis of bovine paratuberculosis in Chilean dairy herds. A, kit A (area under the curve [AUC] 0.899); B, kit B (AUC 0.944); C, kit C (AUC 0.853); D, kit D (AUC 0.815).

Sensitivity and specificity of kits A, B, and D were evaluated based on sera from 53 cases and 345 controls. Kit C was evaluated using all 53 cases but only 299 controls because of failure of positive controls on 2 ELISA plates in 1 kit. Diagnostic sensitivities of the 4 ELISAs ranged from 30.2% (95% confidence interval [CI]: 18.3–44.3) for kit A to 41.5% (95% CI: 28.1–55.9) for kit B and were not significantly different (Table 1). Specificities ranged from 99% (95% CI: 97.10–99.79) for kit C to 99.71% (95% CI: 98.40–99.99) for kit A and were not significantly different (Table 1). Assay agreement was done by pairwise analysis of results among results for all 4 ELISA kits. More than 98% of all assays were in agreement with κ values ranging between 0.84 and 0.95. The lowest agreement (98.29%) was between kits A and C, and the highest agreement (99.43%) was between kits B and C (κ value: 0.95). No significant differences among kits were found by McNemar's chisquare analysis (Table 2). Receiver operating characteristic curves for all 4 ELISA kits were plotted, and AUC was calculated (Fig. 1). The AUCs were 0.899 (95% CI: 0.864–0.935), 0.944 (95% CI: 0.915–0.972), 0.853 (95% CI: 0.790–0.916), and 0.815 (95% CI: 0.743–0.887) for kits A, B, C, and D, respectively.

Paratuberculosis control by testing, culling, and herd management are important in limiting the economic impact of MAP infections on dairy herds. 3,4 Serological tests, in particular ELISA-based methods, are an important component of paratuberculosis programs because of their low-cost, quantitative results, which are directly associated with MAP fecal-shedding rates, and rapidity of results. 5 ELISA accuracy is usually reported in reference to fecal culture. 6,15 The gold standard for definition of cattle not infected with MAP was based on multiple whole-herd fecal cultures spaced at least 1 year apart. 5 In the present study, noninfected herds were selected among small-holder farms without previous history of clinical disease based on fecal culture–negative status on 2 consecutive samplings 1 year apart. The diagnostic sensitivity of the 4 paratuberculosis ELISAs ranged from 30.2% to 41.5% (Table 1), with no significant differences among kits. Prior studies where ELISA sensitivity was evaluated found results ranging from 15.4% in cattle classified as light MAP shedders to 80% for heavy shedders. 6,7 These varied ranges can be explained by the strong dependence of the sensitivity with the stage of MAP infection in the sampled animals. 7 In general, most studies report that roughly one-third of fecal culture–positive cattle can be detected by ELISA, consistent with the findings of the present study. 5 Other factors affecting test sensitivity estimation are the reference test and case definition (single or multiple cultures on feces or tissues) and technical aspects of the assays (e.g., antigen concentration and duration for preabsorption of test sera with M. phlei and ELISA conjugate). 1,5,6,15,21 The specificity of all 4 ELISA kits was high (>99%), and there was no significant difference in specificity among the kits. These high values are corroborated by previous studies that reported specificities of 99% 19 and 99.8% 12 for kit A. Kit C had the lowest specificity (99%). Another study reported a specificity of 100% for the same kit. 5 False-positive ELISA results have been associated with recovery of environmental mycobacteria from bovine feces. 16 That study also suggested that the potential for natural exposure of cattle to Mycobacterium spp. may be associated with geographic differences in exposure risk. 16,17 It is essential to estimate sensitivity and specificity for diagnostic tests, but it is equally important to assess test agreement. Pairwise comparison of ELISA results showed a high rate of agreement (>98%) and high κ values (>0.84; Table 2). The ROC curve “depicts the trade-off between correctly identifying diseased animals and falsely identifying non-diseased animals for a given test, for all possible values of the test result.” 9 Such curves are used universally for test comparisons even when the tests have different cutoff values or different units of measurement. The AUC is a measure of overall test performance with a perfect test having an AUC of 1.0, and a nondiscriminating test an area of 0.5. 9 In the present study, kit B had the highest overall accuracy (AUC = 0.944). The other 3 kits had a moderate accuracy with AUC values of 0.899, 0.853, and 0.815 for kits A, C, and D, respectively.

Table 1.

Sensitivity and specificity of 4 commercial enzyme-linked immunosorbent assay (ELISA) kits to detect antibody against Mycobacterium avium subsp. paratuberculosis infection in dairy herds.

ELISA kit	Sensitivity*	95% confidence interval	Specificity*	95% confidence interval
A	30.2	18.3–44.3	99.71	98.40–99.99
B	41.5	28.1–55.9	99.42	97.92–99.93
C	37.7	24.8–52.1	99.00	97.10–99.79
D	37.7	24.7–52.1	99.42	97.92–99.43

*

Lack of superscript letters indicates no significant differences.

Based on the results of the present study on Chilean dairy cattle, it was concluded that there were no significant differences among the 4 commercial ELISA kits for bovine paratuberculosis. This suggests that any of the 4 kits could be appropriate for herd certification and for paratuberculosis control programs on Chilean dairy cattle. Other important considerations in kit selection include, but are not limited to, kit versatility (e.g., ability to test for antibodies in serum and milk), cost, availability, and frequency of use in other laboratories in the same country or around the world.

Acknowledgements. Grant research supported by the National Commission for Scientific and Technological Research/Chile (FONDECYT), Project 1050639/7060015.

Table 2.

Pairwise comparison of 4 commercial enzyme-linked immunosorbent assay (ELISA) kits for the diagnosis of bovine paratuberculosis in dairy herds in agreement and κ value.

ELISA kits	No. of samples	Percent agreement	κ value
A vs. B	398	98.49	0.84
A vs. C	352	98.29	0.84
A vs. D	398	98.49	0.84
B vs. C	352	99.43	0.95
B vs. D	398	98.99	0.91
C vs. D	352	98.86	0.91