Evaluation of two commercial enzyme-linked immunosorbent assay kits for the detection of serum antibodies against Akabane virus in cattle

Abstract

In New Zealand, an arbovirus surveillance program has been operating for more than 20 years, which includes testing of cattle with the Akabane virus neutralization test. With the aim to replace this laborious test by an easier-to-perform enzyme-linked immunosorbent assay (ELISA), 2 commercial ELISA kits, ELISA-1 from France (originally from Australia) and ELISA-2 from Japan, were compared, using 334 serum samples from noninfected New Zealand cattle, and 548 serum samples from naturally infected cattle herds in Australia. Diagnostic specificities for the test methods were high, ranging from 99.4% to 100%. The diagnostic sensitivities varied considerably between the test methods and differed from the values reported by the manufacturers (94% for each ELISA). The diagnostic sensitivities relative to the virus neutralization test (n = 378) were 96.0% for ELISA-1 or 98.9% when suspect samples were included, and 78.0% for ELISA-2. Differences in the commercial ELISA kits may be explained by the presence of other Simbu serogroup viruses in Australian cattle herds, causing cross-reactions in ELISA-1. Both commercial ELISA kits would be fit for purpose and could replace the virus neutralization test for Akabane virus surveillance in New Zealand. ELISA-1 may be able to detect other Simbu serogroup viruses, should they be present. The current study shows that despite comparable ELISA test characteristics given by the manufacturers, evaluation on the target population revealed marked differences in the ELISA kits test methods’ characteristics.

The New Zealand livestock population is free from Akabane virus (AKAV; family Bunyaviridae, genus Orthobunyavirus), an arthropod-borne (arbo) virus,^5,8,9 and the country is also free from the transmitting vector, biting midges of the genus Culicoides.^10,13 However, with the vector range and virus distribution in neighboring Australia and the evident global warming, incursion of vectors and viruses are a possibility. Therefore, an arbovirus surveillance program that includes AKAV surveillance has been operating in New Zealand since 1990, ¹¹ which includes insect trapping and blood sampling from 17 sentinel cattle herds and testing with the AKAV neutralization test (VNT); to date with negative results. ¹⁰

Laboratory diagnosis of Akabane disease is generally based on serology. The VNT is considered the reference method (gold standard) but is rather laborious and time consuming, requiring live virus and cell culture (Della-Porta AJ, White JR, Gard GP, Kirkland PD: 1993, Akabane disease—histopathology, virology and serology. Available at: http://www.scahls.org.au/__data/assets/pdf_file/0007/1280815/akabane_diseases.pdf). Commercial enzyme-linked immunosorbent assay (ELISA) kits are available, are easier to perform, and could be a good alternative to the VNT. The current study reports the comparative evaluation of 2 commercial ELISA kits on cattle serum samples from noninfected New Zealand herds, and infected Australian herds, with the aim of identifying an ELISA kit that would be best suited for AKAV surveillance on the New Zealand target population. ⁶

In the current study, the VNT was performed according to the Australian standard diagnostic procedure. ⁴ Samples with titers <1/4 were considered negative and titers ≥1/4 as positive. ELISA-1, ^a a commercial blocking ELISA with specificity for the viral glycoprotein Gc (formerly G1; Lunt RA, White JR, Newberry KM, et al.: 2007, Development of a commercial ELISA for the detection of serum antibody to Akabane virus. In: 13th international symposium proceedings, World Association of Veterinary Laboratory Diagnosticians, pp. 11–14, Melbourne, Australia. Available at: http://www.csiro.au/files/files/pm04.pdf), and ELISA-2, ^b another commercial blocking ELISA, were performed according to the manufacturers’ instructions. For both ELISAs, percentage inhibition (%I) values were calculated with the following cutoff values: ELISA-1: %I >70 positive, ≤60 negative, and 60–70 suspect; ELISA-2: %I >40 positive and ≤40 negative, with no suspect range.

The cattle numbers to be tested were based on previously published recommendations ⁶ and known diagnostic sensitivity (DSE) and diagnostic specificity (DSP) values, which were 94% and 98%, respectively, for ELISA-1 and for ELISA-2 (manufacturer’s information). Subsequently, a total of 334 serum samples from noninfected cattle from New Zealand were used as negative population from the sentinel cattle herds used for arbovirus surveillance. ¹⁰ The gold standard for absence of infection was the lack of detectable AKAV VNT antibody in New Zealand. A total of 548 cattle serum samples were collected from 3 herds in a region of Australia where AKAV is recognized to be endemic and were submitted to the Elizabeth Macarthur Agriculture Institute (Camden, New South Wales, Australia). Because Akabane disease is not present in New Zealand, and Australia is the most likely country from where the virus could spread, this population was considered the closest to the New Zealand target population. Of the 548 animals, a total of 378 had tested positive in the VNT at a cutoff of 1/4 and were considered the infected reference panel.

Frequency histograms for noninfected (panel 1) and infected (panel 2) cattle were generated for each ELISA to display the distribution of %I results in both classes of cattle with the aid of a spreadsheet program. Statistical analyses were undertaken using Epitools (Sergeant ESG: 2009, Epitools epidemiological calculators. AusVet Animal Health Services and Australian Biosecurity Cooperative Research Centre for Emerging Infectious Disease. Available at: http://epitools.ausvet.com.au). Diagnostic sensitivity and DSP for each test method were calculated by using the Epitool “test evaluation against a gold standard.” Receiver operating characteristic (ROC) analysis was performed over a range of possible cutoff points, based on ELISA %I values for the noninfected New Zealand cattle (panel 1) and the infected Australian cattle (panel 2) by using the Epitool “calculate test sensitivity and specificity and ROC curves” (Sergeant ESG: 2009).

The DSP was high for each of the 3 serological test methods evaluated. The DSP for the VNT performed at Investigation and Diagnostic Centre, Animal Health Laboratory, Upper Hutt, New Zealand, was 99.4%. Two samples had shown titers close to the cutoff of 1/4 (i.e., titers of 1/6 [intermediate titer of duplicates] and 1/4, respectively). These samples tested negative in both commercial ELISA kits and in a confirmatory VNT performed at the Australian Animal Health Laboratory (Geelong, Victoria, Australia). ELISA-1 possessed a DSP of 99.7%, as 1 sample tested with a suspect result (%I = 60.7). This sample was negative in the VNT and in ELISA-2. The DSP of ELISA-2 was 100%, with all 334 samples from the noninfected serum panel testing with negative results.

Test results for the VNT and the commercial ELISA kits for the infected herds’ serum panel were diverse. With the VNT as reference method (n = 378), the DSE for ELISA-1 was high: 96.0% for positive results and 98.9% when positive and suspect results were counted, which was comparable to the manufacturer’s claim of 94%. For ELISA-2, the DSE was much lower than expected at 78.0% compared to 94%. Furthermore, in the 548 cattle from naturally infected herds, ELISA-1 appeared to detect many more animals as positive or suspect (90.1%) than the VNT (79.0%). ELISA-2 detected 56.2% as positive. It was speculated that the higher number of positive and suspicious results in ELISA-1 may have been caused by antibodies against other Simbu genogroup viruses that are circulating in the Australian cattle population and that are cross-reacting in this ELISA.

In Figure 1, frequency histograms are shown for the 2 commercial ELISAs for infected (VNT-positive) and noninfected cattle. ELISA-1 presents a typical histogram for a well-performing test method where the infected and noninfected populations are fairly well separated. ELISA-2 shows a major peak in the positive %I range and, in addition, a shallow peak in the negative %I range. The noninfected population is clustered around low %I values. Again, these results could be interpreted with the presence of cattle with antibodies other than for AKAV, which are detected by ELISA-1 as positives, and which form the shallow peak in ELISA-2 in the negative range. It is known that even the VNT can suffer from cross-reactivity with other Simbu genogroup viruses because of the antigenic relatedness of these viruses ⁷ and the occurrence of reassorted Simbu genogroup viruses in nature. ¹²

Figure 1.

Frequency histograms and receiver operating characteristic (ROC) curves for 2 commercial blocking Akabane virus enzyme-linked immunosorbent assays (ELISA-1 ^a and ELISA-2 ^b ) for virus neutralization test–positive cattle from Australian Akabane virus naturally infected herds (n = 378) and noninfected New Zealand cattle (n = 334).

The ROC analysis results for the commercial ELISAs are also shown in Figure 1. ELISA-1 presents a curve that is typical for a well-performing test method, where the curve climbs rapidly toward the upper left hand corner of the graph. This can be interpreted that the true positive rate is high and the false negative rate is low. ELISA-2 shows a less favorable curve. The slow rise indicates a problem with sensitivity on the population investigated.

Another possible explanation for the differences in the commercial ELISA results may lie in the different AKAV strains that were used for making the antigen for each ELISA. ELISA-1 uses antigen prepared from the Australian virus strain B8935; ELISA-2 uses antigen prepared from the Japanese virus strain OBE-1. A publication reporting on a dot immunobinding assay showed different patterns for the Australian virus strain B8935 (pattern 2) and the Japanese virus strain OBE-1 (pattern 1), ¹⁴ thus confirming antigenic strain differences. It has been shown by others that antigenic diversity exists among AKAV isolates.^1-3 Furthermore, phylogenetically, both strains are quite distant, with B8935 belonging to genogroup III and OBE-1 to genogroup II. ⁹ Perhaps OBE-1 lacks epitopes that are present in AKAV strains circulating in Australia or the monoclonal antibody 19B-4 in ELISA-2 does not block serum antibodies against certain epitopes present on Australian virus strains. For both assumptions, animals with antibody responses to such epitopes would not be detected by ELISA-2. Further investigations would be necessary to answer these questions.

In conclusion, the current study has highlighted the importance of evaluating test methods for the target population for which they will be used. Even though evaluation data existed for other populations, and DSP values were comparable, DSE values for the various populations turned out to be quite different. In regard to fitness for purpose for the New Zealand arbovirus surveillance program, both commercial ELISA kits could be used. In view of the higher sensitivity of ELISA-1, this ELISA should be the preferred test method. As it is more likely that false-positive results will occur with ELISA-1, such results could be clarified by using the VNT as a confirmatory test method.