Abstract
Recently, screening tests for monitoring the prevalence of transmissible spongiform encephalopathies specifically in sheep and goats became available. Although most countries require comprehensive test validation prior to approval, little is known about their performance under normal operating conditions. Switzerland was one of the first countries to implement 2 of these tests, an enzymelinked immunosorbent assay (ELISA) and a Western blot, in a 1-year active surveillance program. Slaughtered animals (n = 32,777) were analyzed in either of the 2 tests with immunohistochemistry for confirmation of initial reactive results, and fallen stock samples (n = 3,193) were subjected to both screening tests and immunohistochemistry in parallel. Initial reactive and false-positive rates were recorded over time. Both tests revealed an excellent diagnostic specificity (> 99.5%). However, initial reactive rates were elevated at the beginning of the program but dropped to levels below 1% with routine and enhanced staff training. Only those in the ELISA increased again in the second half of the program and correlated with the degree of tissue autolysis in the fallen stock samples. It is noteworthy that the Western blot missed 1 of the 3 atypical scrapie cases in the fallen stock, indicating potential differences in the diagnostic sensitivities between the 2 screening tests. However, an estimation of the diagnostic sensitivity for both tests on field samples remained difficult due to the low disease prevalence. Taken together, these results highlight the importance of staff training, sample quality, and interlaboratory comparison trials when such screening tests are implemented in the field.
Three types of transmissible spongiform encephalopathy (TSE) occur in small ruminants (sheep and goats): classical scrapie, atypical scrapie, and infections with bovine spongiform encephalopathy (BSE) agent. 5,6 During the BSE epidemic in cattle, an etiological link to the scrapie agent as a potential origin of BSE had been hypothesized. 9 Moreover, it was later shown that BSE transmitted naturally between sheep within an experimental flock. 1 Small ruminants were likely exposed to BSE-contaminated meat-and-bone meal, 4 and, thus, concerns were raised that the BSE agent might circulate within the small ruminant population unrecognized (indistinguishable form scrapie), thereby acting as a reservoir for the agent and presenting a zoonotic risk. Consequently, active surveillance programs were implemented in many countries to monitor TSEs in small ruminants in addition to BSE in cattle. 2
For a precise estimation of the disease prevalence and timely reporting of the test results, reliable, highly sensitive, specific, and rapid screening tests are required. The diagnostic performance of the tests is usually assessed on the basis of validation dossiers provided by the test producer and/or on independent laboratory evaluations prior to approval by the authorities. However, such evaluations often run under ideal laboratory conditions and do not necessarily reflect the day-to-day performance in routine testing laboratories in the field. To overcome these limitations, extensive field trials are now required for the evaluation of cattle BSE screening tests in the European Union (European Food Safety Agency: 2004, Scientific report on the evaluation of seven new rapid post mortem BSE tests. EFSA J 18:1–13). To the authors' knowledge, this aspect has not been addressed by any of the small ruminant TSE screening test evaluations conducted so far (European Food Safety Agency: 2005, Evaluation of rapid post mortem TSE tests intended for small ruminants. EFSA J 31:1–17). The present communication intends to partially fill this gap and reports on field experiences with 2 commercial small ruminant TSE screening tests in the first year after their implementation in a comprehensive, active small ruminant TSE surveillance program in Switzerland.
For the approval of screening tests, companies were asked to submit test performance data to the authorities covering 1) diagnostic specificity, 2) diagnostic sensitivity for classical scrapie, 3) capacity to detect atypical scrapie and small ruminant BSE, and 4) repeatability to the authorities. After an assessment of the data and a limited laboratory evaluation in the Swiss national reference laboratory (NeuroCentre, Vetsuisse Faculty, University of Berne), 2 tests were approved for small ruminant TSE screening: the TeSeE sheep and goat enzyme-linked immunosorbent assay (ELISA) a and the Check Western SR b Western blot (WB). Both are run on brain tissue (i.e., postmortem) to indentify the pathologic prion protein (PrPsc) in the central nervous system (CNS).
Screening test performances in fallen and slaughtered small ruminants in a 1-year active transmissible spongiform encephalopathy surveillance program in Switzerland, 2004–2005.
Percentages in relation to the number of samples tested in the respective test format are indicated in parentheses if >0.1%.
Fallen stock samples were analyzed in both screening tests in parallel.
Slaughterhouse samples were analyzed on either of the 2 screening tests.
Between July 2004 and July 2005, 2 types of surveys, a slaughterhouse survey and a fallen stock survey, were conducted in adult (i.e., age ≥12 months) sheep and goats. In the slaughterhouse survey, caudal brainstem samples including the obex, the main target site for the detection of PrPsc in classical scrapie and BSE, were removed from healthy slaughtered animals and tested in 1 of 5 approved small ruminant TSE testing laboratories. Three laboratories used the ELISA and 2 the WB technique. Both tests were performed and interpreted according to the package inserts as approved by the authorities. The laboratories were instructed to cut the brainstem samples in half sagittally to ensure that both halves contained the dorsal motor nucleus of the vagus nerve. While one-half was sampled at the level of the obex for the rapid tests and was subsequently stored at −20°C, the second half was preserved in formalin for confirmatory purposes. All initial reactive (IR) samples in the ELISA were forwarded to the NeuroCentre, where the test was repeated in duplicate. A sample was considered ELISA positive if at least 1 duplicate scored positive and inconclusive if at least 1 duplicate reached a 10% range below the cut-off value. Similarly, IR samples in the WB were repeated in the reference laboratory and regarded positive when a typical 3-band pattern for classical scrapie and BSE or a multiband pattern for atypical scrapie was observed. Faint and diffuse reactivity was interpreted as inconclusive. Confirmatory diagnosis by immunohistochemistry (IHC) according to the standards of the World Animal Health Organization 10 was performed on all ELISA and WB IR samples irrespective of the final test outcome. For the fallen stock survey, heads of perished animals were sent to the reference laboratory, and the brains were prepared as described previously. 8 Briefly, they were cut sagittally into 2 equal halves. One-half was fixed in formalin and processed for histopathology and IHC, whereas the second was stored at −20°C. All fallen sheep and goats were analyzed in both screening tests in obex samples in parallel and were also systematically subjected to IHC examination of different brain tissues (obex, cerebellum, thalamus, midbrain, ammonshorn, frontal cerebral cortex, occipital cerebral cortex) and lymphatic tissues (retropharyngeal lymph nodes, tonsils). As a part of the histopathologic examination by experienced neuropathologists, the sample autolysis was assessed on a scale from 0 (no autolysis) to 4 (severe autolysis) using criteria defined elsewhere 3 : perivascular and perineuronal retraction, white matter vacuolation, superficial spongiosis, and vascular erythrocyte appearance.
The performance of the screening tests in both surveys is summarized in Table 1. Eight cases of TSE were identified. A follow-up study classified 7 of them as atypical scrapie and 1 as classical scrapie; none turned out as BSE like. 7 Only a small proportion of the IR samples were test positive or inconclusive after repetition in the reference laboratory. To investigate the development of IR rates over time, they were stratified by month of submission (Fig. 1A, 1B). In the beginning of the program, the IR rate was as high as 4–7%, depending on the type of survey and the test format. High IR and false-positive rates lead to laborious and time-consuming retesting or confirmatory diagnostics. In some countries, this may interfere with the requirement to retain the carcasses in the slaughterhouse until the test result is reported. In the slaughterhouse survey, the ELISA revealed a considerably higher IR rate in the first quarter of the surveillance program (July–September 2004) when compared with the WB (Fig. 1A). This resulted in repeated staff training sessions by the reference laboratory and the test manufacturer. Although all laboratories were experienced in BSE diagnostics and had accredited quality management systems in place, some technical aspects were identified as critical. Insufficiently homogenized tissue samples occurred at a high frequency, and in some laboratories, the management of sample handling under high throughput operation with respect to incubation times was difficult. Hence, thereafter all samples were systematically homogenized twice and the sample management was restructured to ensure a robust compliance with the prescribed process. In one laboratory, an aldehyde-based disinfectant was used extensively during sample preparation. Reactive vapors of acids, alkalis, and aldehydes are known to alter the enzymatic reactivity of the ELISA conjugate. This product was subsequently banned from that laboratory, and decontamination procedures were modified to use 2M NaOH and 70% ethanol instead. Because formaldehyde (formalin) was indispensible in all laboratories to preserve brainstem tissues for contingent confirmatory IHC, its handling was limited to periods without ELISA sample preparation in the same laboratory environment. These corrective measures resulted then in a steady reduction of the ELISA IR rates to levels <1% in the slaughterhouse survey. Six ELISA IR samples in this survey were still classified as test positive or inconclusive after repetition but were negative in the confirmatory IHC (Table 2). They were also negative in a commercial Western blot c (data not shown), and all but 1 (ID 38115) scored only in 1 of the 2 replicates above the cut off (Table 2). Together with the finding that 4 of them were identified in the first month of the program, it was assumed that those positive scores in the test repetition were a result of the overall elevated IR rate at that time. Sample 38115 was analyzed next in line after a strong positive sample that was later on confirmed as classical scrapie and was likely contaminated with positive material during the sampling procedure. However, by definition all 6 samples are to be considered ELISA false positive or inconclusive, resulting in a diagnostic specificity of the ELISA of 99.96% (95% confidence limits: 99.92–99.98%)in the slaughterhouse survey.
Monthly initial reactive rates of 2 transmissible spongiform encephalopathy screening tests for small ruminants and sample autolysis indices in the course of a 1-year active surveillance program in Switzerland, 2004–2005.
For the WB, IR samples were not necessarily related to technical problems but to difficulties in the interpretation of the signal. Especially in atypical scrapie cases, the signal may be faint and the bands not always clearly identifiable. 7 Four of 38 IR samples remained inconclusive after repetition and were negative in confirmatory IHC tests. Based on this, the diagnostic specificity of the WB in the slaughterhouse survey was calculated to 99.98% (95% confidence limits: 99.94–99.99%).
Under the setup of the slaughterhouse survey, predictions of the diagnostic sensitivity are difficult, because negative results are not followed up and consequently false-negative samples remain unrecognized. The WB detected 3 TSE cases and the ELISA 2 TSE cases in almost identically sized sample subsets (Table 1). Under the assumption that the TSE prevalence in both subsets was comparable, the 2 tests did not appear to dramatically differ in their diagnostic sensitivity under field conditions, although the small number of cases does not allow any further interpretation.
Details on false-positive or inconclusive samples in the enzyme-linked immunosorbent assay (ELISA) and confirmatory diagnosis by immunohistochemistry (IHC) in a study on diagnostic properties of 2 screening tests for small ruminant transmissible spongiform encephalopathy.*
OD= optical density. Values above the cut off are shown in bold. – = negative.
Inconclusive OD scores in a 10% range below the cut off.
In the fallen stock survey, the overall IR rates in both screening tests were higher when compared with the slaughterhouse survey (Table 1). For the WB, a steadily decreasing trend was observed over the 1-year period. In contrast, the ELISA IR rates decreased during the winter but increased again in spring 2005 (Fig. 1B). Tissue autolysis has been considered a critical parameter affecting the diagnostic specificity in cattle BSE tests, and therefore testing of bad quality samples has become part of their evaluation (Opinion of the Scientific Steering Committee on the field trial evaluation of two new rapid BSE post mortem tests, http://europa.eu.int/comm/food/fs/sc/ssc/out246_en.pdf, last accessed 8/1/2008). To investigate a possible effect of the tissue autolysis on the rapid test performances, the fallen stock samples were similarly stratified by month of submission and the average autolysis indices were plotted over time. They were significantly lower during the winter period from November to January when compared with the warmer summer period (Fig. 1C). An interesting question was whether IR or false-positive screening test results were correlated to the degree of tissue autolysis. Indeed, the group of false-positive samples revealed significantly higher autolysis indices compared with the group of negative samples for the ELISA (Fig. 1D) but not for the WB (data not shown). Five WB- and 3 ELISA-positive or inconclusive samples were negative in confirmatory IHC tests (Table 1), resulting in a diagnostic specificity of 99.84% (95% confidence limits: 99.63–99.93%) and 99.99% (95% confidence limits: 99.72–99.99%), respectively. As described previously, all 3 TSE cases that were identified in the fallen stock survey and later classified as atypical scrapie also scored positive in the ELISA, whereas only 2 were positive in the WB, 7,8 which may reflect potential differences in the sensitivity of the 2 test formats for the detection of atypical scrapie in obex samples. However, it must be emphasized that the obex cannot be regarded as an ideal target structure for the diagnosis of atypical scrapie and that the case that was missed by the WB in the obex was readily detected by the same test in the cerebrum. 8 These numbers are not sufficient to calculate a diagnostic sensitivity with confidence. It is noteworthy that no atypical scrapie cases were missed by the screening tests on obex samples that scored positive with the IHC; the latter in all fallen stock samples was conducted not only on slides from the obex region but also in the main target sites for atypical scrapie, namely the cerebellar cortex and the cerebrum. In the course of classical scrapie and small ruminant BSE, PrPsc is detectable in the lymphatic tissues at earlier stages of the infection as compared to the CNS. None of the fallen stock animals was found positive by IHC in the lymphatic tissues under investigation, indicating that the CNS rapid tests did not miss any cases at that stage of the disease. By contrast, in the slaughterhouse survey, lymphatic tissues were not analyzed, and therefore it cannot be excluded that a proportion of affected animals were not identified in this population.
Taken together, the experiences from the Swiss active small ruminant TSE surveillance show that 2 widely used screening tests, an ELISA and a WB technique, demonstrate an excellent operative (field) diagnostic specificity (>99.5%) on slaughterhouse, as well as fallen stock, CNS samples. However, in the early phase of the slaughterhouse survey, the ELISA exhibited elevated IR rates due to technical problems that presumably were also responsible for some of the false-positive results. Uncertainties in the interpretation of WB signals obtained from fallen stock samples especially with regard to weak positive atypical scrapie cases led to a series of IR and inconclusive samples in both surveys. Based on the experiences presented in the current study, it is recommended that large-scale ad-hoc test implementation in the screening laboratories be omitted. Small-scale pilot studies and training, as well as troubleshooting by the manufacturers and the reference laboratory, may help identify critical points in the test procedure and the laboratory environment before large sample numbers are analyzed. For the ELISA, a seasonal correlation of the overall average tissue autolysis and the IR rates in fallen stock was observed, and the autolysis indices were significantly higher in ELISA false-positive samples than in negative samples. Both findings support the notion that tissue autolysis affects the performance of this specific test. Samples should therefore be removed from the animals as soon as possible after death and should ideally be stored and transported continuously in a cooled environment (∼4°C) until further processing in the laboratories. In the fallen stock survey that was designed to compare the performance of the 2 screening tests with the confirmatory IHC, only atypical scrapie cases were identified at a very low prevalence, making an assessment of the field diagnostic sensitivities very difficult for atypical scrapie and impossible for BSE and classical scrapie. These limitations were addressed by 2 interlaboratory comparison trials in the fall of 2004 and spring of 2005 that included blinded panels of classical scrapie-positive and TSE-negative samples, as well as dilution series of the positive samples. All laboratories identified these samples correctly (data not shown); obtained similar results in 1:10, 1:50, and 1:100 dilutions; and thus demonstrated a good sensitivity for classical scrapie. By contrast, due to limited amounts of reference materials, similar investigations were not possible for atypical scrapie and small ruminant BSE. In conclusion, this brief communication highlights high performance standards of current small ruminant TSE screening tests in terms of their diagnostic specificity and points to the importance of training, quality assurance measures, and sample quality to maintain these characteristics after their implementation in the field.
Acknowledgements. The authors gratefully acknowledge the participation of the laboratories UFAG (Sursee), SQTS (Courtepin), Labor Güntert (Luzern), Institut Galli Valerio (Lausanne), and Natura-Chimica (Vevey) in the surveillance program, as well as the continuous support of the test manufacturers. The authors are also indebted to Doris Ambühl, Valerie Juillerat, and Christoph Prisi for technical assistance in the NeuroCentre. This study was funded by the Swiss Federal Veterinary Office.
Footnotes
a.
Bio-Rad, Reinach, Switzerland.
b.
Prionics, Schlieren, Switzerland.
c.
TeSeE sheep and goat Western Blot, Bio-Rad, Reinach, Switzerland.