Sage Journals: Discover world-class research

Abstract

A blinded interlaboratory assessment of the diagnostic agreement and accuracy of serologic tests for routine detection of antibodies against Porcine circovirus-2 (PCV-2), including indirect fluorescent antibody tests (IFATs) and enzyme-linked immunosorbent assays (ELISAs) was conducted in 7 North American laboratories. Serum samples were collected weekly, on trial days 0, 7, 14, 21, 28, 35, 42, and 49, from the following groups of animals: 1) negative controls (n = 7), 2) PCV-2a (n = 8), 3) PCV-2b (n = 8), 4) PCV-1 (n = 8), 5) PCV-2 vaccine A (n = 8; Ingelvac® CircoFLEX™), 6) PCV-2 vaccine B (n = 8; Circumvent® PCV2), and 7) PCV-2 vaccine C (n = 8; Suvaxyn® PCV2 One Dose). Results from each laboratory were analyzed by kappa and receiver operating characteristic (ROC) analysis. Kappa analysis indicated that, by trial day 49, IFATs had almost perfect agreement, in-house ELISAs had fair to almost perfect agreement, and commercially available anti–PCV-2 immunoglobulin G ELISAs (I or S) had moderate to substantial agreement. From trial days 14–49, the area under the ROC curve for the 2 laboratories that offered IFATs, the 4 laboratories that offered in-house ELISAs, and the 3 laboratories that used commercially available ELISAs ranged from 0.94 to 1.00, 0.72 to 1.00, and 0.95 to 1.00, respectively. However, test sensitivities varied based on laboratory-specific cutoffs that were used to dichotomize test results.

Keywords

Enzyme-linked immunosorbent assay indirect fluorescent antibody test laboratory comparison pigs serology

Introduction

Porcine circovirus-2 (PCV-2; family Circoviridae; genus Circovirus) is a small, nonenveloped, single-stranded circular DNA virus. 19 Since the first reports of PCV-2-associated disease in 1996, 6 PCV-2 has been related to a number of disease manifestations in swine, including enteritis, respiratory disease, systemic infections, porcine dermatitis and nephropathy syndrome, and reproductive problems. 13

Several vaccines were introduced into the United States market in 2006 1-3 and have proven to be highly effective. 7 Whereas, most herds are seropositive for PCV-2 either because of natural infection 9 or vaccination, commercial serologic assays, including the indirect fluorescent antibody test (IFAT) and enzyme-linked immunosorbent assay (ELISA), are widely used. Common uses include determination of the PCV-2 serologic status of herds (specific pathogen–free herds and research studies), cross-sectional serologic surveys to determine the prevalence and levels of anti–PCV-2 antibodies for optimal timing of vaccination, and assessing vaccination compliance. Although the IFAT and ELISA have advantages over standard cultivation methods, including rapid detection of antibodies in a wide range of samples, 21 the IFAT is not automated and has a subjective end point, whereas the ELISA is automated and has an objective end point.

In a previous study, 20 serum samples taken from both experimentally and naturally infected pigs were analyzed by 5 different laboratories by using IFAT and/or immunoperoxidase monolayer assay (IPMA). 11 Analysis of results indicated that there was a wide variation of anti–PCV-2 antibody detection and that use of different fixatives (paraformaldehyde, acetone, or ethyl alcohol) and fixation methods may be the cause for the different results. 11 No statistical analysis was performed in that trial. In addition, other studies have been published that compared newly developed serologic tests with either an IFAT 12,16,17,20 or an IPMA. 10 In a 2008 study in which 2 commercially available ELISAs and 1 in-house ELISA were compared, it was found that the assays were able to detect both PCV-2a and PCV-2b, and that all assays had high diagnostic accuracies based on receiver operating characteristic (ROC) analysis. 15 To the authors' knowledge, an interlaboratory comparison between currently offered in-house and commercial ELISAs and IFATs has not been conducted to date. The objective of the current study was to evaluate the agreement and diagnostic accuracy of PCV-2 serologic assays from 7 diagnostic laboratories using known positive and negative samples from experimentally infected specific pathogen–free conventional pigs.

Table 1.

Summary of enzyme-linked immunosorbent assay (ELISA) protocols and indirect fluorescent antibody tests (IFAT) used by each of the 7 diagnostic laboratories.*

Assay	Laboratory	Protocol	Assay use	Reporting format
In-house ELISA	1	Adapted from previously described protocol 12	Commercial	S/P ratio
	3	Indirect ELISA by using plates coated with an ORF2 gene amplified from a PCV-2b isolate and anti-pig peroxidase HRP conjugate at a 1:2,000 dilution	Research	S/P ratio
	4	ELISA by using plates coated with N-truncated ORF2 gene from a PCV-2b isolate and anti-pig peroxidase-HRP conjugate at a 1:5,000 dilution	Research	S/P ratio
	5	Adapted from previously described protocol. 12	Commercial	S/P ratio
I-ELISA	1	Manufacturer's protocol f	Research	OD value
S-ELISA	1	Manufacturer's protocol g	Research	S/N ratio
	6	Manufacturer's protocol g	Commercial	S/N ratio
	7	Manufacturer's protocol g	Commercial	S/N ratio
	IFAT	1	Previously described protocol 15	Commercial	Titer
	2	Previously described protocol 7	Commercial	Titer
	5	Adapted from previously described protocol 15	Commercial	Positive or negative

Commercial = routine use in diagnostic laboratory setting; S/P = sample to positive; ORF2 = open reading frame 2; PCV-2 = Porcine circovirus-2; Research = modified use for individual studies; HRP = horseradish peroxidase; I-ELISA = Ingezim f ; OD = optical density; S-ELISA = Serelisa g ; S/N = sample to negative.

Materials and methods

Diagnostic laboratories

Each of the 7 participating laboratories (coded 1–7) voluntarily participated in the research project. Each laboratory was asked to perform a currently available PCV-2 IFAT and/or ELISA by using laboratory-specific protocols (Table 1). The samples were coded with a numerical system that had no association with the original treatment system.

Study samples

Four hundred and forty serum samples from a previous trial were used for the present research project. Briefly, 3-week-old conventional pigs purchased from a PCV-2–negative herd were randomly divided into 7 groups and inoculated or vaccinated at trial day 0 as follows: 1) negative controls (n = 7), 2) PCV-2a (inoculated with ISU-40895; GenBank accession no. AF264042; n = 8), 3) PCV-2b (inoculated with NC-16845; GenBank accession no. EU340258, n = 8), 4) PCV-1 (inoculated with a well-characterized PCV-1 clone 2 ; n = 8), 5) vaccine A a (n = 8), 6) vaccine B b (n = 8), and 7) vaccine C c (n = 8). All the pigs were bled at weekly intervals until trial day 49. The experimental protocol was approved by the Iowa State University Institutional Animal Care and Use Committee.

Sample preparation and submission

After blood collection, the samples were centrifuged at 1,700 x g for 10 min. The serum samples from all pigs from trial days 0–49 (n = 440 total samples) were each divided into 4-ml aliquots and stored at −80°C. Before shipment to the various diagnostic laboratories, each sample was further divided into seven 50-μl aliquots and assigned an individual identification number to which all laboratories were blinded. Coded samples were packaged with ice packs for overnight shipment to the various diagnostic laboratories.

Definition of positive and negative samples

A correctly identified positive sample (n = 280) was defined as a sample obtained from pigs in the PCV-2–infected groups (PCV-2a, PCV-2b) and the PCV-2–vaccinated groups (vaccines A–C; n = 40) on trial days 7, 14, 21, 28, 35, 42, and 49. A correctly identified negative sample (n = 160) was defined as a sample obtained from any pig on trial day 0 (n = 55 independent observations) and from pigs in the negative control group and pigs infected with PCV-1 (n = 15 dependent observations) on trial days 7, 14, 21, 28, 35, 42, and 49.

Table 2.

Percentage of correctly identified Porcine circovirus-2 antibody negative (specificity) or positive (sensitivity) samples by different serologic assays used in 7 diagnostic laboratories by assay use (commercial or research).*

			Specificity		Sensitivity on different trial days
	Assay use	Assay	Laboratory	Independent	Dependent†	7	14	21	28	35	42	49	Total
Commercial	In-house	1	55/55 a ‡	105/105	0/40 a	18/40 a	37/40 a	38/40 a	40/40 a	40/40 a	40/40 a	213/280
	ELISA	5	53/55 a	103/105	4/40 a	33/40 b	36/40 a	38/40 a	40/40 a	40/40 a	40/40 a	231/280
	S-ELISA	6	55/55 a	105/105	4/40 a	12/40 a	5/40 b	12/40 b	16/40 b	26/40 b	30/40 b	105/280
		7§	51/51 a	96/96	0/40 a	11/40 a	2/40 b	12/40 b	19/40 b	8/39 c	18/40 c	70/279
	IFAT	1	55/55 a	105/105	0/40 a	38/40 b	40/40 a	37/40 a	40/40 a	40/40 a	39/40 a b	234/280
		2	55/55 a	104/105	0/40 a	36/40 b	39/40 a	40/40 a	40/40 a	40/40 a	40/40 a	235/280
		5	55/55 a	82/105	1/40 a	15/40 a	24/40 c	24/40 b	28/40 b	40/40 a	40/40 a	172/280
Research	In-house	3	38/55 a	97/105	7/40 a	6/40 ab	16/40 a	14/40 a	19/40 a	22/40 a	26/40 a b	110/280
	ELISA	4	55/55 b	105/105	0/40 b	0/40 a	6/40 b	14/40 a	15/40 a	19/40 a	21/40 a	75/280
	I-ELISA	1	55/55 b	105/105	1/40 ab	13/40 b	26/40 c	30/40 b	34/40 b	35/40 b	34/40 b	173/280
	S-ELISA	1	55/55 b	105/105	2/40 ab	9/40 b	1/40 b	11/40 a	16/40 a	17/40 a	23/40 a	79/280

ELISA = enzyme-linked immunosorbent assay; S-ELISA = Serelisa g ; IFAT = indirect fluorescent antibody test; I-ELISA = Ingezim. f

†

n = 15 pigs over time; McNemar test is based on independent assumption of observations and was not done.

‡

Assays not sharing common superscripts ( a,b,c ) within each of the assay use groups are significantly different in classification probability (sensitivity by day or specificity).

Only 426 samples were tested by laboratory 7.

Statistics

Differences in sensitivity and specificity among assays were tested for significance by the pairwise McNemar test, with Bonferroni correction for multiple comparisons. A kappa statistic was calculated for paired tests by using dichotomized data. For tests in which results are identified as positive, negative, or suspect, the suspect samples were considered negative. Values for kappa range from –1 to 1, in which –1 indicates perfect disagreement, 0 equals agreement no better than expected by chance, and 1 equals perfect agreement. 18 The following arbitrary standards for the strength of agreement as described by Landis and Koch 8 were used: ≤0 = poor, 0.01–0.2 = slight, 0.21–0.4 = fair, 0.41–0.60 = moderate, 0.61–0.80 = substantial, and 0.81–1 = almost perfect. Statistical analysis was performed by using JMP software. d Diagnostic accuracy of each assay was also evaluated by using ROC analysis. This method results in area under the ROC curve (AUC) values that range from 0.5 to 1: 0.5 indicates that the test is not discriminating between positive and negative animals, and 1 indicates perfect discrimination. 22 All ROC calculations were performed on continuous data with infection status as the indicator of true infection status by using MedCalc. e Specifically, samples from pigs in the PCV-2a, PCV-2b, vaccine A, vaccine B, and vaccine C groups were considered “true positive”; negative controls and PCV-1 were considered “true negative” samples.

Results

The percentage of correctly identified negative and positive samples for all assays is provided in Table 2. Kappa values for all assays on trial days 14 and 49 samples are summarized in Tables 3 (commercial setting) and 4 (research setting). The percentage of correctly identified test results for trial day 49 on PCV-2–infected animals, PCV-2–vaccinated animals, and PCV-2–negative animals for each assay appear in Figure 1. Results of ROC analysis on trial days 14–49 are summarized in Table 5. The sensitivity and specificity generated through ROC analysis by using recommended cutoffs for each assay at trial days 14 and 49 are presented in Table 6.

Enzyme-linked immunosorbent assays

In-house ELISAs. The overall specificity was 100% (55/55 correctly identified negative samples) for laboratories 1 and 4, 69.1% (38/55 correctly identified negative samples) for laboratory 3, and 96.4% (53/55 correctly identified negative samples) for laboratory 5 (Table 2). The overall sensitivity was 76.1% (213/280 correctly identified positive samples) for laboratory 1, 39.3% (110/280 correctly identified positive samples) for laboratory 3, 26.8% (75/280 correctly identified positive samples) for laboratory 4, and 82.5% (231/280 correctly identified positive samples) for laboratory 5. It is of interest to note that the assays with sensitivities of 76.1–82.5% (laboratories 1 and 5) were used for commercial diagnostic purposes, and those with sensitivities of 26.8–39.3% (laboratories 3 and 4) were used for research purposes. The commercially used assays had improved sensitivity at an early trial day (45% and 82.5%; trial day 14) compared with assays used for research purposes (0% and 15%; trial day 14; Table 2). The agreement on results was almost perfect for laboratories 1 and 5 (Table 3) and moderate for laboratories 3 and 4 (Table 4).

Table 3.

Overall measure of statistical agreement (kappa statistic ± standard error) trial days 14 and 49 among laboratories (commercial assay use).*

			In-house ELISA	IFAT			S-ELISA
Trial day	Assay	Laboratory	Laboratory 5	Laboratory 1	Laboratory 2	Laboratory 5	Laboratory 6	Laboratory 7
14	In-house ELISA	1	0.42 ± 0.10	0.34 ± 0.10	0.38 ± 0.09	0.29 ± 0.14	0.01 ± 0.13	−0.07 ± 0.12
		5		0.81 ± 0.08	0.77 ± 0.09	0.19 ± 0.11	0.25 ± 0.09	0.27 ± 0.08
	IFAT	1				0.80 ± 0.09	0.12 ± 0.10	0.19 ± 0.08	0.19 ± 0.06
			2				0.04 ± 0.10	0.24 ± 0.07	0.20 ± 0.07
			5					−0.16 ± 0.11	−0.15 ± 0.12
	S-ELISA	6						0.61 ± 0.13
49	In-house ELISA	1	1.00 ± 0.00	1.00 ± 0.00	1.00 ± 0.00	0.90 ± 0.07	0.62 ± 0.10	0.25 ± 0.07
		5	1.00 ± 0.00	1.00 ± 0.00	0.90 ± 0.07	0.62 ± 0.10	0.25 ± 0.07
	IFAT	1			1.00 ± 0.00	0.90 ± 0.07	0.62 ± 0.10	0.27 ± 0.08
		2				0.90 ± 0.07	0.62 ± 0.10	0.25 ± 0.07
	S-ELISA	6						0.48 ± 0.10

ELISA = enzyme-linked immunosorbent assay; IFAT = indirect fluorescent antibody test; S-ELISA = Serelisa. g The following arbitrary standards for the strength of agreement previously described 8 were used: ≤0 = poor, 0.01–0.2 = slight, 0.21–0.4 = fair, 0.41–0.60 = moderate, 0.61–0.80 = substantial, and 0.81–1 = almost perfect.

Commercially produced ELISAs. The overall specificity was 100% for all commercially produced ELISAs (laboratories 1 [55/55], 6 [55/55], and 7 [53/53]; Table 2). The overall sensitivity was 61.8% (Ingezim, f or I-ELISA) and 28.2% (Serelisa, g or S-ELISA) with 173 and 79, respectively, of 280 correctly identified positive samples for laboratory 1; 37.5% (105/280 correctly identified positive samples) for laboratory 6; and 25.1.0% (70/279 correctly identified positive samples) for laboratory 7. Regardless of purpose, S-ELISAs appeared to perform very similarly (Table 2). By trial day 49, the agreement on results was moderate for laboratories 6 and 7 with S-ELISA (Table 3). Similarly, laboratory 1 (S-ELISA and I-ELISA) had moderate agreement on trial day 49 (Table 4).

Indirect fluorescent antibody tests

The overall specificity was 100% (55/55 correctly identified negative samples) for laboratories 1, 2, and 5 (Table 2). The overall sensitivity was 83.6% (234/280 correctly identified positive samples) for laboratory 1, 83.9% (235/280 correctly identified positive samples) for laboratory 2, and 61.4% (172/280 correctly identified positive samples) for laboratory 5 (Table 2). By trial day 49, the 3 IFATs had almost perfect agreements, which ranged from 0.90 ± 0.07 to 1.00 ± 0.00 (Table 3).

Discussion

To reflect the actual outcomes (positive or negative for anti–PCV-2 antibodies) obtained by 7 North American laboratories, kappa analysis was used in the current study to compare dichotomous test data. Conversely, ROC analysis, which has become an increasingly valuable tool for comparison of diagnostic tests, 4,5,22 uses information on the sensitivity and specificity across the entire range of possible cutoffs and can provide an optimized cutoff, which maximizes the sensitivity and specificity based on the continuous data. 4 Therefore, the discrepancy noted in the present study between the results of ROC and kappa analyses on the commercial and in-house ELISAs was because of the selection of the cutoff values for data dichotomization. This concept is clearly evident in Table 4, in which sensitivities differ among laboratories that had similar high AUC values. Because this concept is not applicable for comparing the results obtained from diagnostic laboratories, only sensitivities and specificities generated at laboratory-specific cutoffs were used in the present study, and AUC values, although provided, were not compared among laboratories.

Table 4.

Overall measure of statistical agreement (kappa statistic ± standard error) on trial days 14 and 49 among laboratories (research assay use).*

			In-house ELISA	S-ELISA	I-ELISA
Trial day	Assay	Laboratory	Laboratory 4	Laboratory 1	Laboratory 1
14	In-house ELISA	3	Not done†	0.27 ± 0.17	0.28 ± 0.15
		4		Not done†	Not done†
	S-ELISA	1				0.10 ± 0.14
49	In-house ELISA	3	0.78 ± 0.08	0.49 ± 0.12	0.60 ± 0.10
		4		0.38 ± 0.09	0.55 ± 0.01
	S-ELISA	1			0.54 ± 0.10

ELISA = enzyme-linked immunosorbent assay; S-ELISA = Serelisa g ; I-ELISA = Ingezim. f The following arbitrary standards for the strength of agreement previously described 8 were used: ≤0 = poor, 0.01–0.2 = slight, 0.21–0.4 = fair, 0.41–0.60 = moderate, 0.61–0.80 = substantial, and 0.81–1 = almost perfect.

†

Laboratories 1 and 3 were not compared with laboratory 4 on trial day 14, because laboratory 4 had no positive samples.

Figure 1.

Percentage of correctly identified test results, which correspond to classification based on established cutoffs on trial day 49 from 7 different diagnostic laboratories in the United States. The data are divided into 3 main categories: PCV2, VACCINE, and NEG. A correctly identified Porcine circovirus-2 (PCV-2) sample was defined as a sample from a pig infected with PCV-2 (combined data from PCV-2a and PCV-2b groups). VACCINE samples were from vaccinated animals (combined data from vaccine A, B, and C groups), and NEG samples were from negative animals (combined data from Porcine circovirus-1 and negative controls). Assays are organized by submitting laboratory and type of assay; indirect fluorescent antibody tests (IFA), in-house enzyme-linked immunosorbent assays (ELISA), and commercially available (Serelisa g [S] or Ingezim f [I]) ELISAs.

When the assays were compared by using kappa analysis, all 3 laboratories that offered IFATs had almost perfect agreement by trial day 49. The IFATs also had almost perfect agreement with the in-house ELISAs from laboratories 1 and 5. However, these data were generated by experimentally inoculating naïve animals with PCV-2, and ancillary data from one of the participating laboratories indicated that agreement between ELISA and IFAT at early trial days may not be as high in piglets with passively acquired maternal antibodies (R. Hesse, personal communication, 2009). Based on previous experimental studies, 14 there were likely consistently high levels of antibodies present at this point, which enhanced uniformity in test results in comparison with samples taken from animals shortly after PCV-2 infection as evident in the kappa values (poor to almost perfect) obtained on trial day 14.

Table 5.

Area under the receiver operating curve values for each of the assays on different trial days.

			Trial day‡
Assay use	Assay*	Laboratory†	14	21	28	35	42	49
Commercial	In-house ELISA	1	0.94 (0.84; 0.98)	0.99 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)
		5	0.95 (0.86; 0.99)	0.89 (0.78; 0.96)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)
	S-ELISA	6	0.96 (0.87; 0.99)	0.95 (0.86; 0.99)	0.96 (0.87; 0.99)	0.96 (0.86; 1.00)	0.98 (0.90; 1.00)	1.00 (0.93; 1.00)
		7	0.96 (0.87; 0.99)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)
	IFAT	1	0.98 (0.89; 1.00)	1.00 (0.9; 1.00)	0.96 (0.87; 0.99)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	0.99 (0.91; 1.00)
		2	0.94 (0.85; 0.99)	0.99 (0.91; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)
Research	In-house ELISA	3	0.75 (0.61; 0.86)	0.84 (0.71; 0.92)	0.84 (0.71; 0.92)	0.83 (0.70; 0.92)	0.73 (0.59; 0.84)	0.99 (0.79; 0.96)
		4	0.72 (0.58; 0.83)	0.86 (0.74; 0.94)	0.86 (0.74; 0.94)	0.88 (0.76; 0.95)	0.91 (0.80; 0.97)	0.89 (0.78; 0.96)
	S-ELISA	1	0.97 (0.89; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)
	I-ELISA	1	0.96 (0.87; 0.99)	0.99 (0.93; 1.0)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)	1.00 (0.93; 1.00)

ELISA = enzyme-linked immunosorbent assay; S-ELISA = Serelisa g ; I-ELISA = Ingezim f ; IFAT = indirect fluorescent antibody test.

†

Data from laboratory 1 have been previously published. 14 Data from the IFAT from laboratory 5 were not included because continuous data were not provided.

‡

Numbers in parentheses are 95% confidence intervals; receiver operating characteristic analysis was performed by denoting PCV-2a, PCV-2b, vaccine A, vaccine B, and vaccine C pigs as “true positive” and negative controls, and PCV-1 as “true negative” animals. Area under the receiver operating curve values range from 0.5 to 1, where 1 indicates that, when using a given sample, the diagnostic test would perfectly discriminate between PCV-2-positive and PCV-2-negative pigs, and 0.5 indicates that there is no discrimination between groups.

Table 6.

Recommended cutoffs and associated sensitivity and specificity for trial days 14 and 49 for each assay.* †

				Trial day 14‡		Trial day 49‡
	Assay use	Assay	Laboratory	Recommended cutoff	Sensitivity (%)	Specificity (%)	Sensitivity (%)	Specificity (%)
Commercial	In-house ELISA	1	≥0.20	42.5 (27.1; 59.1)	100.0 (78.0; 100.0)	100.0 (91.1; 100.0)	100.0 (78.0; 100.0)
		5	≥0.4	82.5 (67.2; 92.6)	100.0 (78.0; 100.0)	100.0 (91.1; 100.0)	100.0 (78.0-100.0)
	S-ELISA	6	≤0.15	30.0 (16.6; 46.5)	100.0 (78.0; 100.0)	75.0 (58.8; 87.3)	100.0 (78.0-100.0)
		7	≤0.15	25.0 (12.7; 41.2)	100.0 (76.7; 100.0)	45.0 (29.3; 61.5)	100.0 (76.7; 100.0)
	IFAT	1	≥0.90§	95.0 (83.0; 99.2)	100.0 (78.0; 100.0)	97.5 (86.8; 99.6)	100.0 (78.0; 100.0)
		2	≥1.3‖	90.0 (76.3; 97.1)	93.3 (68.0; 98.9)	100.0 (91.1; 100.0)	100.0 (78.0; 100.0)
Research	In-house ELISA	3	>0.3	0.0 (0.0; 8.9)	100.0 (78.0; 100.0)	52.5 (36.1; 68.5)	97.5 (86.8; 99.6)
		4	≥0.4	0.0 (0.0; 8.9)	100.0 (78.0; 100.0)	0.0 (0.0; 8.9)	100.0 (78.0; 100.0)
	S-ELISA	1	≤0.15	25.0 (12.7; 41.2)	100.0 (78.0; 100.0)	60.0 (43.3; 75.1)	100.0 (78.0; 100.0)
	I-ELISA	1	≥0.33¶	27.5 (14.6; 43.9)	100.0 (78.0; 100.0)	82.5 (67.2; 92.6)	100.0 (78.0-100.0)

ELISA = enzyme-linked immunosorbent assay; S-ELISA = Serelisa g ; IFAT = indirect fluorescent antibody test; I-ELISA = Ingezim. f

†

Data from laboratory 5 (IFAT) was not included because continuous data were not provided.

‡

Numbers in parentheses are 95% confidence intervals; sensitivity and specificity were generated by using receiver operating characteristic analysis.

Corresponds to a cutoff of ≥1:8.

‖

Corresponds to a cutoff of ≥1:20.

Plate-specific cutoff was determined by the average optical density value of the positive control wells multiplied by 0.3; the number represents an average of plate-specific cutoffs when using the experimental study data.

In comparison with the ELISAs, kappa analysis revealed differences among the laboratories. Specifically, the commercially available S-ELISAs from laboratories 6 and 7 had only moderate agreement at trial day 49. Reasons for the interlaboratory variability of the commercial assays in the present study need to be further elucidated but may include variability among technicians because the laboratories were using the same commercially available kits according to the manufacturers' directions. Among in-house ELISAs, the sensitivity varied based on the main purpose (76.1–82.5% for commercial purpose and 26.8–39.3% for research purpose) for which the assay was designed and being used. Reasons for the interlaboratory variability of the in-house assays may have been because of differences in strain of virus used in the coating of the plates, the length of the PCV-2 capsid protein used in the assay (laboratories 1 and 5 used an intact capsid, whereas laboratory 4 used a truncated capsid), antibody specificity, selection of cutoff value, or variability between technicians. In addition, laboratories designated as research (laboratories 3 and 4) were routinely modifying assays for individual studies; therefore, the conditions used in the current study do not necessarily predict future performance.

Based on data from trial day 14, the IFATs in laboratories 1 and 2, and the in-house ELISA used in laboratory 5 detected the highest number of positive samples. Therefore, the use of one of these assays may result in a higher likelihood of early disease detection when monitoring negative herds. However, before trial day 14, few samples (0, 0, and 4, respectively) were correctly identified as being positive by these 3 assays, which indicates that the use of serology for routine monitoring of a PCV-2–negative population may be of limited use for detection of early infections (Table 2).

These results demonstrate that, whereas differences among laboratories do exist, in-house ELISAs (excluding in-house ELISAs categorized for research use) and IFATs had high diagnostic accuracy based on kappa analysis. Commercial ELISAs used in the current study had fair to substantial agreement with the in-house ELISAs.

Acknowledgements

The authors thank Paul Thomas and Jeremy Johnson for assistance with animal work and Boehringer Ingelheim Vetmedica Inc., for funding of this study. The authors also thank Dr. Chong Wang for statistical advice.

Footnotes

a.

Ingelvac® CircoFLEX™, Boehringer Ingelheim Vetmedica Inc., St. Joseph, MO.

b.

Circumvent® PCV, Intervet Inc., Millsboro, DE.

c.

Suvaxyn® PCV2 One Dose, Fort Dodge Animal Health, Fort Dodge, IA.

d.

JMP® version 6.0.0, SAS Institute Inc., Cary, NC.

e.

MedCalc® Software version 9.1.0.1, Mariakerke, Belgium.

f.

Ingezim Circovirus IgG, Ingenasa, Madrid, Spain.

g.

Serelisa® PCV2 Ab Mono Blocking, Synbiotics Europe SAS, Lyon, France.

References

Fachinger

Bischoff

Jedidia

: 2008, The effect of vaccination against porcine circovirus type 2 in pigs suffering from porcine respiratory disease complex. Vaccine 26:1488–1499.

Fenaux

Opriessnig

Halbur

Meng

: 2003, Immunogenicity and pathogenicity of chimeric infectious DNA clones of pathogenic porcine circovirus type 2 (PCV2) and nonpathogenic PCV1 in weanling pigs. J Virol 77:11232–11243.

Fort

Sibila

Allepuz

: 2008, Porcine circovirus type 2 (PCV2) vaccination of conventional pigs prevents viremia against PCV2 isolates of different genotypes and geographic origins. Vaccine 26:1063–1071.

Gardner

Greiner

: 2006, Receiver-operating characteristic curves and likelihood ratios: Improvements over traditional methods for the evaluation and application of veterinary clinical pathology tests. Vet Clin Pathol 35:8–17.

Hanley

McNeil

: 1983, A method of comparing the areas under the receiver operating characteristic curves derived from the same cases. Radiology 148:839–843.

Harding

Clark

: 1997, Recognizing and diagnosing postweaning multisystemic wasting syndrome (PMWS). Swine Health Prod 5:201–203.

Horlen

Dritz

Nietfeld

: 2008, A field evaluation of mortality rate and growth performance in pigs vaccinated against porcine circovirus type 2. J Am Vet Med Assoc 232:906–912.

Landis

Koch

: 1977, The measurement of observer agreement for categorical data. Biometrics 33:159–174.

Larochelle

Magar

D'Allaire

: 2003, Comparative serologic and virologic study of commercial swine herds with and without postweaning multisystemic wasting syndrome. Can J Vet Res 67:114–120.

10.

Liu

Ihara

Nunoya

Ueda

: 2004, Development of an ELISA based on the baculovirus-expressed capsid protein of porcine circovirus type 2 as antigen. J Vet Med Sci 66:237–242.

11.

McNair

Marshall

McNeilly

: 2004, Interlaboratory testing of porcine sera for antibodies to porcine circovirus type 2. J Vet Diagn Invest 16:164–166.

12.

Nawagitgul

Harms

Morozov

: 2002, Modified indirect porcine circovirus (PCV) type 2-based and recombinant capsid protein (ORF2)-based enzyme-linked immuno-sorbent assays for detection of antibodies to PCV. Clin Diagn Lab Immunol 9:33–40.

13.

Opriessnig

Meng

Halbur

: 2007, Porcine circovirus type 2 associated disease: Update on current terminology, clinical manifestations, pathogenesis, diagnosis, and intervention strategies. J Vet Diagn Invest 19:591–615.

14.

Opriessnig

Patterson

Elsener

: 2008, Influence of maternal antibodies on efficacy of porcine circovirus type 2 (PCV2) vaccination to protect pigs from experimental infection with PCV2. Clin Vaccine Immunol 15:397–401.

15.

Patterson

Johnson

Ramamoorthy

: 2008, Comparison of three enzyme-linked immunosorbent assays to detect porcine circovirus-2 (PCV-2)-specific antibodies after vaccination or inoculation of pigs with distinct PCV-1 or PCV-2 isolates 2. J Vet Diagn Invest 20:744–751.

16.

Pogranichnyy

Yoon

Harms

: 2000, Characterization of immune response of young pigs to porcine circovirus type 2 infection. Viral Immunol 13:143–153.

17.

Shang

Guo

: 2007, Development and validation of a recombinant capsid protein-based ELISA for detection of antibody to porcine circovirus type 2. Res Vet Sci 84:150–157.

18.

Sim

Wright

: 2005, The kappa statistic in reliability studies: Use, interpretation, and sample size requirements. Phys Ther 85:257–268.

19.

Tischer

Gelderblom

Vettermann

Koch

: 1982, A very small porcine virus with circular single-stranded DNA. Nature 295:64–66.

20.

P-C

Chein

M-S

Tseng

Y-Y

: 2008, Expression of the porcine circovirus type 2 capsid protein subunits and application to an indirect ELISA. J Biotechnol 133:58–64.

21.

Yolken

Viscidi

: 1989, Enzyme immunoassay and radioimmunoassay, In: Diagnostic procedures for viral, rickettsial and chlamydial infections, ed. Schmidt

Emmons

, 6th ed., pp. 157–178. American Public Health Association, Washington, DC.

22.

Zweig

Campbell

: 1993, Receiver-operating characteristic (ROC) plots: A fundamental evaluation tool in clinical medicine. Clin Chem 39:562–557.

Interlaboratory Comparison of Porcine Circovirus-2 Indirect Immunofluorescent Antibody Test and Enzyme-Linked Immunosorbent Assay Results on Experimentally Infected Pigs

Abstract

Keywords

Introduction

Materials and methods

Diagnostic laboratories

Study samples

Sample preparation and submission

Definition of positive and negative samples

Statistics

Results

Enzyme-linked immunosorbent assays

Indirect fluorescent antibody tests

Discussion

Acknowledgements

Footnotes

a.

b.

c.

d.

e.

f.

g.

References