Performance of Commercial Mycoplasma hyopneumoniae Serum Enzyme-Linked Immunosorbent Assays under Experimental and Field Conditions

Mycoplasma hyopneumoniae is an economically significant pathogen of swine. M. hyopneumoniae serum antibody detection via commercial enzyme-linked immunosorbent assays (ELISAs) is widely used for routine surveillance in commercial swine production systems. Samples from two studies were used to evaluate assay performance. In study 1, 6 commercial M. hyopneumoniae ELISAs were compared using serum samples from 8-week-old cesarean-derived, colostrum-deprived (CDCD) pigs allocated to the following 5 inoculation groups of 10 pigs each: (i) negative control, (ii) Mycoplasma flocculare (strain 27399), (iii) Mycoplasma hyorhinis (strain 38983), (iv) Mycoplasma hyosynoviae (strain 34428), and (v) M. hyopneumoniae (strain 232).

M. flocculare (strain 27399) inoculum consisted of a live culture propagated in Friis broth media in a shaking water bath (37°C for 48 h) diluted to a concentration at 1.0 ϫ 10 5 CCU/ml, as determined by titration in Friis media. The inoculum was administered intranasally (1 ml in each nostril) and intratracheally (1 ml in the trachea). In addition, on 0 and 4 days postinoculation (dpi), the inoculum was "scrubbed" on the tonsils of the soft palate using a 16-in. (41-cm) large-tip cotton swab (Scopettes 16-in. absorbent; Birchwood Laboratories, Inc., Eden Prairie, MN, USA) saturated with 2 ml of the inoculum.
M. hyorhinis (strain 38983) inoculum consisted of a live culture propagated in Friis broth media (37°C for 48 h) and diluted to a concentration of 3.2 ϫ 10 8 CCU/ml, as determined by titration in Friis agar media. The inoculum was administered into the peritoneal cavity (2 ml) and "scrubbed" on the tonsil of the soft palate using a 16-in. (41-cm) large-tip cotton swab (Scopettes 16-in. absorbent) saturated with 2 ml of the inoculum.
M. hyosynoviae (strain 34428) field isolate was cultivated in Difco medium containing turkey serum (d-TS) broth (37°C for 48 h) and diluted to a concentration of 2.1 ϫ 10 9 CCU/ml, as determined by titration in Friis agar media. Pigs were inoculated intranasally (1 ml per naris) and intravenously (1 ml via the auricular vein) and "scrubbed" on the tonsils of the soft palate using a 16-in. (41 cm) large-tip cotton swab (Scopettes16-in. absorbent) saturated with 2 ml of the inoculum.
(ii) Study 2. The M. hyopneumoniae (strain 232) inoculum used in study 2 was identical to that in study 1 except that the concentration was estimated to be 1 ϫ 10 5 CCU/ml, and each of the 10 M. hyopneumoniae-inoculated pigs was intratracheally administered 10 ml.
Sample collection. Blood samples were collected using single-use collection tubes. In the laboratory, samples were centrifuged (1,500 ϫ g for 15 min), and the serum was aliquoted into 2 ml cryogenic tubes (Cryo.s; Greiner Bio-One, Monroe, NC, USA) for storage at Ϫ80°C.
Pen-based oral fluid samples were collected using 3-strand (1.6 cm) 100% cotton rope (Web Rigging Supply, Lake Barrington, IL, USA) suspended from a bracket fixed to the side of the enclosure (17). The oral fluid sample was recovered from the rope after 30 min exposure to the pigs. In the laboratory, oral fluid samples were vortexed and aliquoted into cryogenic tubes (Cryo.s) and stored at Ϫ80°C.
Tracheal mucosal samples were collected using a sheathed, single-use porcine cervical artificial insemination catheter (Nasco, Fort Atkinson, WI, USA). After sampling, the tip containing the collected material was severed from the catheter and placed in a tube containing 1 ml sterile phosphate-buffered saline (PBS). In the laboratory, tracheal samples were transferred to 2 ml cryogenic tubes (Cryo.s) for storage at Ϫ80°C.
In study 1, Blood samples were collected from all pigs on Ϫ3, 0, 3,7,10,14,17,21,24,28,35,42,49, and 56 dpi, and pen-based oral fluid samples were collected daily. Lung tissues were collected at necropsy (56 dpi) and stored at Ϫ80°C until processed for PCR testing. . Distinct from the other ELISAs, one assay (Civtest Suis Mhyo, Laboratorios Hipra, S.A., Girona, Spain) was based on a two-well indirect ELISA format. As described by the manufacturer, one well contained an M. hyopneumoniae-specific antigen and the other contained generic mycoplasma antigen. The response was then determined as the proportion of M. hyopneumoniae-specific antibodies in the sample.
The same ELISA equipment, i.e., plate washer (ELx405; Biotek Instruments Inc., Winooski, VT USA), ELISA reader (EMax Plus microplate reader; Molecular Devices, San Jose, CA USA), and reader software (SoftMax Pro 7.0; Molecular Devices) were used throughout. One commercial M. hyopneumoniae ELISA was run per day following the instructions provided by the manufacturer. Assay validity criteria were checked and verified for each plate according to the instructions provided by each manufacturer. Serum samples were randomized to avoid systematic errors, and no samples were retested.  water). A test result was considered valid when the internal positive cycle threshold (C T ) values were Յ36. A sample was considered M. hyopneumoniae positive when C T values were Յ40.
Detection of M. hyopneumoniae DNA in tracheal samples and lung homogenates. Lung tissues were collected at necropsy (56 dpi) and stored at Ϫ80°C until they were processed for PCR testing. As described elsewhere (18), lung (3 by 3 cm) containing both normal and affected tissue was minced using sterile scissors and then placed in a 50-ml conical tube with 30 ml of Earle's balanced salt solution (Sigma-Aldrich, St. Louis, MO, USA) at a concentration of 10% (wt/vol). The sample was homogenized (2 min at 1,000 rpm; Geno/Grinder; Spex SamplePrep, Metuchen, NJ, USA) and then centrifuged (10 min at 4,200 ϫ g).
M. hyopneumoniae DNA detection for both tracheal and lung homogenates was based on commercial kits (MagMax-96 Pathogen RNA/DNA kit, PCR VetMax-Plus qPCR master mix, VetMax Mycoplasma hyopneumoniae reagents; Applied Biosystems) performed as directed by the manufacturer. DNA was extracted on the Kingfisher Flex system and amplified on Applied Biosystems 7500 real-time PCR. Each plate included a known M. hyopneumoniae-positive sample (VetMax-Plus qPCR master mix kit includes Xeno DNA Control, Applied Biosystems) and a negative control sample (RNA-free water). A test result was considered valid when the internal positive C T value was Յ36. A sample was considered M. hyopneumoniae positive when C T values were Յ37.
Data analysis. (i) Study 1. Serum antibody data from 680 samples were identified by inoculation group (negative control, M. flocculare, M. hyorhinis, M. hyosynoviae, and M. hyopneumoniae), pig number (1 to 50), and the standardized response for each of the six ELISAs, i.e., sample-to-positive ratio (S/P), percent blocking (%B), or relative index (RI). Evaluation of the six ELISA results revealed that the data were not normally distributed (Shapiro-Wilk W test; P Ͻ 0.05). Transformation using logarithms or the Box-Cox method was not successful but was achieved using "ordered quantile normalization" (19). Ordered quantile normalization was performed using the function orderNorm of the bestNormalize package (20) in R (R version 3.6.0, R Core Team 2019) for each of the six sets of ELISA results. Transformed ELISA results were normally distributed, i.e., lain on a straight line in quantile-quantile plots, and the Shapiro-Wilk W test was not significant (P Ͼ 0.05).
The evaluation of test performance was constrained by the small number of M. hyopneumoniaeinfected animals (n ϭ 8 pigs) and the delayed antibody response in these pigs. Initially, test results from non-M. hyopneumoniae groups (n ϭ 40 pigs) were used to evaluate the impact of alternative cutoffs on the false-positive rate for each manufacturer's assay. As described elsewhere (21), alternative cutoffs for BioChek, Eurofins Ingenasa, IDEXX, IDvet, and Hipra ELISAs were established as the mean of the results from non-M. hyopneumoniae groups plus two or three standard deviations (x ϩ 2, 3 SD), i.e., the values localized at the 97.5% and 99.7% quantiles of the normal distribution (upper tail values), respectively. Because Oxoid ELISA defined positives as results with Ͻ50% blocking, the alternative cutoffs were established as (x Ϫ 2 and 3 SDs), i.e., the values located at the 2.5% and 0.3% quantiles of the normal distribution, respectively (lower tail values). Thereafter, these alternative cutoffs were back transformed to the original ELISA units (S/P, %B, or RI), using the predict function in R.
Overall misclassification error rates (false positives and false negatives) for the six commercial M. hyopneumoniae ELISAs were determined by Poisson regression using generalized estimating equations (PROC GENMOD, SAS v.9.4; SAS Institute, Cary, NC). For the misclassification analysis, false-positive results were defined as any positive result from non-M. hyopneumoniae-inoculated pigs or from M. hyopneumoniae-inoculated pigs at Ͻ21 dpi ("suspect" results on 14 or 17 dpi were not penalized). False-negative results were defined as negative results from M. hyopneumoniae-inoculated pigs Ն21 dpi. Two pigs in the M. hyopneumoniae group did not become M. hyopneumoniae infected via inoculation or contact, i.e., were free of M. hyopneumoniae infection on the basis of M. hyopneumoniae PCR testing of lung homogenate collected at the termination of the experiment, and were excluded from the misclassification error rate analysis. The Poisson regression model used the count of misclassification responses as the dependent variable, ELISA as the independent variable, and dpi as a repeated measure, and it assumed a compounded symmetry correlation structure across dpi (22). The model goodness of fit was evaluated by residual deviance testing (chi-square test). The misclassification error rate was calculated as the exponential of beta coefficients estimated by the Poisson regression model and interpreted as the overall incident rate of misclassification errors over time. The analyses were considered statistically significant at P Յ 0.05.
(ii) Study 2. Serum samples (44 from inoculated pigs, 318 from noninoculated pigs) were tested on BioChek, IDEXX, and Hipra ELISAs. For noninoculated pigs, individual animal M. hyopneumoniae status was established by PCR testing of tracheal samples (positive or negative).
The diagnostic sensitivity and specificity of each of the three commercial M. hyopneumoniae ELISAs were estimated by logistic regression using GEE (PROC GENMOD; SAS v.9.4) with the qualitative ELISA result as the dependent variable, sample status (based on PCR testing of tracheal samples) as the independent variable, and pen as a repeated measure (1 to 46), and assumed a heterogeneous first-order autoregressive covariance-variance structure (22). Model goodness of fit was evaluated by residual deviance testing (chi-square test). Diagnostic sensitivity (or specificity) was estimated by modeling the probability of a positive (negative) ELISA result given a positive (negative) tracheal sample PCR testing result. Receiver operating characteristic (ROC) and area under the curve (AUC) analyses were based on the marginal predicted probabilities from the diagnostic sensitivity models using PROC LOGISTIC (SAS v.9.4).

RESULTS
Study 1. Clinical signs and pathological lesions at necropsy were reported in a previous study (15). In brief, no clinical signs were observed in pigs in the negative control, M. flocculare, or M. hyopneumoniae groups, and no gross lesions were observed at necropsy. In M. hyorhinis-inoculated pigs, mild front and hind limb lameness, swollen joints, rough hair coat, and loss of condition were observed in eight pigs. Two pigs in this group were humanely euthanized at 24 dpi due to anorexia and reluctance to move. Necropsy of these animals revealed polyarthritis and polyserositis. Likewise, in the M. hyosynoviae group, swollen joints (hocks) were observed in seven pigs, and increased joint fluid volume was observed at necropsy. One M. hyosynoviae-inoculated pig died during blood collection on 10 dpi.  Table 1, but there was no evidence of infection in M. flocculare-exposed pigs.
A total of 680 serum samples were collected over the course of the study (Ϫ3 to 56 dpi) from the pigs in the 5 treatment groups. For the analysis of the ELISAs, the 540 serum testing results from the non-M. hyopneumoniae inoculated pigs were used to evaluate the false-positive rates in the 6 ELISAs. Four cutoffs were considered: (i) manufacturer's cutoff with suspect considered negative, (ii) manufacturer's cutoff with suspect considered positive, (iii) cutoff based on the mean of the results from non-M. hyopneumoniae groups plus two standard deviations (x ϩ 2 SD), and (iv) cutoff based on the mean of the results from non-M. hyopneumoniae groups plus three standard deviations (x ϩ 3 SD).
As shown in Table 3, the manufacturer's recommended cutoffs resulted in zero or few false positives, especially if suspect results were interpreted as negative. Cutoffs calculated as x Ϯ 2 or 3 SD were generally less stringent than the manufacturer's cutoffs and, with exception of IDvet ELISA, resulted in more false-positive results.  Table 4. A detectable antibody was slow to develop, and its appearance was inconsistent among pigs and ELISAs. Because of limitations in the experimental design, i.e., relatively few samples from M. hyopneu-moniae antibody-positive pigs, the overall ELISA comparison was based on misclassification errors under two scenarios, i.e., with suspect results interpreted as negative or positive (Table 5). For both cases, no significant difference (P Ͻ 0.05) in the misclassification rate was detected among BioChek, IDEXX, Hipra, and Oxoid ELISAs, whereas significantly higher misclassification rates were observed in Eurofins Ingenasa and IDvet ELISAs. With suspect considered negative, the misclassification error rates for Eurofins Ingenasa and IDvet ELISAs were 2.07 and 3.28, respectively, i.e., 107% and 228% higher than the other ELISA used in the comparison. With suspect considered positive, the misclassification rates for Eurofins Ingenasa and IDvet ELISAs were 2.36 and 3.00, respectively, i.e., 126% and 200% higher than the comparison ELISAs. Study 2. M. hyopneumoniae DNA testing of tracheal samples from M. hyopneumoniaeinoculated pigs (n ϭ 10) confirmed productive infection, i.e., all M. hyopneumoniaeinoculated pigs were positive for M. hyopneumoniae DNA testing at 7 dpi and a noninoculated pen mate at 15 dpi. In noninoculated pens, M. hyopneumoniae DNA testing of tracheal samples verified transmission to other animals throughout the room over time, with the first positive at 28 dpi (Fig. 1).
Based on the results from study 1, ELISA serum testing results were interpreted based on the manufacturers' cutoffs, with suspect interpreted as a positive result for IDEXX and Hipra ELISAs. Serum samples (n ϭ 8) from inoculated pigs were antibody negative at 0 dpi, with the exception of one sample positive on BioChek ELISA. At 15 dpi, 5 of 10 serum samples were antibody positive on IDEXX and Hipra ELISAs, and 7 of 10 were antibody positive on BioChek. All M. hyopneumoniae-inoculated pigs were antibody positive on all the three ELISAs at 42 and 56 dpi. In the noninoculated pigs, all ELISAs were negative until 42 dpi, including pigs sharing the pen with the 10 M. hyopneumoniae-inoculated pigs, with the exception of one BioChek-positive sample at 15 dpi (Fig. 1). Overall, 318 serum test results from noninoculated pigs were used to estimate the diagnostic sensitivity and specificity of BioChek, IDEXX, and Hipra ELISAs. As shown in Table 6, the analyses found no statistically significant difference in diagnostic sensitivity, diagnostic specificity, or AUC among the 3 ELISAs.

DISCUSSION
The objectives of this study were to compare the performance of commercial M. hyopneumoniae serum antibody ELISAs and to evaluate cross-reactivity to other my-  Table 2). d Percent blocking (%B) (see Table 2). e Relative index (RI) (see Table 2). f This manufacturer does not include a suspect classification (see Table 2). g False-negative and false-positive results (n ϭ 4 and n ϭ 5, respectively) were from one pig in the negative control group. coplasma species under experimental conditions (study 1) and then compare the best performing assays using field samples (study 2).
Study 1 was performed with samples from cesarean-derived, colostrum-deprived (CDCD) pigs inoculated with M. flocculare, M. hyorhinis, M. hyosynoviae, or M. hyopneumoniae under experimental conditions, which were used to achieve these objectives. The use of CDCD pigs, i.e., animals raised and maintained in a highly controlled environment free of common swine pathogens (e.g., porcine circovirus type 2 [PCV2], porcine reproductive and respiratory syndrome virus [PRRSV], swine influenza A virus (SIAV), swine mycoplasmas, etc.), provided assurance that all pigs were mycoplasma  free at the beginning of the study. Moreover, using colostrum-deprived (CD) pigs is a good alternative strategy to circumvent passively acquired immunity against mycoplasmas or exposure to any mycoplasma that may occur shortly after birth. Collecting and testing serum and oral fluid specimens over the course of the observation period and lung tissue recovered from pigs at euthanasia allowed for establishing the timeline and infection status of individual animals. Serum samples of a precisely known status were then used to evaluate and compare ELISA performance.  -3 0 3 7 10 14 17 21 24 28 35 42 49 56  Suspect assumed negative 1  0  0  0  0  0  0  0  8  8  3  3  2  1  1   Animals in the M. hyopneumoniae group were intratracheally inoculated with using lung homogenate strain 232 (1.0 ϫ 10 6 CCU/ml), a well-characterized and mildly virulent isolate (23). Inoculation of pigs with lung homogenate has been extensively used to reproduce M. hyopneumoniae disease in challenge studies under controlled (13,16) and field conditions (24,25). Infection using pure M. hyopneumoniae culture resulted in similar gross lesions in previous challenge models (26,27). The timeline of productive M. hyopneumoniae infection was confirmed by PCR testing of pen-based oral fluid samples (2 pigs per pen), with the first PCR-positive sample collected at 9 dpi. At euthanasia, no lung lesions were observed, but M. hyopneumoniae was confirmed in lung homogenate by PCRs in 8 of 10 pigs. Two M. hyopneumoniae-inoculated pigs were PCR negative for lung homogenate and likewise showed no detectable antibody response on any of the ELISAs. These data supported the conclusion that these animals did not become infected either through inoculation or through exposure to M. hyopneumoniae-infected animals housed in the same room. Therefore, the analyses were based on the 8 pigs demonstrated to have become infected with M. hyopneumoniae.
In other inoculation groups, PCR testing for species-specific DNA confirmed productive infection with M. hyorhinis and M. hyosynoviae, with pen-based oral fluids first positive for M. hyorhinis and M. hyosynoviae DNA by 2 and 4 dpi, respectively (15). In contrast, exposure of pigs to M. flocculare did not produce infection, as confirmed by the negative M. flocculare DNA PCR results on oral fluids (n ϭ 280) and the absence of antibody in serum samples (n ϭ 140) evaluated (15), which represents a limitation of this study. As previously reported, productive infection by single inoculation with M. flocculare is challenging, presumably due to inefficient colonization of the respiratory tract (28)(29)(30). Regardless, serum samples from the M. flocculare group were tested on the commercial M. hyopneumoniae serum ELISAs and the data included in the analyses, but the specific question of the cross-reactivity of M. flocculare antibodies on M. hyopneumoniae serum ELISAs could not be addressed as previously reported (31).
The M. hyopneumoniae ELISA comparisons were based on 680 serum samples collected over the course of the study (3 to 56 dpi) from 50 CDCD pigs in 5 defined mycoplasma exposure groups. Two sources of variation in M. hyopneumoniae antibody detection were apparent: pig-to-pig variability in response to exposure to M. hyopneumoniae and assay variability performance. As described by Pieters and Maes (1), M. hyopneumoniae antibody is typically detected 3 to 8 weeks after exposure but may be absent in infected animals. In the present study, M. hyopneumoniae serum antibodypositive results were observed as early as 21 and as late as 28 dpi, albeit Oxoid ELISA produced 6 suspect results (Table 4). Most typically for studies using samples of known infection status, assay performance assessment is based on receiver operating characteristic (ROC) analysis. This approach permits the calculation of diagnostic sensitivity and diagnostic specificity as a function of assay cutoff and allows for evaluations of assay performance using area under the curve (AUC) comparisons (32). In this study, ROC analysis was precluded because of the small number of M. hyopneumoniaeinfected animals. A further complication was the fact that 5 of 6 ELISAs included a suspect classification. In the field, suspect is not a viable option; animals are either M. hyopneumoniae infected or not. Therefore, the analyses were performed with suspect considered positive and suspect considered negative. For these reasons, ELISA results were analyzed by comparing alternative cutoffs to the manufacturers' recommended cutoffs. After determining the optimum cutoffs, results were evaluated in terms of misclassification error rates. Alternative assay cutoffs were calculated as the 97.5% and 99.7% quantiles of the normal distribution (upper tail values) for 5 ELISAs (BioChek, Eurofins Ingenasa, IDEXX, IDvet, and Hipra) and 2.5% and 0.3% quantiles (lower tail values) for Oxoid ELISA, as described in reference 21. A comparison of alternative cutoffs to the manufacturers' recommended cutoffs in terms of false-positive results showed that the manufacturers' recommended cutoffs were usually more stringent, i.e., produced fewer false positives for both the suspect considered positive and suspect considered negative conditions. Exceptions to this general observation were IDvet and Oxoid ELISAs (Table 3). Thereafter, the M. hyopneumoniae ELISAs were analyzed in terms of misclassification error rate based on the manufacturers' cutoffs. The misclassification error rate was calculated as the total number of false negatives and false positives among samples tested for both suspect conditions, i.e., as positive or as negative (Table 5). False positive was defined as a positive result on samples from M. flocculare, M. hyorhinis, M. hyosynoviae, and negative control groups. A false negative was defined as a negative result from the M. hyopneumoniae group at Ն21 dpi. Thus, misclassification error rate simultaneously accounted for both types of diagnostic errors that occur in routine testing. Misclassification errors (false positives or false negatives) were observed in all assays evaluated. No significant difference in misclassification error rate was observed among BioChek, IDEXX, Hipra, and Oxoid ELISAs (Table 5). Among these four, the performance of Oxoid ELISA differed both in the early onset of detection and in the number of false positives in samples from non-M. hyopneumoniae-inoculated pigs.
In study 2, the three ELISAs (BioChek, IDEXX, and Hipra) that provided the highest performance in study 1 were evaluated under field conditions. Specifically, M. hyopneumoniae infection was established in a commercial wean-to-finish population (1,250 pigs in 46 pens in one room) free of M. hyopneumoniae, porcine reproductive and respiratory syndrome virus, and influenza A virus infection by intratracheal inoculation of 10 pigs with M. hyopneumoniae. Thereafter, the collection and DNA testing of tracheal samples from all pens over time was used to establish the M. hyopneumoniae infection status of individual animals and compare serum ELISAs.
The control and/or elimination of M. hyopneumoniae from commercial production systems requires ongoing testing to establish the true status of populations and detect the introduction of the pathogen. In study 1 (CDCD pigs), the three ELISAs (BioChek, IDEXX, and Hipra) with the best performance were equivalent when comparing diagnostic specificity and false-positive rates. Notably, IDEXX and Hipra performed best by interpreting suspect results as positive. Likewise, in study 2, although the BioChek ELISA produced 2 false-positive results based on DNA testing, no statistically significant differences were detected in the diagnostic sensitivity or specificity of BioChek, IDEXX, and Hipra ELISAs. The point of conflict in the monitoring process has been (and continues to be) maximizing early detection while minimizing false-positive reactions. Future research should focus on improving diagnostic methods in order to be able to improve time to detection and overall diagnostic sensitivity. For the present, the data reported in this study will help users understand ELISA performance and select the assay (or combination of assays) most suited to their testing objective(s).