Comparison and clinical utility evaluation of four multiple allergen simultaneous tests including two newly introduced fully automated analyzers

Background We compared the diagnostic performances of two newly introduced fully automated multiple allergen simultaneous tests (MAST) analyzers with two conventional MAST assays. Methods The serum samples from a total of 53 and 104 patients were tested for food panels and inhalant panels, respectively, in four analyzers including AdvanSure AlloScreen (LG Life Science, Korea), AdvanSure Allostation Smart II (LG Life Science), PROTIA Allergy-Q (ProteomeTech, Korea), and RIDA Allergy Screen (R-Biopharm, Germany). We compared not only the total agreement percentages but also positive propensities among four analyzers. Results Evaluation of AdvanSure Allostation Smart II as upgraded version of AdvanSure AlloScreen revealed good concordance with total agreement percentages of 93.0% and 92.2% in food and inhalant panel, respectively. Comparisons of AdvanSure Allostation Smart II or PROTIA Allergy-Q with RIDA Allergy Screen also showed good concordance performance with positive propensities of two new analyzers for common allergens (Dermatophagoides farina and Dermatophagoides pteronyssinus). The changes of cut-off level resulted in various total agreement percentage fluctuations among allergens by different analyzers, although current cut-off level of class 2 appeared to be generally suitable. Conclusions AdvanSure Allostation Smart II and PROTIA Allergy-Q presented favorable agreement performances with RIDA Allergy Screen, although positive propensities were noticed in common allergens.


Introduction
The detection of allergen-specific IgE, along with the patient's chief complaints and medical history, is diagnostically valuable for allergic diseases, such as allergic rhinitis, atopic dermatitis, and asthma [1,2]. Although in vivo skin test has been traditionally used in the clinical environments, there are several limitations of in vivo skin test including error-prone results Contents lists available at ScienceDirect journal homepage: www.elsevier.com/locate/plabm

Inter-method comparison of four analyzers
We compared a pair of analyzers each time in order to maximize the comparison efficiency because different allergen lists are available by four analyzers. Furthermore, we focused on comparison of two specific analyzers (AdvanSure Al-loScreen versus AdvanSure Allostation Smart II), because AdvanSure Allostation Smart II is the upgraded version of Ad-vanSure AlloScreen, both of which are developed by the same manufacturer (LG Life Science). Afterwards, we compared two newly introduced analyzers (i.e. AdvanSure Allostation Smart II and PROTIA Allergy-Q) with currently widely utilized assay (RIDA Allergy Screen) as reference values.

Comparison among different cut-off levels for positive interpretation
No standardized specific cut-off level for positive result is defined worldwide until today [17]. Moreover, previous studies which compared various MAST assays utilized different cutoff levels. For instance, several studies used class 1 as the cutoff level for positive results [8,11,12], whereas class 2 was adopted as the cutoff level for positive results in other studies [6,10,32]. Considering the natural characteristics of semi-quantitative results in MAST assays, comparison of different cut-off levels in the paired results might provide clinical clues for more precise diagnostic interpretation. Therefore, we applied cutoff levels of class 1, class 2, and class 3 as minimal requirement for positive results for all comparison analyses.

Statistical analysis
We analyzed the concordance degree by calculating total agreement percentage following the same methodology used in a previous study [18]; total agreement percentage ¼(total number of results Ànumber of discrepancies) Â 100/total number of results. Additionally, concordant positive rates were calculated with the proportions of agreement for positive responses because low frequency of positive results can affect the total agreement percentage. Furthermore, agreement of detection results between two analyzers was determined by Cohen's kappa analysis [19]. Finally, the presence of propensity toward positive results in specific assay for certain allergen was determined when the difference between discrepant results accounted for over 10% of all pairs. For example, when assay A and assay B are compared for allergen C, [(number of samples with A positive, B negative result) À (number of samples with A negative, B positive result)] Â 100/ total number of results Z10% can be interpreted as the positive propensity of assay A for allergen C compared to assay B.

Characteristics of study participants and paired sets
The serum samples from a total of 53 and 104 patients were tested for food panel and inhalant panel in this study. Characteristics of study participants are summarized in Table 2. Although several patients presented multiple allergic

Comparison between AdvanSure AlloScreen and AdvanSure Allostation Smart II for evaluation of upgrade
When we compared qualitative results between AdvanSure AlloScreen and AdvanSure Allostation Smart II, we used class 2 as the cut-off level for positive result since the manufacturer suggested the possibility of class 1 result indicating insufficient clinical significance to trigger allergic progression. A total of 43 and 90 paired serum samples were tested for 39 and 41 allergens in food and inhalant panel, respectively (Table 3). All allergens showed total agreement percentages over 93.0% and 92.2% in food and inhalant panel, respectively, which indicates good concordance between old and new versions of AdvanSure assays. However, 6 allergens (Candida albicans, cheddar cheese, chicken, Cladosporium herbarum, pork, yeast) in food panel and 4 allergens (dog, egg white, mackerel, soy bean) in inhalant panel showed no concordant positive result, possibly due to rare frequency of specific IgE antibodies to these allergens among Koreans and restricted number of paired samples in this study. On the contrary, two most common allergens in both food and inhalant panels which were Dermatophagoides pteronyssinus and Dermatophagoides farina showed high total agreement percentages of over 95.0% and high agreement levels with kappa indices over 0.9. However, total agreement percentage and kappa index decreased to 93.0% and 0.8, respectively, for house dust, which was the third most common allergen.

Comparison of AdvanSure Allostation Smart II or PROTIA Allergy-Q with RIDA Allergy Screen applying cut-off level of class 2
We evaluated concordance rate of two newly developed fully automated assays (i.e. AdvanSure Allostation Smart II and PROTIA Allergy-Q) with results by RIDA Allergy Screen considered as the reference values in this study utilizing class 2 for the cut-off level for positive result (Table 4). Total agreement percentages were over 90.0% in most allergens in both assays for food and inhalant panels. However, allergens with the most frequent positive results (i.e. D. farina, D. pteronyssinus, house dust, and storage mite) presented concordance rates ranging from 69.6% to 90.0% for both AdvanSure Allostation Smart II and PROTIA Allergy-Q in food panel as well as inhalant panel.
Furthermore, several allergens which showed propensity toward positive result in specific assay were noticed in both comparison analyses (Table 5). While AdvanSure Allostation Smart II and PROTIA Allergy-Q showed positive propensity for

Effects of lowering or raising up the cut-off level for positive result
To evaluate the effects of various cut-off levels for positive result determination, we applied two more cut-off levels other than the conventional criteria of class 2 as minimal requirement for positive result; class 1 and class 3 as cut-off levels. Total agreement percentages and concordant positive rates were fairly influenced by application of both higher and lower cut-off levels ( Figs. 1 and 2). Since higher cut-off level led to more negative results, concordant positive rates decreased naturally. However, the changes of total agreement percentage according to the increase in cut-off level varied among allergens by different analyzers.

Evaluation of positive rates for unique antigens in specific analyzer
Since different analyzers include various allergens, analyzer-specific allergens present diverse frequencies among patients ( Table 6). Among the accretional allergens introduced in Advansure Allostation Smart II, Acarus siro and apple in inhalant panel showed significant positive rates of 20.0% and 8.9%, respectively, when class 1 was utilized as cut-off level. When cut-off level was increased to class 2, these positive rates decreased to 12.2% and 4.4%, respectively (data not shown).

Discussion
During the last decade, there have been several remarkable introductions of new MAST assays by different manufacturers into the clinical field of allergic diseases. Accordingly, evaluation and comparison studies of these novel MAST analyzers were reported by few groups [6,8]. Until today, a total of four MAST assays [i.e. AdvanSure AlloScreen, RIDA Allergy Screen, MAST Optigen (Hitachi), and Polycheck (Biocheck)] were frequently evaluated with each other and showed comparable clinical performances [10,11,20]. Recently, Lee and colleagues presented favorable performance of newly developed PROTIA Allergy-Q [12]. In this current trend, we evaluated four MAST analyzers including two newly developed and fully automated assays. This study is the first evaluation report for AdvanSure Allostation Smart II and only the second comparison study for PROTIA Allergy-Q. Also, our study is unique for evaluating upgraded version of specific assay to ensure the improvement by including both AdvanSure AlloScreen and AdvanSure Allostation Smart II.
Based on our results, most results of comparison analyses presented good concordance levels by means of total agreement percentages over 90.0%. Satisfactory agreements were observed not only in the comparison between AdvanSure AlloScreen and AdvanSure Allostation Smart II, but also in the evaluation of AdvanSure Allostation Smart II and PROTIA Allergy-Q compared with RIDA Allergy Screen. Although four allergens with the most frequent positive results, which were D. farina, D. pteronyssinus, house dust, and storage mite, showed slightly lower concordance rates, these different results could be sufficiently overcome by careful interpretation of MAST results in association with clinical manifestations.
One interesting finding we focused on in this study was positive propensity of each analyzer. In the midst of various available MAST analyzers with comparable diagnostic performance, it is important for laboratory physicians to recognize the unique propensity of each analyzer which might easily lead to positive results for particular allergen. Our study suggests that AdvanSure Allostation Smart II and PROTIA Allergy-Q are more sensitive or prone to report positive results for three common allergens (i.e. D. farina, D. pteronyssinus, and house dust) in both food and inhalant panels than RIDA Allergy Screen. Considering the multiple positive result frequencies related with cross-reactivity among similar allergens as possible mechanism for explanation [21][22][23], positive propensity of each analyzer should be cautiously understood. Moreover, variations in the allergen extraction method by different manufacturers might have caused this phenomenon of diverse positive propensity in each analyzer. Adding new allergens in the panel list is another issue for future development of MAST analyzers. Candidate allergens should be assessed based on evidences for continuous and dramatic changes in the environment and socio-behavioral lifestyle of modern individuals [24,25]. At the same time, cost-effective approach is required for choices of clinically efficient allergen with reference to epidemiologic results of geographically characteristic allergen studies [26][27][28][29]. Our results support the significant positive rate for Acarus siro among Korean population [30], which is unique allergen included only in Advansure Allostation Smart II inhalant panel. Further investigations for Acarus siro as an inhalant allergen in general population might highlight the advantage of Advansure Allostation Smart II.
One of the most important approaches we performed in this study was the re-evaluation of cut-off levels in order to avoid false positive results. Besides the conventional cut-off level of class 2 as the minimal positive result criteria, we analyzed the changes of total agreement percentages and concordant positive rates according to cut-off level decrease to class 1 or increase to class 3. Although the increase of cut-off level seemed to make clinical circumstance more simple and concise by presenting only the definitive positive allergens, this modification resulted in lower concordant positive rates with possibility of missing potentially critical allergens. On the other hand, decrease of cut-off level produced lower total agreement percentages in most allergens, which might obscure physicians from clear identification of clinically relevant allergens. Detailed evaluation with similar approach for optimal cut-off class should be conducted for each analyzer according to specific regional frequency and distribution of allergens in the future.
A critical limitation of this study was the use of RIDA Allergy Screen assay as the reference value for evaluation of newly developed analyzers. Among the comparison studies published until today, most studies included the ImmunoCAP system (Phadia, Uppsala, Sweden) for comparison analyses as an empirically reference method [8,[10][11][12]20]. However, the Im-munoCAP system is neither the official nor the definite reference procedure for measurement of allergen-specific IgE antibodies despite its good reliability and reproducibility. While the ImmunoCAP system might be impractically expensive for efficient clinical service for small to medium sized clinical laboratories [31], RIDA Allergy Screen assay has been continuously evaluated and reported for favorable clinical correlation with not only the ImmunoCAP system, but also serum total IgE [10,32]. We anticipated that objective comparison between currently available MAST analyzers might provide sufficient information for clinical use in the practical medical field. In conclusion, AdvanSure Allostation Smart II maintained steady concordant performance in the upgrade process from AdvanSure AlloScreen, with the uniquely extended allergen list including Acarus siro which showed certain positive rates. AdvanSure Allostation Smart II and PROTIA Allergy-Q presented favorable agreement performances with RIDA Allergy Screen, although positive propensities were noticed in some allergens. The conventional cut-off level of class 2 as the minimal positive result criteria appeared to be suitable for current MAST analyzers in the clinical interpretation. Table 6 List of analyzer-specific allergens and frequencies of positive results using cut-off level of class 1.