The Quansys multiplex immunoassay for serum ferritin, C-reactive protein, and α-1-acid glycoprotein showed good comparability with reference-type assays but not for soluble transferrin receptor and retinol-binding protein

The Quansys multiplex (Q-Plex) measures ferritin (Fer), soluble transferrin receptor (sTfR), C-reactive protein (CRP), α-1-acid glycoprotein (AGP), and retinol-binding protein (RBP). We compared Q-Plex results with reference-type assays and evaluated Q-Plex performance. Pearson correlation and Lin’s concordance coefficients between the Q-Plex and reference assay were: Fer 0.98 and 0.91, sTfR 0.88 and 0.35, CRP 0.98 and 0.98, AGP 0.82 and 0.81, and RBP 0.68 and 0.31, respectively. The median relative difference between the Q-Plex and reference assay were: Fer -2.4%, sTfR 107%, CRP 0.03%, AGP -1.3%, and RBP 51%. The Q-Plex intra-assay CVs were <5%; the inter-assay CVs were higher: Fer 11%, sTfR 14%, CRP 9.3%, AGP 7.5%, and RBP 19%. EDTA plasma produced 74% higher Q-Plex sTfR concentrations compared to serum. Analyte stability was good for ≤5 freeze-thaw cycles. After adjusting Q-Plex data to the reference assays, sensitivity and specificity were >85% for Fer and CRP; specificity was >85% for sTfR, AGP, and RBP. Using performance criteria derived from biologic variation, Fer, CRP, and AGP met the minimum allowable imprecision (<10.7%, <31.7%, and <8.5%, respectively) and difference from the reference assay (<±7.7%, <±32.7%, and <±10.3%, respectively), while sTfR and RBP exceeded these thresholds (<8.5% and <7.8% for imprecision and <±7.7% and <±12% for difference, respectively). The Q-Plex measures multiple biomarkers simultaneously, is easy to perform, and uses small sample volumes. With some improvements in accuracy and precision (i.e., sTfR and RBP), this assay could be a useful tool for low-resource laboratories conducting micronutrient surveys for epidemiologic screening applications. These findings need to be verified using other populations, particularly those with inadequate micronutrient status.


Introduction
Iron and vitamin A deficiency have negative consequences for human health and child development [1,2]. Biomarkers such as serum ferritin (Fer), soluble transferrin receptor (sTfR), and retinol or retinol-binding protein (RBP) are measured to assess population nutritional status. RBP and retinol generally circulate at one to one molar ratio [2]. RBP is sometimes used as a less expensive surrogate for retinol [2]. Inflammation markers such as C-reactive protein (CRP) and α-1-acid glycoprotein (AGP) are also measured to interpret inflammation-dependent nutritional biomarkers [3]. Often adequate laboratory facilities, equipment and supplies, and properly trained staff are lacking to conduct biomarker analyses using widely-accepted laboratory methods and survey samples need to be shipped out of country for analysis, unless there is a restriction for sample export. Furthermore, the analysis of multiple biomarkers by individual assays is costly and requires relatively high sample volumes and each run takes several hours per biomarker (Table 1). Access to reliable, easy-to-use, and affordable assays is needed to accurately assess the population nutritional status in-country.
To address this problem and allow low-and-middle income countries to become self-sufficient in conducting micronutrient surveys, companies have attempted to develop commercially available 'all-in-one' instrument platforms that conduct multiple micronutrient tests in a single sample aliquot [4][5][6]. These instruments need to be inexpensive, of low complexity, and operable by a laboratory technician without requiring specialized training. As importantly, the data produced by these platforms need to be of sufficient quality, reproducibility, and validity for interpretation.
An early version of a multiplex assay for 5 micronutrients (Fer, sTfR, CRP, AGP, and RBP), the Q-Plex Human Micronutrient (5-plex) protein microarray [7], was published in 2014 [6]. In 2017, an expanded version 7-plex including thyroglobulin and HRP2 for malaria was presented [8] and recently the group applied this assay to dried blood spot matrices [9]. The 7-plex microarray was validated against a laboratory-developed test (LDT) sandwich-ELISA from the VitMin Lab [10], which is widely used in micronutrient surveys, using 206 heparinized plasma samples from Nigerian pregnant women [8]. In another comparison of these 2 assays (7-plex and LDT) using 180 serum samples from women and children from Burkina Faso, Cambodia, and Malaysia, poor comparability between the methods was reported [11].
We compared the Quansys 5-plex array, a non-diagnostic research tool, to results obtained with well-established and widely-used clinical commercial assays and for RBP to retinol by HPLC. We consider these particular methods as reference-type assays because they are used as part of the National Health and Nutrition Examination Survey (NHANES) to assess micronutrient status in the US population and/or have been shown to produce result close to the target values of international reference materials. Given that countries often desire to compare their national survey results to NHANES, it is helpful to understand the relationship between the Quansys assay and these CDC assays used in NHANES. Thus, we provide conversion equations from the Q-Plex assay to the reference-type assays. We also evaluated the Q-Plex assay for key method performance parameters and selected pre-analytical factors.

Biological specimens and reference materials
Eighty-five anonymous serum specimens from adult male and female blood donors were purchased from two U.S. commercial blood banks Tennessee Blood Services (Memphis, Tennessee) and BiolVT (Westbury, NY). Of these, 25 specimens had matrix-matched pairs of serum, heparinized plasma (HEP-P) and EDTA plasma (EDTA-P). We obtained the following international reference materials: 94/572 3 rd international standard for recombinant ferritin and 07/202 recombinant soluble transferrin receptor reference reagent from the National Institute for Biological Standards and Control (NIBSC); ERM-DA474/IFCC for CRP (spiked) and ERM-DA470k/IFCC for AGP from the European Reference Materials, Institute for Reference Materials and Measurements; and Standard Reference Material (SRM) 968e for retinol from the NIST.

Statistical analysis
Due to assumption violations (non-constant difference and non-constant variance) with the standard Bland-Altman limits of agreement (LoA) method, we employed a variation of the LoA method to assess agreement between 2 assays (S3 Text). We used linear regression of the difference between the assays on the average of the assays to derive prediction equations from the test assay to the reference assay [12]. Where appropriate (i.e., Fer, sTfR, CRP, and RBP), we log (natural) transformed the data to address the assumption violations. In these cases, after back-transforming the model estimates, the relationship between the 2 assays and the prediction limits were non-linear. We used the prediction equations to calculate predicted values and 95% prediction intervals at selected measured values (minimum, 25 th , 50 th , and 75 th percentiles, and maximum as measured by the reference assay). To minimize the impact of influential points on the prediction equations and subsequent analyses, we removed 4 data pairs (Fer: n = 2; CRP n = 1, sTfR: n = 1). The criterion for removal was based on the calculated Cook's D (distance) for each data value from a regression of the difference between the assays on the average of the assays. Cook's D measures the effect of omitting the data pair on the estimated regression coefficients. The Cook's D of each of the data pairs removed from the analysis all exceeded 5 times the traditional cutoff for Cook's D (4/n) A total of 77, 72, 83, 85, and 85 serum samples were used in this analysis for Fer, CRP, sTfR, AGP, and RBP. Additionally, we used a non-parametric approach to describe the agreement between the 2 methods by reporting the proportion of the relative differences that fall within selected limits (e.g., within ±5% of the reference assay). P values �0.05 were considered statistically significant.
To assess the Q-Plex assay difference to each reference assay, we calculated the median relative difference across serum samples. We assessed the acceptability of the difference by comparing to the minimum allowable difference based on biologic variation: difference = 0.375 � (within-individual CV 2 +between-individual CV 2 ) 1/2 [13]. For objective quality goals for method performance, see S2 Table. We evaluated the diagnostic characteristics of the Q-Plex assay by calculating sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) using results from the reference assay as a gold-standard and commonly used cutoff values to define deficiency or inflammation: <15 μg/L for Fer [1], >5.3 mg/L for sTfR [14], >5 mg/L for CRP [3], >1 g/L for AGP [3], and <0.7 μmol/L for RBP [2].
To assess the Q-Plex assay imprecision, we calculated the mean CV across 4 serum samples for 5 replicates measured in 1 run (intra-assay CV) and for 1 replicate measured in 10 runs (inter-assay CV). The acceptability of the imprecision was judged by comparing to the minimum allowable imprecision based on biologic variation: analytical CV = 0.75 � within-individual CV [13].

Accuracy and diagnostic characteristics of Q-Plex based on comparison to reference assays
The concentration range of the 85 serum samples as measured by the reference assays covered normal and abnormal values, except for retinol, where none of the samples had low retinol concentrations <0.7 μmol/L ( Table 2). For Fer, sTfR, and CRP we had incomplete sample sets. The Q-Plex assay had 5, 1, and 12 no reportable results for Fer, sTfR, and CRP, respectively.  Table 2).
We established conversion equations between the Q-Plex assay to reference-equivalent assay (S3 Table). We applied the conversion equations to selected biomarker values (S4 Table). For example, for a measured value of Fer by the Q-Plex assay of 13.6 μg/L there is 95% probability that the reference-equivalent value of Fer is between 13.0 μg/L and 26.7 μg/L. A graphical display of the conversion equations and 95% prediction limits are shown in Fig 1. The median relative difference between the Q-Plex and Roche assay was: Fer -2.4%, sTfR 107%, CRP 0.03%, AGP -1.3% ( Table 2). The median relative difference between the Q-Plex Table 2. Agreement between Q-Plex and reference assay for serum samples a . RBP and HPLC retinol assay was 51%. Compared to the allowable difference based on biologic variation, Fer (±7.7%), CRP (±33%), and AGP (±10%) achieved, while sTfR (±7.7%) and RBP (±12%) exceeded the minimum threshold. A different way to look at this is by calculating the percentage of Q-Plex results that agreed with the reference assay results within certain agreement limits (S5 Table). For Fer, nearly 1/3 of samples agreed within ±10%, while close to 2/3 agreed within ±30%. For CRP and AGP, nearly 50% of samples agreed within ±10% and >90% agreed within ±30%. For RBP and sTfR there were consistent assay differences; onlỹ 30% and~2.5% of samples agreed within ±30%, respectively. The original sensitivity and specificity of the Q-Plex Fer (100% and 85.7%, respectively) and CRP (93.8% and 98.2%, respectively) assays was good and changed only slightly for Fer and not for CRP when we used predicted values derived from the prediction equations (Table 3). For sTfR and AGP, using predicted values led to big improvements: the original specificity for sTfR increased from 7.55% to 86.8% (sensitivity decreased); the original sensitivity for AGP increased from 44.4% to 61.1% (specificity decreased slightly). Because our sample set did not include samples with low retinol concentrations, we could not evaluate the sensitivity, but the original and predicted specificity of the Q-Plex RBP assay was 100%.

Effect of specimen matrices
While some matrix differences were statistically significant (P <0.05), they were not biologically relevant and all were within ±5% of the serum results (Table 6). However, the Q-Plex sTfR results for EDTA-P were 74% higher than serum results. The Roche sTfR assay did not show such an effect.

Freeze-thaw stability of Q-Plex
For serum samples, we observed for all analytes acceptable differences of �±10% compared to the reference condition for up to 5 freeze-thaw cycles, with AGP showing a small but consistent positive difference (S6 Table). For HEP-P samples, we observed slightly larger differences of �±17% compared to the reference condition, with Fer and sTfR showing consistent positive differences and CRP, AGP, and RBP showing consistent negative differences. For EDTA-P samples, nearly all results were within ±10% compared to the reference condition, but we observed consistent negative differences for sTfR, CRP, and RBP. The sTfR results in EDTA-P were again much higher than in serum (62%), but they did not increase with additional freeze-thaw cycles.

Effect of incubation temperature
When we subjected serum, HEP-P, and EDTA-P samples to an elevated incubation temperature of 30˚C compared to our room temperature of 18˚C (suitable per manufacturer), the Q-Plex assay produced similar results for CRP (5.8% higher), slightly lower results for Fer (-8.6%), much lower results for AGP (-39%) and RBP (all results were <LOD and could not be calculated), and much higher results for sTfR (88%) (S7 Table). Moreover, the calibration curves at 30˚C displayed different shapes and the background noise was increased, resulting in lower assay sensitivity. As such, we had to exclude 3 samples with low Fer and 2 samples with low CRP concentrations from this experiment, because results could not be calculated after the 30˚C incubation.

Discussion
This study is to our knowledge the first to compare the Quansys 5-plex microarray with wellestablished and validated reference-type assays. Furthermore, we carefully conducted the statistical analysis to assess the agreement between the test and reference assay, ensuring that we appropriately address assumption violations such as non-constant variance and non-constant difference. Because previous method comparison studies [8,11] used different approaches, it is difficult to compare the findings. As such, the median relative differences between the Q-Plex and reference assay in our study (Fer -2.4%, sTfR 107%, CRP 0.03%, AGP -1.3%, and RBP 51%) do not correspond well with the 2 previously reported relative differences between the Q-Plex and the commonlyused sandwich ELISA [8,11]: Fer 88% and 108%, sTfR 70% and 148%, CRP -33% and -1%, AGP -53% and -37%, and RBP -16% and 12%. Several reasons could explain these discrepancies: different antibodies with different specificities and affinities used in these comparisons; different sample sets may result in different assay relationships either due to different concentration ranges and/or due to sample composition; the statistical approach used to assess the assay agreement varies across studies and non-constant variance and/or non-constant differences may not have been addressed in previous studies; in previous studies, the authors used the slope to describe the proportional difference between the assays without giving consideration to the intercept, which in some cases was quite large. It would be interesting to see how our findings compare to the previous studies if those studies used the same statistical approach.
The prediction equations derived in this study allow the conversion of Q-Plex data to reference assay-equivalent data, but they should be considered preliminary until further confirmation. Conversion equations may be helpful in the future to allow comparison of Q-Plex assay data with US population data generated with the reference-type assays in the National Health and Nutrition Examination Survey. Previous studies that compared the Q-Plex and sandwich-ELISA assay with samples from African and Asian countries, appeared to cover similar concentration ranges to our study based on visual inspection of the scatter plots or Bland-Altman plots [8,11]. Still, the equations derived in this study should be confirmed with other sample sets, particularly from countries where analyte concentrations are different from those in the United States.
Assessing the accuracy of an assay with international reference materials is difficult due to potential commutability issues, i.e., the assay responds differently to the reference material compared to a native sample. The Roche CRP assay measured 11% lower than the certificate target value, due to the non-commutability of the ERM reference material, possibly because it is a spiked material. However, Roche has confirmed that the CRP (Gen.3) assay measures accurately for patient samples (personal communication with Guenter Trefz, Roche, 06/08/ 2016). Similarly, the Q-Plex CRP assay measured closely to the Roche assay (0.03% difference) in serum samples, but 65% higher in the ERM reference material, suggesting non-commutability of the reference material for the Q-Plex assay. Furthermore, the Q-Plex sTfR assay measured 107% higher than the Roche assay in serum samples, but 55% lower in the NIBSC reference material, possibly indicating non-commutability of the reference material for the Q-Plex assay. The Q-Plex RBP assay measured 51% higher than the HPLC retinol assay in serum samples and 29-37% higher in the NIST reference material, a difference of similar magnitude, suggesting that the reference material is likely commutable for the Q-Plex assay. Another difficulty in interpreting data from reference materials is the lack of assay standardization. The Roche sTfR assay has not been calibrated to the NIBSC reference material and measures much higher (174%) than the certificate target value [15]. However, the assay has largely remained stable over the last 10 years, still producing the same value as during the original material characterization in 2008.
The performance of the Q-Plex CRP assay was best among the 5 analytes, meeting the performance criteria for precision and difference to the reference assay, as well as showing high (>90%) sensitivity and specificity before and after adjusting the data with the prediction equation.
The Q-Plex Fer assay performed second best, meeting the performance criteria for precision (borderline) and difference to the reference assay, as well as both showing high (>85%) sensitivity and specificity before and after adjusting the data with the prediction equation. The relationship between the 2 assays was most complicated for Fer because it was non-linear and showed increasing variance. Even though the Q-Plex assay measured lower at low Fer and higher at high Fer concentrations compared to the Roche assay, the diagnostic performance of the assay was still satisfactory.
The Q-Plex AGP assay met the performance criteria for precision and difference to the reference assay, showed high specificity (>85% before and after adjustment), but showed only moderate sensitivity (44.4% before and 61.1% after adjustment).
The Q-Plex sTfR assay did not meet the performance criteria for precision and difference to the reference assay and showed poor specificity (7.55%) due to an apparent calibration difference; after adjusting the data with the prediction equation, the specificity improved (>85%) at the cost of a loss in sensitivity (100% to 73.3%). An improvement in sTfR precision would be desirable.
Lastly, the Q-Plex RBP assay did not meet the performance criteria for precision and difference to the reference assay, and we could not assess the sensitivity of the assay because our sample set did not contain samples with low retinol concentrations. Given that the difference to the reference assay was of similar magnitude for serum samples and the NIST reference material, it may be possible to adjust the Q-Plex assay. However, an improvement in RBP precision would still be necessary. Because our comparison of RBP to retinol is a combined biomarker validity/analytical comparability assessment, it is hard to separate between the methodological and the potential physiologic components. However, our ultimate goal was to assess how the RBP Q-Plex assay compared to serum retinol, which is currently the generally accepted biomarker for vitamin A deficiency.
While the method comparison information is the center-piece of this paper, we also tested relevant pre-analytical factors that users need to be aware of. We confirmed that serum samples cannot be diluted less than 1:10 and that a 1:20 dilution leads to >±15% deviation for sTfR and RBP. We showed that EDTA-P is not a suitable sample matrix for the Q-Plex assay because of invalid high sTfR results. A similar effect has been observed previously with other ELISA sTfR assays [16] and is possibly due to an interference of EDTA with the antibody. If serum cannot be obtained, HEP-P is a suitable alternative. All 5 analytes showed good stability for �5 freeze-thaw cycles, with serum showing less difference from the reference condition than the 2 plasma matrices. The Q-Plex assays did not show any notable interference from repeated freeze-thawing. Lastly, we showed that elevated temperature (30˚C) during the performance of the Q-Plex assay is problematic, as it leads to invalid high sTfR, low AGP, and non-detectable RBP results.
In conclusion, the Quansys 5-plex microarray has a number of advantages compared to conventional laboratory assays that makes it attractive for low-resource settings: it is easy to perform, requires minimal analyst training, needs only a small sample volume, is relatively inexpensive, and measures all 5 biomarkers at the same time (Table 1). However, some improvements in accuracy and precision are still desirable and the inclusion of quality control materials in the kit is needed so that the user can monitor consistency and assay performance.  Table. Selected conversion equations between the Q-Plex and reference assay for serum samples. AGP, α-1-acid glycoprotein; CRP, C-reactive protein; Fer, ferritin; RBP, retinolbinding protein; sTfR, soluble transferrin receptor; Roche clinical analyzer assays used as reference assays for Fer, sTfR, CRP, and AGP; retinol measured by HPLC used as reference assay for RBP; prediction error (PE) is provided in parentheses and can be used to construct 95% prediction intervals for a selected value x using ± t n-1,0.025 PE, where t n-1,0.025 is the 97.5 th percentile from the Student t distribution with n-1 degrees of freedom. (DOCX) S4 Table. Predictions and 95% prediction intervals from conversion equations. Results in this table can be interpreted as follows: for a future measured value by assay x there is a 95% probability that the future measured value by assay y would be contained in the presented prediction interval. AGP, α-1-acid glycoprotein; CRP, C-reactive protein; Fer, ferritin; RBP, retinol-binding protein; sTfR, soluble transferrin receptor; Roche clinical analyzer assays used as reference assays for Fer, sTfR, CRP, and AGP; retinol measured by HPLC used as reference assay for RBP. (DOCX) S5 Table. Percentage of Q-Plex serum sample results that agree with the reference assay results within selected limits. AGP, α-1-acid glycoprotein; CRP, C-reactive protein; Fer, ferritin; RBP, retinol-binding protein; sTfR, soluble transferrin receptor; Roche clinical analyzer assays were used as reference assays for Fer, sTfR, CRP, and AGP; retinol measured by HPLC was used as reference assay for RBP. (DOCX) S6 Table. Q-Plex freeze-thaw stability of serum, heparin plasma, and EDTA plasma samples. AGP, α-1-acid glycoprotein; CRP, C-reactive protein; Fer, ferritin; RBP, retinol-binding protein; sTfR, soluble transferrin receptor; 5 serum, heparin plasma, and EDTA plasma samples subjected to up to 5 freeze-thaw cycles (3 h at room temperature/cycle), then analyzed together with the reference samples (no freeze-thaw cycle) in the same run; samples stored at -70˚C when not in use. Mean concentration across 5 samples; the SD estimates the variability at the reference condition for each matrix. Percent difference to reference condition was calculated for each sample and then averaged across 5 samples. (DOCX) S7 Table. Q-Plex results comparing incubation at 30˚C with incubation at 18˚C. AGP, α-1-acid glycoprotein; CRP, C-reactive protein; Fer, ferritin; RBP, retinol-binding protein; sTfR, soluble transferrin receptor. Mean concentration across~35-40 samples consisting of about 1/ 3 serum, 1/3 heparin plasma, and 1/3 EDTA plasma samples; the SD estimates the variability at the 2 incubation temperatures. Percent difference to incubation temperature of 18˚C was calculated for each sample and then averaged across all samples. Analysis is based on same number of samples analyzed at both incubation temperatures.  Table. (XLSX) S7 Data. Incubation temperature_S7 Table. (XLSX)