Method validation of a set of 12 GEM® Premier™ 4000 blood gas analyzers for point-of-care testing in a university teaching hospital

Background Blood gas analyzers are o0.ften integrated into point-of-care testing provisions. International standards (ISO 22870 and 15189) as adapted to French COFRAC regulations make accreditation of point-ofta-care testintag obligatory. We installed and assessed 12 GEM PREMIER 4000 analyzers for pH, pCO2, pO2, Na+, K+, Cl-, Ca2+, lactate, hemoglobin and oxyhemoglobin (O2Hb) at Clermont-Ferrand Hospital. These instruments were distributed across 11 care sites in the hospital. Methods Precision was studied at two control levels for each parameter. Comparisons between GEM analyzers were performed (on 30 samples) for pH, pCO2, pO2, Na+, K+, Cl-, Ca2+, lactate, hemoglobin and O2Hb; and between GEM analyzers and the central laboratory for Na+, K+, Cl-, Ca2+ and hemoglobin (on 30–50 samples). Uncertainty in measurement (UM) was evaluated with an approach using reproducibility and accuracy data. Results The coefficients of variation (CVs) were in line with recommendations, except for the repeatability CV for pO2. All CVs were below 4%. All comparisons complied with recommendations. Uncertainties of measurement were also validated. Conclusion Our results met standard requirements and the 12 analyzers were assessed as suitable for point-of-care testing in services of academic medical centers, as exemplified at Clermont-Ferrand hospital.


Introduction
In France, all biomedical analysis laboratories are governed by the standardization and accreditation regulations set by COFRAC to meet the requirements of the standard EN ISO 15189. However, point-of-care testing (POCT), governed by the standard EN ISO 22870, also requires accreditation by 2020. Although biomedical laboratories have moved forward in the accreditation of analyses carried out in the laboratory, currently very few of them are accredited for POCT. Although Normes Françaises (NF) 22870 is an extension of 15189, and shares many of its requirements, there are some additional requirements specific for POCT. For example, an oversight committee must be created and established prior to installing POCT equipment [1,2]. The training and accreditation of the

Analyzers
The GEM PREMIER 4000 analyzers (Werfen, Le Pré-Saint-Gervais, France) are cassette blood gas analysis instruments. Proton concentration [H+] (for Hydrogen potential (pH)), partial pressure of CO 2 (pCO 2 ), sodium (Na + ), potassium (K + ), chloride (Cl -) and ionized calcium (Ca 2+ ) levels are measured by potentiometry; partial pressure of O 2 (pO 2 ) and lactate are measured by amperometry; hemoglobin (Hb) and oxyhemoglobin (O 2 Hb) by optical absorbance. Twelve analyzers were deployed over 11 sites of the Clermont-Ferrand university teaching hospital system between February and June 2015. At these 11 deployment sites, each analyzer was assigned a name according to the clinical ward where it was located ( Fig. 1). At Gabriel Montpied hospital, the instruments were located in the medical intensive care unit (MICU or G2), surgical intensive care unit (SICU or G3), neurological intensive care unit (NICU or G4), cardiovascular surgery intensive care unit (CSICU or G5), cardiovascular surgery operating theater (CVS or G6), medico-technical center (MTC or G7), emergency department (ED or G8), and sport medicine and functional exploration department (SMFED or G9). A backup instrument (BACKUP or G1) was also installed at the SICU. At Estaing hospital, the instruments were installed in the adult intensive care unit (aICU or G10), pediatric intensive care unit (pICU or G11) and maternity ward (MATER or G12). To allow for different specific clinical requirements, not all the analytical values were available on all the analyzers (Fig. 1).

Repeatability
Tests to evaluate repeatability were carried out using vials of two distinct levels of controls supplied by Werfen. These were titered aqueous solutions, with physiological analyte concentrations for level 2 and pathological concentrations for level 3, designated respectively GEM System Evaluator 2 (GSE2) and 3 (GSE3). For all the instruments, 30 vials of GEM System Evaluator 2 (Reference no. 00025000102) and 30 vials of GEM System Evaluator 3 (Reference no. 00025000103) were analyzed. The mean of the expected values for our different parameters in solutions GSE2 and GSE3 are presented in Table 1.

Reproducibility
The reproducibility of the GEM PREMIER 4000 analyzers was assessed independently as soon as the cartridge was loaded (6)(17) (18). Three internal quality control solutions (A B and D) were programmed for measurements at regular time intervals. For [H + ], pCO 2 , pO 2 , Na + , K + , Ca 2+ and Cl -, assays were carried out on solutions A and B. The assays of lactate, hemoglobin (Hb) and oxyhemoglobin (O 2 Hb) were carried out on solutions A and D. The data were then retrieved and processed using the PCSar software supplied by Werfen (V2.0). The means of the expected values for the various parameters in solutions A, B and D are presented in Table 1.

Accuracy
The GEM PREMIER 4000 analyzers were enrolled in an external quality assessment (EQA) program from ASQUALAB association C. Oris et al.

Uncertainty in measurement
Uncertainty in measurement (UM) was evaluated with an approach using both reproducibility and accuracy data. For each parameter and at 2 levels (the same as those studied for reproducibility), results were calculated for concentrations and as a percentage.

Method comparison
The comparisons made between analyzers are shown in Fig. 1. First, so as to serve as the reference analyzer, BACKUP (G1) was compared with the automated analyzers of the central laboratory. Na + , K + , Cland lactate were assayed at the Medical Biochemistry and Molecular Biology Laboratory on a Vista®-type analyzer (Siemens, Saint-Denis, France). Ca 2+ was assayed in the same laboratory on a RAPIDPoint500® analyzer (Siemens, Saint-Denis, France). Hb was assayed on an XN® analyzer (Sysmex, Villepinte, France) in the Hematology Laboratory. pH, pO 2 , pCO 2 and O 2 Hb were only determined as point-of-care testing in the clinical departments at the Clermont-Ferrand university teaching hospital. All the analyzers were then compared with G1, except for the one in the maternity ward (G12), which was compared with G11 ( Fig. 1). For each comparison, the time interval between the two assays was negligible because analyzers were first installed side by side. The comparisons between GEM PREMIER 4000 analyzers were made using 30 samples (35 for Ca 2+ ). The comparisons between G1 and the central laboratory were made using 50 samples for Na + and Cl -, 40 samples for K + and lactate, and 30 samples for Ca 2+ . For the comparison between the central laboratory and G1, samples of whole blood were taken with a heparinized syringe (for analysis on G1) and plasma from the same blood sample was taken for the central laboratory analysis. For the comparisons between GEM PREMIER 4000 analyzers, samples in heparinized syringes were collected from SICU, MICU and NICU. All the samples were anonymized. To obtain a broader range of results for several of the parameters such as pO 2 , pCO 2 , lactates and O 2 Hb, some samples were left to age at room temperature. In parallel, jointly with the hemodialysis and cardiovascular surgery intensive care units, we collected samples from pre-and post-dialysis patients to obtain extreme values of K + and low values of Ca 2+ (post-filtered samples from extra-renal clearance with citrate [19]). Lastly, jointly with the NICU, we collected samples from patients under osmotherapy to obtain extreme values of Na + [20]. We followed the French Society of Clinical Biology (SFBC) guidelines for the spread of points over the range for each parameter [21].
A second method comparison between the 12 GEM PREMIER 4000 analyzers was made using GEM System Evaluator vials. The comparisons between GEM PREMIER 4000 analyzers were made on 43 vials (5 of level 1, 19 of level 2 and 19 of level 3), all from the same batch (batch 1512 for level 1, 2513 for level 2 and 3514 for level 3), run on all 12 analyzers.

Statistics
The data was analyzed using the VISKALI® software (Viskali ACC, Lyon, France, V5.0). For the accuracy study, we calculated means ( ± SD) of repeatability and reproducibility, together with coefficients of variation (CV) expressed as a percentage. The acceptability criteria chosen were either the desirable specifications of Ricos based on biological variation, in particular for hemoglobin in repeatability and reproducibility tests, and also K + for reproducibility [22], or the specifications of the French Society of Clinical Biology (SFBC) for all the other repeatability and reproducibility tests as well as accuracy tests [21]. The method comparison was studied by least rectangles regression and calculation of means (+ SD) of differences and ratios. The corresponding graphs were plotted. The comparison of the means was obtained after analysis of the 43 GEM System Evaluator vials on each of the 12 analyzers, and were made with an ANOVA test. Statistical significance of differences was set at p < 0.05.

Accuracy
The bias values obtained met SFBC requirements for [H + ], pO 2 , pCO 2 , Na + , K + , Cl -, Ca 2+ , lactate and O 2 Hb (Table 4). The bias values for Hb were considered as "good" by ASQUALAB. Table 2 Repeatability studies on 12 GEM PREMIER 4000 analyzers (G1-G12) at 2 levels. The results are presented as coefficients of variation (%). Thirty vials of GEM System Evaluator 2 and 30 vials of GEM System Evaluator 3 were analyzed.  Result meets desirable specifications of Ricos et al. [22] after conversion of a reproducibility objective into a repeatability objective according to SFBC recommendations [21] (application of the formula CV repeatability = CV reproducibility * 0.75).

C. Oris et al.
Practical Laboratory Medicine 10 (2018) 21-33 Table 3 Reproducibility study on the 12 GEM PREMIER 4000 analyzers (G1-G12) on two levels. The results are presented as coefficients of variation in percentage. The reproducibility is evaluated independently as soon as the cartridge is loaded. Three internal quality control solutions were programmed to be measured at regular time intervals.             Table 6 Method comparisons between 12 GEM PREMIER 4000 analyzers (G1-G12) and between GEM PREMIER 4000 analyzers and those of the central laboratory (    (CVs) we obtained complied with the SFBC recommendations [22]. As there were no SFBC recommendations for Hb, the CV values were compared and found to be compliant with the recommendations of Ricos et al. [22]. For pO 2 , CV values for reproducibility were compliant with SFBC recommendations [21]. By contrast, the CV values for repeatability were compliant only for 3 levels out of the 24 tested; the other CV values for pO 2 conformed to the supplier's recommendations (4.6%). These supplier's recommendations were based upon guidelines published by the Clinical and Laboratory Standards Institute (Wayne, PA, USA) [23]. All the CV values were below 4%. In addition, in our external quality control surveys (ASQUALAB), no nonconformity was established for this parameter (and the others); since it is highly labile on contact with ambient air it is difficult to perform repeat determinations is. Indeed, the bias values obtained in accuracy study met SFBC requirements for [H + ], pO 2 , pCO 2 , Na + , K + , Cl -, Ca 2+ and lactate [21]. This being the case, the criteria of Vassault et al. seem difficult to meet [14,15]. The fact that all these reproducibility values, calculated from controls tested automatically on solutions carried by iQM cartridges, fulfill SFBC requirements, validates the self-monitoring strategy managed by the iQM [13,17,18]. A quantitative statement of the uncertainty in measurement (UM) is required to help interpret patients' results. The results obtained for the UM determination of all analytes met available Ricos requirements with the exception of Na + and [H + ]. For Na + , we observed all the analyzers were consistent with data published by Matar et al. [24]. For [H + ], only one value was outside Ricos' requirements, with no impact on patient's results. There are no recommendations (Ricos, literature data or other) for pO 2 and O 2 Hb. In this study, we propose 24 values of UM (12 values at each of 2 levels calculated with an approach using both reproducibility and accuracy dat a ) for the first time for these 2 parameters. Our values were conformed with the UM calculated by the long-term analytical CV method (using only external quality control material) [25] at 8.4% and 2.9% for pO 2 and O 2 Hb respectively. For method comparisons, because of the large number of analyzers in the Hospital, we opted to make comparisons using a transitive model. In this model, the BACKUP (G1) analyzer was considered as the reference GEM PREMIER 4000 instrument. It was first compared with the automated analyzers at the central laboratory (Vista®, RapidPoint® and XN®), and then with all the other GEM PREMIER 4000 analyzers. All our comparisons between GEM PREMIER 4000 analyzers proved conformant, in both analysis of graphs and processing of numerical data. The recent COFRAC recommendations prescribe simultaneous comparison using an ANOVA test of all the analyzers performing the same analysis in a laboratory. The large number of instruments in our study, the low stability of some parameters, such as pO 2 , and the geographical locations of our analyzers make it difficult to perform such a comparison with blood samples. We decided to compare all the analyzers with 516 quality control vials (43 per analyzer) at three different levels and from the same batch (no statistically significant differences).
The comparison between GEM PREMIER 4000 analyzers and automated central laboratory analyzers was made for the first time. For COFRAC, the analyses can be considered different according to whether they were performed at the central laboratory or at the point of care: neither the matrix (heparinized whole blood for POCT versus heparinized plasma or serum for the central laboratory) nor the analysis method were the same. A comparative study thus remains to be conducted so that hospital laboratory staff can confidently advise clinicians on the simultaneous interpretation of tests done at the central laboratory and at point of care. Overall, the comparisons were conformant. This is particularly important for some parameters such as blood sodium: variations in concentration can occur according to the assay method (direct or indirect potentiometry) [26]. With indirect potentiometry, the results of Na + determinations can vary owing to high blood lipid or protein levels, which modify the plasma water content and underestimate the Na + titer [26]. In addition, as emphasized when vetting users, preanalytical errors can cause discrepancies in results [14].
In conclusion, in view of the compliance of our data with standard recommendations, this work supports the use of 12 GEM PREMIER 4000 analyzers for POCT in services of academic medical centers, as exemplified by Clermont-Ferrand university teaching hospital.