Clinical precision, accuracy, number and durations of exhalations for a novel electrochemical monitor for exhaled nitric oxide

Background: Exhaled nitric oxide (FeNO) is a validated marker of eosinophilic inflammation. Fenom ProTM is a novel FDA-cleared monitor for FeNO. The American Thoracic Guidelines from 2005 recommend at least 6 s exhalation for adults and in some cases up to 10 s, and 4 s for children, and that the average of the first two valid exhalations is taken as the FeNO value. Methods: Clinical precision, 6 versus 10 s exhalations, the first versus the average of the first two valid exhalation methods comparison were evaluated for Fenom ProTM, as well as a methods comparison to the NIOX VERO® monitor. Results: The intent-to-treat population (n = 126) consisted of 83 adults, and 43 pediatric subjects with 16 subjects under 12 years of age. Clinical precision for 10 s exhalations on Fenom ProTM was excellent with a within-subject standard deviation (SD) range of 0.57–3.73 ppb and mean coefficient of variation (CV) range of 4.21% to 9.65%. The clinical precision for the separate adult and pediatric groups as well as for the 6 s exhalations were similar. The 10 and 6 s exhalation comparisons and one versus the average of two valid exhalations showed a high level of agreement. The Fenom ProTM and the NIOX VERO® monitors also demonstrated a high level of agreement with the values from the latter slightly lower (mean bias of −3.2 ppb). Conclusion: Fenom ProTM demonstrated eminently acceptable performance supporting its clinical utility. The data suggests that 6 s exhalations can be used in adults and children, and that one exhalation is adequate rather than obtaining the average of two exhalations on Fenom ProTM.


Introduction
Exhaled nitric oxide (FeNO) is an established marker of airway inflammation in allergic asthma. The major benefits of using FeNO include helping diagnose asthma, predicting and confirming the response to medications that target eosinophilic inflammation and identifying suboptimal compliance with maintenance asthma medications. The first NO analyzers used to measure FeNO were ozone chemiluminescent analyzers with a rapid enough response time to display real-time exhalation profiles of NO versus time termed 'online measurement'. A typical NO versus time profile has an initial washout phase where NO rises as the anatomical dead space is cleared followed by manual methods or software to select the steady state washout plateau which is taken as the FeNO measurement. Subsequently, the technical complexity, size and cost of chemiluminescent analyzers resulted in the development of alternative methods of NO detection predominantly based on electrochemistry. Electrochemical sensors have slower response times, and therefore are unable to be used for online measurement. Instead, a sample from a qualifying exhaled breath is taken into the sensor chamber, and a FeNO result is generated after some nominal delay.
The American Thoracic Society (ATS) and European Respiratory Society (ERS) guidelines from 2005 [1] were written principally for chemiluminescent analyzers, and included a recommendation that repeated, reproducible exhalations should be performed to obtain at least two NO plateau values that agree within 10% of each other. Exhaled NO is then calculated as the mean of two values. Additionally, the duration of exhalation was recommended to be at least 4 s for children >12 years and >6 s for children >12 years and adults. The guidelines also recognized that a 10 s exhalation could be necessary for a FeNO measure in some patients.
An ERS technical standard on exhaled biomarkers from 2016 states that the ATS/ERS guideline recommendation for two replicate measurements is still valid, although it was appreciated that if only one measurement could be performed owing to financial or other constraints that this could provide valuable data [2].
A novel electrochemical FeNO monitor, Fenom Pro TM , Spirosure Inc., Pleasanton CA, was recently cleared in the USA by the Food and Drug Administration.Version 3 of Fenom Pro TM has CE marking for commercialization in the EU, is pending approval in the USA, and was used in the studies in this report. Studies were performed for precision, agreement with the NIOX VERO ® (Circassia, Morrisville, NC, USA) and the impact of single versus replicate exhalations, as well as 6 versus 10 s exhalations, and are presented herein.

Study design
This was a cross-sectional study with a single visit conducted at three outpatient clinical sites in the USA with a planned enrollment of at least 100 patients aged 5 and above.

Primary and secondary objectives
The primary objective was to evaluate the clinical precision of the Spirosure Fenom Pro TM device in adult and pediatric subjects. The secondary objectives were to compare exhalations of 6 and 10 s for the Fenom Pro TM device, to perform a method comparison of the Fenom Pro TM and the NIOX VERO ® devices, and to compare the first measure versus the average of two FeNO measures for the Fenom Pro TM device.

Study devices
The Fenom Pro TM and NIOX VERO ® devices both employ electrochemical sensing technology.

Principal inclusion and exclusion criteria
A study population was planned of approximately 60 adult and at least 40 pediatric (age 5-17) male and female physician-diagnosed asthma subjects (with at least 12 patients under 12 years of age) with stable asthma, on any combination of asthma controller medications and as-needed bronchodilators, or on no asthma medications. Subjects were excluded if they currently had clinically significant unstable asthma, were cigarette smokers within the past six months, had a 10 pack-year history of smoking, had other respiratory conditions or clinically significant unstable medical conditions.

Safety observations
Adverse events (AEs), serious adverse events (SAEs) and unanticipated adverse device effects (UADEs) were monitored during the testing and in the approximately seven day period following the single visit.
Order of procedures

Primary Objective
Clinical Precision Fenom Pro TM For each subject, a median, standard deviation (SD) and percent coefficient of variation (%CV) for the two measurements were calculated. Subject medians were then classified into FeNO ranges (e.g. 0 to <10, 10 to <20 K 50 ppb). Within each of the ranges, the mean SD, and mean %CVs were provided, as well as their associated 95% confidence intervals (95% CIs).

Secondary objectives
The assessment of clinical accuracy comparing values for the Fenom Pro TM from 6 and 10 s exhalations utilized unweighted Deming regression, Bland-Altman analysis and the agreement analyses . The 10 s exhalation was considered as the reference, and these values were placed on the x-axis for the Deming regression. For both the 6 and 10 s exhalations, the first valid measures were compared. The same agreement analyses were performed for the method comparison of the Fenom Pro TM to the NIOX VERO ® , with the NIOX VERO ® results placed on the x-axis for the Deming regression. The average of two 10 s exhalation measurements (y-axis) was compared to the first 10 s exhalation measurement (x-axis), again employing unweighted Deming regression, Bland-Altman analysis and other agreement methods.

Results
The study was performed at three clinical research sites in the USA. The study was approved by a central institutional review board (Aspire IRB, Santee, CA), and all subjects or caregivers for children under 12 years of age signed an informed consent. Children of sufficient capacity also signed an assent form approved by the IRB.

Subject disposition
One hundred and twenty-six subjects were enrolled, with one screen failure occurring due to current enrollment in another investigational study. Of these, an eight year old Caucasian male was unable to provide a valid exhalation on any of the three FeNO devices and thus was excluded from the per protocol (PP) population (n=125). Data from the intent-to-treat population (ITT; n=126) is presented henceforth. Of the 126 enrolled, 124 (98.4%) completed all study procedures. No major protocol violations occurred during the study, and there were no SAEs.  FeNO concentrations, the within-subject mean of the SDs of the measurements, the within-subject mean of the calculated % CVs of the measurements and the associated two-sided 95% CIs for both the SD and % CV parameters. The population SDs vary by category of median FeNO concentration with higher SDs resulting from higher median concentration categories. However, the %CV results display an acceptable performance within each concentration category with point estimates ranging from 4.21%-9.25%, with lower %CVs generally corresponding to higher concentrations, as would be expected for a continuous scale quantitative variable. The 95% CIs by concentration category vary according to the N within each category, with wider CIs resulting from categories with lower N, as expected. Nevertheless, the upper bounds of the CIs remain below 12% in all concentration categories, indicating excellent clinical within-subject precision of FeNO measurements on the Fenom Pro TM device. Tables 3 and 4 present the clinical precision results in the pediatric and adult sub-cohorts of the ITT population using the 10 s exhalation. In the pediatric sub-cohort, several FeNO median concentration categories display N's12 (10 to <20, 20 to <30, 30 to <40, 40 to <50, and 50 ppb). In these latter categories, the upper bound of the 95% CI of the %CV exceeds 12% with a maximum of 26.73% for the 10 to <20 ppb category. However, the point estimates for all FeNO categories are below 7% ranging from 4.44%-6.63%, except for the 10 to <20 category (15.74%). For the adult sub-cohort, point estimates range from 2.93% in the next-to-highest FeNO concentration category (40 to <50 ppb) to 8.58% in the lowest category (0 to <10 ppb). In this sub-cohort, the upper bounds of the 95% CI for the %CV tend to display higher results with lower N with the highest upper bound of 31.97% occurring in the lowest FeNO category and at the lowest observed N of four subjects within the category. With the limitations of the number of subjects and focusing on the point estimates, these data demonstrate excellent clinical precision.

Demographic and baseline characteristics
Tables 5-7 present the clinical precision results using Fenom Pro TM for the combined ITT population (table 5) and the pediatric and adult sub-cohorts for a 6 s exhalation (tables 6 and 7. In general, the observed precision appears similar to the 10 s exhalation results with point estimates ranging from 4.29%-10.46%. Again, higher FeNO values tend to provide better precision estimates as do those FeNO categories with more subjects since both factors serve to narrow the confidence intervals and provide more accurate point estimates. Of note, the results for the pediatric subcohort indicate slightly better precision for a 6 versus a    to 1.468 ppb), R=0.978 and Sy|x = 6.3 ppb. Of note, the 95% CIs for the slope and y-intercept include the identity targets of 1.0 and 0.0, respectively, thus indicating the interchangeability of the two exhalation times. Figure 2 presents the results of the Bland-Altman analysis of paired differences (y) versus mean FeNO concentrations in ppb (x) in the ITT population for 6 versus 10 s exhalations on Fenom Pro TM . The points generally distribute evenly around the line of identity (0.0 ppb bias) but without any visual trend of either increasing or decreasing deviation with increasing measurement range. The mean bias is 0.4 ppb with 95% limits of −1.9 to 12.7 ppb. Since the 95% limits of the agreement include 0.0 ppb, the data indicate substantive agreement between the 6 and 10 s exhalations on the Fenom Pro TM in the ITT population.

Comparison of 6 and 10 s exhalations on Fenom Pro TM in the adult and pediatric ITT populations
There was also a high level of agreement for 6 versus 10 s exhalations, as shown in table 9 and comparable to the data in the combined ITT population.
Comparison of first valid 10 s exhalations on Fenom Pro TM to NIOX VERO ® Figure 3 presents the Deming regression comparing the first valid 10 s exhalation ppb values (y-axis) for the Fenom Pro TM to the NIOX VERO ® ppb values (x-axis) in the ITT population. The plot presents the individual points, the fitted regression and the line of identity. Table 10 presents the point estimates for the slope and y-intercept and their associated 95% CIs, as well as point estimates for the correlation coefficient (R), and Sy|x comparing the first valid 10 s exhalations on the Fenom Pro TM to the NIOX VERO ® . The results demonstrate a tight correlation between the two devices (albeit with slightly higher values in the Fenom Pro TM ) with a slope point estimate=1.149 (95% CI: 1.077-1.221), y-intercept=−1.211 (95% CI: −2.911 to 0.4895) ppb, R=0.981 and Sy|x=5.7 ppb. Figure 4 presents the results of the Bland-Altman analysis of paired differences (y) versus mean FeNO concentrations in ppb (x) in the ITT population comparing the first valid 10 s exhalations on the Fenom Pro TM to the NIOX VERO ® .
The points generally distribute evenly around the line of identity (0.0 ppb bias) with a slight trend toward increasing positive differences with increasing measurement range. The mean bias is 3.2 ppb with 95% limits of −9.6 to 16.0 ppb, with slightly higher values for the Fenom Pro TM . Since the 95% limits of the agreement include 0.0 ppb, the data nevertheless indicate substantive agreement between the Fenom Pro TM and NIOX VERO ® in the ITT population.
Comparison to ozone chemiluminescence   280i, demonstrating a tight agreement between the two analyzers.

One versus two exhalations on Fenom Pro TM
The first valid 10 s exhalation was compared to the average of the first two valid 10 s exhalations on the Fenom Pro TM to evaluate agreement. The ATS 2005 guidelines recommend evaluating the average of two exhalations [1]. Figure 6 and table 11 present the Deming regression for one measure versus the average of two measures. The points generally distribute evenly around the line of identity (0.65 ppb bias) with a slightly observable trend for increased divergence with increasing measurement range. The mean bias is 0.65 ppb with 95% limits of −3.66 to 4.96 ppb. The results demonstrate a tight correlation between one and the average of two exhalations with a slope point estimate = 1.026 (95% CI: 1.012-1.041), y-intercept=−0.2096 (95% CI: −0.7230 to 0.3038) ppb, R=0.998 and Sy| x=1.8 ppb. The data indicate substantive agreement between the one and the average of two exhalations in the ITT population. Figure 7 presents a Bland-Altman plot with a mean difference of 0.65 ppb and 95% limits of agreement ranging from 3.19−4.49 ppb, and as this includes 0, this supports the substantial agreement for the comparison.
One versus two exhalations on Fenom Pro TM for the separate adult and pediatric ITT populations For the separate adult and pediatric ITT populations, there was also a high level of agreement between the first valid and the average of the first two valid 10 s exhalations on the Fenom Pro TM , as shown in table 12.

Adverse Events for Fenom Pro TM
Two of 126 subjects (1.6%) in the safety population experienced self-limited mild AEs, headache and lightheadedness. The AE for lightheadedness was judged related to the investigational device and appeared associated with repeated breathing into the machine. The subject recovered within 2 min with no adverse sequelae.

Discussion
This study presents for the first time device performance for a recently cleared electrochemical officebased FeNO device, Fenom Pro TM . The primary objective was to evaluate the clinical precision of within-subject FeNO measurements using the Fenom Pro TM device for a mixed study population of pediatric subjects (ages 5-17 years) and adults. The secondary objectives were to evaluate the agreement between 6 and 10 s exhalations as the former would be easier for subjects to perform. In addition, one measure versus two measures were compared to see if one measure is adequate for routine practice. Finally, a method comparison was performed against the NIOX VERO ® , a commonly used marketed FeNO device.
Clinical precision for the combined, adult and pediatric ITT populations for 10 s exhalations was  For the Fenom Pro TM device, our results also indicate comparable clinical precision for 6 s exhalations to 10 s exhalations for the combined and separate adult and pediatric ITT populations, and importantly, better precision for 6 versus 10 s exhalations in children. Secondly, the comparison between 6 and 10 s exhalations demonstrated very close agreement in the combined and separate adult and pediatric ITT populations. Thus, 6 s exhalations are suitable for wide application in adults and pediatric populations. This will convey the benefit of shorter procedures and perhaps superior comfort for patients (less effort). We observed a much higher success rate with 6 versus 10 s exhalations as (72.8%) achieved success in two attempts on the Fenom Pro TM with a 10 s exhalation versus 93.6% with a 6 s exhalation (data not shown).
Lastly, we have convincingly demonstrated that one valid exhalation is in substantial agreement with the mean of two valid exhalations in both the combined and separate adult and pediatric ITT populations using Fenom Pro TM . The ATS recommendations from 2005 recommend performing two valid exhalations that agree within 10%. However, the recent ERS technical standard proposes that one valid exhalation is adequate [2]. Our data support this proposal. The  ability to perform a single exhalation will convey advantages, including a shorter overall FeNO test time, less procedures for patients and easier workflow in the clinic.
The method comparison of the Fenom Pro TM to the NIOX VERO ® indicates a high level of agreement with a mean bias of 3.2 ppb (range −9.6-16.0) in the ITT population, and these confidence limits include zero, indicating substantive agreement between the two devices with slightly higher values on average for the Fenom Pro TM .
On the issue of 6 versus 10 s exhalation, this may prove of particular importance for young children.  [4]. Thus, our demonstration of the excellent precision and equivalence of the 6 to the 10 s exhalation agrees with these reports [3] and opens the way to application of great relevance to very young children. Ten seconds of exhalation at 50 ml s −1 leads to a rapid deflation of the lung in very young children, which is the most plausible explanation of the lower success rate compared to 6 s exhalations.
There have been multiple [5][6][7][8][9][10][11][12][13][14][15][16][17][18] reports comparing different FeNO monitors, including diverse sensor technologies. Early studies focused on comparison of electrochemical sensors to ozone chemiluminescent analyzers, which are regarded as the 'gold standard'. Alving et al compared the NIOX MINO® (a handheld electrochemical device) to the NIOX ® (a chemiluminescent device), and reported that the mean bias was −1.2 ppb with higher readings for the NIOX MINO ® and 95% limits of agreement ranging from −9.8 to 8.0 ppb [19]. Boot et al compared the NIOX MINO ® to the Ecomedics ® ozone chemiluminescent device, and reported a high correlation (r=0.975, p<0.0001) but the mean bias was −10% with the NIOX MINO ® demonstrating lower values but the 95% confidence limits were broad ranging between −36% to +28% [5]. Inoue    when comparing devices, in general, the levels of correlation are high, but there may be significant differences between the absolute values obtained on one analyzer versus another. The reasons for the betweenanalyzer differences are probably multifactorial but could include divergent sensor technologies and calibration reliability, sensor drift and different precision in controlling an exhalation flow rate of 50 ml s −1 , as FeNO is markedly flow dependent [20]. This means that analyzers cannot be assumed to be interchangeable, although the between-analyzer differences are small. Certainly, for clinical research in a multisite setting, the same FeNO monitors would be a sensible requirement.
The limitations of the study are as follows. We only compared the Fenom Pro TM to the NIOX VERO ® as this is the most prevalent FeNO meter in use in the USA, but comparison to other FeNO devices would add more information for users outside the US. Also, we did not include a comparison to ozone chemiluminescent analyzers as these are not widely used in the physician offices where the study is performed, but we have included a benchtop comparison between the Fenom Pro TM and Sievers 280i devices. While the sample size was adequate for the purpose of this study, we had smaller numbers of subjects with FeNO values in the <10 ppb and 40-50 ppb and >50 ppb ranges. Also, the number of pediatric subjects under 12 and aged 5 and 6 was low. Finally, this data was collected in stable adult and pediatric subjects. However, we have also evaluated the Fenom Pro TM analyzer in unstable pediatric and adult subjects before and after corticosteroid therapy, representing a more diseased population and demonstrating a marked decline in FeNO after this intervention (data on file).
In summary, FeNO measurements on the Fenom Pro TM monitor demonstrated excellent clinical precision for both adults and children, whether for a 10 s exhalation or a 6 s exhalation, and the 6 s and 10 s exhalations show close agreement. Furthermore, the first valid exhalation on the Fenom Pro TM demonstrates good agreement with the average of the first two exhalations across adult and pediatric subjects. Lastly, the Fenom Pro TM FeNO values are in good agreement with the NIOX VERO ® device.