Lung cancer mutation testing: a clinical retesting study of agreement between a real-time PCR and a mass spectrometry test

To investigate the clinical validity and utility of tests for detecting Epidermal Growth Factor Receptor (EGFR) gene mutations in non-squamous non-small cell lung cancer patients, tumour DNA extracts from 532 patients previously tested by the cobas EGFR Mutation Test (RT-PCR test) were retested by the Sequenom/Agena Biosciences MassArray OncoFocus mass spectrometry test (MS test). Valid results from both tests were available from 470 patients (88%) for agreement analysis. Survival data were obtained for 513 patients (96%) and 77 patients (14%) were treated with EGFR tyrosine kinase inhibitors (TKIs). Agreement analysis revealed moderately high positive (79.8%), negative (96.9%) and overall percentage agreement (93.2%) for the detection of EGFR mutations. However, EGFR mutations were detected by one test and not by the other test in 32 patients (7%). Retesting of discordant samples revealed false-positive and false-negative results generated by both tests. Despite this, treatment and survival outcomes correlated with the results of the RT-PCR and MS tests. In conclusion, this study provides evidence of the clinical validity and utility of the RT-PCR and MS tests for detection of EGFR mutations that predict prognosis and benefit from EGFR-TKI treatment. However, their false-positive and false-negative test results may have important clinical consequences.


INTRODUCTION
Epidermal Growth Factor Receptor (EGFR) gene mutation testing is a critical first step in the personalised treatment of patients with non-squamous non-small cell lung cancer (NSCLC). The testing is required for identifying patients with EGFR gene mutation-positive non-squamous NSCLC, who are candidates for first-line treatment with EGFR tyrosine kinase inhibitors (EGFR-TKIs). Significant numbers of non-squamous NSCLC patients can be expected to test positive for EGFR gene mutations, although the exact proportion varies widely between different ethnic groups and geographical regions. For example, in a population-based registry www.impactjournals.com/oncotarget/ Oncotarget, 2017, Vol. 8, (No. 60), pp: 101437-101451 Research Paper cohort of non-squamous NSCLC patients presenting in northern New Zealand, we had previously shown that EGFR gene mutations were detected in 109 of 500 tested patients (22%) [1]. Randomised trials had previously shown increased tumour response rates and progressionfree survival with EGFR-TKI treatment in EGFR gene mutation-positive patients [2][3][4][5][6][7][8], and with platinumdoublet combination chemotherapy in EGFR gene mutation-negative patients [9,10], who had previously untreated metastatic non-squamous NSCLC. In the same New Zealand population-based study mentioned above [1], we showed that the introduction of EGFR gene mutation testing was associated with improved quality of prescribing of EGFR-TKIs, and with improved health outcomes, including prolongation of overall survival and increased duration of benefit from EGFR-TKI treatment. Currently, no gold standard EGFR gene mutation testing methodology exists, and international clinical practice guidelines recommend use of any validated testing method with sufficient performance characteristics, but without recommending one or more individual methods to the exclusion of any others [11]. Surveys of real-world testing practices have revealed wide variation with the use of many different testing methodologies for EGFR gene mutation detection in the routine setting [12,13].
The cobas EGFR Mutation Test (Roche Molecular Systems Inc., Branchburg, NJ, USA) (RT-PCR test) is an oncogene mutation detection protocol based on multiplexed allele-specific PCR and a pre-validated set of primers to amplify and detect 41 variant sequences in the tyrosine kinase domain (exons [18][19][20][21] of the EGFR gene [14]. This RT-PCR test achieved CE-IVD regulatory status in Europe in October 2011 and FDA-USIVA approval of a modified version of the test in April 2013 [15]. Clinical validation studies were undertaken by retrospective analyses of tumour specimens (often surgical) sourced from vendors or clinical trial collections, and the data compared to other testing methods [16][17][18][19].
The Sequenom/Agena Biosciences OncoFocus mass spectrometry test (Agena Bioscience, San Diego, CA, USA) (MS test) comprises of a set of prevalidated genotyping assays designed for the simultaneous detection of 128 EGFR gene mutations and 63 KRAS, NRAS and BRAF gene mutations using a PCR-based mass spectrometry method. This method uses a two-step reaction protocol in which DNA sequences of interest are first amplified by PCR, followed by a single base primer extension and termination reaction across variant nucleotide positions, before specific detection of the amplified allele-specific oligonucleotide reaction products by MALDI-TOF mass spectrometry. This method is now widely used for lung cancer mutation testing due to the need for rapid detection of an increasing number of therapeutically targetable genetic abnormalities across multiple lung cancer genes [20]. However, the MS-test is not yet approved by regulatory authorities for diagnostic use and limited data have been published on its clinical validity and utility [21].
With this background, this study sought to evaluate the clinical performance of the RT-PCR and MS tests in the setting of everyday testing of tumour specimens from lung cancer patients for EGFR gene mutations. To do so, tumour DNA extracts from a large and unselected group of lung cancer patients (n=532) previously tested by the RT-PCR test were retested by the MS test. Recently, we reported on the impact and uptake of EGFR gene mutation testing during the implementation of clinical practice guidelines for testing in this population of patients from northern New Zealand [1]. The clinical validity and utility of the tests were evaluated by agreement analysis and by correlating the test results with the treatment and survival outcomes of tested patients. These outcomes comprised of the patient overall survival and duration of benefit from EGFR-TKI treatment.

Study populations
Tumour DNA extracts from 532 NSCLC patients previously tested by the RT-PCR test were retested by the MS test (retested population). Valid results from both tests were available from 470 (88%) patients for an agreement analysis (agreement analysis population). Of 62 patients (12%) excluded from the agreement analysis, 2 (0.4%) had invalid MS test results, 9 (1.7%) had invalid RT-PCT test results and RT-PCR results were missing for 51 patients (10%). Survival data was available for analysis from 513 (96%) patients (survival population). The retested, agreement analysis and survival populations had similar demographic and clinical profiles (Table 1). Seventy-seven (14%) patients were treated with EGFR-TKIs (EGFR-TKItreated population). The EGFR-TKI-treated population were younger, and had a higher proportion of females, Asians and Pacific people than the retested, agreement analysis or survival study populations (Table 1).

Agreement analysis
There was moderately high agreement between the RT-PCR test and the MS test for the detection of EGFR gene mutations. Among 470 patients with valid results available from both tests, 367 (78%) had no EGFR gene mutations detected in both tests and 71 (15%) had an EGFR gene mutation detected in both tests. The remaining 32 (7%) patients had an EGFR gene mutation detected by one test but not by the other test. From these data, the estimated levels of positive percentage agreement, negative percentage agreement and overall percentage agreement between the RT-PCR and MS tests for the detection of EGFR gene mutations were 79.8%, 96.9% or 93.2%, respectively ( Table 2).
There was high agreement between the RT-PCR and MS test results for the identification of specific EGFR gene mutations among patients who had an EGFR gene  (Table 3).

Disagreement
As mentioned above, there was disagreement between the RT-PCR and MS test results for the detection of EGFR gene mutations in a total of 32 (7%) patients (    The discordant patient samples were retested by a third assay (Biocartis Idylla EGFR Mutation Test) ( Table  4). The Idylla test DNA Quality Control check was passed by all except three of the retested samples. The RT-PCR and MS test results were then re-categorised as true or false positive or negative tests according to the results of the third test (Table 4). Accordingly, nine patients had true-positive RT-PCR results and false-negative MS results; six patients had true-negative RT-PCR results and false-positive MS results; four patients had true-positive MS results and false-negative RT-PCR results, and; seven patients had true-negative MS results and false-positive RT-PCR results. Six discordant patients were not retested in the third assay because no specimen was available or failure of the DNA Quality Control check.
There was disagreement between the RT-PCR and MS tests for the identification of specific EGFR gene mutations in 6 of 71 (8%) patients who had EGFR gene mutations detected in both tests (Table 5). Four patients had double EGFR gene mutations detected by one test but only one mutation was detected by the other test. One patient had double mutations detected by both tests but one of those two mutations differed between the two tests. One patient had a single EGFR gene mutation detected by both tests but the specific mutation detected differed between the tests.

KRAS, NRAS and BRAF gene mutation detection
The MS test identified a large number of patients with KRAS, NRAS and BRAF gene mutations that could not be expected to be detected by the RT-PCR test. Among 367 patients who had no EGFR gene mutations detected in both tests, a total of 127 patients (35%) had KRAS, NRAS and BRAF gene mutations identified by the MS test. The spectrum and frequency of specific KRAS, NRAS and BRAF mutations is shown in Table 6.

Overall survival
Overall survival was correlated with the results of the RT-PCR test ( Figure 1A), the MS test ( Figure 1B) and agreement analysis ( Figure 1C). The detection of an EGFR gene mutation was associated with significantly prolonged overall survival compared to when no EGFR gene mutations were detected, either by the RT-PCR test (adjusted HR=0.55 (95% CI 0.39 to 0.75))( Figure 1A), the MS test (adjusted HR=0.58 (95% CI 0.41 to 0.79)) ( Figure 1B) or by both tests (HR=0.47 (95% CI 0.32 to 0.66))( Figure 1C). The detection of a KRAS, NRAS or BRAF gene mutation was associated with a trend towards shorter overall survival compared to when no mutations were detected by the MS test ( Figure 1B). The discordant detection of an EGFR gene mutation by one test but not the other test was associated with a trend for immediate overall survival compared to that of patients with concordantly positive or negative EGFR gene mutation tests ( Figure 1C).

Duration of EGFR-TKI treatment
The duration of benefit from EGFR-TKI treatment was correlated with the results of the RT-PCR test (Figure  2A), the MS test ( Figure 2B) and agreement analysis ( Figure 2C). The detection of an EGFR gene mutation was associated with trends toward longer duration of benefit from EGFR-TKI treatment compared to when no mutations were detected, either by the RT-PCR test (Figure 2A), the MS test ( Figure 2B) or by both tests ( Figure 2C). The detection of a KRAS, NRAS or BRAF gene mutation was associated with a trend toward shorter duration of benefit from EGFR-TKI treatment compared to when no mutations were detected by the MS test ( Figure  2B). The discordant detection of an EGFR gene mutation by one test but not the other test was associated with a trend of intermediate duration of benefit from EGFR-TKI treatment compared to that of patients with concordantly positive or negative EGFR gene mutation tests ( Figure  2C).

DISCUSSION
The retesting study described here demonstrates moderately high agreement between a RT-PCR and a MS test for detecting EGFR gene mutations in lung cancer patients. The study was the first to directly compare these RT-PCR and MS tests despite both being commercially available and widely used in clinical practice for the detection of EGFR gene mutations in lung cancer patients. Both tests agreed in the detection of EGFR gene mutations in most patients with levels of positive, negative and overall agreement ranging from 79.8% to 96.9%. Identical EGFR gene mutations were detected by both tests in 92% of patients who were EGFR gene mutation-positive in both tests. In this way, this agreement study has provided evidence of the clinical validity of the RT-PCR and MS tests for detecting EGFR gene mutations in lung cancer patients.
However, this retesting study also demonstrates less than full concordance between the RT-PCR and MS tests for detecting EGFR gene mutations in lung cancer patients. The tests disagreed in the detection of EGFR gene mutations in some patients, with levels of positive, negative and overall disagreement ranging from 3.1% to www.impactjournals.com/oncotarget 20.2%. The causes of this discordance were unclear but interesting to speculate upon. Most of the discordant results arose during the testing of small histological or cytological biopsy specimens, which are known to be less reliable for testing than surgical specimens [22]. The DNA quality control checks that were performed during the RT-PCR testing and Idylla retesting were passed by almost all of the discordant samples. The specific EGFR gene mutations identified in the discordant samples were most frequently exon 19 deletions, exon 20 insertions and exon 21 L858R point mutations, which are the most prevalent EGFR gene mutations associated with lung cancer [23] and most were expected to be detected by both the RT-PCR and MS tests. Oncogene mutation detection is known for being error-   prone, due to, for example, contamination, PCR inhibitors and genetic polymorphisms. Contamination may occur during handling, DNA extraction or analysis [24,25]. Reagents or reaction products used or produced during the testing may inadvertently inhibit PCR amplification [26]. Polymorphisms in genetic sequences targeted by primer and probe oligonucleotides used in a test protocol may also produce variation in PCR amplification [27,28].
Retesting of the discordant samples by a third test showed that false-positive and false-negative test results were generated by both the RT-PCR and the MS test. Similar numbers of false-positive and falsenegative results were produced by the RT-PCR and MS tests. Patients with discordant EGFR gene mutation test results had survival and treatment outcomes that were intermediate of patients with concordantly positive or negative EGFR gene mutation tests. While several factors may have influenced these intermediate outcomes, these findings suggest that some patients with discordant EGFR gene mutation test results survived and responded to EGFR-TKI treatment as would be expected of patients with true-positive EGFR gene mutation test results, whereas others did so as would be expected of patients with true-negative EGFR gene mutation test results. In these regards, false-positive and false-negative EGFR gene mutation test results may have important clinical consequences. If treatment decisions are based on falsepositive results then such patients may receive less benefit from EGFR-TKI treatment but bear the same risks of adverse events compared to patients with true-positive EGFR gene mutation test results. Furthermore, if treatment decisions are based on false-negative test results then such patients may be denied substantial benefits of EGFR-TKI treatment, and bear increased risk of disease progression and death compared to patients with true-positive EGFR gene mutation test results. Other studies have also shown false-positive and false-negative results being generated by tests used for detecting EGFR gene mutations [29].
Despite less than full concordance, and falsepositive and false-negative test results in some patients, treatment and survival outcomes of tested patients correlated with the results of the RT-PCR and MS tests in this study. The detection of an EGFR gene mutation by the RT-PCR or MS test was associated with significantly (Continued ) prolonged overall survival and trends toward more durable benefit from EGFR-TKI treatment, compared to when no EGFR gene mutations were detected. The demonstration of these associations with treatment and survival outcomes provides evidence of the clinical utility of the RT-PCR and MS tests for predicting the prognosis and clinical benefits of EGFR-TKI treatment in patients with lung cancer.
In conclusion, this study has provided evidence of the clinical validity and utility of a RT-PCR and a MS test for the detection of EGFR gene mutations that predict the prognosis and clinical benefits of EGFR-TKI treatment in lung cancer patients. However, caution and awareness are required regarding the possibility of false-positive and false-negative results being generated by these tests, which run the risk of important clinical consequences.

Study design, context and population
The research described here was a clinical laboratory test-retesting study and clinical outcomes correlative study. Eligible patients were those referred to a New Zealand provider of EGFR gene mutation testing services between August 2012 and July 2014 (n=826), who had remnant tissue DNA extracts available for retesting after the completion of clinical diagnostic testing (n=532). The main study endpoint was the result of mutation testing and retesting. The main study outcomes were overall survival and the duration of treatment with an EGFR-TKI, either erlotinib or gefitinib. The Northern B Health and Disability Ethics Committee approved the study and required no individual patient consent. The study was registered with the Australian New Zealand Clinical Trials Registry (ACTRN12615000998549).

Mutation testing MS Test
The MS test was undertaken using the Sequenom/ Agena MassARRAY System and Typer 4 software (Agena Bioscience, San Diego, CA, USA). The OncoFOCUS

RT-PCR test
The RT-PCR test involved PCR amplification and detection of target DNA using complementary primer pairs and oligonucleotide probes. DNA was extracted using a Roche FFPE extraction kit and following manufacturer's instructions [15]; essentially this involved de-paraffinization and macro-dissection of tissue from slides previously marked by a histopathologist, thus ensuring the correct area was sampled. Following incubation in tissue lysis buffer, the lysate was bound to a filter column, washed and eluted. DNA was quantified using a Nanodrop® 1000 and diluted to 2 ng/ μL prior to performing the RT-PCR test on the cobas® z 480 analyzer. The RT-PCR test consisted of three simultaneous PCRs encompassing 41 mutations within exons 18-21 of the EGFR gene. A mutant control and negative control were included in each run to confirm the validity of the run.

Third test
Patient samples with discordant results for the detection of EGFR gene mutations were retested by a third assay, the Biocartis Idylla EGFR Mutation Test [30]. The Idylla® System is an in vitro diagnostic (IVD) device, consisting of a console and instrument. The system uses disposable, single sample cartridges, in which paraffin embedded tissue is macro-dissected onto small filter paper circles which are inserted directly into a gene-specific cartridge. This is then placed in to the instrument, and all the processing steps take place within the hermeticallyclosed cartridge. The steps comprise liquefaction, cell lysis, DNA/RNA extraction, real-time amplification/ detection, data analysis and reporting. Mutant and negative controls were incorporated into the cartridge. The EGFRspecific cartridge detects 52 mutations in exons 18-21 of the EGFR gene. The testing was conducted according to the manufacturer's instructions.

Data sources
Mutation testing results were linked with other prospectively archived electronic data from healthcare administrative databases using individual patient-unique healthcare identifier numbers. Data extractions from the New Zealand Cancer Registry on tested patients provided information on the date of diagnosis and death, and codes for gender, ethnicity, place of domicile, morphology, basis of diagnosis, notifying laboratory, extent of disease, TNM, tumour grade and additional notes. Records of erlotinib and gefitinib dispensing were extracted from the Pharmaceutical Information database (PHARMS), which contains claims and payment information from pharmacists for subsidised dispensing in New Zealand, and was found in our pilot study to be a near-complete source of EGFR-TKI dispensing data. Where possible, electronic medical records were viewed to verify extracted data and provide additional information.

Data analysis
Mutation testing results were tabulated by patient numbers and proportions were calculated according to whether or not a mutation was detected by each gene and by each specific mutation. The concordance between the mutation test results was analysed by agreement analysis [31]. A sample size of more than 400 patients was estimated to be sufficient for estimating proportions of agreement and population subgroups with 95% confidence intervals of less than 5%.
Cox proportional hazards regression modelling was used to assess hazards of overall mortality by the gene mutation status (EGFR gene mutation, KRAS, NRAS or BRAF gene mutation and no mutation) determined by the MS test and by agreement analysis results (concordant detection of an EGFR gene mutation by both RT-PCR and MS tests, discordant detection of an EGFR gene mutation by one test and not the other test, and no EGFR gene mutation). Overall survival was measured from the date of diagnosis to the date of death. Surviving patients were right-censored on 5 August 2016. Hazard ratios were adjusted by multivariate analysis for age, gender, ethnicity, histology, site and extent of disease, and diagnosis period.
Similar analyses were performed in a subgroup of patients who were treated with EGFR-TKIs and the duration of treatment was assessed against the gene mutation status determined by the MS test and by agreement analysis results. The duration of EGFR-TKI treatment was measured from the date that the drug was first dispensed to the date of the last dose (calculated as the date of last dispense plus the number of days of treatment dispensed on the last occasion) or death. Surviving patients who were continuing treatment with EGFR-TKIs were right-censored on 1 June 2016. No multivariate analysis was done because of the small sample sizes. www.impactjournals.com/oncotarget

Author contributions
PS, GRL, NJK, CAL, DRL JME and MJM, were responsible for the study concept and design, and obtained funding. PS and KLS undertook the retesting of the samples. PS, KLS, PK, PSA and AL collected and assembled the data. STT, PSA, JME and MJM did the statistical analyses. PS, STT, PSA, GRL, JME, DRL and MJM contributed to the data interpretation. PS, STT and MJM wrote the first draft of the report that was critically reviewed, and the final version approved, by all coauthors.