Validation of the 12-gene Predictive Signature for Adjuvant Chemotherapy Response in Lung Cancer Free

Lung cancer is the leading cause of death in the United States and around the world, and non–small cell lung cancer (NSCLC) accounts for up to 85% of such deaths (1–3). Large-scale clinical trials have demonstrated the survival benefits of adjuvant chemotherapy and have established adjuvant chemotherapy as the standard of care for completely resected stage II–IIIA NSCLC, and shown the benefits of adjuvant chemotherapy for stage IB patients with tumors larger than 4 centimeters (4–9). However, the effect of adjuvant chemotherapy on prolonging survival is modest, while such treatment is also associated with serious adverse effects (4, 8, 10, 11). Therefore, the routine use of adjuvant chemotherapy is not justified for all patients with resected NSCLC (9), and it is critical to identify the subgroup of patients who will most likely benefit from adjuvant chemotherapy (12).

Recently, using a cohort of 442 patients with stage I–III NSCLC from the Lung Cancer Director Challenge Dataset (13), we discovered a 12-gene signature that predicts adjuvant chemotherapy response (14) in patients with NSCLC. In addition, this gene signature predicts patient prognosis in patients with lung adenocarcinoma, but not in patients with squamous cell carcinoma (SCC). This signature was derived and validated using microarray technology from fresh frozen tumor samples. The crucial next step in translating this discovery to clinical practice is to develop a clinical assay for this 12-gene signature in formalin-fixed, paraffin-embedded (FFPE) samples. In this study, we developed a clinical-grade assay using the NanoString nCounter GX platform to measure the expression of the 12-gene signature from FFPE samples. Both the prognostic and predictive performance of this clinical assay together with the original algorithm were validated in FFPE samples from patients with early-stage (defined as stage I and II) NSCLC, and we demonstrated that the benefit group predicted by this gene signature showed significant improvement in survival after adjuvant chemotherapy, while the predicted nonbenefit group actually showed worse survival after adjuvant chemotherapy.

A list of 12 genes (ATP8A1, AURKA, C1orf116, COL4A3, DOCK9, HOPX, HSD17B6, IFT57, MBIP, NKX2-1, RRM2, and TTC37) and the predefined risk prediction algorithm were extracted from our previous study (14). The NanoString nCounter platform was used to develop an assay that could measure the expression levels of these 12 genes from FFPE tissue samples. The assay was developed and optimized using a cohort of 30 patients with NSCLC (with both FFPE and fresh frozen tissues), and the quality control measures were determined from this cohort. A cohort of 327 patients with early-stage (stage I and II) NSCLC was used for assay validation. The assay development and validation procedures are summarized in Fig. 1 and also detailed in the following sections. The resulting tables and figures are summarized in Supplementary Table S1.

The patients gave written consent; the study was approved by the ethics committee at the University of Texas MD Anderson Cancer Center (IRB #LAB90-020 and #LAB03-0320; Houston, TX); and the study was conducted in accordance with the Declaration of Helsinki.

A cohort of 30 patients with NSCLC (with matched FFPE and fresh frozen tissues) was used for the assay development. The expression levels of the 12-gene set were measured in both FFPE and fresh frozen tissues using the NanoString nCounter assay. The correlation between expression levels measured from FFPE tissue samples and those measured from corresponding fresh frozen samples was calculated for each individual gene to validate the measuring accuracy of the assay for FFPE samples.

The FFPE tissue samples of 327 patients with early-stage (stage I and II) NSCLC were obtained from the University of Texas Lung Cancer Specialized Program of Research Excellence (SPORE) Tissue Bank at the MD Anderson Cancer Center (MDACC, Houston, TX). This study was approved by the MD Anderson Institutional Review Board. Among these 327 early-stage patients, 69 (21.1%) were treated with adjuvant chemotherapy, and the remaining 258 (78.9%) were not treated with adjuvant chemotherapy. None of the patients received neoadjuvant chemotherapy. The 258 patients without adjuvant chemotherapy (166 adenocarcinoma, 86 SCC, and 6 others) were used as the validation cohort for prognostic performance of the 12-gene signature.

Because this is a retrospective cohort, to minimize the confounding factors, we used a propensity score matching technique to estimate the effect of adjuvant chemotherapy by accounting for the covariates (detailed in a later section). From the original 327 patients, we derived a cohort of 207 propensity score–matched patients, among which 69 patients (33.3%) were treated with adjuvant chemotherapy and 138 patients (66.7%) were not treated with adjuvant chemotherapy, to validate the predictive performance of the assay in FFPE samples.

Detailed information on the patients in the development cohort (n = 30), the prognostic validation cohort (258 patients without adjuvant chemotherapy), and the predictive validation cohort (207 propensity score–matched patients) are summarized in Table 1.

We used a propensity score matching technique (15, 16) to match the patients with and without adjuvant chemotherapy, to adjust for potential confounding factors in patient selection for adjuvant chemotherapy. The clinical variables, including histology, gender, age, smoking history, and stage, were used in the propensity score matching, and the coefficient and P value of each variable in the logistic regression model for propensity score matching are summarized in Supplementary Table S2. In this study, we used a 1:2 ratio for adjuvant chemotherapy treated and untreated patients, and the final matched patients included 69 adjuvant chemotherapy treated (one patient was removed because of missing values in covariates) and 138 patients without adjuvant chemotherapy. The propensity score matching results are summarized in Supplementary Table S3. These 207 propensity score–matched patients were used in the following analyses to validate the predictive performance of the 12-gene assay. The propensity score matching was implemented using R package matchit (17).

An unstained section was obtained from FFPE tissue blocks at 10-μm thickness, attached to a noncoated glass slide, and stored at −80°C in a freezer. The section was briefly baked at 65°C for 30 minutes and then deparaffinized with CitriSolv (Thermo Fisher Scientific, #22-143-975). Tumor areas were macrodissected and collected in a fresh Eppendorf microcentrifuge tube containing lysis buffer. Total RNAs were extracted using AllPrep DNA/RNA FFPE Extraction Kit (Qiagen, catalog no. 80234). RNA was quantified by NanoDrop 2000 and its quality was assessed by Agilent Bioanalyzer 1000 with Agilent RNA 6000 Nano Kit (Agilent, catalog no. 5067-1511).

NanoString nCounter technology uses unique color-coded molecular barcodes that can hybridize directly to target nucleic acid molecules, such as mRNA, to discover gene expression level with no need for amplification. The nCounter gene expression assay can investigate multiple genes in a single reaction. Probe sequences were custom designed for target genes and manufactured by NanoString Technologies. This CodeSet includes 7 housekeeping genes to correct for RNA input amount and/or quality differences. Housekeeping genes were selected from publicly available databases based on stability and detectable expression levels across the tissue type of interest. The manufacturer's protocol was followed to perform an assay. A master mix of hybridization buffer and Reporter CodeSet was prepared at 10 μL each, then 5 μL of total RNA sample and 5 μL of Capture ProbeSet were added to the reaction sequentially. After incubating at 65°C for hybridization for 20to 22 hours, the reaction products were loaded to GEN2 Prep Station for washing and immobilizing signals to cartridge. The cartridge was imaged in Digital Analyzer at 555 fields of view, whose images would be interpreted to count for specified targets by nSolver software (NanoString Technologies).

We followed the procedure by Veldman-Jones and colleagues (18) for gene expression data preprocessing. NAPPA R package (https://cran.r-project.org/) from the CRAN (Comprehensive R Archive Network) was used to normalize nCounter data in three steps: (i) a truncated Poisson correction was used to adjust the background signals in raw NanoString counts using internal-negative controls. (ii) The data were then normalized using internal-positive controls. (iii) A sigmoid shrunken slope normalization was used to correct the data for input amount variation using the mean expression of housekeeping genes. If a raw count was below the average of the eight internal-negative control raw counts plus 2 SDs, the transcript was designated as not detected. Transcripts below the limit of detection were not included in the data analysis. Finally, the data were log₂ transformed.

This is a prospective validation study, so the exact prediction model described in our previous study (14) was implemented. Specifically, predefined supervised principal component analysis was performed for the 12 genes. Using this predefined model, a risk score was assigned to each patient on the basis of the patient's gene expression level of the 12-gene signature.

Patients were separated into the high-risk (i.e., adjuvant chemotherapy benefit) or low-risk (i.e., adjuvant chemotherapy nonbenefit) group based on the patient's risk score. A predefined cutoff for risk score was needed to validate the prognostic and predictive performances of the developed assay. In this study, we determined the risk score cutoff based on the assay development cohort (n = 30), and applied this predefined cutoff into the prognostic performance validation cohort (n = 258) and the predictive performance validation cohort (n = 207). The risk score was calculated by the expression level of the 12-gene signature in FFPE samples for each patient in the assay development cohort. The predicted risk scores of the 30 patients were plotted in Supplementary Fig. S1A, which indicates a bimodal distribution of the predicted risk scores. To determine a data driven threshold value of the high- and low-risk patient groups, a model-based clustering method (implemented by R package mclust) was used to fit the risk score distribution with a mixture model of two normal distributions, one corresponding to the high-risk group (red) and the other corresponding to the low-risk group (green). On the basis of the fitting of the mixture model, the probability of a specific patient being in the high-risk group could be calculated from the predicted risk score (derived from the expression of the 12-gene signature) of the patient. This functional relationship is shown in Supplementary Fig. S1B.

Recurrence-free survival time (RFS) was calculated from the date of surgery until recurrence of lung cancer or death or the date of last follow-up contact. Survival curves were estimated using the Kaplan–Meier product limit method (19). Differences in the survival outcomes between the predicted high- and low-risk groups were compared using a log-rank test. A multivariate Cox proportional hazards model (20) was used to determine the association between the factors of interest (i.e., adjuvant chemotherapy, predicted risk-group, and their interaction term) and the patient survival outcomes adjusted for other clinical variables, including age, gender, smoking status, histology, and stage. Multivariate Cox proportional hazards models were also used to determine the benefit of adjuvant chemotherapy adjusted for other clinical variables in the predicted high- and low-risk groups, respectively.

The time-dependent ROC curves (21) at 24 months were calculated using R package survivalROC. The AUC of each time-dependent ROC curve was calculated as a measure of prediction accuracy for patient prognosis. The ROC curve for the predictive analysis was calculated using R package tsm (22).

The concordance index (CI), defined by Concordance = |$\displaystyle{\frac{{{\rm{\# all \ concordant \ pairs}}}}{{{\rm{\# total \ pairs \ ignoring \ ties}}}}}$| (23), was calculated using the R package survcomp.

The expression levels of the 12-gene set were measured from both FFPE and frozen tissue samples by nCounter gene expression assay in the development cohort (n = 30) to compare the consistency of the measurements from the two types of tissue samples. The dynamic range included the average, maximum, and minimum count of each gene, and the Pearson correlation coefficients between the measurements in FFPE and fresh frozen tissue samples are summarized in Supplementary Table S5. The Pearson correlation coefficients for different genes ranged from 0.5 to 0.8, indicating that the measurements from FFPE samples were reliable and consistent with those from fresh frozen samples.

The high- and low-risk groups defined by the 12-gene signature from FFPE tissue samples showed significant differences in RFS (Fig. 2C) for the 258 patients with NSCLC without adjuvant chemotherapy treatment: the high-risk group had worse prognosis in RFS [HR = 1.83 (1.19–2.95); P = 0.0114]. Prognoses for patients with different histology types' adenocarcinoma and SCC were also analyzed. The high-risk group had significantly worse RFS prognosis for patients with adenocarcinoma [Fig. 2A; HR = 4.12 (2.07–8.22); P = 1.25e-05], and patients with stage I-only adenocarcinoma [Fig. 2B; HR = 3.91 (1.82–8.38); P = 0.000161]. The time-dependent ROCs were calculated in both, the patients with adenocarcinoma (Supplementary Fig. S2A; AUC = 0.75) and the patients with stage I adenocarcinoma (Supplementary Fig. S2B; AUC = 0.74). Furthermore, the RFS differences were still significant in patients with adenocarcinoma [HR = 4.20 (2.01–8.79); P = 0.00014; Table 2] and in patients with stage I adenocarcinoma [HR = 3.23 (1.26–8.30); P = 0.015; Supplementary Table S6] after adjusting for other clinical variables in the multivariate analysis. On the other hand, for patients with SCC (n = 90), the predicted high- and low-risk groups did not show any differences in RFS [Fig. 2D; HR = 1.16 (0.272–4.92); P = 0.83]. In summary, the prognostic value of the 12-gene signature is adenocarcinoma specific, which is consistent with the findings in our original study. Therefore, this study validated the prognostic performance of the 12-gene signature in FFPE tissue samples using a biomarker assay that can be developed into a Clinical Laboratory Improvement Amendments-certified test for use in patients with adenocarcinoma, to move it further toward clinical application.

The 207 patients with early-stage NSCLC (propensity score-matched for adjuvant chemotherapy treatment) in the validation cohort were placed by the assay into two groups: those predicted to benefit from adjuvant chemotherapy (adjuvant chemotherapy benefit) and those predicted not to benefit from adjuvant chemotherapy (nonbenefit) group, using the same risk score and cut-off criteria. Figure 3 shows the RFS curves for patients with and without adjuvant chemotherapy in the predicted adjuvant chemotherapy benefit and nonbenefit groups, respectively. In the predicted adjuvant chemotherapy benefit group (high-risk group; Fig. 3A), the patients who received adjuvant chemotherapy had longer RFS time than those who did not receive adjuvant chemotherapy, while in the predicted–adjuvant chemotherapy nonbenefit group (low-risk group; Fig. 3B), patients who received adjuvant chemotherapy actually exhibited worse survival than those who did not receive adjuvant chemotherapy. Multivariate analyses for the efficacy of adjuvant chemotherapy treatment shows that adjuvant chemotherapy had significant RFS benefit (HR = 0.34; P = 0.016) in the predicted–adjuvant chemotherapy benefit group (Fig. 3C), while adjuvant chemotherapy was not associated with RFS benefit (HR = 1.86; P = 0.139) in the predicted–adjuvant chemotherapy nonbenefit group (Fig. 3D) after adjusting for other clinical variables. To test the interaction between the predicted–adjuvant chemotherapy benefit groups and adjuvant chemotherapy treatment effects, we performed multivariate analysis adjusting for clinical variables, including histology, smoking status, age, gender, tumor size, and stage, for all 207 patients (Table 3). This analysis indicated that, after adjusting for other clinical variables, adjuvant chemotherapy has a survival benefit for patients (HR = 0.40; P = 0.020) and there is significant interaction (P = 0.0056) between the effect of adjuvant chemotherapy and the predicted risk groups. We calculated the ROC curve for the predictive analysis (Supplementary Fig. S3). The AUC for the predictive performance was 0.806 and the 95% confidence interval was 0.694–0.865. This demonstrated that the effects of adjuvant chemotherapy are significantly different between the predicted adjuvant chemotherapy benefit group and predicted nonbenefit group. This validated that the 12-gene signature could predict adjuvant chemotherapy survival benefit in patients with early-stage NSCLC and could be used to identify the patient subpopulation that would benefit from the treatment, thereby assisting the clinical decision for using adjuvant chemotherapy. Furthermore, the 12-gene signature could be used to identify patients in whom adjuvant chemotherapy might be harmful. Figure 3B shows that in the predicted adjuvant chemotherapy nonbenefit group, the patients with adjuvant chemotherapy have significantly worse survival outcomes (P = 0.0154, log-rank test comparing the two survival curves).

Response to standard chemotherapy in lung cancer varies widely among patients. Therefore, it is of substantial clinical importance to be able to predict who will benefit from adjuvant chemotherapy before starting treatment. Multiple studies have been conducted to identify clinical factors that are associated with chemotherapy response in NSCLC (9, 24–27). Cancer and Leukemia Group B recently demonstrated that there is no significant survival benefit of adjuvant chemotherapy in patients with stage IB NSCLC based on a randomized trial (P = 0.12), while a statistically significant survival advantage was observed for stage IB patients with tumors ≥4 cm (9). The Lung Adjuvant Cisplatin Evaluation study showed that the chemotherapy effect was higher in patients with better performance status, and there was no interaction between the chemotherapy effect and gender, age, histology, type of surgery, planned radiotherapy, or planned total dose of cisplatin (27). Currently, a patient's tumor–node–metastasis (TNM) stage is the main clinical variable that provides prognostic information to suggest which patients need adjuvant chemotherapy. However, the TNM information (or the specific tumor histopathologic subtype) does not predict which patients within a TNM-stage category will derive survival benefit from adjuvant chemotherapy. Therefore, identifying and validating molecular markers with clinical assays to predict adjuvant chemotherapy response is important.

Although a large number of cancer biomarkers have been reported, few have been translated into real clinical tools. The major bottleneck in translating biomarker discovery to improved patient outcomes is the availability of accurate clinical tests (assays) that will allow treatments to be optimized and tailored to an individual's needs. In this study, we developed and tested a clinical-grade assay measuring mRNA expression level from FFPE tumor samples, together with a predefined risk stratification algorithm for patient response to adjuvant chemotherapy after tumor resection. The gene signature measured by the clinical-grade assay demonstrated promising results in predicting adjuvant chemotherapy response of patients with NSCLC.

Thus far, the dominant effort to improve the clinical outcomes of patients with lung cancer with adjuvant chemotherapy has been put toward testing the benefits of adding targeted therapies; for example, the National Cancer Institute of Canada JBR.19 trial added gefitinib to adjuvant chemotherapy for patients with resected NSCLC, and the Eastern Cooperative Oncology Group E1505 trial added bevacizumab. Although molecular targeted therapies are promising, only a small proportion of patients with early-stage NSCLC have mutations that are currently targetable for these therapies. Standard adjuvant chemotherapies are currently the primary treatment choice for patients with lung cancer. Therefore, being able to identify a subgroup of patients who would benefit from adjuvant chemotherapy would be an effective way to improve the clinical outcomes of patients with lung cancer.

This is a retrospective study with propensity score matching to minimize the effect of potential confounders for treatment effects and biomarker effects. Prospective clinical trials are needed to further validate the predictive signature for adjuvant chemotherapy. It would also be interesting to test whether this 12-gene signature could predict chemotherapy response in patients with late-stage lung cancer, and other types of cancer. Finally, multiple measurements of the same specimen on different days will be helpful for technical validation of the performance of the clinical-grade assay for future clinical application.

D.H. Johnson is a consultant/advisory board member for Genentech, Merck, Peloton Therapeutics, and Xcovery. No potential conflicts of interest were disclosed by the other authors.

Conception and design: Y. Xie, X. Tang, H. Tang, D.H. Johnson, G. Xiao, J.D. Minna, I.I. Wistuba

Development of methodology: Y. Xie, W. Lu, X. Tang, G. Xiao, I.I. Wistuba

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): Y. Xie, W. Lu, X. Tang, C. Moran, C. Behrens, J.A. Roth, S.G. Swisher, V.A. Papadimitrakopoulou, J.D. Minna, I.I. Wistuba

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): Y. Xie, W. Lu, S. Wang, X. Tang, H. Tang, Y. Zhou, D.H. Johnson, S.G. Swisher, G. Xiao, I.I. Wistuba

Writing, review, and/or revision of the manuscript: Y. Xie, W. Lu, S. Wang, H. Tang, J.A. Roth, Q. Zhou, D.H. Johnson, S.G. Swisher, J.V. Heymach, V.A. Papadimitrakopoulou, G. Xiao, J.D. Minna, I.I. Wistuba

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): Y. Xie, X. Tang, Q. Zhou, D.H. Johnson, S.G. Swisher, J.V. Heymach, I.I. Wistuba

Study supervision: Y. Xie, X. Tang, J.A. Roth, Q. Zhou, I.I. Wistuba

This work was supported by the NIH (5R01CA152301, P50CA70907, 5P30CA142543, 1R01GM115473, and 1R01CA172211), the Cancer Prevention and Research Institute of Texas (RP120732), and the National Key Research and Development Plan of China (2016YEE0103400) and National Natural Science Foundation of China (81572288).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.