Purpose:

Immune checkpoint inhibitors (ICI) are a significant advance to the treatment of advanced non–small cell lung cancer (NSCLC); however, their initiation is associated with heterogeneity in outcomes. This study aimed to develop and validate a prognostic tool of survival in patients with advanced NSCLC treated with ICIs.

Experimental Design:

A pretreatment prognostic model was developed and validated using clinicopathologic data. Development data consisted of patients with advanced NSCLC treated with atezolizumab from the randomised trials OAK and POPLAR (n = 751). Data from the single-arm atezolizumab trials BIRCH and FIR (n = 797) were used for external validation. Prognostic scores were categorized into low, intermediate-low, intermediate, intermediate-high, and high-risk prognostic groups. The primary outcome was overall survival (OS), with progression-free survival (PFS) secondary.

Results:

Pretreatment C-reactive protein (CRP) was the most predictive variable for OS. The prognostic tool was optimally defined by CRP, lactate dehydrogenase, derived neutrophil-to-lymphocyte ratio, albumin, PD-L1 expression, performance status, time since metastatic diagnosis, and metastatic site count. Prognostic groups had significantly different OS (c-statistic = 0.72), with median OS ranging from >24 to 3 months for the low- to high-risk groups. Performance was maintained on validation (c = 0.76). These findings were similar for PFS, with median PFS ranging from 5 months to 1 month for the low- to high-risk groups. Benefit of atezolizumab (vs. docetaxel) was greatest in the low-risk group (>3 months median OS improvement), with little benefit apparent for the highest risk group.

Conclusions:

A prognostic tool was developed and validated to identify patient groups with distinctly different survival following atezolizumab initiation for advanced NSCLC.

Translational Relevance

This study identified pretreatment C-reactive protein (CRP) as the single most prognostic variable for OS in advanced NSCLC patients treated with the immune checkpoint inhibitor (ICI) atezolizumab. Furthermore, a prognostic tool was developed and validated that discriminates patient groups with clinically significant differences in survival outcomes; the tool is the first to incorporate CRP and PD-L1 expression, along with lactate dehydrogenase, derived neutrophil-to-lymphocyte ratio, albumin, performance status, time since metastatic diagnosis, and metastatic sites count. The prognostic tool had substantially improved performance compared to other previously proposed options. For patients considering the initiation of ICIs for the treatment of advanced NSCLC, the prognostic tool presented may enable the provision of more personalized expectations of survival outcomes.

Immune checkpoint inhibitors (ICI) are a significant advance to the treatment arsenal for advanced melanoma, non–small cell lung cancer (NSCLC), and renal cell carcinoma; however, their initiation is still associated with significant heterogeneity in survival outcomes. For example, while there is a significant increase in the likelihood of achieving long-term survival, the proportion of responders is similar to that achieved with conventional chemotherapies (1, 2).

Clinical prediction models integrate clinicopathologic data from many patients to identify subgroups with varying prognosis (3). Thereby, clinical prediction models may facilitate improved decision making by providing patients with personalized realistic expectations of treatment outcomes (4, 5). Prediction models may also be used to define patient groups with predicted good or poor prognosis, and subsequently allow a targeted analysis of genetic markers resulting in unexpectedly bad or good outcomes.

While much research has been conducted to assess pretreatment biomarkers associated with varying prognosis following the initiation of ICIs (6, 7), there has been minimal research on clinical prediction models to provide personalized risk predictions. One prominent recent example utilized the derived neutrophil to lymphocyte ratio (dNLR) and lactate dehydrogenase (LDH) to define a lung immune prognostic index (LIPI; ref. 8). While LIPI incorporates two robust prognostic inflammatory variables (8), its development did not extensively evaluate the multitude of routinely collected prognostic information to define an optimized prediction model. The aim of this study was to develop and validate a pretreatment prognostic tool for survival outcomes in patients with advanced NSCLC treated with ICIs, via the utilization of routinely collected clinicopathologic data.

Population

This study was a pooled post hoc analysis of individual-participant data (IPD) from the clinical trials BIRCH (NCT02031458; May 28, 2015 data cutoff; January 22, 2014 to December 4, 2014 participant enrollment period), FIR (NCT01846416; January 7, 2015 data cutoff; May 14, 2013 to June 27, 2014 participant enrollment period), OAK (NCT02008227; July 7, 2016 data cutoff; March 11, 2014 to April 29, 2015 participant enrollment period), and POPLAR (NCT01903993; May 8, 2015 data cutoff; August 5, 2013 to March 31, 2014 participant enrollment period; refs. 2, 9–11). Model development IPD was from patients with advanced NSCLC who received atezolizumab treatment within OAK and POPLAR. OAK and POPLAR were randomized trials of atezolizumab 1,200 mg i.v. every 3 weeks versus docetaxel 75 mg/m² i.v. every 3 weeks for patients with advanced NSCLC whose disease progressed on platinum-containing therapy (2, 11). IPD from patients with advanced NSCLC who received atezolizumab treatment within BIRCH and FIR were used as an external validation dataset. BIRCH and FIR were single-arm studies of atezolizumab 1,200 mg i.v. every 3 weeks in programmed death-ligand 1 (PD-L1)-positive advanced NSCLC (9, 10). Exploratory analysis of prognostic models was conducted in patients with advanced NSCLC who received docetaxel treatment within OAK and POPLAR.

All trials were done in full accordance with the guidelines for Good Clinical Practice and the Declaration of Helsinki and approval was obtained from an independent ethics committee for each trial site (2, 9–11). All patients gave written informed consent (2, 9–11). Secondary analysis of anonymized clinical trial data was deemed negligible risk research by the Southern Adelaide Local Health Network, Office for Research and Ethics and was exempt from review. Data were accessed according to Roche's policy and process for clinical study data sharing (12).

Predictors and outcomes

The primary outcome predicted was overall survival (OS), with progression-free survival (PFS) assessed as a secondary outcome. Primary study definitions of PFS were utilized; PFS was investigator-assessed as per Response Evaluation Criteria in Solid Tumours (RECIST version 1.1) in POPLAR and OAK (2, 11); an independent review facility–assessed PFS per RECIST (version 1.1) in BIRCH (10), and PFS was investigator-assessed per modified RECIST in FIR (9). Median follow-up was calculated using Kaplan–Meier analysis for the censored times.

Analyzed pretreatment patient and tumor characteristics were prespecified based upon availability, prior evidence (6–8), and biological plausibility. Utilized pretreatment clinicopathologic data were the values closest to, but prior to, the start of treatment. Analyzed clinicopathologic data included sex, age, race, body mass index (BMI), smoking history, Eastern Cooperative Oncology Group performance status (ECOG PS), histology (nonsquamous vs. squamous), stage, time since metastatic diagnosis (months), number of prior treatments, PD-L1 expression, metastatic sites count (i.e., number of organs with metastases), neutrophil-to-lymphocyte ratio (NLR), derived neutrophil-to-lymphocyte ratio [dNLR; calculated as neutrophil count/(white blood cell count – neutrophil count)], lymphocyte-to-monocyte ratio (LMR), platelet-to-lymphocyte ratio (PLR), and eosinophils, lactate dehydrogenase (LDH), C-reactive protein (CRP), alkaline phosphatase (ALP), calcium, hemoglobin, and albumin levels.

PD-L1 expression was categorized according to previously published scoring criteria (2, 9–11). Briefly, PD-L1 expression (0 vs. 1 vs. 2 vs. 3) was defined as the maximum score according to tumor cell scoring (TC0/1/2/3) or tumor-infiltrating immune cell scoring (IC0/1/2/3; refs. 2, 9–11). TC0 corresponds to a percentage of PD-L1–expressing tumor cells <1%, TC1 is ≥1% and <5%, TC2 is ≥5% and <50%, and TC3 is ≥50%. IC0 corresponds to a percentage of PD-L1–expressing tumor-infiltrating immune cells <1%, IC1 is ≥1% and <5%, IC2 is ≥5% and <10%, and IC3 is ≥10% (2, 9–11).

Univariate analysis

Cox proportional hazard models were used to assess the association between clinicopathologic data and survival outcomes within a pooled analysis of OAK and POPLAR. The associations were reported as HR with 95% confidence intervals (95% CI), and P values (likelihood ratio test). Continuous variables were explored for observed nonlinear effects using restricted cubic splines, and skewed data was log transformed. Analyses were conducted with a focus on facilitating clinical use and interpretability. Continuous variables were then categorized based upon model fit [assessed via the Akaike information criterion (AIC)], observed nonlinear effects, and standard/interpretable reference cut-off points. All analyses were stratified by study, allowing separate baseline hazard functions for each study. All P values less than 0.05 were considered statistically significant. As minimal data was missing (<10%) for the assessed variables, complete case analyses were conducted. Discrimination performance of models were assessed via the concordance statistic (c-statistic).

Development of a pretreatment prognostic tool

Pretreatment multivariable Cox proportional hazards models were developed via a stepwise forward inclusion of variables with a P < 0.05 and the greatest increase in the c-statistic; this was followed by a stepwise backwards elimination of univariables with P > 0.05 and those that did not increase the c-statistic by 0.05. To facilitate clinical use, model coefficients were scaled from 0 to 10 to allow calculation of prognostic scores (i.e., calculation of linear predictor score). Prognostic scores were categorized into low, intermediate-low, intermediate, intermediate-high, and high-risk prognostic groups (categorized according to the lower 15th, 15th–35th, 35th–65th, 65th–85th, and upper 15th risk percentiles). All analyses were stratified by study. Kaplan–Meier analysis was used for plotting and estimating probabilities of the prognostic groups. Five-fold cross validation (averaged over 20 repetitions) of a stepwise approach involving forward inclusion and backward elimination based on AIC was applied to estimate internal validity.

Exploratory analyses were conducted to assess the discrimination performance of a defined decision tree, random forest, and least absolute shrinkage and selection operator (LASSO) regression model. A decision tree was defined by means of recursive partitioning analysis (13), with a stop criterion set at a multiplicity-adjusted univariate P value of less than 0.05, with no subgroup size below 100. Random forest analysis, a machine learning regression approach, was conducted using the Breiman-Cutler permutation method for survival analysis, with imputation using a parallel out-of-bag error estimate method (14, 15). A LASSO regression model was evaluated using the glmnet package (16). For comparison, the performance of established immune-related prognostic scores were assessed, including the LIPI (8), Sen and colleagues (17) prognostic model, Gustave Roussy Immune Score (GRIm-Score; ref. 18), and Systemic Immune-Inflammation Index (SII; ≤6 × 1011 vs. > 6 and ≤15 × 1011 vs. >15 × 1011; refs. 19, 20).

All data analysis was conducted using R version 3.4.3.

Data sharing statement

Individual-participant data utilized was accessed according to Roche's policy and process for clinical study data sharing (12).

Patient population

Model development data consisted of 751 patients who received atezolizumab for advanced NSCLC within OAK and POPLAR. Median follow-up was 20 months (95% CI, 19–20) within the cohort. Validation data from BIRCH and FIR consisted of 797 patients. Supplementary Table S1 provides a summary of the patient characteristics from the four trials.

Univariate analysis

Univariable Cox proportional hazard analysis identified CRP, NLR, dNLR, hemoglobin, albumin, PLR, metastatic sites count, LMR, ECOG PS, LDH, months since metastatic diagnosis, ALP, histology, PD-L1 expression, race, and eosinophils as pretreatment prognostic markers of OS in patients who received atezolizumab for advanced NSCLC (P < 0.05; Supplementary Table S2). CRP (<3 vs. 3–9.9 vs. 10–49.9 vs. ≥ 50 mg/L) was the most predictive univariable for OS (c = 0.64). Figure 1 presents Kaplan–Meier estimate of OS for CRP within the development data. Discrimination performance of the derived CRP groups was maintained on validation (c = 0.71; Supplementary Fig. S1). In the validation dataset CRP (c = 0.71), NLR (c = 0.69), LMR (c = 0.66), dNLR (c = 0.65), and PLR (c = 0.65) were the five most prognostic variables of OS in patients who received atezolizumab for advanced NSCLC (P < 0.001).

Figure 1.

Kaplan–Meier estimates of OS by pretreatment CRP (A) and prognostic group (B) for advanced NSCLC patients who received atezolizumab within OAK/POPLAR.

Figure 1.

Kaplan–Meier estimates of OS by pretreatment CRP (A) and prognostic group (B) for advanced NSCLC patients who received atezolizumab within OAK/POPLAR.

Close modal

Univariable Cox proportional hazard analysis identified metastatic sites count, hemoglobin, NLR, CRP, dNLR, LMR, PLR, albumin, ECOG PS, LDH, PD-L1 expression, months since metastatic diagnosis, ALP, smoking history, and eosinophils as pretreatment prognostic markers of PFS in patients who received atezolizumab for advanced NSCLC (P < 0.05; Supplementary Table S3).

Prognostic tool for OS prediction

The optimal OS multivariable Cox proportional hazard model was defined by CRP, LDH, dNLR, albumin, PD-L1 expression, ECOG PS, time since metastatic diagnosis, and metastatic sites count (P < 0.05; Supplementary Table S4). Model coefficients were scaled to an integer between 0 and 10 based on Cox model coefficients (Supplementary Table S5). Table 1 presents the points allocated to each predictor to calculate a prognostic score.

Table 1.

Points allocated to each pretreatment factor to calculate an OS and PFS prognostic score.

012345678910
CRP (mg/L) <3  3 to 9.9    10 to 49.9  ≥50   
LDH (U/L) <230  230 to 459     ≥460    
dNLR <2     2 to 3.49   ≥3.5   
Albumin (g/L) ≥38  34 to 37.9    < 34     
PD-L1 expression level         0 or 1 
ECOG PS    1+       
Months since metastatic diagnosis ≥18    <18       
Metastatic sites count 1 to 2     3 or 4  ≥5    
012345678910
CRP (mg/L) <3  3 to 9.9    10 to 49.9  ≥50   
LDH (U/L) <230  230 to 459     ≥460    
dNLR <2     2 to 3.49   ≥3.5   
Albumin (g/L) ≥38  34 to 37.9    < 34     
PD-L1 expression level         0 or 1 
ECOG PS    1+       
Months since metastatic diagnosis ≥18    <18       
Metastatic sites count 1 to 2     3 or 4  ≥5    

Abbreviations: CRP, C-reactive protein; dNLR, derived neutrophil-to-lymphocyte ratio [dNLR = neutrophil count/(white blood cell count − neutrophil count)]; ECOG PS, Eastern Cooperative Oncology Group performance status; LDH, lactate dehydrogenase.

Figure 1 presents Kaplan–Meier estimates of OS within the development data for defined pretreatment prognostic groups. The discrimination performance of the prognostic groups in the development and validation data were 0.72 and 0.76, respectively. Table 2 presents OS estimates for the prognostic groups in the development data, where median OS ranged from greater than 24 to 3 months for the low- to high-risk prognostic groups. Supplementary Table S6 and Supplementary Fig. S1 present Kaplan–Meier estimates of OS for the prognostic groups in the validation data. Supplementary Fig. S2 presents calibration plots of OS for prognostic groups in the development and validation data. The discrimination performance of the stepwise 5-fold cross validation model was 0.70 in the development data, which is similar to the presented prognostic tool and an internal test that overfitting has not occurred.

Table 2.

OS and PFS estimates by pretreatment prognostic group for patients treated with atezolizumab within OAK/POPLAR.

OSPFS
Prognostic groupPrognostic ScoreMedian [95% CI] T2E (months)24 month [95% CI] OS probability (%)Median [95% CI] T2E (months)12 month [95% CI] PFS probability (%)
Low <18 >24 [22–24] 55 [42–72] 5 [4–8] 25 [18–34] 
Intermediate-low 18–23 19 [15–24] 47 [38–58] 4 [3–5] 23 [17–32] 
Intermediate 24–31 14 [11–16] 29 [22–38] 3 [2–4] 17 [13–24] 
Intermediate-high 32–38 7 [6–10] 7 [1–35] 2 [1–3] 10 [6–17] 
High ≥39 3 [3–5] 0 [NA] 1 [1–1] 6 [2–13] 
OSPFS
Prognostic groupPrognostic ScoreMedian [95% CI] T2E (months)24 month [95% CI] OS probability (%)Median [95% CI] T2E (months)12 month [95% CI] PFS probability (%)
Low <18 >24 [22–24] 55 [42–72] 5 [4–8] 25 [18–34] 
Intermediate-low 18–23 19 [15–24] 47 [38–58] 4 [3–5] 23 [17–32] 
Intermediate 24–31 14 [11–16] 29 [22–38] 3 [2–4] 17 [13–24] 
Intermediate-high 32–38 7 [6–10] 7 [1–35] 2 [1–3] 10 [6–17] 
High ≥39 3 [3–5] 0 [NA] 1 [1–1] 6 [2–13] 

Abbreviations: CI, confidence interval; OS, overall survival; PFS, progression-free survival; T2E, time to event.

Exploratory analyses were undertaken to assess a simpler (decision tree) and a more complex (random forest) modeling approach. With respect to prediction performance on the validation dataset, the decision-tree model (Supplementary Fig. S3) had discrimination (c-statistic) of 0.69 and the random forest model had discrimination (c-statistic) of 0.77. The discrimination performance of a defined LASSO regression model was 0.75 on validation. The discrimination performance of LIPI (Supplementary Fig. S4), Sen and colleagues (17) prognostic model, GRIm-Score, and SII were 0.66, 0.67, 0.71, and 0.69, respectively, on validation data (Supplementary Table S7).

Prognostic tool for PFS prediction

The discrimination performance (c-statistic) of prognostic groups defined according to a scoring tool based upon multivariable Cox proportional hazards prediction of PFS (Supplementary Table S8) was 0.61 and 0.60 within the development and validation data, respectively. Comparatively, the PFS discrimination performance (c-statistic) of the OS prognostic scoring tool groups was 0.62 and 0.63, respectively. As the OS prognostic scoring tool groups performed superiorly, Table 2 presents PFS estimates derived according to the OS prognostic scoring tool in the development data, where median PFS ranged from 5 months to 1 month for the low- to high-risk prognostic groups. Supplementary Figure S5 presents Kaplan–Meier estimates of PFS according to prognostic groups. Supplementary Figure S2 presents calibration plots of PFS for prognostic groups in the development and validation data.

The PFS discrimination performance of LIPI (Supplementary Fig. S5), Sen and colleagues (17) prognostic model, GRIm-Score, and SII were 0.59, 0.59, 0.60, and 0.58, respectively, on validation data (Supplementary Table S7).

CRP and prognostic tool in patients treated with docetaxel

CRP (<3 vs. 3–9.9 vs. 10–49.9 vs. ≥50 mg/L) was significantly associated with OS (P < 0.001, c = 0.66) in a pooled analysis of patients who received docetaxel within OAK/POPLAR (Supplementary Table S9). Supplementary Figure S6 presents Kaplan–Meier estimates of OS by CRP for patients who received docetaxel within OAK/POPLAR.

The prognostic scoring tool groups were significantly associated with OS (P < 0.001, c = 0.67) and PFS (P < 0.001, c = 0.60) for patients who received docetaxel within OAK/POPLAR. Supplementary Table S10 and Supplementary Fig. S7 provide a summary of OS/PFS estimates for patients who received docetaxel within OAK/POPLAR. Comparing the pooled randomized arms of OAK/POPLAR, the observed improvement in median OS for participants randomized to atezolizumab (versus docetaxel) within the low, intermediate-low, and intermediate prognostic groups was >3 (>24 vs. 21), 2 (19 vs. 17), and 4 (14 vs. 10) months, respectively. Little apparent benefit in median OS was observed for atezolizumab (vs. docetaxel) for the intermediate-high (7 vs. 8 months) and high-risk (3 vs. 4 months) prognostic groups (Fig. 2). No benefit in median PFS was observed for atezolizumab versus docetaxel for the low- to high-risk groups (Supplementary Fig. S8).

Figure 2.

A–E, Kaplan–Meier comparisons of OS by pretreatment prognostic groups for pooled randomised arms of OAK/POPLAR (atezolizumab versus docetaxel).

Figure 2.

A–E, Kaplan–Meier comparisons of OS by pretreatment prognostic groups for pooled randomised arms of OAK/POPLAR (atezolizumab versus docetaxel).

Close modal

Using large high-quality data, this study developed and validated a pretreatment prognostic tool able to clearly distinguish patient groups with distinct differences in OS and PFS when considering atezolizumab initiation for advanced NSCLC treatment. To the best of the authors' knowledge, this is the first published NSCLC-ICI–specific prognostic tool that accounts for pretreatment CRP which was demonstrated as the single most predictive variable for OS in advanced patients with NSCLC initiating atezolizumab. Because of this, it is unsurprising that on external validation, the discrimination performance of the developed tool was substantially superior than other previously proposed options (8, 17–22). Importantly, the presented study includes data from trials including randomized treatment allocation, enabling valid assessment of treatment effect of atezolizumab compared to docetaxel by identified prognostic groups.

The identified optimal combination of predictors for OS included CRP, LDH, dNLR, albumin, PD-L1 expression, ECOG PS, months since metastatic diagnosis, and metastatic sites count. This study is the largest to evaluate the predictive performance of CRP within a cohort of advanced NSCLC patients initiating ICI therapy (6, 7, 23–26). Smaller prior studies have suggested that CRP may be an important prognostic factor in patients with advanced cancer initiating ICIs (23–26), and the prognostic significance of CRP in NSCLC is established (27, 28). Of the investigated univariables in this study, CRP was the strongest predictor of OS within the OAK and POPLAR cohort (c = 0.64; Supplementary Table S2), and performance was maintained on validation (c = 0.71). Exemplifying the need to appreciate the prognostic significance of CRP in patients with advanced NSCLC initiating ICI therapy, is that even with enforced easy to use cut-off points, its c-statistic was superior in the development (0.64 vs. 0.61) and validation (0.71 vs. 0.66) data as compared with LIPI, which incorporates only dNLR and LDH.

In addition to presenting the first NSCLC-ICI–specific prognostic tool that accounts for pretreatment CRP, the presented tool is also the first to incorporate the prognostic potential of PD-L1 expression. Thus, while the potential for PD-L1 expression to consistently and conclusively discriminate treatment benefit to ICIs remains under investigation (7, 9, 10, 29), this study demonstrates the clinical application of evaluating PD-L1 expression to provide patients with realistic prognostic information, which has not been accounted for in prior tools. Furthermore, exemplifying the need to evaluate beyond PD-L1 expression, CRP, dNLR, and LDH is that each of albumin, ECOG PS, months since metastatic diagnosis, and metastatic sites count substantially improved model performance (demonstrated by an increase in the c-statistic by at least 0.05). It is not surprising these variables provide important prognostic information given prior literature demonstrating their prognostic significance for advanced NSCLC- or ICI-treated patients (6, 7).

While this study had a focus on optimising prediction performance, equally there was a preference to develop a pretreatment prognostic tool that was practical for clinical use. This included using routinely available clinical data, a preference for a single model for OS and PFS, and avoiding complex numerical derivations. The developed prognostic tool has a similar structure to the widely utilized Hurria and colleagues (30) tool for predicting chemotherapy-related toxicity in older adults, and thus has significant potential for integration within clinical use (31, 32). In addition to evaluating the performance of a developed decision tree, the machine learning approach, random forest, was evaluated. Despite a series of simplifications, including enforcing simple cut-off points to continuous variables, minimizing the number of variables via backward deletion, using a single risk score for both OS and PFS, and scaling coefficients to integers between 0 to 10, the performance of the developed prognostic tool on validation was comparable to the black-box random forest approach.

While the numerical calculations required to utilize the prognostic tool developed herein are greater than LIPI (ref. 8; the most assessed tool to date; refs. 8, 21, 22), the discrimination performance (c-statistic) was markedly superior in the development (0.72 vs. 0.61) and validation (0.76 vs. 0.66) data. The developed prognostic tool was also superior to other previously proposed options (8, 17–22). Demonstrating the importance of this optimized discrimination performance from a patients perspective is that within the pooled OAK and POPLAR cohort, the observed difference in median OS and 24-month OS probability between the high- to low-risk groups was greater than 21 (3 vs. >24) months and 55% (0 vs. 55), respectively, for the tool herein. Comparatively, the difference in median OS and 24-month OS probability was 13 (5 vs. 18) months and 17% (14 vs. 31), respectively, between the LIPI defined poor to good prognostic groups, indicative of a significantly greater spread of risks for the tool developed herein.

The developed prognostic tool allows the simultaneous interpretation of personalized risks for OS and PFS in individuals commencing ICI therapy for advanced NSCLC. While the OS and PFS estimates presented in Table 2 are only applicable to patients who corresponded to the inclusion criteria of OAK and POPLAR (i.e., patients with advanced NSCLC whose disease has progressed on platinum-containing therapy), the maintained discrimination performance within BIRCH and FIR indicates prognostic performance in external cohorts. For example, in contrast to OAK and POPLAR, BIRCH and FIR included only PD-L1–positive patients and was not limited to patients whose disease had progressed on prior platinum-containing therapy. Thus, the developed tool should be explored further as a mechanism to provide patients with realistic expectation of ICIs. The importance of this is highlighted by the difference in median OS and 24-month OS probability being greater than 21 (range = 3 to >24) months and 55% (0 vs. 55) between the high- to low-risk groups within OAK and POPLAR. Another potential utilization of the developed tool is to aid clinical trial randomization, whereby balancing prognostic scores between arms can aid valid treatment comparisons. Future directions of research include evaluation as a prognostic tool or treatment-effect predictor for other ICIs used for advanced NSCLC (durvalumab, nivolumab, pembrolizumab), for first-line use of ICIs, and ICIs used in combination therapies. Moreover, the prognostic tool may be considered as an aid to basic research design, whereby it may be used to select patients with predicted good or bad prognosis, and subsequently allow a targeted analysis of genetic profiles to determine markers resulting in patients with unexpectedly bad or good outcomes (i.e., a targeted utilization of model unpredictability).

The developed and validated prognostic groups had modestly superior discrimination performance in the atezolizumab-treated patients from OAK and POPLAR, albeit a similar prognostic association within docetaxel-treated patients highlights the prognostic groups are largely not specific to ICI treatment. Nonetheless, when comparing the pooled randomized arms of OAK/POPLAR, the benefit of atezolizumab treatment (vs. docetaxel) was greatest in the low-risk group (>3 months median OS improvement), with little benefit apparent for the highest-risk group. Future directions of research should include further evaluation of the prognostic groups as a marker of heterogeneity in ICI treatment effect (both relative and absolute; refs. 33–35) for randomized clinical trials across other ICIs used for advanced NSCLC (durvalumab, nivolumab, pembrolizumab), first-line use of ICIs, and combination use of ICIs.

Clinical trials are a pillar of evidenced-based medicine and the data collected within are high-quality and stringently regulated. Equally, the inclusion criteria of trials can limit generalizability to clinical practice (36). For example, ECOG PS is an important cancer prognostic marker; however, no participants within the OAK, POPLAR, BIRCH, or FIR trials had scores of three or four (2, 9–11). Furthermore, this study is a post hoc analysis, and assessed predictors were limited to available data. Nonetheless, data from OAK and POPLAR were pooled to increase study power and generalizability; the extent of commonly collected clinicopathologic data was vast and ultimately the full prediction model required reduction to facilitate clinical potential. The data also contained PFS which is often not collected in registry data, yet is an important outcome to patients (5).

The discrimination performance of the developed tool for OS prediction was consistent with a strong-performing model in both the development and external validation cohorts (c > 0.72; ref. 37). Ideally the discrimination performance for PFS would have a c-statistic greater than 0.7, albeit a significant association between prognostic groups and PFS was identified. Differences in inclusion criteria (e.g., prior treatments received), nuances of disease progression definitions between trials, and variable subsequent therapies between patients following study completion may have contributed to the poorer prediction of PFS. While this study was able to validate the developed tool within data external to the development data, future research will ideally validate and recalibrate, if necessary, using real-world population data to optimize generalizability and performance. Nonetheless, the study presents a model developed using data which underpins the use of atezolizumab within the advanced NSCLC population, and thus the developed tool may be used to better understand likely survival outcomes from therapy than is currently possible.

In conclusion, a pretreatment prognostic tool for OS and PFS in patients with advanced NSCLC initiating atezolizumab was developed and validated. Importantly the prognostic tool incorporates a multitude of factors to provide more nuanced survival predictions than previously presented options, and notably these factors include CRP and PD-L1 expression which are among the most prognostic factors in advanced NSCLC patients treated with ICIs. There is the potential for the developed prognostic tool to provide patients with advanced NSCLC with more realistic expectations of long-term disease control and survival when considering the initiation of ICIs.

G. Kichenadasse is an advisory board member/unpaid consultant for Merck. C.S. Karapetis is an advisory board member/unpaid consultant for Roche, Bristol-Myers Squibb, Merck Serono, AstraZeneca, and Amgen. A. Rowland reports receiving commercial research grants from Pfizer. No potential conflicts of interest were disclosed by the other authors.

Conception and design: A.M. Hopkins, A. Rowland, M.J. Sorich

Development of methodology: A.M. Hopkins, M.J. Sorich

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): A.M. Hopkins, A. Rowland, M.J. Sorich

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): A.M. Hopkins, G. Kichenadasse, E. Garrett-Mayer, C.S. Karapetis, A. Rowland, M.J. Sorich

Writing, review, and/or revision of the manuscript: A.M. Hopkins, G. Kichenadasse, E. Garrett-Mayer, C.S. Karapetis, A. Rowland, M.J. Sorich

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): A. Rowland, M.J. Sorich

Study supervision: A.M. Hopkins, A. Rowland, M.J. Sorich

This work was produced with the financial and other support of Cancer Council SA's Beat Cancer Project on behalf of its donors and the State Government of South Australia through the Department of Health. Andrew Rowland is supported by a Beat Cancer Mid-Career Research Fellowship from Cancer Council SA. Ashley Hopkins is supported by a Fellowship from the National Breast Cancer Foundation, Australia (PF-17-007).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
Park
Y-J
,
Kuen
D-S
,
Chung
Y
. 
Future prospects of immune checkpoint blockade in cancer: from response prediction to overcoming resistance
.
Exp Mol Med
2018
;
50
:
109
.
2.
Rittmeyer
A
,
Barlesi
F
,
Waterkamp
D
,
Park
K
,
Ciardiello
F
,
von Pawel
J
, et al
Atezolizumab versus docetaxel in patients with previously treated non-small-cell lung cancer (OAK): a phase 3, open-label, multicentre randomised controlled trial
.
Lancet
2017
;
389
:
255
65
.
3.
Adams
ST
,
Leveson
SH
. 
Clinical prediction rules
.
BMJ
2012
;
344
:
d8312
.
4.
Pavlou
M
,
Ambler
G
,
Seaman
SR
,
Guttmann
O
,
Elliott
P
,
King
M
, et al
How to develop a more accurate risk prediction model when there are few events
.
BMJ
2015
;
351
:
h3868
.
5.
Paladino
J
,
Lakin
JR
,
Sanders
JJ
. 
Communication strategies for sharing prognostic information with patients: beyond survival statistics
.
JAMA
2019
;
322
:
1345
6
.
6.
Hopkins
AM
,
Rowland
A
,
Kichenadasse
G
,
Wiese
MD
,
Gurney
H
,
McKinnon
RA
, et al
Predicting response and toxicity to immune checkpoint inhibitors using routinely available blood and clinical markers
.
Br J Cancer
2017
;
117
:
913
.
7.
Prelaj
A
,
Tay
R
,
Ferrara
R
,
Chaput
N
,
Besse
B
,
Califano
R
. 
Predictive biomarkers of response for immune checkpoint inhibitors in non-small-cell lung cancer
.
Eur J Cancer
2019
;
106
:
144
59
.
8.
Mezquita
L
,
Auclin
E
,
Ferrara
R
,
Charrier
M
,
Remon
J
,
Planchard
D
, et al
Association of the lung immune prognostic index with immune checkpoint inhibitor outcomes in patients with advanced non-small cell lung cancer
.
JAMA Oncol
2018
;
4
:
351
7
.
9.
Spigel
DR
,
Chaft
JE
,
Gettinger
S
,
Chao
BH
,
Dirix
L
,
Schmid
P
, et al
FIR: efficacy, safety, and biomarker analysis of a phase II open-label study of atezolizumab in PD-L1-selected patients with NSCLC
.
J Thorac Oncol
2018
;
13
:
1733
42
.
10.
Peters
S
,
Gettinger
S
,
Johnson
ML
,
Janne
PA
,
Garassino
MC
,
Christoph
D
, et al
Phase II trial of atezolizumab as first-line or subsequent therapy for patients with programmed death-ligand 1-selected advanced non-small-cell lung cancer (BIRCH)
.
J Clin Oncol
2017
;
35
:
2781
9
.
11.
Fehrenbacher
L
,
Spira
A
,
Ballinger
M
,
Kowanetz
M
,
Vansteenkiste
J
,
Mazieres
J
, et al
Atezolizumab versus docetaxel for patients with previously treated non-small-cell lung cancer (POPLAR): a multicentre, open-label, phase 2 randomised controlled trial
.
Lancet
2016
;
387
:
1837
46
.
12.
Strom
BL
,
Buyse
M
,
Hughes
J
,
Knoppers
BM
. 
Data sharing, year 1—access to data from industry-sponsored clinical trials
.
N Engl J Med
2014
;
371
:
2052
4
.
13.
Hothorn
T
,
Hornik
K
,
Zeileis
A
. 
Unbiased recursive partitioning: a conditional inference framework
.
J Comput Graph Statist
2006
;
15
:
651
74
.
14.
Bischl
B
,
Lang
M
,
Kotthoff
L
,
Schiffner
J
,
Richter
J
,
Studerus
E
, et al
mlr: machine learning in R
.
J Mach Learn Res
2016
;
17
.
15.
Kogalur
U
,
H
I
. 
2019 randomForestSRC version 2.9.1; Random Forests for Survival, Regression, and Classification; A Parallel Package for a General Implemention of Breiman's Random Forests Theory and Specifications
. Available from: https://kogalur.github.io/randomForestSRC/theory.html.
16.
Friedman
J
,
Hastie
T
,
Simon
N
,
Tibshirani
R
. 
2018
02/04/2018. Package ‘glmnet’: Lasso and Elastic-Net Regularized Generalized Linear Models
. Available from: https://cran.r-project.org/web/packages/glmnet/glmnet.pdf.
02/04/2018
.
17.
Sen
S
,
Hess
K
,
Hong
DS
,
Naing
A
,
Piha-Paul
S
,
Janku
F
, et al
Development of a prognostic scoring system for patients with advanced cancer enrolled in immune checkpoint inhibitor phase 1 clinical trials
.
Br J Cancer
2018
;
118
:
763
.
18.
Minami
S
,
Ihara
S
,
Ikuta
S
,
Komuta
K
. 
Gustave Roussy Immune Score and Royal Marsden Hospital Prognostic Score are biomarkers of immune-checkpoint inhibitor for non-small cell lung cancer
.
World J Oncol
2019
;
10
:
90
100
.
19.
Zhang
Y
,
Chen
B
,
Wang
L
,
Wang
R
,
Yang
X
. 
Systemic immune-inflammation index is a promising noninvasive marker to predict survival of lung cancer: a meta-analysis
.
Medicine
2019
;
98
:
e13788
.
20.
Hu
B
,
Yang
X-R
,
Xu
Y
,
Sun
Y-F
,
Sun
C
,
Guo
W
, et al
Systemic immune-inflammation index predicts prognosis of patients after curative resection for hepatocellular carcinoma
.
Clin Cancer Res
2014
;
20
:
6212
22
.
21.
Sorich
MJ
,
Rowland
A
,
Karapetis
CS
,
Hopkins
AM
. 
Evaluation of the lung immune prognostic index for prediction of survival and response in patients treated with atezolizumab for NSCLC: pooled analysis of clinical trials
.
J Thorac Oncol
2019
;
14
:
1440
6
.
22.
Kazandjian
D
,
Gong
Y
,
Keegan
P
,
Pazdur
R
,
Blumenthal
GM
. 
Prognostic value of the lung immune prognostic index for patients treated for metastatic non–small cell lung cancer
.
JAMA Oncol
2019
;
5
:
1481
5
.
23.
Iivanainen
S
,
Ahvonen
J
,
Knuuttila
A
,
Tiainen
S
,
Koivunen
JP
. 
Elevated CRP levels indicate poor progression-free and overall survival on cancer patients treated with PD-1 inhibitors
.
J ESMO Open
2019
;
4
:
e000531
.
24.
Naqash
AR
,
Stroud
CRG
,
Cherry
CR
,
Sharma
N
,
Butt
MU
,
Muzaffar
M
, et al
Evaluating the utility of pretreatment C-reactive protein (CRP) in survival stratification of advanced non-small cell lung cancer (NSCLC) treated with immune checkpoint blockade (ICB): a prospective cohort study
.
J Clin Oncol
2018
;
36
:
e15122
.
25.
Oya
Y
,
Yoshida
T
,
Kuroda
H
,
Mikubo
M
,
Kondo
C
,
Shimizu
J
, et al
Predictive clinical parameters for the response of nivolumab in pretreated advanced non-small-cell lung cancer
.
Oncotarget
2017
;
8
:
103117
28
.
26.
Weber
JS
,
Tang
H
,
Hippeli
L
,
Qian
M
,
Wind-Rotolo
M
,
Larkin
JMG
, et al
Serum IL-6 and CRP as prognostic factors in melanoma patients receiving single agent and combination checkpoint inhibition
.
J Clin Oncol
2019
;
37(15_suppl)
:
100
100
.
27.
Jin
Y
,
Sun
Y
,
Shi
X
,
Zhao
J
,
Shi
L
,
Yu
X
. 
Prognostic value of circulating C-reactive protein levels in patients with non-small cell lung cancer: a systematic review with meta-analysis
.
J Cancer Res Ther
2014
;
10(suppl)
:
S160
6
.
28.
Jing
X
,
Huang
C
,
Zhou
H
,
Li
C
,
Fan
L
,
Chen
J
, et al
Association between serum C-reactive protein value and prognosis of patients with non-small cell lung cancer: a meta-analysis
.
Int J Clin Exp Med
2015
;
8
:
10633
9
.
29.
Reck
M
,
Rodríguez-Abreu
D
,
Robinson
AG
,
Hui
R
,
Csőszi
T
,
Fülöp
A
, et al
Pembrolizumab versus chemotherapy for PD-L1–positive non–small-cell lung cancer
.
N Engl J Med
2016
;
375
:
1823
33
.
30.
Hurria
A
,
Togawa
K
,
Mohile
SG
,
Owusu
C
,
Klepin
HD
,
Gross
CP
, et al
Predicting chemotherapy toxicity in older adults with cancer: a prospective multicenter study
.
J Clin Oncol
2011
;
29
:
3457
65
.
31.
Hurria
A
,
Mohile
S
,
Gajra
A
,
Klepin
H
,
Muss
H
,
Chapman
A
, et al
Validation of a prediction tool for chemotherapy toxicity in older adults with cancer
.
J Clin Oncol
2016
;
34
:
2366
71
.
32.
Mohile
SG
,
Dale
W
,
Somerfield
MR
,
Schonberg
MA
,
Boyd
CM
,
Burhenn
PS
, et al
Practical assessment and management of vulnerabilities in older patients receiving chemotherapy: ASCO guideline for geriatric oncology
.
J Clin Oncol
2018
;
36
:
2326
47
.
33.
Dahabreh
IJ
,
Hayward
R
,
Kent
DM
. 
Using group data to treat individuals: understanding heterogeneous treatment effects in the age of precision medicine and patient-centred evidence
.
Int J Epidemiol
2016
;
45
:
2184
93
.
34.
Kent
DM
,
Nelson
J
,
Dahabreh
IJ
,
Rothwell
PM
,
Altman
DG
,
Hayward
RA
. 
Risk and treatment effect heterogeneity: re-analysis of individual participant data from 32 large clinical trials
.
Int J Epidemiol
2016
;
45
:
2075
88
.
35.
Sorich
MJ
,
Coory
M
. 
Interpreting the clinical utility of a pharmacogenomic marker based on observational association studies
.
Pharmacogenomics J
2013
;
14
:
1
.
36.
Kibbelaar
RE
,
Oortgiesen
BE
,
van der Wal-Oost
AM
,
Boslooper
K
,
Coebergh
JW
,
Veeger
N
, et al
Bridging the gap between the randomised clinical trial world and the real world by combination of population-based registry and electronic health record data: a case study in haemato-oncology
.
Eur J Cancer
2017
;
86
:
178
85
.
37.
Hosmer
DW
 Jr
,
Lemeshow
S
,
Sturdivant
RX
.
Applied logistic regression
, 3rd Edition.
John Wiley & Sons
; 
2013
.