Early Prediction of Disease Progression in Small Cell Lung Cancer: Toward Model-Based Personalized Medicine in Oncology

Lung cancer is the leading cause of cancer-related mortality around the world (1–3). In the United States and Europe, small cell lung cancer (SCLC) accounts for 13% to 20% of all new cases and deaths from lung cancer (4, 5). Treatment of SCLC is based on chemotherapy and radiotherapy. In the past 30 years, few therapeutic strategies have been successful in clinical trials (6) and many important questions regarding effective treatment strategies remain open (7, 8). These include the duration of the treatment, the role of maintenance and consolidation therapies, the role of chemoradiotherapy, alternative combination chemotherapy agents, and dose intensification. Although response rates (RR) to first-line treatment (generally combinations of platinum compound and etoposide) are high, RRs to second-line treatment are generally lower due to rapid onset of resistance (9). Among other factors, RRs to second-line treatment depend on the time elapsed between the end of first-line treatment and the detection of disease recurrence (9). Therefore, early identification of the individual risk of disease recurrence is a key component of effective clinical management.

Prognosis of untreated patients with SCLC is poor, with a median survival of 2 to 4 months after diagnosis. Currently, the main prognostic factor of SCLC is the disease stage at diagnosis (10). The most widely used staging classification is the dichotomous system developed by the Veterans Administration Lung Study Group. This method classifies patients with limited-stage disease (LD; those with tumor confined to the hemithorax of origin, the mediastinum, or the supraclavicular lymph nodes) and extensive-stage disease (ED; those whose tumor has spread outside these limits; ref. 11). SCLC is characterized by its fast growth and rapid onset of widespread metastases where approximately 30% of patients with SCLC are diagnosed as limited-stage and the remaining 70% as extensive-stage (4). The median survival for LD SCLC is 14 to 20 months, and the 2- and 5-year survival rates are 30% and 10%, respectively. For ED SCLC, median survival is 9 to 12 months, and survival after 2 years occurs in less than 5% of patients (4). In addition to stage at diagnosis, performance status and other clinical parameters, such as age, gender, and number and location of metastatic tumors, have been proposed as predictive and prognostic factors (10, 12–14).

Existing methods to detect disease progression in SCLC and most solid tumors generally rely on computed CT scans, where the change in the sum of longest diameters (SLD) is assessed. In particular, the RECIST criteria define disease progression as change of 20% in SLD or appearance of new metastases (15). Recommendations during follow-up of SCLC include a CT scan every 2 to 3 months (9, 16). This follow-up strategy, however, means that significant disease progression can already occur even before the recurrent disease is detected. In a retrospective study, Sugiyama and colleagues (17) showed that a more intensive follow-up strategy in SCLC patients that had achieved response to first-line treatment resulted in significantly higher postrelapse survival and overall median survival compared with nonintensive follow-up patients. However, more intensive follow-up programs may not be necessary for all patients because early identification of high-risk patients may enable targeted monitoring programs to ultimately improve individual patient survival.

The use of circulating biomarkers has been proposed for predicting patient outcome. However, despite the easy collection of those biomarkers in many tumor types, their use has had limited uptake due to the lack of validated markers of disease progression (18). One of the reasons is that establishing the link between biomarkers and tumor progression is not a trivial task. The prevailing strategy has been to use simple univariate and multivariate analysis to determine biomarker relevance. With SCLC, elevation of serum lactate dehydrogenase (LDH) is found in half of patients, and up to 85% of patients with ED (10, 19). Some studies have also proposed other biomarkers that may have prognostic significance, namely neuron-specific enolase (NSE), chromogranin, circulating tumor cells, and progastrin-releasing peptide (20–22). However, the prognostic and predictive value of these biomarkers has not been widely accepted in the context of standard clinical practice and therefore their use as predictive biomarkers has been limited (23). The empirical methods used to assess these biomarkers, however, may not provide an accurate representation of their utility because appropriate assessment of the predictive capacity of biomarkers should be in the context of mechanistic models (24–26). Indeed, we have previously shown that when these biomarkers are integrated into a mechanistic model of disease time-course, model-based disease predictions showed high correlation with outcome (27). These models allow for time-dependent and nonlinear relationships between biomarker and outcome to be accounted, which may not be detected by a simple assessment of correlation. Mechanistic models also allow data from multiple biomarkers to be integrated together to obtain improved predictors of response. With SCLC, we found that model-based disease predictions showed higher correlation with outcome than either biomarker individually.

The traditional approach for using markers for individual prediction generally relies on the comparison of biomarker levels with an established cutoff value (e.g., derived from receiver operating characteristic curve analysis). Indeed, our previously developed SCLC biomarker model could have been used in this manner for individual prediction. However, with cutoff approaches, individuals with similar disease profiles may have very different predictions of outcome if they lie close to the cutoff value. This unsatisfactory situation arises because a cutoff approach describes risk as a dichotomous rather than continuous variable. In contrast, a probabilistic model-based approach allows the continuous disease-risk relationship to be accurately modeled and provides quantitative predictions of individual risk. Patients predicted to have high probabilities of disease progression may therefore be provided with alternative therapeutic strategies while patients predicted to have less certain outcomes may be more intensively monitored.

On the basis of the above considerations, the main objective is to develop a framework that allows at-risk patients to be identified early so that therapeutic intervention and monitoring can be individualized. We use our previously reported biomarker model (27) to develop and illustrate this framework in SCLC.

Two datasets were available: a training and an external dataset. The training data were identical to the data used to develop and assess the previously reported biomarker model (27). The external dataset was retrospectively collected from the medical records of 22 patients diagnosed in the University Hospital of Navarra (Pamplona, Spain) between 2012 and 2014. We included all treatment-naïve SCLC patients (as determined by histologic examination) receiving first-line chemotherapy (i.e., etoposide plus either cisplatin or carboplatin). No patients satisfying these inclusion criteria were excluded from the analysis. Figure 1A depicts standard treatment and follow-up periods. Treatment typically included six chemotherapy cycles given every 3 weeks. Some patients also received concomitant radiotherapy in the primary tumor (hemithorax). Tumor assessments were performed with CT before the commencement of therapy, between the second and third cycles of chemotherapy, at the end of the entire course of chemotherapy, and every second/third month thereafter in follow-up examinations. Clinical outcome was categorized according to modified RECIST criteria, where the sum of the longest diameters is compared with the value of the previous CT scan. Because the medical records did not contain the exact tumor size measurements, only the derived RECIST categories were used in the data analysis. Biomarker (LDH and NSE) samples used to externally validate the model were obtained during the treatment period. The time courses of LDH and NSE are shown in Supplementary Fig. S1. The external dataset included a total of 137 LDH and 77 NSE observations during treatment period and a total of 78 CT scans, obtained during both treatment and follow-up periods. Out of the 22 patients, 7 patients (31.8%) had CT scans with disease progression (with three patients progressing just after finishing first-line treatment). A more detailed description of the data can be found in Table 1 that lists also the patients and study characteristics of the training dataset.

We have previously developed a population model describing the dynamics of LDH and NSE in SCLC patients (27). A schematic view of the published biomarker model and model equations can be found in Supplementary Fig. S2. In this study, we extend this biomarker model with a logistic model to link model predictions from the biomarker model with clinical outcome (observed disease progression). We then perform an external validation of both the biomarker and the logistic models. We then used the combined biomarker/logistic model to predict, at the end of first-line treatment, individual probabilities of disease progression in follow-up periods. Figure 1B shows a schematic diagram of the workflow.

Our previously published semi-mechanistic model for NSE and LDH dynamics has not yet been assessed using an external dataset (27). We therefore used the newly acquired external dataset to perform an external validation of biomarker predictions in these new patients.

External validation of the biomarker model at the individual level was based on basic goodness of fit plots. Individual parameters for the new set of patients were obtained in NONMEM 7.2 (28) by evaluating the empirical Bayes estimates (EBE). EBEs are conditional estimates of an individual's model parameters given the parameter estimates from the population model (27), the individual observations (LDH and NSE concentrations), and covariate information (radiotherapy and granulocyte colony-stimulating factor therapy) from patients in the external dataset.

The external validation at the population level was based on simulation-based diagnostics, namely prediction- and variability-corrected visual predictive checks (pvc-VPC; ref. 29) as implemented in PsN (30). Briefly, VPCs evaluate the model's ability to describe the central tendency and variability in the observed data by comparison of the observed quantiles to the quantiles of the simulated datasets. We produced pvc-VPCs by simulating 1,000 replicate datasets with the same characteristics of the external dataset (i.e., patient numbers, sample times, intervention, and covariate information) by sampling new sets of individual parameters from the estimated population parameter distributions (27).

RECIST data were dichotomized into two categories (disease progression and non-disease progression) and analyzed using by logistic regression. We fit a linear logit model to relate the probability of having disease progression at the CT scans to the model predicted changes in disease (Eq. 1) between each CT scan and its previous CT scan. The logit transformation (Eq. 2) ensures that the probability of having disease progression as outcome of a CT scan will fall between 0 and 1.

where D_i,j is the predicted disease level for patient j at observation point (CT scan) i, and ΔD_i,j is the predicted change in disease level for patient j between observation point i and i + 1 (i.e., the following CT scan, approximately 8 weeks later). P(DP_ij∣η) represents the probability of disease progression in the jth individual at the ith measurement (CT scan), η is a subject-level random effect (normally distributed with mean 0 and variance OMEGA) describing interpatient variability in the probability of disease progression, θ₁ is the intercept parameter and θ₂ is the slope parameter. To obtain meaningful initial parameter estimates, θ₁ and θ₂, were parameterized in terms of ΔD₈₀ and ΔD₅₀, which describe the ΔD predicted to cause a 50% and 80% probability of disease progression [P(DP)], respectively. Thus, the intercept and the slope take the form of θ₁ = −ΔD₅₀ × θ₂ and θ₂ = log(4)/(ΔD₈₀ − ΔD₅₀).

The logistic model was obtained using the CT scans from treatment and follow-up periods of patients belonging to the training dataset. Model parameters were estimated in NONMEM 7.2 using the SAEM algorithm (28). Model evaluation was based on the construction of VPCs, in a similar manner to the biomarker model evaluation. The VPC compared the observed proportion of patients showing disease progression in each CT scan with the model-predicted distribution of the same proportion attained in 1,000 simulated datasets. Because a model may perform well at the population level (i.e., it may predict well the proportion of patients progressing at each CT scan) but it may perform poorly with individual patient prediction, we also evaluated the predictive accuracy of the model at the individual level. This was achieved by applying the framework detailed in the next section (individual early prediction of disease progression) but modified to use all data available (i.e., biomarker and CT scans outcomes). The final output was a graphical comparison of the predicted individual distribution of P(DP) with the actual outcome of each CT scan.

To simulate decision making after completion of first-line treatment, we evaluated P(DP) predictions of the first two follow-up CT scans (i.e., the period 6 months after end of treatment) using data up to the end of first-line treatment, by comparing the predicted P(DP) distributions at the first and the second follow-up CT scans with the observed CT scan outcomes.

Biomarker (LDH and NSE) samples from patients in the external dataset obtained during the treatment period were used to create patient-specific disease profiles during treatment and follow-up using the Bayesian algorithm in NONMEM 7.2 (28). Changes in the distribution of predicted disease levels were computed in accordance with Eq. 1. The logistic model was then used to convert predicted changes in disease levels to probabilities of disease progression at the times where CT scans were collected (Eq. 2). In order to account for uncertainty in population and individual model parameters, we obtained the full uncertainty distribution of P(DP) predictions for each individual using Markov chain Monte Carlo (MCMC) evaluation in NONMEM 7.2 (28). In total, 1,000 MCMC samples were obtained from the stationary distribution with Gibbs sampling to obtain individual uncertainty distributions.

Results of the external validation (individual and population level) of the previously developed biomarker model (27) are shown in Fig. 2 and indicate that individual profiles (Fig. 2A), typical tendencies, and data dispersion (Fig. 2B) are well captured by the model.

Predicted P(DP) was obtained by logistic regression where the expression for the logit included a linear function of ΔD_i,j and ΔD_i,j is the change in predicted disease between two consecutive CT scans using the biomarker model (27). Estimation of interpatient variability in the logistic model was not supported by the data. The estimates of the transformed parameters (see Materials and Methods) ΔD₅₀ and ΔD₈₀ were 0.292 and 0.510 with relative standard errors of 20.2% and 30.2%, respectively.

Figure 3A shows the relationship between the typical probability of disease progression and ΔD_i,j. In Fig. 3B, the performance of the logistic model at the population level is shown for the training and external datasets. The predictive accuracy of the model at the individual level is shown in Supplementary Fig. S3. Results indicate adequate model performance in both cases.

P(DP) predictions for follow-up visits were obtained for new SCLC cancer patients (external dataset) assuming that only treatment period data were available (see Fig. 1). Figure 4 shows the schematic of the Bayesian framework developed for prediction. P(DP) uncertainty distributions were obtained using the full nonparametric distribution of the predicted disease in accordance with Eq. 2 (Fig. 4B–D).

Figure 5 shows the results comparing the predicted P(DP) with the observed patient status (i.e., disease progression vs. non-disease progression) at the first and second follow-up scans. For demonstration purposes, P(DP) predictions higher than 80% were deemed “high confidence” or “reliable” predictions of a disease progression outcome. Similarly, P(DP) predictions lower than 20% were deemed “high confidence” predictions of a non-disease progression outcome. Out of 24 CT scan measurements, 75% of the P(DP) predictions were prospectively identified as “high confidence” predictions. As can be seen in Fig. 5, all “high confidence” P(DP) predictions correctly predicted follow-up outcomes with 100% accuracy using treatment only data. The remaining 25% of the P(DP) predictions were deemed to be of insufficient confidence to provide a reliable prediction of outcome. A total of 4 individuals experienced disease progression in the two follow-up scans. Two of these four outcomes were prospectively identified as having “high confidence” P(DP) predictions.

In the past decade, great advances have been made in the area of personalized medicine, thanks to pharmacogenomics and targeted therapies (31–33). Likewise, great efforts have been made to develop new predictive biomarkers to personalize therapy to individuals (34, 35). In this work, we used known biomarkers together with probabilistic nonlinear mixed-effects models (NLME) within a framework to improve personalized disease monitoring.

The disease we have investigated together with the current monitoring strategy would traditionally be viewed as challenging for prediction due to the heterogeneity in response and highly dynamic nature of SCLC (36). On a conceptual level, the predictive capability of the model may be restricted by the ability to infer underlying disease processes from the available data. However, our success with predicting SCLC outcome suggests that the integrated NLME approach (combining individual biomarker and scan data) may have wider utility to other cancers even when available biomarkers have not historically been considered as predictive markers. In fact, NLME models have been used in the past to describe circulating biomarker dynamics and clinical outcome in different cancer indications. Some examples include the use of mathematical models in prostate cancer to personalize vaccination regimens to stabilize prostate-specific antigen levels (37, 38), the use of CA125 as a predictive marker of ovarian cancer (39), the kinetic modeling of human chorionic gonadotropin as an early predictor of methotrexate resistance in low-risk gestational trophoblastic neoplasia patients (40), and the use of soluble VEGF receptor 3 to monitor adverse events and clinical response in patients with imatinib-resistant gastrointestinal stromal tumors (41, 42). A key strength of NLME models is that they provide a description of individual variability in population level trends. NLME modeling also provides a rigorous statistical basis to individual prediction using Bayesian methodology, so that information from available individual level observations is balanced with knowledge and uncertainty in population level trends (43, 44).

The external dataset was used to evaluate the predictive accuracy of the previously reported biomarker model (27). Interestingly, NSE values in Table 1 are lower in the external dataset compared with the training dataset. This is likely due to differences in diagnostic techniques used between the two datasets leading to the Will Rogers phenomenon (45), where stage migration due to new methods of diagnosis leads to a systemic bias in measures of central tendency. Patients in the training dataset were staged using imaging techniques while patients included in the external dataset (more recently diagnosed) were staged according to the PET (Positron emission tomography) technique. Due to the highly metabolic nature of SCLC, it has been shown that the use of PET can increase staging accuracy, resulting in a higher percentage of patients upstaged from limited- to extensive-disease compared with patients downstaged from extensive- to limited-disease (46). Nevertheless, the biomarker and the RECIST model were capable to describe observations from the external dataset.

After successful external validation of the combined biomarker/logistic model, we investigated the feasibility of using data up to end of first-line treatment to obtain early prediction of disease progression in follow-up visits. Although we defined thresholds for “high confidence” P(DP) predictions to assess overall predictive accuracy in our external dataset, we do not recommend establishing arbitrary thresholds, such as P(DP)>80%, in clinical practice. This is because personalized dosing decisions should account for all available individual information, including patient characteristics, available treatments, and their expected tolerability and efficacy. We therefore envisage quantitative prediction of probability of progression as being one factor for oncologists to consider.

The surprising result of this investigation was that biomarker data gathered only during the treatment period was sufficient to provide reliable predictions for 75% of follow-up tumor assessment results. Moreover, disease progression was confidently predicted at the end of treatment for two out of the four cases present in the external dataset. Although the sample size here is limited, this suggests that the approach can be used to optimize the timing of commencement of second-line therapy in a significant proportion of patients. As an example, ID patient 107 completed first-line treatment on week 13 and was detected to be in disease progression in week 24. As shown in Fig. 5, our framework could have confidently predicted this outcome from the treatment-only data. Figure 6 corroborates this by showing that the model-predicted disease profile was increasing up to 13 weeks prior to the follow-up CT scan, a period where the patient was receiving no treatment. These model predictions could have been made at the end of first-line treatment, providing oncologists with the option to continue first-line therapy or commence second-line therapy immediately at the end of first-line treatment for this particular patient.

For the 25% of assessments where P(DP) were not deemed to be of “high confidence” (i.e., 20% < P(DP) < 80%), the framework fails to provide reliable predictions of outcome. However, physicians can still derive relevant information from the model: there is insufficient individual-level information for these patients and therefore these patients should be considered “at risk.” Intensive monitoring plans can then be followed to gather more individual level information. For example, for ID 100 (Fig 5), who progressed at the first follow-up scan, the uncertainty intervals indicate that the true P(DP) may be above 80%, so the decision could be taken to continue weekly or biweekly visits for additional biomarker monitoring as a precautionary measure. Another example of a “non-high confidence” prediction is patient 104, who had an expected P(DP) at second follow-up scan of 30.3% and progressed at this scan. Although this P(DP) does not fall into the P(DP)>80% range, 30.3% is not a small probability of progression and therefore this patient should also be considered at risk and monitored carefully. It is the opinion of the authors that although many patients were predicted not to progress with high confidence, a reduced monitoring schedule relative to existing practices would not be advisable due to the reliance of the RECIST criteria on an approximately fixed interval between CT scans.

On a statistical level, the main distinction between the proposed framework and traditional binary classification systems is that the output is a continuous probability measure associated with a binary state rather than a binary state itself. The consequence of unreliable predictions in binary classification systems is the potential for inappropriate treatment decisions for individuals. As discussed, though the primary strength of the proposed framework is the ability to identify a priori individuals with reliable/unreliable predictions and tailor treatment/monitoring strategies as appropriate, providing value to physicians even when there is significant individual uncertainty in underlying predictor variables.

Several limitations have already been stated in this discussion section and in the discussion section of Buil-Bruna and colleagues (27). All limitations applying to the previously reported biomarker model also apply here. Briefly, the limited numbers of patients available for the training dataset means that the ability to accurately quantify interpatient variability effects is reduced. Individual prediction in some instances may therefore be limited if the patient is atypical of individuals in the training dataset. A K-PD model was used to describe the relationship between treatment and drug effects due to the lack of pharmacokinetic information. The absence of PK information likely hampers the interpretation of the variability seen in the response. There was also insufficient information to incorporate differences in radiotherapy administration into the model.

In this work, the latent disease variable representing tumor size dynamics that was used to predict disease progression was inferred from two biomarkers. Although our results suggest that predictive biomarkers allow inference of the latent disease variable, actual tumor size measurements, when available, should be more informative. In fact, a recent review article summarizes several population models describing tumor growth dynamics using tumor size measurement data (47). However, medical records of the SCLC patients did not include exact tumor size measurements, which is a frequent occurrence with routinely collected medical data. In addition, there is a high degree of heterogeneity in the number, time, and frequency of samples (biomarker and CT scans) between patients, which translates into heterogeneity in the precision of individual P(DP) predictions. The proposed framework, however, accounts for these differences by quantifying the uncertainty in P(DP) predictions.

Optimal design methodology (48) could be applied to identify the optimum number and time of measurements to narrow prediction uncertainty.

We modeled the RECIST endpoint as a binary variable for simplicity. However, the additional step of modeling the four different RECIST categories (i.e., complete response, partial response, stable disease, and disease progression) may further increase the individual predictive accuracy.

Efficacy and toxicity associated with second-line therapies have not been formally considered in this analysis. Therefore, it is currently not possible to quantify what early commencement of second-line therapy would mean in terms of expected patient outcome (e.g., increased survival times or remission rates). In order to quantify the added value of early treatment in terms of measureable endpoints, a model-based analysis of efficacy and toxicity endpoints from second-line therapy would be required.

We recommend the use of the proposed prediction framework to support proactive personalized medicine in SCLC patients during and following first-line treatment. Development of the model-based framework required only patient medical record data and did not require commissioning of a new study. The feasibility of the approach should be investigated with other cancers.

No potential conflicts of interest were disclosed.

This work does not necessarily represent the view of all DDMoRe partners.

Conception and design: N. Buil-Bruna, T. Sahota, J.-M. López-Picazo, B. Ribba, I.F. Trocóniz

Development of methodology: N. Buil-Bruna, J.-M. López-Picazo, B. Ribba, I.F. Trocóniz

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): J.-M. López-Picazo, M. Moreno-Jiménez, S. Martín-Algarra

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): N. Buil-Bruna, T. Sahota, B. Ribba, I.F. Trocóniz

Writing, review, and/or revision of the manuscript: N. Buil-Bruna, T. Sahota, J.-M. López-Picazo, M. Moreno-Jiménez, S. Martín-Algarra, B. Ribba, I.F. Trocóniz

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): N. Buil-Bruna, I.F. Trocóniz

Study supervision: J.-M. López-Picazo, S. Martín-Algarra, I.F. Trocóniz

The authors thank M.C. Martínez-Cosgaya, I. Echenique-Gubía. and M.J. Martínez Alvarez-Nava for their kind help with data collection from University Clinic of Navarra.

N. Buil-Bruna was supported by a predoctoral fellowship from Asociación de Amigos de la Universidad de Navarra. This work was supported by the Innovative Medicines Initiative Joint Undertaking under grant agreement no. 115156, resources of which are composed of financial contributions from the European Union's Seventh Framework Programme (FP7/2007-2013) and EFPIA companies' in-kind contribution. The DDMoRe project is also supported by financial contribution from academic and SME partners.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.