Purpose: To develop a single validated model for survival in metastatic renal cell carcinoma (mRCC) using a comprehensive international database.

Experimental Design: A comprehensive database of 3,748 patients including previously reported clinical prognostic factors was established by pooling patient-level data from clinical trials. Following quality control and standardization, descriptive statistics were generated. Univariate analyses were conducted using proportional hazards models. Multivariable analysis using a log–logistic model stratified by center and multivariable fractional polynomials was conducted to identify independent predictors of survival. Missing data were handled using multiple imputation methods. Three risk groups were formed using the 25th and 75th percentiles of the resulting prognostic index. The model was validated using an independent data set of 645 patients treated with tyrosine kinase inhibitor (TKI) therapy.

Results: Median survival in the favorable, intermediate and poor risk groups was 26.9 months, 11.5 months, and 4.2 months, respectively. Factors contributing to the prognostic index included treatment, performance status, number of metastatic sites, time from diagnosis to treatment, and pretreatment hemoglobin, white blood count, lactate dehydrogenase, alkaline phosphatase, and serum calcium. The model showed good concordance when tested among patients treated with TKI therapy (C statistic = 0.741, 95% CI: 0.714–0.768).

Conclusions: Nine clinical factors can be used to model survival in mRCC and form distinct prognostic groups. The model shows utility among patients treated in the TKI era. Clin Cancer Res; 17(16); 5443–50. ©2011 AACR.

Translational Relevance

Understanding of factors prognostic for survival among patients with metastatic renal cell cancer (mRCC), in particular, those reported here which have been captured over many years in many parts of the globe, will contribute to more thorough understanding of mechanisms of metastasis and better treatment for the disease. Although the studies reported here predate the current era of treatment with tyrosine kinase inhibitors (TKI), the continued relevance of these factors is shown.

Renal cell carcinoma (RCC) is one of the most common adult cancers, with an incidence of 58,240 new cases and 13,040 deaths in the United States in 2011 (1). Worldwide, there were 273,518 new cases and 116,368 deaths in 2008, the most recent year for which global statistics are available (2). Although the proportion of patients with metastatic disease at diagnosis has declined due to more intense screening and incidental case ascertainment, the relationship between early diagnosis and mortality is confounded by the length and lead-time biases that accompany estimates of pre-clinical and clinical time intervals in the presence of advancing technology (2). Unfortunately, approximately one-third of patients will eventually develop metastatic disease. Even with the recent introduction of vascular endothelial growth factor directed therapies, the long-term prognosis for patients with metastatic renal cell carcinoma (mRCC) is poor, with median survival under 2 years.

The natural history of mRCC is quite variable, and therefore an important consideration in the evaluation and development of new treatment strategies is the role of factors that are predictive of outcome. Use of known prognostic factors can help direct therapies to patient groups most likely to benefit from them and can help identify patients for which watchful waiting is a suitable alternative, thus potentially preserving alternatives for later use. Understanding prognostic factors can aid in the interpretation of clinical trials by helping to assess the magnitude of differences between treatments. In addition, consideration of known prognostic factors can help determine the extent to which new therapies are altering the natural history of the disease, and can be used as a backdrop against which new, potentially important prognostic factors can be evaluated to determine if they add independent information, or if their prognostic value is due simply to correlations with known factors.

A number of large retrospective studies of prognostic factors in mRCC had been reported prior to the initiation of this project (3–9). These investigations focused primarily on factors prognostic for survival and generally concentrated on clinical data that were collected as part of the normal patient work-up. Unfortunately, although these investigations consistently identified several factors, such as performance status and metastasis-free interval as being independent predictors of survival, the definitions of these predictors varied, and none of the investigations evaluated the same set of factors. In addition, few investigations were validated; for example, by using statistical re-sampling methods such as the bootstrap (6), or by applying the results to an independent set of data (3). Consequently, a comprehensive, well-characterized, validated set of prognostic factors that could be utilized to help evaluate clinical investigations of mRCC and further our understanding of the natural history of the disease remained to be defined.

Under the auspices of the Kidney Cancer Association, the International Kidney Cancer Working Group was convened and the first meeting held in October 2002, in Lyon, France, to discuss this issue. The consensus of the meeting was that development of a prognostic factors scoring system for survival in mRCC was needed and a collaborative study group was formed with the objective of defining such a system.

The primary goals of this project were to establish a comprehensive database of potential prognostic factors for survival in mRCC, and to derive a single, validated prognostic scoring system that could be used to define risk groups of patients. A secondary goal was to establish the database as a resource that could be used to study the natural history of mRCC and as an aid in the design, analysis, and interpretation of clinical studies of this disease.

Project organization and database assembly

On October 17 and 18, 2002, researchers from the United States, France, England, and Israel gathered in Lyon, France, to discuss how they could work together to identify prognostic factors in metastatic renal cell cancer. The meeting was organized by the Kidney Cancer Association, based in Evanston, Illinois, and representatives from Centre Léon Bérard in Lyon. Participants included investigators who had previously identified prognostic factors based on retrospective experience, leaders in clinical research and pathology in renal cell cancer, and representatives from Cooperative Groups with experience conducting randomized trials in renal cell cancer. Following the meeting, an international consortium of institutions/organizations with special interest and expertise in mRCC was established. A steering committee was established to oversee the project and to prioritize use of the database once the primary goals of the project were achieved. A protocol was then written which outlined the variables to be collected, the acceptable platforms for data collection and transmission, and how patient confidentiality would be assured. Criteria for data set inclusion in the project were also specified and consisted of the following:

  • Minimum of 100 patients available for submission;

  • Patients diagnosed with advanced renal cell cancer, defined as Stage IV disease according to the staging system in use at the time of study entry, metastatic disease, or disease with advanced tumor stage and positive lymph nodes;

  • Enrolled on an institutional review board (IRB)-approved clinical trial between January 1, 1975 and December 31, 2002;

  • No prior systemic therapy; and

  • Data from the clinical trial should have been previously published.

An effort was made to identify all available data sets available to the consortium that met these criteria. In addition, other groups with potentially eligible data sets were also contacted. After all known groups had been given an opportunity to participate, final data sets were identified; and each participant was sent specific instructions regarding data collection and transmission. The patient-specific data collected included type of treatment on trial, year of entry, patient and disease characteristics, and laboratory parameters, with their associated normal limits. Data sets were forwarded to statisticians at Cleveland Clinic Foundation, who collated data into a single analysis data set. The collated data set was then forwarded to the statistical offices of the Eastern Cooperative Oncology Group (ECOG), Memorial Sloan-Kettering Cancer Institute, and the United Kingdom Medical Research Council for analysis. All analysis sites were responsible for assuring the confidentiality and security of data stored at the site, and all analysis sites operated under oversight of a local IRB. A patient index number was employed, but no patient identifiers such as name or medical record number were included in the data sets.

Data standardization and transformation

Some standardization was done to assure consistency of reporting units. Karnofsky performance status (KPS) was converted to ECOG performance status (PS) by considering KPS of 100 equal to ECOG PS of 0, KPS of 80 to 90 equal to ECOG PS of 1, and KPS 70 or less equal to ECOG PS 2 or more (10). Values of hemoglobin, creatinine, calcium and bilirubin reported in SI units were converted to the more commonly reported standard units using published conversion factors.

Exploratory and univariate analyses

The primary endpoint was survival, defined as time from the date of entry onto the clinical trial to the date of death or the date last known alive.

Descriptive statistics such as means, medians, CIs, ranges, and proportions were used to characterize patients. The method of Kaplan and Meier (11) was used to estimate survival and generate plots, and the log-rank test was used to determine differences attributable to prognostic factors. Hazard ratios and 95% CIs were estimated using Cox proportional hazards models (12).

Multivariable analyses and model construction

Five variables [hemoglobin, white blood count, lactate dehydrogenase (LDH), alkaline phosphatase, and calcium] were standardized by dividing by their site- and gender-specific median value and then multiplying by their gender-specific median to transform the values to a common scale. Stata's ice package (Stata Corporation, College Station, TX; ref. 13) was used to generate 10 imputations of the data set. Survival time was represented in the imputation model by the empirical cumulative hazard function for each individual, and the censoring indicator. Variables having a skew distribution (white blood count, alkaline phosphatase, days from diagnosis to study entry, and lactate dehydrogenase) were transformed toward normality using Box–Cox methodology before imputation and back-transformed afterwards.

Cox proportional hazards models were used for initial exploratory analyses. After observing significant nonproportionality of the hazard in the Cox model and exploring several other models, a log–logistic survival model, stratified by participating center, was determined to fit the data well. Next, multivariable fractional polynomial models were built for each of the 10 imputations separately, using Stata's “streg” and “mfp” commands with option “distribution(llogistic)” to fit the log–logistic survival model. A significance level of 0.0044 was used for selection of variables. This significance level gives results similar to use of the BIC (Bayesian information criterion) and helps to avoid creating overcomplex models. A model was selected using each imputed data set, and the final model was obtained as the majority vote across the imputations. Rubin's rules (14) for multiply imputed estimates were used to combine the estimates and compute SE. Just as in a logistic regression model, each estimate (regression coefficient) represented the change in the log odds of death up to any given time point for a unit change in the prognostic factor.

Once a final model was defined, patients were divided into risk groups in 2 ways: 3 groups according to low, medium, and high risk (placing cut points at the 25th and 75th percentiles of the model's risk score distribution); and 10 groups, using Cox's (15) cut points. The latter minimize the loss of information for a given number of groups. Because the use of 3 risk groups is familiar in the clinical setting, the 3-group paradigm is used hereafter to characterize the model.

Internal and external validation and calibration

The model's discriminatory value was estimated using the C index and its 95% CI, as implemented by Pencina and d'Agostino (16). Proportion of variability explained by the model was further explored using the D index developed by Royston and Sauerbrei (17).

The model was also externally validated using an independent data set of 645 patients treated with tyrosine kinase inhibitor (TKI) therapy (18). To do this, we calculated a risk score for each patient in the validation data set using the final model equation described above. That is, weights derived from the model were applied to the value of each factor for each patient, with factors transformed if necessary (see Table 3). Missing factors were imputed for each patient as the mean value of the (transformed) factor in the derivation data set. The model's discriminatory value in the validation data set was assessed using the C index. Patients were divided into risk groups using the 25th and 75th percentiles of the distribution of scores in the validation data set.

Table 1.

Contributing institutions/groups

Institution/group and country or regionPatients
Groupe Français d'Immunothérapie, France 777 
Eastern Cooperative Oncology Group, United States 743 
Memorial Sloan-Kettering Cancer Institute, United States 551 
Cleveland Clinic Foundation, United States 351 
Medical Research Council, United Kingdom 341 
Cytokine Working Group, United States 199 
EORTC, Europe 176 
University of Washington, United States 170 
UCLA, United States 159 
Norwegian Radium Hospital 153 
Our Lady of Mercy Cancer Center, United States 128 
Institution/group and country or regionPatients
Groupe Français d'Immunothérapie, France 777 
Eastern Cooperative Oncology Group, United States 743 
Memorial Sloan-Kettering Cancer Institute, United States 551 
Cleveland Clinic Foundation, United States 351 
Medical Research Council, United Kingdom 341 
Cytokine Working Group, United States 199 
EORTC, Europe 176 
University of Washington, United States 170 
UCLA, United States 159 
Norwegian Radium Hospital 153 
Our Lady of Mercy Cancer Center, United States 128 
Table 2.

Univariate analyses of variables included in the model

Parameter% MissingLevelMedian survival, yTotalFailedLog-rank P
Protocol treatment: immunotherapy None 0.55 1030 1008 <0.0001 
  IFN only 1.01 1034 918  
  IL2 only 1.22 705 590  
  IFN and IL2 1.19 979 796  
Prior radiotherapy 9.6% No 1.02 2780 2398 <0.0001 
  Yes 0.74 608 577  
Lung metastases 1.0% No 0.96 1033 919 0.29 
  Yes 0.91 2676 2367  
Liver metastases 3.4% No 1.00 2915 2571 <0.0001 
  Yes 0.59 706 657  
Bone metastases 4.3% No 1.01 2508 2198 <0.0001 
  Yes 0.74 1077 996  
Retroperitoneal or mediastinal lymph node or local recurrence 38.6% No 0.94 644 571 0.13 
  Yes 0.87 1656 1487  
Number of metastatic sites 1.21 1387 1173 <0.0001 
  0.92 1392 1243  
  0.74 684 623  
  0.53 232 224  
  5 or more 0.56 53 49  
Sex Male 0.94 2631 2305 0.13 
  Female 0.91 1117 1007  
Performance status 1.3% 1.50 1467 1198 <0.0001 
  0.77 1836 1686  
  ≥2 0.36 395 384  
Year on study 1975–1984 0.55 661 653 <0.0001 
  1985–1994 0.96 1727 1580  
  1995–2002 1.17 1360 1079  
Age at study entry 0.1% ≤58 0.96 1988 1740 0.01 
  >58 0.89 1755 1569  
Days from diagnosis to study entrya 0.6% ≤134 0.73 1865 1685 <0.0001 
  >134 1.19 1862 1612  
Hemoglobin (g/dL)a 5.4% ≤12.5 0.64 1801 1677 <0.0001 
  >12.5 1.42 1746 1472  
Alkaline phosphatase (U/L)a 40.8% ≤99.0 1.46 1124 947 <0.0001 
  >99.0 0.84 1093 975  
Lactate dehydrogenase (U/L)a 37.0% ≤193.0 1.20 1190 1059 <0.0001 
  >193.0 0.76 1170 1075  
       
White blood count (K/μL)a 42.1% ≤7.6 1.37 1132 929 <0.0001 
  >7.6 0.82 1129 1003  
Calcium (mg/dL)a 31.6% ≤9.5 0.94 1301 1157 0.16 
  >9.5 0.88 1263 1110  
    % Failed   
Group 1.25 80.6% <0.0001  
  0.55 99.2%   
  0.89 95.5%   
  1.09 84.6%   
  1.80 64.1%   
  0.83 69.9%   
  1.17 80.4%   
  0.67 94.1%   
  1.11 98.0%   
  1.08 79.9%   
  1.47 85.9%   
Parameter% MissingLevelMedian survival, yTotalFailedLog-rank P
Protocol treatment: immunotherapy None 0.55 1030 1008 <0.0001 
  IFN only 1.01 1034 918  
  IL2 only 1.22 705 590  
  IFN and IL2 1.19 979 796  
Prior radiotherapy 9.6% No 1.02 2780 2398 <0.0001 
  Yes 0.74 608 577  
Lung metastases 1.0% No 0.96 1033 919 0.29 
  Yes 0.91 2676 2367  
Liver metastases 3.4% No 1.00 2915 2571 <0.0001 
  Yes 0.59 706 657  
Bone metastases 4.3% No 1.01 2508 2198 <0.0001 
  Yes 0.74 1077 996  
Retroperitoneal or mediastinal lymph node or local recurrence 38.6% No 0.94 644 571 0.13 
  Yes 0.87 1656 1487  
Number of metastatic sites 1.21 1387 1173 <0.0001 
  0.92 1392 1243  
  0.74 684 623  
  0.53 232 224  
  5 or more 0.56 53 49  
Sex Male 0.94 2631 2305 0.13 
  Female 0.91 1117 1007  
Performance status 1.3% 1.50 1467 1198 <0.0001 
  0.77 1836 1686  
  ≥2 0.36 395 384  
Year on study 1975–1984 0.55 661 653 <0.0001 
  1985–1994 0.96 1727 1580  
  1995–2002 1.17 1360 1079  
Age at study entry 0.1% ≤58 0.96 1988 1740 0.01 
  >58 0.89 1755 1569  
Days from diagnosis to study entrya 0.6% ≤134 0.73 1865 1685 <0.0001 
  >134 1.19 1862 1612  
Hemoglobin (g/dL)a 5.4% ≤12.5 0.64 1801 1677 <0.0001 
  >12.5 1.42 1746 1472  
Alkaline phosphatase (U/L)a 40.8% ≤99.0 1.46 1124 947 <0.0001 
  >99.0 0.84 1093 975  
Lactate dehydrogenase (U/L)a 37.0% ≤193.0 1.20 1190 1059 <0.0001 
  >193.0 0.76 1170 1075  
       
White blood count (K/μL)a 42.1% ≤7.6 1.37 1132 929 <0.0001 
  >7.6 0.82 1129 1003  
Calcium (mg/dL)a 31.6% ≤9.5 0.94 1301 1157 0.16 
  >9.5 0.88 1263 1110  
    % Failed   
Group 1.25 80.6% <0.0001  
  0.55 99.2%   
  0.89 95.5%   
  1.09 84.6%   
  1.80 64.1%   
  0.83 69.9%   
  1.17 80.4%   
  0.67 94.1%   
  1.11 98.0%   
  1.08 79.9%   
  1.47 85.9%   

aShown split at the median. These factors were used as continuous variables in the model. More information about the distribution of the variables is available in the supplemental tables.

Table 3.

Parameter estimates and standard errors–final model

βSE
Square root of days from diagnosis to study entry −0.0192 0.002 
ECOG performance status 0 −1.524 0.11 
ECOG performance status 1 −0.838 0.11 
Number of metastatic sites 0.324 0.032 
Protocol immunotherapy −0.574 0.094 
Natural log of hemoglobin −2.47 0.20 
Natural log of LDH 0.611 0.062 
Square root of white blood count 0.623 0.071 
1/Square root of alkaline phosphatase −6.665 1.39 
Serum calcium 0.105 0.033 
βSE
Square root of days from diagnosis to study entry −0.0192 0.002 
ECOG performance status 0 −1.524 0.11 
ECOG performance status 1 −0.838 0.11 
Number of metastatic sites 0.324 0.032 
Protocol immunotherapy −0.574 0.094 
Natural log of hemoglobin −2.47 0.20 
Natural log of LDH 0.611 0.062 
Square root of white blood count 0.623 0.071 
1/Square root of alkaline phosphatase −6.665 1.39 
Serum calcium 0.105 0.033 

The model was then recalibrated using the proportional odds recalibration model described by Miller and Hui (19). The fitted model contains an intercept term to represent the baseline level of risk in the population, a term representing the gradient of risk over time across all prognostic subgroups, and a term for the slope on the prognostic index, representing the predictive ability of the index. The recalibrated model was used to evaluate the fit of the prognostic index model and can be used to estimate survival probabilities for individual patients.

Final database and model

A total of 3,748 patients were treated by 11 groups and contributed information to the project. Table 1 summarizes the contributing groups and data sets. Patients began treatment between 1975 and 2002. European groups enrolled 39% of the patients and U.S. groups enrolled 61%.

Variables for which more than 40% of the observations were missing were not considered as potential predictors. Other variables or alternative parameterizations of factors were considered but not included in the final model-building process. Although not considered for the model, the information contained in these variables may be of use to other investigators; therefore, these are described in Supplemental Tables S1 through S4.

Table 2 provides the frequency of predictor variables tested for inclusion, including median survival, percent missing, number of events, and log-rank P values from univariate analyses.

The final model was a composite derived from each of 10 imputed data sets; variables included in each of the candidate models are shown in Supplemental Table S5. Table 3 shows the form of the variables used, and the associated parameter estimates (regression coefficients or weights) and SE from the final model. The prognostic index from the model is the sum of the variables, each multiplied by its weight. A positive weight attached to a prognostic factor indicates higher mortality with higher levels of the factor, and a negative weight, lower mortality; similarly for the prognostic index. Three risk groups were formed by splitting at index values of −2.755 (25th percentile) and −1.253 (75th percentile), where patients with index values less than −2.755 had the most favorable outcome. Figure 1 (solid line) shows overall survival for each of the risk groups.

Figure 1.

Overall survival by risk category for the original data set (solid line) and validation data set (dashed line).

Figure 1.

Overall survival by risk category for the original data set (solid line) and validation data set (dashed line).

Close modal

Internal and external validation

Internal validation of the model yielded a C index of 0.712 (95% CI, 0.703–0.721). This indicates that the model has good discriminatory ability and suggests that subjects with longer predicted survival times actually lived longer. Using the measure proposed by Royston and Sauerbrei to estimate the proportion of variability explained by the model, we estimate that the model's D index is 0.320 (95% CI, 0.295–0.345), which is quite reasonable for oncology studies.

External validation was conducted using data from Heng and colleagues (18). Characteristics of the patients in the validation data set are shown in Table 4. The regression equation resulting from the model fitting process described above was applied to each patient in the validation data set. Weighted values used in place of missing observations are shown in Supplemental Table S6. Some differences between the data set used to build the model and the validation data set should be noted. First, the validation data set did not contain information about patients' baseline alkaline phosphatase. White blood count was not available, but neutrophils were measured. An equation was derived for estimating white blood count from neutrophils using a separate kidney cancer data set where both were available. As a result, white blood count was defined as 0.69 times the neutrophil count plus 1.90 times the square root of neutrophils. Performance status was provided using the Karnofsky index; the rules provided in the methods section were used to transform the values to ECOG performance status. Whereas the regression equation treated the number of metastatic sites as a whole number (1, 2, 3…), the validation data set used a binary coding, where 1 represented greater than 1 metastatic site. A value of 0 was considered to represent 1 metastatic site and a value of 1 was considered to represent 2 metastatic sites. The model contained total calcium, whereas the validation data set contained corrected calcium.

Table 4.

Characteristics and univariate analyses of model variables for patients in validation data set

Parameter% MissingLevelMedian survival, yTotalFailedLog-rank P
Days from diagnosis to treatment starta ≤506 1.31 212 111 <0.0001 
  >506 2.44 167 155  
Karnofsky performance status 4.7% ≤80 1.07 336 243 <0.0001 
  >80 3.24 279 115  
Number of metastatic sites 2.16 156 81 0.11 
  >1 1.71 489 298  
Prior immunotherapy No 1.71 431 234 0.82 
  Yes 1.83 214 145  
Hemoglobin (g/dL)a 5.1% ≤12.9 1.23 310 218 <0.0001 
  >12.9 2.71 302 142  
LDH (U/L)a 15.7% ≤197.0 2.21 273 144 0.0016 
  >197.0 1.62 271 173  
Neutrophils (K/μL)a 9.6% ≤4.86 2.55 293 148 <0.0001 
  >4.86 1.19 290 193  
Corrected calciuma 7.4% ≤9.48 2.41 305 159 <0.0001 
  >9.48 1.36 292 192  
Parameter% MissingLevelMedian survival, yTotalFailedLog-rank P
Days from diagnosis to treatment starta ≤506 1.31 212 111 <0.0001 
  >506 2.44 167 155  
Karnofsky performance status 4.7% ≤80 1.07 336 243 <0.0001 
  >80 3.24 279 115  
Number of metastatic sites 2.16 156 81 0.11 
  >1 1.71 489 298  
Prior immunotherapy No 1.71 431 234 0.82 
  Yes 1.83 214 145  
Hemoglobin (g/dL)a 5.1% ≤12.9 1.23 310 218 <0.0001 
  >12.9 2.71 302 142  
LDH (U/L)a 15.7% ≤197.0 2.21 273 144 0.0016 
  >197.0 1.62 271 173  
Neutrophils (K/μL)a 9.6% ≤4.86 2.55 293 148 <0.0001 
  >4.86 1.19 290 193  
Corrected calciuma 7.4% ≤9.48 2.41 305 159 <0.0001 
  >9.48 1.36 292 192  

aShown split at the median. These factors were used as continuous variables in the model.

The overall C index was estimated for the validation data set to be 0.741 (95% CI, 0.714–0.768). The D index for the model was 0.321 (95% CI, 0.296–0.345). Thus, despite the differences between the 2 data sets, we conclude that the model also has good discriminatory ability among patients treated with TKI therapy. The proportion of variability explained by the model in the 2 data sets is quite similar. Figure 1 (dotted lines) plots survival for each of the risk groups derived from the validation data set.

To further evaluate the model, a proportional odds recalibration model was applied to the validation data set. If the original model fits the validation data set well, then (a) the constant terms will be similar, reflecting similar baseline risks, (b) the gradients of risk over time in the 2 data sets will be similar, and (c) the slope on the prognostic index will be similar between the 2 data sets. Coefficients for these parameters for both data sets are shown in Supplemental Table S7. While the gradients over time were similar between the 2 data sets (1.733 vs. 1.737) and the coefficients corresponding to the slope were similar (1.0 vs. 1.049), the baseline risk for patients in the validation data set was much lower than in the derivation data set (1.019 vs. 2.005). This is consistent with Figure 1 and our belief that treatment with TKI therapy confers an improvement in overall survival compared with previous therapies.

The constant parameter from the calibrated model was used with the other parameter estimates from the derivation model to provide a prediction model suitable for use with patients treated in the TKI era.

Given evidence that the baseline risk of death is lower among patients treated in the TKI era, cutpoints for the risk categories were adjusted to balance the group sizes. Table 5 provides a summary of overall survival by risk category, showing the cutpoints for each group. Risk categories should be assigned differently for patients treated in the pre-TKI era and the TKI era. Figure 1 (dotted line) shows overall survival by risk group for the validation data set. Table 5 also provides a summary of overall survival for each of the data sets.

Table 5.

Summary of overall survival by risk category

Cut pointsPatientsDeathsMedian survival, y95% CI
Model data set Overall  3748 3312 0.93 0.88–0.98 
 Good risk ≤ −2.755 937 698 2.24 2.10–2.44 
 Intermediate risk > −2.755 and ≤ −1.253 1874 1703 0.96 0.89–1.01 
 Poor risk > −1.253 937 911 0.35 0.33–0.37 
       
Validation data set Overall  645 379 1.78 1.62–1.99 
 Good risk ≤−2.67 161 61 3.72 2.94–nyra 
 Intermediate risk > −2.67 and ≤ −1.05 324 188 1.88 1.65–2.25 
 Poor risk >1.05 160 130 0.56 0.49–0.78 
Cut pointsPatientsDeathsMedian survival, y95% CI
Model data set Overall  3748 3312 0.93 0.88–0.98 
 Good risk ≤ −2.755 937 698 2.24 2.10–2.44 
 Intermediate risk > −2.755 and ≤ −1.253 1874 1703 0.96 0.89–1.01 
 Poor risk > −1.253 937 911 0.35 0.33–0.37 
       
Validation data set Overall  645 379 1.78 1.62–1.99 
 Good risk ≤−2.67 161 61 3.72 2.94–nyra 
 Intermediate risk > −2.67 and ≤ −1.05 324 188 1.88 1.65–2.25 
 Poor risk >1.05 160 130 0.56 0.49–0.78 

anyr = not yet reached.

Supplemental Table S6 can be used as a worksheet to calculate the risk score for a patient treated in the TKI era. The cutpoints used for deriving and validating the model were established using data sets with patterns of missing data unlikely to exist for individual patients. For purposes of illustrating and validating the model, missing values were naively estimated, presenting a conservative portrait of the test and validation data sets. To provide more accurate cutpoints for the prediction model shown in Supplemental Table S6, we applied the previously described multiple imputation techniques to the validation data set to minimize the impact of missing data while establishing cutpoints. The “complete-data” cutpoints shown in Supplemental Table S6 (−2.70 and −0.98) are appropriate for prospectively assigning risk categories to future patients treated in the TKI era.

Supplemental Figures S1 and S2 and Supplemental Table S8 show how the model performs in subgroups defined by protocol treatment. The model distinguishes risk groups within all of these subsets. As expected, patients in favorable subsets have improved survival within each risk category compared with those in the contrasting, less-favorable subsets.

Through this international effort, a database of 3,748 patients has been assembled. We believe that this is the largest database ever assembled for prognostic factor evaluation in mRCC. Using this database, we proposed a single model that accurately estimates the risk of death from kidney cancer, which is based on a comprehensive assessment of previously reported clinical prognostic factors. We showed that it retains value in the era of patients treated with TKIs.

The number of reports of factors prognostic for survival among patients treated prior to the TKI era is extensive; such reports among patients treated with TKIs are only beginning to emerge as information about survival endpoints on randomized trials accumulates. The report by Heng and colleagues (18), whose data were used to validate our model, is among the first of such reports. The model they reported, based on consecutive case series from multiple institutions, included ECOG performance status, time from diagnosis to treatment, and the laboratory parameters of hemoglobin, corrected calcium, neutrophil count, and platelet count. All of these except platelet count were represented in our model; platelets were unavailable for over half the patients in our data set and were thus not considered. While the numbers are not directly comparable since different methods were used for calculating them, the c-index of 0.741 achieved by this model in the validation data set was very similar to the bias-adjusted concordance statistic of 0.73 reported by Heng and colleagues themselves. Patil and colleagues (20) report prognostic factors for overall survival from a phase III trial of sunitinib versus interferon-alpha as first-line therapy for mRCC. Among patients randomized to sunitinib, prognostic factors included time from diagnosis to treatment, serum LDH, corrected calcium, hemoglobin level, ECOG performance status, and the presence of bone metastases. All of these except ECOG performance status were also prognostic among patients randomized to interferon-alfa. Motzer and colleagues (21), in their report of a second-line trial of everolimus for mRCC, identified poor performance status, high corrected calcium, low hemoglobin, and prior treatment with sunitinib as prognostic for shorter survival. These reports substantiate our finding that factors that were important before the introduction of TKI therapy remain so in the current era.

In the prior era, it was generally acknowledged that, while important in the localized setting, biologic features of the tumor were not important prognostic factors in the metastatic setting. This was suggested by the current analysis as well. Although the rate of missing information was too high to consider the factors for inclusion in the model, univariate assessments of histology, and grade failed to show prognostic significance. In the modern era, the advent of molecular-based technologies has led to a better understanding of the underlying biology of RCC, and an increasing number of molecular level factors are being reported to be of prognostic and/or predictive value in the metastatic setting. It will be important, therefore, to assess these new markers within the context of the current model to evaluate fully their clinical utility.

The study has limitations that should be acknowledged. Patients were treated between 1975 and 2002, representing a heterogeneous period for treatment of the disease. While the use of high-dose interleukin-2 was approved for treatment of mRCC during this time, many of the other experimental treatments were ineffective. Thus, the database may, to a great extent, represent the natural history of the disease in the absence of an effective therapy. Heterogeneity of the data sources, while enhancing the generalizability of the model, imposes some restrictions on the choice of potential prognostic factors by limiting the model to the set of commonly available factors. Use of the validation data set required transforming some variables and making assumptions about others. The analysis is thus not ideal, but the ability of the model to show such good concordance in the face of these compromises is encouraging with respect to the model's future utility.

While it is possible that differences or imbalances between the data sets exist, the improvement in overall survival reflected in the validation data set suggests that patients treated today have a favorable shift in prognosis compared with previously treated patients, almost equivalent to a risk category. As illustrated in Figure 1, today's intermediate risk patients have survival that is similar to that of good risk patients treated during the chemotherapy and cytokine eras. While this is encouraging, the need remains for additional treatment options as second-line therapy and for more effective treatments in general for patients with mRCC.

T. Eisen has other commercial research support from Bayer, Pfizer, and AZ and has honoraria from Bayer, Pfizer, AZ, Roche, Boehringer, Ingleheim, and BMS. The other authors disclosed no potential conflicts of interest.

The authors would like to thank the Kidney Cancer Association for its support, and in particular the family of Maita Lyn Gerber, who provided financial support for the project. We are grateful to Wanling Xie and the contributors of the International mRCC Database Consortium for their contributions related to the external validation database. The advice of Hajime Uno in the use and interpretation of concordance statistics was valuable. Finally, we extend sincere appreciation to the groups who contributed data and to the patients who were treated on the clinical trials.

Travel support for the project was provided by the Kidney Cancer Association.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
Jemal
A
,
Siegel
R
,
Xu
J
,
Ward
E
Cancer statistics, 2010
.
CA Cancer J Clin
2010
;
60
:
277
300
.
2.
Ferlay
J
,
Shin
HR
,
Bray
F
,
Forman
D
,
Mathers
C
,
Parkin
DM
. 
GLOBOCAN 2008, Cancer incidence and mortality worldwide: IARC cancer base No. 10 [Internet]
.
Lyon, France
:
International Agency for Research on Cancer
; 
2010
.
Available from
: http://globocan.iarc.fr.
3.
Hollingsworth
JM
,
Miller
DC
,
Daignault
S
,
Hollenbeck
BK
. 
Rising incidence of small renal masses: a need to reassess treatment effect
.
J Natl Cancer Inst
2006
;
98
:
1331
4
.
4.
Elson
PJ
,
Witte
RS
,
Trump
DL
. 
Prognostic factors for survival in patients with recurrent or metastatic renal cell carcinoma
.
Cancer Res
1988
;
48
:
7310
3
.
5.
Palmer
PA
,
Vinke
J
,
Philip
T
,
Negrier
S
,
Atzpodien
J
,
Kirchner
H
, et al
Prognostic factors for survival in patients with advanced renal cell carcinoma treated with recombinant interleukin-2
.
Ann Oncol
1992
;
3
:
475
80
.
6.
Fossa
SD
,
Kramar
A
,
Droz
J-P
. 
Prognostic factors and survival in patients with metastatic renal cell carcinoma treated with chemotherapy or interferon-a
.
Eur J Cancer
1994
;
30
:
1310
4
.
7.
Motzer
RJ
,
Mazumdar
M
,
Bacik
J
,
Russo
P
,
Berg
WJ
,
Metz
EM
. 
Effect of cytokine therapy on survival for patients with advanced renal cell carcinoma
.
J Clin Oncol
2000
;
18
:
1928
35
.
8.
Negrier
S
,
Escudier
B
,
Gomez
F
,
Douillard
JY
,
Ravaud
A
,
Chevreau
C
, et al
Prognostic factors of survival and rapid progression in 782 patients with metastatic renal carcinomas treated by cytokines: a report from the Groupe Francais d'Immunotherapie
.
Ann of Oncol
2002
;
13
:
1460
8
.
9.
Atzpodien
J
,
Royston
P
,
Wandert
T
,
Reitz
M
,
Reitz
M
. 
Metastatic renal carcinoma comprehensive prognostic system
.
Br J Cancer
2003
;
88
:
348
53
.
10.
Ma
C
,
Bandukwala
S
,
Burman
D
,
Bryson
J
,
Seccareccia
D
,
Banerjee
S
, et al
Interconversion of three measures of performance status: an empirical analysis
.
Eur J Cancer
2010
;
46
:
3175
83
.
11.
Kaplan
EL
,
Meier
. 
Nonparametric estimation from incomplete observations
.
J Am Stat Assoc
1958
;
53
:
457
81
.
12.
Cox
DR
. 
Regression models and life tables
.
J Royal Stat Soc B
1972
;
34
:
181
220
.
13.
Royston
P
,
Sauerbrei
W
. 
Building multivariable regression models with continuous covariates in epidemiology—with an emphasis on fractional polynomials
.
Methods Inf Med
2005
;
44
:
561
71
.
14.
Rubin
DB
Multiple Imputation for Nonresponse in Surveys
.
Wiley
,
New York
: 
1987
.
15.
Cox
DR
. 
Note on grouping
.
J Am Stat Assoc
1957
;
52
:
543
7
.
16.
Pencina
MJ
,
D'Agostino
RB
. 
Overall c as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation
.
Stat Med
2004
;
23
:
2109
23
.
17.
Royston
P
,
Sauerbrei
W
. 
A new measure of prognostic separation in survival
.
Stat Med
2004
;
23
:
723
48
.
18.
Heng
DY
,
Xie
W
,
Regan
MM
,
Warren
MA
,
Golshayan
AR
,
Sahi
C
, et al
Prognostic factors for overall survival in patients with metastatic renal cell carcinoma treated with vascular endothelial growth factor-targeted agents: results from a large, multicenter study
.
J Clin Oncol
2009
;
27
:
5794
9
.
19.
Miller
ME
,
Hui
SL
. 
Validation techniques for logistic regression models
.
Stat Med
1991
;
10
:
1213
26
.
20.
Patil
S
,
Figlin
RA
,
Hutson
TE
,
Michaelson
MD
,
Négrier
S
,
Kim
ST
, et al
Prognostic factors for progression-free and overall survival with sunitinib targeted therapy and with cytokine as first-line therapy in patients with metastatic renal cell carcinoma
.
Ann Oncol
2011
;
22
:
295
300
.
21.
Motzer
RJ
,
Escudier
B
,
Oudard
S
,
Hutson
TE
,
Porta
C
,
Bracarda
S
, et al
for the RECORD-1 Study Group. Phase 3 trial of everolimus for metastatic renal cell carcinoma: final results and analysis of prognostic factors
.
Cancer
2010
;
116
:
4256
65
.