Aims: To develop multivariate nomograms that determine the probabilities of all-cause and bladder cancer–specific survival after radical cystectomy and to compare their predictive accuracy to that of American Joint Committee on Cancer (AJCC) staging.

Methods: We used Cox proportional hazards regression analyses to model variables of 731 consecutive patients treated with radical cystectomy and bilateral pelvic lymphadenectomy for bladder transitional cell carcinoma. Variables included age of patient, gender, pathologic stage (pT), pathologic grade, carcinoma in situ, lymphovascular invasion (LVI), lymph node status (pN), neoadjuvant chemotherapy (NACH), adjuvant chemotherapy (ACH), and adjuvant external beam radiotherapy (AXRT). Two hundred bootstrap resamples were used to reduce overfit bias and for internal validation.

Results: During a mean follow-up of 36.4 months, 290 of 731 (39.7%) patients died; 196 of 290 patients (67.6%) died of bladder cancer. Actuarial all-cause survival estimates were 56.3% [95% confidence interval (95% CI), 51.8-60.6%] and 42.9% (95% CI, 37.3-48.4%) at 5 and 8 years after cystectomy, respectively. Actuarial cancer-specific survival estimates were 67.3% (62.9-71.3%) and 58.7% (52.7-64.2%) at 5 and 8 years, respectively. The accuracy of a nomogram for prediction of all-cause survival (0.732) that included patient age, pT, pN, LVI, NACH, ACH, and AXRT was significantly superior (P = 0.001) to that of AJCC staging–based risk grouping (0.615). Similarly, the accuracy of a nomogram for prediction of cancer-specific survival that included pT, pN, LVI, NACH, and AXRT (0.791) was significantly superior (P = 0.001) to that of AJCC staging–based risk grouping (0.663).

Conclusions: Multivariate nomograms provide a more accurate and relevant individualized prediction of survival after cystectomy compared with conventional prediction models, thereby allowing for improved patient counseling and treatment selection.

Transitional cell carcinoma of the urinary bladder is a significant cause of morbidity and mortality worldwide. In the United States, bladder transitional cell carcinoma is the fifth most commonly diagnosed new cancer with an incidence of 63,210 new cases and 13,180 cancer-related deaths yearly (1). Radical cystectomy with bilateral pelvic lymphadenectomy is the most commonly applied primary treatment modality for individuals with muscle-invasive bladder cancer or refractory, high-grade nonmuscle invasive cancer. However, the 5-year all-cause survival rate in patients with pathologically staged T2 (pT2; i.e., invasion into the bladder muscle) tumors is only 60% to 75% (26).5

5

S. Shariat, P. Karakiewicz, G. Palapattu, et al. Radical cystectomy for transitional cell carcinoma of the bladder: a contemporary series from the bladder cancer research consortium. Journal of Clinical Oncology, submitted for publication 2005.

Tumors that exhibit extravesicle extension (i.e., pT3) are associated with 36% to 58% 5-year survival, and pT4 (i.e., invasion into contiguous structures) or node-positive tumors show 4% to 35% 5-year survival. Failure to cure is often due to the presence of occult metastases at the time of primary local therapy.

Progression to measurable metastatic disease occurs, on average, 1 to 2 years after radical cystectomy and is fatal in the majority of patient despite a high initial response rate to systemic chemotherapy (26).5 Contemporary clinical trails strongly support an integrated treatment approach with perioperative chemotherapy and definitive locoregional therapy but these trials relied on rigid pathologic entry criteria associated with usually >50% probability of progression with cystectomy alone (714). New predictive models that incorporate a wide range of variables that may affect both all-cause and cancer-specific survival after cystectomy may be useful in assessing risk of all patients. A set of tools that can be validated prospectively can serve to expand hypothesis testing of multimodality treatment to all patients undergoing cystectomy rather than applying arbitrary pathologic criteria. These tools also offer potential benefits with patient counseling and postoperative surveillance strategies (15, 16).

Currently, pathologic stage (pT) and lymph nodes status (pN), which form the bladder cancer American Joint Committee on Cancer (AJCC) stages, represent the gold standard prognostic variables after radical cystectomy (26).5 However, various other tumor features have also been shown to be associated with bladder cancer outcomes. For example, lymphovascular invasion (LVI) represents a highly significant, independent predictor of cancer-specific survival in lymph node negative cystectomy patients after adjusting for pathologic stage (17, 18). Thus, the addition of other variables, in addition to the tumor-node-metastasis stage, may improve the prediction of outcomes in patients treated with radical cystectomy.

Recently, prognostic nomograms have been introduced to the urologic literature and have been shown to be valuable tools for estimating the likelihood of cancer being a diagnosis or prognosis, as well as tumor pathologic features (16, 1926). This approach readily quantifies the probability of a given outcome based on defined patient characteristics and allows simultaneous consideration of multiple aspects of numerous variables (27). Such predictive tools can be used for patient counseling, follow-up scheduling, and clinical trial design and analysis.

We hypothesized that survival in patients treated with radical cystectomy for bladder transitional cell carcinoma is predictable with satisfactory precision and may improve the predictive accuracy over conventional prognostic tools. Therefore, the primary aim of this study was to develop prognostic tools capable of accurately predicting the probability that a patient would die of any cause or of bladder cancer at specific time points after cystectomy. Towards this end, we collected comprehensive radical cystectomy data and outcomes from a large, contemporary, consecutive series of patients who were treated with pelvic lymphadenectomy and radical cystectomy at three U.S. academic centers. In addition, we compared the predictive ability of multivariable nomograms with that of current AJCC staging risk groupings.

Patient population. All studies were undertaken with the approval and oversight of the Institutional Review Board for the Protection of Human Subjects at each institution. Nine hundred fifty-eight consecutive patients who underwent radical cystectomy and pelvic lymphadenectomy with curative intent to treat their bladder cancer by urologic cancer surgeons at The University of Texas Southwestern (Dallas, TX), Johns Hopkins Hospital (Baltimore, MD), and Baylor College of Medicine (Houston, TX) during the period from 3/11/1984 through 1/24/2003 and who had data available were potential candidates for this analysis. All patients underwent radical cystectomy and bilateral pelvic lymphadenectomy with the common iliac bifurcation as the minimum proximal limit of dissection. For each patient, comprehensive clinical and pathologic information was collected and entered into an institutional review board–approved database. Multiple internal and external data reviews and quality checks were done to ensure the accuracy and completeness of data elements. The indications for radical cystectomy were tumor invasion into the muscularis propria or prostatic stroma or Ta, T1, or carcinoma in situ refractory to transurethral resection with intravesical chemotherapy and/or immunotherapy. No patient had distant metastatic disease at the time of cystectomy (cN0 and cM0). Clinical stage was assigned by the operative surgeon according to the 1997 tumor-node-metastasis system. Of 958 patients, 227 were excluded for the following reasons: 135 due to pathologic cell type other than transitional cell carcinoma, 78 due to missing pT stage, 5 due to missing pathologic WHO tumor grade, 33 due to missing pN stage, 16 due to missing LVI status, 86 due to missing carcinoma in situ status, 38 due to unknown neoadjuvant chemotherapy (NACH) status, 50 due to unknown adjuvant chemotherapy (ACH) status, 2 due to unknown adjuvant external beam radiotherapy (AXRT) status, and 9 due to missing follow-up information. This left 731 patients for analysis.

NACH and ACH were administered to 5.2% and 25.6% of patients, respectively. Chemotherapy regimens included methotrexate, vinblastine, doxorubicin, and cisplatin (MVAC); methotrexate, cisplatin, and vinblastine; gemcitabine and cisplatin; taxol and carboplatin; and cisplatin and carboplatin. AXRT was administered to 4.7% of the study population according to institutional protocols.

Pathology. Staff pathologists from each institution with expertise in genitourinary pathology examined all specimens according to institutional protocols. Multiple, well-oriented quadrant sections from the tumor, adjacent and distant bladder wall, ureters, and urethra were processed. Pelvic lymph node dissections were examined grossly and all lymphoid tissues were submitted for histologic examination. Tumors were staged according to the criteria in the fifth edition of the AJCC staging manual (28) and graded according to the 1973 WHO classification. Patients who had surgery before 1997 had their pathologic stage updated to reflect the 1997 tumor-node-metastasis staging system. LVI was defined as the unequivocal presence of tumor cells within an endothelium-lined space without underlying muscular walls (17, 29). Equivocal cases and tumor cells that merely encroached on a vascular lumen were considered negative, and a perivascular reaction was not required. In males, the prostate and seminal vesicles were evaluated in accordance with the guidelines of the College of American Pathologists (30). In females, the ovaries, uterus, and vagina were evaluated when included in the surgical specimen. To ensure validity of the pathologic outcomes, the pathology reports of 219 consecutive patients were read by two clinicians who were blinded to patient clinical variables and the finding of the other reviewer. Interreader reliability measured using the intraclass correlation coefficient was >0.95 for all pathologic variables. The AJCC staging was applied according the Sixth Edition AJCC Cancer Staging Manual (31).

Follow-up. Follow-up was done according to institutional protocols. Patients generally were seen postoperatively at least every 3 to 4 months for the first year, semiannually for the second year, and annually thereafter. Follow-up visits consisted of a physical examination and serum chemistry evaluation, including liver function tests and alkaline phosphatase. Diagnostic imaging of the upper tracts (e.g., ultrasonography and/or i.v. pyelography, computed tomography of abdomen/pelvis with IV contrast) and chest radiography were done at least annually or when clinically indicated. Additional radiographic evaluation, such as bone scan and/or computerized tomography, was done at the discretion of the treating physician. Detection of cancer in the ureter and/or urethra was coded as a second (metachronous) primary and not as local or distant recurrence. When patients died, the cause of death was determined by the treating physicians by chart review corroborated by death certificates, or by death certificates alone. Most patients who were identified as having died of bladder cancer had progressive, widely disseminated, and often highly symptomatic metastases at the time of death. Perioperative mortality (death within 30 days of surgery) was censored at time of death for bladder cancer–specific survival analyses.

Statistical analyses. Separate univariable and multivariable Cox regression models addressed two types of events: time to all-cause mortality and time to cancer-specific mortality. Main predictors consisted of pT and pN stages, which form the basis for the AJCC stages. Additional variables included age, gender, tumor grade, LVI, carcinoma in situ, NACH, ACH, and AXRT. Actuarial survival probabilities were estimated using the Kaplan-Meier method. Reduced model selection was done using a backward step-down selection process, which used as stopping rule the Akaike's information criterion (32, 33). Proportional hazards assumptions were systematically verified for all proposed models using the Grambsch-Therneau residual-based test (34). Multivariate Cox regression coefficients were then used to generate prognostic nomograms. Predictive accuracy of these nomograms was quantified with receiver operating characteristic curve (ROC)–derived area under the curve (AUC) estimates (27, 35). In Cox regression models, the AUC is substituted with Harrell's concordance index, which was used in this analysis (33, 34). A value of 1.0 indicates perfect predictions, whereas 0.5 is equivalent to a toss of a coin. Internal validation was done using 200 bootstrap resamples (36). The predictive abilities of the nomogram and the AJCC staging were compared by computing the concordance index for each method and testing for a difference by using the DeLong method (37). Calibration plots were generated to explore nomogram performance. All statistical tests were done with S-Plus Professional (MathSoft, Inc., Seattle, WA) and statistical significance was set at 0.05.

The descriptive characteristics of the 731 patients are shown in Table 1. Median follow-up from the date of cystectomy was 24.9 months (mean, 36.9; range, 0.1-183.4). Of 731 patients, 290 (39.7%) died of any cause. One hundred ninety-six of the 290 deaths (67.6%) were attributable to bladder cancer. Median survival time was 73 months (mean, 83; range, 0.1-171.9). Of the 731 patients, 373, 168, and 49 were alive and at risk of dying at 2, 5, and 8 years, respectively. Actuarial all-cause survival probabilities were 71.8% (95% CI, 68.1-75.1%), 56.3% (51.8-60.6%), and 42.9% (37.3-48.4%) at 2, 5, and 8 years after cystectomy, respectively (Fig. 1A). Actuarial cancer-specific survival probabilities were 77.0% (95% CI, 73.4-80.1%), 67.3% (62.9-71.3%), and 58.7% (52.7-64.2%) at 2, 5, and 8 years after cystectomy, respectively (Fig. 1B). Figure 2 shows the Kaplan Meier plots of all-cause and cancer-specific survival according to 2002 AJCC stages.

Table 1.

Clinical and pathologic characteristics of 731 patients treated with radical cystectomy for transitional cell carcinoma of the urinary bladder

VariablesNo. patients (%)
Gender  
    Female 129 (17.6) 
    Male 602 (82.4) 
Age of patient at cystectomy (y)  
    Mean (median) 64.5 (65.9) 
    Range 33.1-89.2 
Pathologic grade  
    Absence of cancer 56 (7.7) 
    1 and 2 56 (7.7) 
    3 619 (84.6) 
Pathologic T stage  
    pT0 56 (7.7) 
    pTis 92 (12.6) 
    pTa 23 (3.1) 
    pT1 94 (12.9) 
    pT2 164 (22.4) 
    pT3 216 (29.5) 
    pT4 86 (11.8) 
Pathologic N stage  
    pN0 557 (76.2) 
    pN1 67 (9.2) 
    pN2 93 (12.7) 
    pN3 14 (1.9) 
AJCC stage  
    0* 54 (7.4) 
    0is 89 (12.2) 
    0a 23 (3.1) 
    I 88 (12.0) 
    II 130 (17.8) 
    III 169 (23.1) 
    IV 178 (24.4) 
Presence of LVI 274 (37.5) 
Concomitant carcinoma in situ at cystectomy 392 (53.6) 
NACH 38 (5.2) 
ACH 187 (25.6) 
Adjuvant radiotherapy 34 (4.7) 
VariablesNo. patients (%)
Gender  
    Female 129 (17.6) 
    Male 602 (82.4) 
Age of patient at cystectomy (y)  
    Mean (median) 64.5 (65.9) 
    Range 33.1-89.2 
Pathologic grade  
    Absence of cancer 56 (7.7) 
    1 and 2 56 (7.7) 
    3 619 (84.6) 
Pathologic T stage  
    pT0 56 (7.7) 
    pTis 92 (12.6) 
    pTa 23 (3.1) 
    pT1 94 (12.9) 
    pT2 164 (22.4) 
    pT3 216 (29.5) 
    pT4 86 (11.8) 
Pathologic N stage  
    pN0 557 (76.2) 
    pN1 67 (9.2) 
    pN2 93 (12.7) 
    pN3 14 (1.9) 
AJCC stage  
    0* 54 (7.4) 
    0is 89 (12.2) 
    0a 23 (3.1) 
    I 88 (12.0) 
    II 130 (17.8) 
    III 169 (23.1) 
    IV 178 (24.4) 
Presence of LVI 274 (37.5) 
Concomitant carcinoma in situ at cystectomy 392 (53.6) 
NACH 38 (5.2) 
ACH 187 (25.6) 
Adjuvant radiotherapy 34 (4.7) 

NOTE: Absence of cancer indicates that no grade could be assigned in pT0 cases.

*

Denotes pT0 cases where no grade could be assigned due to absence of residual cancer.

Fig. 1.

Kaplan-Meier estimates of all-cause (A) and bladder cancer–specific (B) survival for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder. Dotted lines, 95% CIs.

Fig. 1.

Kaplan-Meier estimates of all-cause (A) and bladder cancer–specific (B) survival for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder. Dotted lines, 95% CIs.

Close modal
Fig. 2.

Kaplan-Meier estimates of all-cause (A) and bladder cancer–specific (B) survival for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder according to 2002 AJCC stages. Stage 0 includes 0a, 0is, and pT0 cases.

Fig. 2.

Kaplan-Meier estimates of all-cause (A) and bladder cancer–specific (B) survival for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder according to 2002 AJCC stages. Stage 0 includes 0a, 0is, and pT0 cases.

Close modal

Table 2 shows univariate and multivariate models for predicting all-cause survival after cystectomy. Patient age, pathologic grade, pT, pN, LVI, ACH, and AXRT were all significantly associated with all-cause survival by univariate analysis. Two different multivariate models were constructed: a full model that included all variables that were significant by univariate analyses and a reduced model that used backward variable elimination with the intent of increasing parsimony while maintaining maximum accuracy. In the full multivariate analysis, NACH achieved statistical significance in addition to all previous predictors, except for age of patient, which did not retain statistical significance. Figure 3 shows the univariate effect of these stratifying variables on the rate of all-cause survival. We then used univariate Cox regression coefficients to generate prognostic nomograms. The predictive accuracy of these nomograms was quantified with ROC-derived AUC estimates (i.e., concordance index; refs. 27, 35). Analyses of predictive accuracy of each individual predictor revealed that pT had the highest univariate predictive accuracy (0.699), followed by LVI (0.648) and pN (0.612). The predictive accuracy of the full multivariate model was 0.729 and exceeded (P < 0.001) that of each individual predictor and that of AJCC staging–based risk grouping (0.635). Backward variable elimination yielded an all-cause survival model with fewer variables that included age of patient, pT, LVI, pN, NACH, ACH, and AXRT with a predictive accuracy for all-cause survival after cystectomy was 0.732. The accuracy of the reduced model was superior to that of each individual variable and that of AJCC staging–based risk grouping (P < 0.001).

Table 2.

Univariate and multivariate Cox regression analyses for prediction of all-cause survival after cystectomy for bladder cancer

PredictorsUnivariate
Full multivariate model
Reduced multivariate model
Hazard ratioPPredictive accuracy*Hazard ratioPHazard ratioP
Male gender 1.270 0.116 0.508 1.251 0.151 — — 
Age of patient at cystectomy 1.021 <0.001 0.578 1.016 0.011 1.017 0.006 
Pathologic grade 3 vs other 2.139 <0.001 0.547 1.218 0.359 — — 
Pathologic T stage — <0.001 0.699 — <0.001 — <0.001 
    pT2 vs pT1 or lower 1.883 <0.001  1.519 0.043 1.610 0.016 
    pT3 vs pT1 or lower 3.950 <0.001  2.740 <0.001 2.891 <0.001 
    pT4 vs pT1 or lower 6.341 <0.001  3.534 <0.001 3.621 <0.001 
Pathologic N stage — <0.001 0.612 1.669 0.001 1.643 0.001 
    pN1 vs pN0 2.101 <0.001  — 0.002 — 0.001 
    pN2 vs pN0 3.062 <0.001  1.405 0.092 1.451 0.064 
    pN3 vs pN0 4.969 <0.001  1.867 0.001 1.901 <0.001 
AJCC stage — <0.001 0.615     
    0is vs 0 1.787 0.205      
    0a vs 0 1.973 0.248      
    I vs 0 2.898 0.014      
    II vs 0 3.478 0.003      
    III vs 0 7.469 <0.001      
    IV vs 0 10.955 <0.001      
LVI 3.060 <0.001 0.648 2.434 0.006 2.471 0.005 
Concomitant carcinoma in situ at cystectomy 1.020 0.866 0.518 1.156 0.252 — — 
NACH 1.454 0.086 0.512 1.801 0.01 1.678 0.021 
ACH 1.743 <0.001 0.546 0.673 0.009 0.670 0.009 
Adjuvant radiotherapy 3.297 <0.001 0.532 1.574 0.036 1.571 0.035 
Predictive accuracy of multivariate models*    0.729  0.732  
PredictorsUnivariate
Full multivariate model
Reduced multivariate model
Hazard ratioPPredictive accuracy*Hazard ratioPHazard ratioP
Male gender 1.270 0.116 0.508 1.251 0.151 — — 
Age of patient at cystectomy 1.021 <0.001 0.578 1.016 0.011 1.017 0.006 
Pathologic grade 3 vs other 2.139 <0.001 0.547 1.218 0.359 — — 
Pathologic T stage — <0.001 0.699 — <0.001 — <0.001 
    pT2 vs pT1 or lower 1.883 <0.001  1.519 0.043 1.610 0.016 
    pT3 vs pT1 or lower 3.950 <0.001  2.740 <0.001 2.891 <0.001 
    pT4 vs pT1 or lower 6.341 <0.001  3.534 <0.001 3.621 <0.001 
Pathologic N stage — <0.001 0.612 1.669 0.001 1.643 0.001 
    pN1 vs pN0 2.101 <0.001  — 0.002 — 0.001 
    pN2 vs pN0 3.062 <0.001  1.405 0.092 1.451 0.064 
    pN3 vs pN0 4.969 <0.001  1.867 0.001 1.901 <0.001 
AJCC stage — <0.001 0.615     
    0is vs 0 1.787 0.205      
    0a vs 0 1.973 0.248      
    I vs 0 2.898 0.014      
    II vs 0 3.478 0.003      
    III vs 0 7.469 <0.001      
    IV vs 0 10.955 <0.001      
LVI 3.060 <0.001 0.648 2.434 0.006 2.471 0.005 
Concomitant carcinoma in situ at cystectomy 1.020 0.866 0.518 1.156 0.252 — — 
NACH 1.454 0.086 0.512 1.801 0.01 1.678 0.021 
ACH 1.743 <0.001 0.546 0.673 0.009 0.670 0.009 
Adjuvant radiotherapy 3.297 <0.001 0.532 1.574 0.036 1.571 0.035 
Predictive accuracy of multivariate models*    0.729  0.732  

NOTE: Rate ratios indicate the increase in the rate of mortality relative to reference categories for which the rate ratio is 1.0. An overall statistic (P value) is provided for categorical variables (pT, pN, and AJCC stages). For these variables, the rate ratios are provided for each variable level. Pathologic T stage 0 (absence of cancer in the specimen) was defined as AJCC stage 0.

*

Predictive accuracy is the AUC estimate of the ROC derived from these nomograms (i.e., concordance index).

Predictive accuracy of AJCC-based risk grouping.

Fig. 3.

Kaplan-Meier survival estimates showing the effect of predictor variables on all-cause survival for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder.

Fig. 3.

Kaplan-Meier survival estimates showing the effect of predictor variables on all-cause survival for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder.

Close modal

Table 3 shows the univariate and multivariate models for prediction of bladder cancer–specific survival after cystectomy. In univariate analyses, pathologic grade, pT, pN, LVI, NACH, ACH, and AXRT were significantly associated with cancer-specific survival. Figure 4 shows the univariate effect of these variables on the rate of cancer-specific survival. In multivariate analyses, pT, pN, LVI, NACH, and AXRT remained significantly associated with cancer-specific survival. Nomogram-based predictive accuracy of each individual predictor revealed that pT had the highest concordance index (0.754), followed by LVI (0.676) and pN (0.666). The predictive accuracy of the full multivariate model was 0.789 and significantly exceeded (P < 0.001) the accuracy of each individual predictor and that of AJCC staging–based risk grouping (0.663). Backward variable elimination of the multivariate model yielded a reduced cancer-specific survival model with an accuracy of 0.791 that included pT, pN, LVI, NACH, and AXRT. The accuracy of the reduced model was higher than that of each individual variable and that of the AJCC staging–based risk grouping (P < 0.001).

Table 3.

Univariate and multivariate Cox regression analyses for prediction of cancer-specific survival after cystectomy for bladder cancer

PredictorsUnivariate analyses
Full multivariate model
Reduced multivariate model
Hazard ratioPPredictive accuracy*Hazard ratioPHazard ratioP
Male gender 1.165 0.408 0.502 1.229 0.281 — — 
Age of patient at cystectomy 1.012 0.093 0.553 1.011 0.156 — — 
Pathologic grade 3 vs other 2.778 <0.001 0.557 1.148 0.638 — — 
Pathologic stage — <0.001 0.754 — <0.001 — <0.001 
    pT2 vs pT1 or lower 2.742 0.001  2.012 0.026 2.059 0.017 
    pT3 vs pT1 or lower 8.884 <0.001  5.073 <0.001 5.125 <0.001 
    pT4 vs pT1 or lower 14.238 <0.001  6.401 <0.001 6.608 <0.001 
Pathologic N stage — <0.001 0.666 — <0.001 — <0.001 
    pN vs pN0 3.435 <0.001  1.928 0.003 1.802 0.005 
    pN2 vs pN0 5.120 <0.001  2.515 <0.001 2.247 <0.001 
    pN3 vs pN0 7.916 <0.001  3.196 0.001 3.015 0.001 
AJCC stage — <0.001 0.630     
    0is vs 0 0.522 0.476      
    0a vs 0 0.855 0.892      
    I vs 0 1.980 0.313      
    II vs 0 2.450 0.159      
    III vs 0 11.026 <0.001      
    IV vs 0 19.686 <0.001      
LVI 4.070 <0.001 0.676 1.540 0.016 1.488 0.026 
Concomitant carcinoma in situ at cystectomy 0.857 0.283 0.524 1.039 0.802 — — 
NACH 2.157 0.001 0.528 2.599 <0.001 2.428 <0.001 
ACH 2.759 <0.001 0.591 0.826 0.264 — — 
Adjuvant radiotherapy 4.415 <0.001 0.551 1.620 0.031 1.635 0.025 
Predictive accuracy of multivariate models*    0.789  0.791  
PredictorsUnivariate analyses
Full multivariate model
Reduced multivariate model
Hazard ratioPPredictive accuracy*Hazard ratioPHazard ratioP
Male gender 1.165 0.408 0.502 1.229 0.281 — — 
Age of patient at cystectomy 1.012 0.093 0.553 1.011 0.156 — — 
Pathologic grade 3 vs other 2.778 <0.001 0.557 1.148 0.638 — — 
Pathologic stage — <0.001 0.754 — <0.001 — <0.001 
    pT2 vs pT1 or lower 2.742 0.001  2.012 0.026 2.059 0.017 
    pT3 vs pT1 or lower 8.884 <0.001  5.073 <0.001 5.125 <0.001 
    pT4 vs pT1 or lower 14.238 <0.001  6.401 <0.001 6.608 <0.001 
Pathologic N stage — <0.001 0.666 — <0.001 — <0.001 
    pN vs pN0 3.435 <0.001  1.928 0.003 1.802 0.005 
    pN2 vs pN0 5.120 <0.001  2.515 <0.001 2.247 <0.001 
    pN3 vs pN0 7.916 <0.001  3.196 0.001 3.015 0.001 
AJCC stage — <0.001 0.630     
    0is vs 0 0.522 0.476      
    0a vs 0 0.855 0.892      
    I vs 0 1.980 0.313      
    II vs 0 2.450 0.159      
    III vs 0 11.026 <0.001      
    IV vs 0 19.686 <0.001      
LVI 4.070 <0.001 0.676 1.540 0.016 1.488 0.026 
Concomitant carcinoma in situ at cystectomy 0.857 0.283 0.524 1.039 0.802 — — 
NACH 2.157 0.001 0.528 2.599 <0.001 2.428 <0.001 
ACH 2.759 <0.001 0.591 0.826 0.264 — — 
Adjuvant radiotherapy 4.415 <0.001 0.551 1.620 0.031 1.635 0.025 
Predictive accuracy of multivariate models*    0.789  0.791  

NOTE: Rate ratios indicate the increase in the rate of mortality relative to reference categories for which the rate ratio is 1.0. An overall statistic (P value) is provided for categorical variables (pT, pN, and AJCC stages). For these variables, the rate ratios are provided for each variable level. Pathologic T stage 0 (absence of cancer in the specimen) was defined as AJCC stage 0.

*

Predictive accuracy is the AUC estimate of the ROC derived from these nomograms (i.e., concordance index).

Predictive accuracy of AJCC-based risk grouping.

Fig. 4.

Kaplan-Meier survival estimates showing the effect of predictor variables on bladder cancer–specific survival for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder.

Fig. 4.

Kaplan-Meier survival estimates showing the effect of predictor variables on bladder cancer–specific survival for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder.

Close modal

Figure 5 (top) shows the most accurate and the most parsimonious nomogram for prediction of all-cause survival after radical cystectomy. Figure 6 shows the most accurate and the most parsimonious nomogram for prediction of cancer-specific survival. Both nomograms are based on their respective reduced, multivariate models and are accompanied by their respective calibration plots. Assessment of axes in both nomograms shows an intuitive relation between pT and all-cause and cancer-specific survival. Conversely, a counterintuitive relation exists between NACH and survival outcomes, where delivery of NACH is associated with worse outcomes. The calibration plots of the two nomograms show performance characteristics, which are close to ideal predictions (denoted by the 45-degree line).

Fig. 5.

A, all-cause survival nomogram based on the reduced multivariate model. Instructions for nomogram use: locate patient values at each axis. Draw a vertical line to the “Point” axis to determine how many points are attributed for each variable value. Sum the points for all variables. Locate the sum on the “Total Points” line. Draw a vertical line towards the 2Yrs.Surv.Prob., 5Yrs.Surv.Prob., and 8Yrs.Surv.Prob. axes to determine respectively the 2-, 5-, and 8-year survival probabilities. B, calibration plot, where nomogram predictions are compared with observed fractions surviving. Diagonal line, performance of an ideal nomogram. The line containing error bars (95% CI) represents the performance of the nomogram applied to the observed fractions surviving.

Fig. 5.

A, all-cause survival nomogram based on the reduced multivariate model. Instructions for nomogram use: locate patient values at each axis. Draw a vertical line to the “Point” axis to determine how many points are attributed for each variable value. Sum the points for all variables. Locate the sum on the “Total Points” line. Draw a vertical line towards the 2Yrs.Surv.Prob., 5Yrs.Surv.Prob., and 8Yrs.Surv.Prob. axes to determine respectively the 2-, 5-, and 8-year survival probabilities. B, calibration plot, where nomogram predictions are compared with observed fractions surviving. Diagonal line, performance of an ideal nomogram. The line containing error bars (95% CI) represents the performance of the nomogram applied to the observed fractions surviving.

Close modal
Fig. 6.

A, bladder cancer–specific survival nomogram based on the reduced multivariate model. Instructions for nomogram use: locate patient values at each axis. Draw a vertical line to the “Point” axis to determine how many points are attributed for each variable value. Sum the points for all variables. Locate the sum on the “Total Points” line. Draw a vertical line towards the 2Yrs.Surv.Prob., 5Yrs.Surv.Prob., and 8Yrs.Surv.Prob. axes to determine respectively the 2-, 5-, and 8-year survival probabilities. B, calibration plot, where nomogram predictions are compared with observed fractions surviving. The diagonal line represents the performance of an ideal nomogram. The line containing error bars (95% CI) represents the performance of the nomogram applied to the observed fractions surviving.

Fig. 6.

A, bladder cancer–specific survival nomogram based on the reduced multivariate model. Instructions for nomogram use: locate patient values at each axis. Draw a vertical line to the “Point” axis to determine how many points are attributed for each variable value. Sum the points for all variables. Locate the sum on the “Total Points” line. Draw a vertical line towards the 2Yrs.Surv.Prob., 5Yrs.Surv.Prob., and 8Yrs.Surv.Prob. axes to determine respectively the 2-, 5-, and 8-year survival probabilities. B, calibration plot, where nomogram predictions are compared with observed fractions surviving. The diagonal line represents the performance of an ideal nomogram. The line containing error bars (95% CI) represents the performance of the nomogram applied to the observed fractions surviving.

Close modal

Figures 7 and 8 compare the reduced nomograms with the predictions of the AJCC stage groupings for both all-cause and cancer-specific survival, respectively. Nomogram probabilities were created for each stage, showing the heterogeneity within each AJCC stage. It is notable that for all-cause and cancer-specific predictions, every AJCC stage overlaps with neighboring AJCC stages when histograms of nomogram predicted probabilities are compared. AJCC staging does not result in a clean separation of patients with differing risks. The most notable degree of overlap may be seen between stage III and IV groupings. This suggests that the discriminatory properties of AJCC stages III and IV are the poorest.

Fig. 7.

Box plots comparing the distribution of nomogram predictions with the predictions of the AJCC stage for all-cause survival at 2, 5, and 8 years for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder. X axes, AJCC stage groupings; Y axes, nomogram probability of survival. Top, 2-year predictions. Middle, 5-year predictions. Bottom, 8-year predictions. Note the heterogeneity of predicted survival probabilities within each AJCC stage.

Fig. 7.

Box plots comparing the distribution of nomogram predictions with the predictions of the AJCC stage for all-cause survival at 2, 5, and 8 years for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder. X axes, AJCC stage groupings; Y axes, nomogram probability of survival. Top, 2-year predictions. Middle, 5-year predictions. Bottom, 8-year predictions. Note the heterogeneity of predicted survival probabilities within each AJCC stage.

Close modal
Fig. 8.

Box plots comparing the distribution of nomogram predictions with the predictions of the AJCC stage for bladder cancer–specific survival at 2, 5, and 8 years for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder. X axes, AJCC stage groupings; Y axes, nomogram probability of survival. Top, 2-year predictions. Middle, 5-year predictions. Bottom, 8-year predictions. Note the heterogeneity of predicted survival probabilities within each AJCC stage.

Fig. 8.

Box plots comparing the distribution of nomogram predictions with the predictions of the AJCC stage for bladder cancer–specific survival at 2, 5, and 8 years for 731 patients treated with radical cystectomy and bilateral lymphadenectomy for transitional cell carcinoma of the urinary bladder. X axes, AJCC stage groupings; Y axes, nomogram probability of survival. Top, 2-year predictions. Middle, 5-year predictions. Bottom, 8-year predictions. Note the heterogeneity of predicted survival probabilities within each AJCC stage.

Close modal

We found that radical cystectomy with bilateral lymphadenectomy provides durable local control and long-term survival with all-cause and bladder cancer–specific survival rates within the range reported in previous studies (26).5 Prognosis in patients treated with radical cystectomy is currently estimated on the basis of the AJCC staging system, which considers the T, N, and M stages only. However, there is a significant variability of survival probabilities within AJCC stage categories that imposes a lack of accuracy in predicting outcomes on an individual basis. Moreover, despite the established prognostic significance of AJCC stages, several reports showed that other variables might at least be equally important (26, 17).5 Recently, two groups independently reported that LVI represents a marker of biologically and clinically aggressive behavior after controlling for pT and pN stages (17, 18). By integrating additional significant prognostic factors, a predictive tool, such as a nomogram, could be used to better assess an individual patient's disease-specific survival following surgery for bladder cancer.

In the present study, we constructed and internally validated highly accurate and discriminating nomograms for prediction of both all-cause and cancer-specific survival by using all routinely available variables of a large, consecutive cohort of patients treated with radical cystectomy for transitional cell carcinoma. Both survival nomograms were based on the simultaneous interaction of variables yielding highly accurate prognostic models that far exceeded the accuracy of individual predictors and showed excellent performance characteristics, which virtually corresponded to ideal predictions (Figs. 5 and 6). The most accurate and parsimonious nomograms predicting all-cause and cancer-specific survival were based on reduced models, from which noninformative variables were removed. To identify the least informative variables, we applied backward variable selection, with the intent of maximizing accuracy and promoting parsimony (27, 32, 33, 35). This method yielded highly accurate and informative tools, which included only the key predictors without sacrificing accuracy or performance. These predictive tools offer a user-friendly interface for determining the risk of all-cause and cancer-specific survival at specific times after cystectomy, especially when numerous variables are considered simultaneously (16, 1926). The advantage of this method over standard multivariate regression models resides in the provision of individual probability of outcome, instead of using a conceptually more challenging notion of relative risk (27, 32, 33, 35). The bootstrapped multivariate nomogram method offers a variety of advantages over conventional Cox regression models (16, 23, 24). For example, nomogram accuracy may be assessed with Harrell's concordance index, a global measure of model accuracy that eliminates the need for assessing of risk strata that lump patients with different risk profiles. Finally, the probability approach relies on nomogram predictions and offers individual estimates of all-cause and cancer-specific survival at specific time points after cystectomy. The predictive accuracy of our nomograms compares very favorably with that of nomograms predicting sarcoma, breast cancer, prostate, renal cell, and gastric cancer outcomes (16, 1926).

We compared the predictive ability of our nomograms with that of the AJCC classification and found our model to be quantitatively more discriminating than the AJCC staging risk grouping. The incremental gains in accuracy for the nomograms over these conventional prognostic models were statistically significant. The finding that the nomograms discriminated with higher accuracy further suggests that the nomograms can resolve the heterogeneity of outcome prediction within each AJCC staging risk group (Figs. 7 and 8). Furthermore, the nomogram predictions are tailored to the risk posed by the characteristics of an individual's cancer, which is more relevant to the patient than are group-level probabilities. For some patients, the change in prognosis will be clinically meaningful. Accurate prediction can aid in individual patient counseling and in follow-up scheduling. It also may play a role in designing future trials and identifying subsets of patients within known AJCC stages who have different prognoses and who may have different responses to established and novel adjuvant treatment strategies.

Nomogram axes allow one to examine and visualize the magnitude and the direction of the association between each predictor variable and all-cause and cancer-specific survival. The larger the covariate axis, the higher the corresponding shift in prognosis when moving across the levels (or values) of the covariate, holding all the others fixed. For example, the axis corresponding to pT is the largest one that underlies the prognostic importance of this factor. The direction of pT stage axis is intuitively related to increasing risk of all-cause and cancer-specific mortality. The effects of NACH and of AXRT show an inverse relation with cancer-specific and all-cause survival, where the delivery of these treatment modalities is associated with worse outcome. We advise caution when interpreting the effect of chemotherapy in our series, as selection for neoadjuvant and/or adjuvant therapy was based on high-risk clinical and or pathologic features usually associated with worse prognosis. The benefit of administering neoadjuvant and/or adjuvant therapy can only be assessed in randomized, controlled trials.

There are several limitations to our study. First and foremost are the limitations inherent to retrospective analyses. Although we have done multiple internal and external reviews of our consortium data set, we excluded from this analysis patients for whom we could not obtain complete information, which could possibly create selection bias. In addition, the population in this study underwent radical cystectomy by multiple surgeons and had their specimens evaluated by multiple pathologists. However, all surgeons were specialty trained in urologic oncology (38). Moreover, whereas prognostic factors may perform well in the select group of patients operated on by a single surgeon, it remains to be determined whether these are applicable to the greater population of patients with bladder cancer. Similarly, whereas it may be preferable for a single pathologist specialized in genitourinary pathology to review each cystectomy specimen, the setup in the present study reflects a real-world practice, in which various pathologists review tissue specimens and their interpretation is then used in clinical decision making with the patient. Furthermore, all specimens were examined by dedicated genitourinary pathologists. It may be postulated that lack of central pathology represents a potential weakness of our work. All of the pathologic data entered into our database were generated by expert genitourinary pathologists at each of the three institutions. Outside biopsy slides used for clinical (precystectomy) staging were also rereviewed by these pathologists. It is therefore unlikely that central pathology review would have materially altered assignment of pathologic variables. In the absence of genitourinary pathology expert review, central pathology may in fact decrease variability in stage and grade assignment and thereby improve the ability of pathologic variables to predict the probability of recurrence. The multi-institutional nature of our study and the use of “local” pathologic interpretation may make our tool more relevant and applicable in both the academic and community settings. Finally, as the study period spans over 20 years, the data in the present study may not represent current practice patterns. For example, NACH is relatively underused in our series compared with current recommendations. Moreover, surgical techniques, such as nerve-sparing radical cystectomy, and number of lymph nodes removed, indication for surgery, and follow-up protocols have changed over time.

The accuracy of our nomograms is not perfect, a shortcoming shared with all prognostic models (16, 1926). For the individual patient, the nomogram predicts the likelihood that a population of similar patients will survive a defined period of time but not the certitude that this will occur. Nevertheless, we believe that nomograms provide a more accurate prediction of what the patient might expect, as Cox regression provides superior predictions compared with neural networks and other machine learning techniques (16, 23, 24). However, there can never be enough predictive variables included in such nomograms to give absolute predictions. Moreover, known variables may not be included because of the lack of numbers or observations, or there may be markers as yet unidentified that predict outcome. In addition, our nomograms cannot be applied to patients treated with treatment modalities other than radical cystectomy, such as partial or salvage cystectomy, after failed bladder preservation.

A relative disadvantage of the nomogram-based approach resides in the requirement of specifying points in time at which predictions are made. We chose 5-year survival, as it is an established reference end point (26)5 and as 92% of cancer-specific deaths had occurred at that time in our study. We also chose the 2-year end point to identify patients at risk for early relapse. At 2 years, the shape of the cancer-specific survival curve changed from its steepest to a less pronounced slope (Fig. 1) and 73% of the cancer-specific deaths had occurred within this time frame. Finally, the follow-up of our cohort allowed us to include an 8-year time point at which 49 individuals remained at risk of death and 1,203 patient-months of follow-up remained. Moreover, cancer-specific survival reached a near-plateau phase at 9 years, as virtually all disease-specific events had occurred [194 of 196 events (99%)] by that time.

The major pitfall in the development of clinically relevant nomograms is the potential lack of generalizability of the model in clinical practice. These data are not applicable to patients with non–transitional cell carcinoma histology as these patients were excluded as their biological behavior and etiology can be quite different from transitional cell carcinoma. Using data from a single institution is subject to criticism that similarities in terms of diagnosis and treatment preferences bias the nomogram. We constructed our nomogram by using a data set from a large multi-institutional collaborative group. The multi-institutional nature of our data set may, however, be interpreted as a limitation, as it groups the contribution of multiple surgeons and pathologists and relies on different types of chemotherapeutic regimens, in addition to other differences that might distinguish the three contributing centers. To minimize interpretation-related difference in data, we used a single database platform with strict criteria to acquire these data across centers. Nevertheless, it is important that such nomograms be validated in other data sets.

We constructed highly discriminative and valid nomograms that predict the individualized probability of both all-cause and cancer-specific survival for patients treated with radical cystectomy for bladder transitional cell carcinoma. The nomograms represent a significant improvement over counseling on the basis of the AJCC staging system by offering a more discriminating and accurate prediction method. The multi-institutional nature of our data set makes our findings more likely to be generalizable to patients treated with radical cystectomy at other centers. Nonetheless, external and prospective validation is needed to control for differences in diagnosis and treatment preferences.

Grant support: The Austrian Program for Advanced Research and Technology (S.F. Shariat), the Fonds de la Recherche en Santé du Québec, the Centre Hospitalier de l'Université de Montréal Foundation, and the Department of Surgery and Les Urologues Associés du Centre Hospitalier de l'Université de Montréal (P.I. Karakiewicz).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

Note: S.F. Shariat and P.I. Karakiewicz contributed equally to this study.

1
Jemal A, Ward E, Hao Y, Thun M. Trends in the leading causes of death in the United States, 1970-2002.
Jama
2005
;
294
:
1255
–9.
2
Stein JP, Lieskovsky G, Cote R, et al. Radical cystectomy in the treatment of invasive bladder cancer: long-term results in 1,054 patients.
J Clin Oncol
2001
;
19
:
666
–75.
3
Madersbacher S, Hochreiter W, Burkhard F, et al. Radical cystectomy for bladder cancer today—a homogeneous series without neoadjuvant therapy.
J Clin Oncol
2003
;
21
:
690
–6.
4
Dalbagni G, Genega E, Hashibe M, et al. Cystectomy for bladder cancer: a contemporary series.
J Urol
2001
;
165
:
1111
–6.
5
Vazina A, Dugi D, Shariat SF, Evans J, Link R, Lerner SP. Stage specific lymph node metastasis mapping in radical cystectomy specimens.
J Urol
2004
;
171
:
1830
–4.
6
Gschwend JE, Fair WR, Vieweg J. Radical cystectomy for invasive bladder cancer: contemporary results and remaining controversies.
Eur Urol
2000
;
38
:
121
–30.
7
Neoadjuvant cisplatin, methotrexate, and vinblastine chemotherapy for muscle-invasive bladder cancer: a randomised controlled trial. International collaboration of trialists.
Lancet
1999
;
354
:
533
–40.
8
Grossman HB, Natale RB, Tangen CM, et al. Neoadjuvant chemotherapy plus cystectomy compared with cystectomy alone for locally advanced bladder cancer.
N Engl J Med
2003
;
349
:
859
–66.
9
Malmstrom PU, Rintala E, Wahlqvist R, Hellstrom P, Hellsten S, Hannisdal E. Five-year follow-up of a prospective trial of radical cystectomy and neoadjuvant chemotherapy: Nordic Cystectomy Trial I. The Nordic Cooperative Bladder Cancer Study Group.
J Urol
1996
;
155
:
1903
–6.
10
Sherif A, Rintala E, Mestad O, et al. Neoadjuvant cisplatin-methotrexate chemotherapy for invasive bladder cancer—Nordic cystectomy trial 2.
Scand J Urol Nephrol
2002
;
36
:
419
–25.
11
Skinner DG, Daniels JR, Russell CA, et al. The role of adjuvant chemotherapy following cystectomy for invasive bladder cancer: a prospective comparative trial.
J Urol
1991
;
145
:
459
–67.
12
Stockle M, Meyenburg W, Wellek S, et al. Advanced bladder cancer (stages pT3b, pT4a, pN1 and pN2): improved survival after radical cystectomy and 3 adjuvant cycles of chemotherapy. Results of a controlled prospective study.
J Urol
1992
;
148
:
302
–7.
13
Studer UE, Bacchi M, Biedermann C, et al. Adjuvant cisplatin chemotherapy following cystectomy for bladder cancer: results of a prospective randomized trial.
J Urol
1994
;
152
:
81
–4.
14
Freiha F, Reese J, Torti FM. A randomized trial of radical cystectomy versus radical cystectomy plus cisplatin, vinblastine and methotrexate chemotherapy for muscle invasive bladder cancer.
J Urol
1996
;
155
:
495
–9; discussion 9–500.
15
Kattan MW, Eastham JA, Wheeler TM, et al. Counseling men with prostate cancer: a nomogram for predicting the presence of small, moderately differentiated, confined tumors.
J Urol
2003
;
170
:
1792
–7.
16
Kattan MW. Nomograms are superior to staging and risk grouping systems for identifying high-risk patients: preoperative application in prostate cancer.
Curr Opin Urol
2003
;
13
:
111
–6.
17
Lotan Y, Gupta A, Shariat SF, et al. Lymphovascular invasion is independently associated with overall survival, cause-specific survival, and local and distant recurrence in patients with negative lymph nodes at radical cystectomy.
J Clin Oncol
2005
;
23
:
6533
–9.
18
Quek ML, Stein JP, Nichols PW, et al. Prognostic significance of lymphovascular invasion of bladder cancer treated with radical cystectomy.
J Urol
2005
;
174
:
103
–6.
19
Kattan MW, Eastham JA, Stapleton AM, Wheeler TM, Scardino PT. A preoperative nomogram for disease recurrence following radical prostatectomy for prostate cancer.
J Natl Cancer Inst
1998
;
90
:
766
–71.
20
Kattan MW, Wheeler TM, Scardino PT. Postoperative nomogram for disease recurrence after radical prostatectomy for prostate cancer.
J Clin Oncol
1999
;
17
:
1499
–507.
21
Kattan MW, Reuter V, Motzer RJ, Katz J, Russo P. A postoperative prognostic nomogram for renal cell carcinoma.
J Urol
2001
;
166
:
63
–7.
22
Kattan MW, Leung DH, Brennan MF. Postoperative nomogram for 12-year sarcoma-specific death.
J Clin Oncol
2002
;
20
:
791
–6.
23
Kattan MW, Shariat SF, Andrews B, et al. The addition of interleukin-6 soluble receptor and transforming growth factor β1 improves a pretreatment nomogram for predicting recurrence in patients with clinically localized prostate cancer.
J Clin Oncol
2003
;
21
:
1
–7.
24
Kattan MW. Comparison of Cox regression with other methods for determining prediction models and nomograms.
J Urol
2003
;
170
:
S6
–9; discussion S10.
25
Kattan MW, Karpeh MS, Mazumdar M, Brennan MF. Postoperative nomogram for disease-specific survival after an R0 resection for gastric carcinoma.
J Clin Oncol
2003
;
21
:
3647
–50.
26
Gail MH, Brinton LA, Byar DP, et al. Projecting individualized probabilities of developing breast cancer for white females who are being examined annually.
J Natl Cancer Inst
1989
;
81
:
1879
–86.
27
Harrell FE, Jr., Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.
Stat Med
1996
;
15
:
361
–87.
28
Sobin LH, Fleming ID. TNM classification of malignant tumors. 5th ed. Union Internationale Contre le Cancer and the American Joint Committee on Cancer.
Cancer
1997
;
80
:
1803
–4.
29
Shariat SF, Khoddami SM, Saboorian H, et al. Lymphovascular invasion is a pathological feature of biologically aggressive disease in patients treated with radical prostatectomy.
J Urol
2004
;
171
:
1122
–7.
30
Henson DE, Hutter RV, Farrow G. Practice protocol for the examination of specimens removed from patients with carcinoma of the prostate gland. A publication of the cancer committee, college of american pathologists. Task Force on the Examination of Specimens Removed From Patients with Prostate Cancer.
Arch Pathol Lab Med
1994
;
118
:
779
–83.
31
Greene FL, Page DL, Fleming ID, et al. AJCC cancer staging manual. 6th ed. New York: Springer-Verlag; 2002.
32
Van Houwelingen JC, Le Cessie S. Predictive value of statistical models.
Stat Med
1990
;
9
:
1303
–25.
33
Atkinson AC. A note on the generalized information criterion for choice of a model.
Biometrics
1980
;
67
:
413
–8.
34
Grambsch PM, Therneau TM. Proportional hazards tests and diagnostics based on weighted residuals.
Biometrics
1994
;
81
:
515
–9.
35
Harrell FE, Jr., Califf RM, Pryor DB, Lee KL, Rosati RA. Evaluating the yield of medical tests.
JAMA
1982
;
247
:
2543
–6.
36
Bradley E, Tibshirani RJ. Monographs on statistics and applied probability: an introduction to the bootstrap. Champman and Hall/CRC; 1993.
37
DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.
Biometrics
1988
;
44
:
837
–45.
38
Herr HW, Faulkner JR, Grossman HB, et al. Surgical factors influence bladder cancer outcomes: a cooperative group report.
J Clin Oncol
2004
;
22
:
2781
–9.