Background:

Population-based metastatic recurrence rates for patients diagnosed with nonmetastatic colorectal cancer cannot be estimated directly from population-based cancer registries because recurrence information is not reported. We derived population-based colorectal cancer recurrence rates using disease-specific survival data based on our understanding of the colorectal cancer recurrence-death process.

Methods:

We used a statistical continuous-time multistate survival model to derive population-based annual colorectal cancer recurrence rates from 6 months to 10 years after colorectal cancer diagnosis using relative survival data from the Surveillance, Epidemiology, and End Results Program. The model was based on the assumption that, after 6 months of diagnosis, all colorectal cancer–related deaths occur only in patients who experience a metastatic recurrence first, and that the annual colorectal cancer–specific death rate among patients with recurrence was the same as in those diagnosed with de novo metastatic disease. We allowed recurrence rates to vary by post-diagnosis time, age, stage, and location for two diagnostic time periods.

Results:

In patients diagnosed in 1975–1984, annual recurrence rates 6 months to 5 years after diagnosis ranged from 0.054 to 0.060 in stage II colon cancer, 0.094 to 0.105 in stage II rectal cancer, and 0.146 to 0.177 in stage III colorectal cancer, depending on age. We found a statistically significant decrease in colorectal cancer recurrence among patients diagnosed in 1994–2003 compared with those diagnosed in 1975–1984 for 6 months to 5 years after diagnosis (hazard ratios between 0.43 and 0.70).

Conclusions:

We derived population-based annual recurrence rates for up to 10 years after diagnosis using relative survival data.

Impact:

Our estimates can be used in decision-analytic models to facilitate analyses of colorectal cancer interventions that are more generalizable.

Improvements in colorectal cancer care have prolonged patient survival since 1975 (1, 2), but many patients still develop (metastatic) recurrence (3–5), from which patients can die from their disease. Patients with progressive, recurrent, or second colorectal cancer may differ in their prognosis and need different treatment regimens. Although progression of colorectal cancer indicates that the primary cancer has spread or worsened, colorectal cancer recurrence is when the cancer has returned after it was initially cured and undetectable for a period of time and can be local (same location as original tumor), regional (lymph node involvement near the initial site), or distant (in another part of the body). Further, recurrent cancer differs from a second cancer in that the former has the same type of cancer cells as the primary cancer, as opposed to the latter, which is unrelated to the primary cancer. The focus of our study was on distant recurrences.

Current evidence on recurrence rates comes from randomized controlled trials (RCT) in which disease-free survival is a common endpoint (6, 7). Because of RCTs' strict eligibility criteria, RCT-based colorectal cancer recurrence rates do not reflect population-based rates (8), which are difficult to ascertain directly because they are not tracked by population-based cancer registries. Evidence on population-based recurrence rates can better inform management of recurrent disease and may play an important role in decision-analytic models to capture the economic and clinical effects of colorectal cancer recurrence.

In this article, we applied a multistate survival modeling approach to derive population-based annual rates of colorectal cancer recurrence using relative survival curves from the Surveillance, Epidemiology, and End Results (SEER) Program. Multistate survival models (MSM) have been widely used to estimate disease recurrence rates that cannot be directly estimated from data because of unobserved events or health states (9–12). Using data on colorectal cancer patients diagnosed in 1975–1984 and 1994–2003 with up to 10 years of follow-up for each patient sample, we estimated recurrence rates by disease stage, cancer location, and patient age for the two diagnosis periods to reflect the impact of changes in colorectal cancer care on colorectal cancer recurrence rates in the United States.

Overview

The objective of our analysis was 2-fold. First, we sought to derive population-based recurrence rates from relative survival curves. Second, we compared the changes in rates for two time periods (1975–1984, 1994–2003). Recurrence was defined as the detection of metastatic disease following an initial diagnosis of nonmetastatic colorectal cancer (stage II or stage III). We used data on relative survival from the time of initial diagnosis, which reflects the probability of not dying from cancer over time in the absence of other causes. We made the assumption that, after 6 months of diagnosis, all colorectal cancer–related deaths occur only in patients who experience a recurrence, and assumed that the time to cancer death among persons with recurrence is the same as for persons with de novo metastatic colorectal cancer. The latter assumption was supported by a study showing no statistically significant difference in survival between patients with metastatic colorectal cancer recurrence and patients with de novo metastatic colorectal cancer (13). These assumptions enabled us to derive annual recurrence rates for groups of patients defined by age, stage, location, and diagnosis time period.

We used relative survival to represent disease-specific survival as opposed to using cause-specific survival. Both estimates have been shown to be close for colorectal cancer, with cause-specific survival estimates slightly higher on average (14). Further, our multistate survival modeling approach assumes that all colorectal cancer patients face a risk of experiencing a disease recurrence, although many patients will not experience a recurrence over the 10-year period. Incorporating a fraction of cured patients cannot experience a recurrence (i.e., a cure model) would represent an alternative approach (15). Because our aim was to estimate annual recurrence rates, we opted for the more parsimonious approach that estimates these rates directly instead of deriving them from two parameters (i.e., cure fraction, recurrence rate among those not cured). Given that the SEER Program implemented a collaborative staging approach in 2004 that resulted in staging discontinuity (16), we selected a second time window of 1994–2003 to represent some improvement in colorectal cancer care, although it does not represent current colorectal cancer care. This allowed us to derive annual recurrence rates for 10 years after diagnosis without having to make parametric extrapolations, and allowed us to avoid assumptions to overcome the limitations of the staging discontinuity that occurred in 2004 (16).

Study population

We used data collected for stage II and stage III colorectal cancer patients from nine SEER registries: Atlanta, Connecticut, Detroit, Hawaii, Iowa, New Mexico, San Francisco–Oakland, Seattle–Puget Sound, and Utah (17). We identified all individuals diagnosed with colorectal cancer without prior cancer diagnosis in two periods: 1975–1984 and 1994–2003. The first period includes patients diagnosed during an era with limited options for colorectal cancer care beyond surgical resection, and the second period represents an era with the additional availability of adjuvant therapies, improved surgical techniques, and use of radiotherapy (2, 18–28). The sample included data from 36,479 individuals diagnosed with colorectal cancer in 1975–1984 and 38,977 in 1994–2003. In both periods, over 50% of these individuals were diagnosed at ages 65 to 79 years, and over 69% of patients with colorectal cancer stage II or III had their primary tumor located in colon (Table 1). We used a 10-year follow-up time for the two cohorts.

Table 1.

Characteristics of patient population diagnosed with colorectal cancer in 1975–1984 and 1994–2003 and reported to the SEER program.

Year of diagnosis
1975–19841994–2003
Stage IIStage IIIStage IVaStage IIStage IIIStage IVa
No. 20,343 16,136 15,574 19,532 19,445 13,933 
Patient characteristics used in stratification 
Age at diagnosis 
 20–49 years 7.2% 9.3% 8.6% 9.7% 13.2% 13.8% 
 50–64 years 34.9% 38.5% 37.1% 29.2% 34.1% 34.2% 
 65–79 years 57.8% 52.2% 54.4% 61.0% 52.6% 52.0% 
Cancer location 
 Colon 74.2% 68.5% 74.0% 76.5% 71.0% 74.0% 
 Rectum 25.8% 31.6% 26.0% 23.5% 29.0% 26.0% 
Additional patient characteristicsb 
Sex 
 Female 48.3% 47.9% 47.3% 47.2% 47.0% 44.8% 
 Male 51.8% 52.1% 52.7% 52.8% 53.0% 55.2% 
Racec 
 Nonwhite 10.6% 12.6% 12.9% 17.8% 20.4% 22.0% 
 White 89.5% 87.4% 87.1% 82.2% 79.6% 78.1% 
Year of diagnosis
1975–19841994–2003
Stage IIStage IIIStage IVaStage IIStage IIIStage IVa
No. 20,343 16,136 15,574 19,532 19,445 13,933 
Patient characteristics used in stratification 
Age at diagnosis 
 20–49 years 7.2% 9.3% 8.6% 9.7% 13.2% 13.8% 
 50–64 years 34.9% 38.5% 37.1% 29.2% 34.1% 34.2% 
 65–79 years 57.8% 52.2% 54.4% 61.0% 52.6% 52.0% 
Cancer location 
 Colon 74.2% 68.5% 74.0% 76.5% 71.0% 74.0% 
 Rectum 25.8% 31.6% 26.0% 23.5% 29.0% 26.0% 
Additional patient characteristicsb 
Sex 
 Female 48.3% 47.9% 47.3% 47.2% 47.0% 44.8% 
 Male 51.8% 52.1% 52.7% 52.8% 53.0% 55.2% 
Racec 
 Nonwhite 10.6% 12.6% 12.9% 17.8% 20.4% 22.0% 
 White 89.5% 87.4% 87.1% 82.2% 79.6% 78.1% 

aData on patients with stage IV colorectal cancer (i.e., de novo metastatic colorectal cancer) defined by age, location, and diagnosis year were used to inform transitions from Recur to death (⁠|{\rm{\lambda }}_{23}^{{\rm{ct}}}$|⁠).

bThese additional characteristics of patient population were not used to stratify patients in the multistate models but are provided for descriptive purposes.

cDue to a high proportion of white patients in the sample, we classified all patients into white and nonwhite race. The nonwhite race group included patients recorded as black and other (American Indian/Alaskan Native and Asian/Pacific Islander).

We stratified patients by age, stage at diagnosis, and cancer location. Patients were grouped into three age categories: 20–49, 50–64, and 65–79 years. We did not include ages greater than 79 years as that age group presented methodological challenges because of the high rates of other-cause mortality that varied sharply with age (29). We based colorectal cancer staging on the fifth edition of the American Joint Committee on Cancer (AJCC) TNM-based system (30). Because this staging system was implemented in the SEER Program from 1988, we used algorithms developed by Dr. Deborah Schrag to reconstruct TNM staging, using its tumor (T) and nodal (N) component measures, for patients diagnosed between 1975 and 1984 (31). Given the differences in treatment of stage II colon and rectal cancer (32), we categorized patients with colorectal cancer stage II by their primary tumor site location. Colon location included patients with cancers in cecum, ascending colon, hepatic flexure, transverse colon, splenic flexure, descending colon, sigmoid colon, and large intestine, not otherwise specified. Rectum location included patients with cancers in rectum and rectosigmoid junction. Rectosigmoid junction site consisted of cancers located above and below the peritoneal reflection and might include a number of patients who would therefore be clinically categorized as colon. Because the differences in recurrent stage III colon and rectal cancer were minor, we did not stratify these patients by their primary tumor site location to avoid creating overly complex MSM with insufficient data (33). We excluded non-colorectal cancer histologic subtypes such as carcinoid, pure squamous and small cells. SEER*Stats codes for data extraction and histologic types based on ICD-03 coding schema included in the analysis are shown in Supplementary Tables S1 and S2, respectively.

Statistical analysis

We first derived 18 distinct relative survival curves for patient samples defined by age (20–49, 50–64, and 65–79 years), stage and location (stage II colon, stage II rectum, and stage III), and diagnosis year (1975–1984 and 1994–2003), which in turn were used to inform 18 MSMs (34). For each MSM, we defined three health states to characterize the colorectal cancer recurrence-death process: (i) no evidence of disease (NED), (ii) symptomatic recurrence (Recur), and (iii) death (Fig. 1). By defining these health states and imposing our understanding of transitions among them, we derived the unobserved annual recurrence rates, defined by transitions occurring from NED to Recur (⁠|\lambda _{12}^{ct}$| in Fig. 1). Specifically, we assumed that all patients dying from colorectal cancer transitioned to the recurrence state first. To inform transitions from Recur to death (⁠|{\rm{\lambda }}_{23}^{{\rm{ct}}}$| in Fig. 1), we used data from SEER on patients with de novo metastatic colorectal cancer defined by age, location, and diagnosis year. We accounted for background mortality (⁠|{\rm{\lambda }}_{13}^{{\rm{ct}}}$| in Fig. 1) using expected survival (i.e., survival in the absence of cancer diagnosis) for each subpopulation considered in our analysis, directly obtained from SEER*Stat from the NCI (35). We estimated the transition rates between the three health states of the MSM using a maximum-likelihood estimation approach (36).

Figure 1.

Structure of continuous-time MSMs. In the continuous-time MSM, all patients start in the NED health state and are at risk of either developing colorectal cancer (CRC) recurrences (Recur; transition |\lambda _{12}^{ct}$|⁠) or dying due to other causes (Death; transition |\lambda _{13}^{ct}$|⁠). Patients who develop recurrences are at risk of dying due to colorectal cancer (transition |\lambda _{23}^{ct}$|⁠). Transitions from NED to Recur health state (i.e., |\lambda _{12}^{ct}$|⁠) are not directly observed in SEER data and are estimated with the MSM. Transitions from Recur to death health state (i.e., |\lambda _{23}^{ct}$|⁠) are informed with SEER data on patients with de novo metastatic colorectal cancer, which are assumed to represent outcomes of patients with recurrent colorectal cancer based on the study published by Hassett et al. (13). c, cohort of patients diagnosed in either 1975–1984 or 1994–2003; Death, death state that includes death due to colorectal cancer and other causes; NED, no evidence of disease health state; Recur, colorectal cancer recurrence health state; t, piecewise-exponential time-dependent transition; |\lambda _{12}^{ct}$|⁠, transition from NED to Recur health state; |\lambda _{23}^{ct}$|⁠, transition from Recur to death due to colorectal cancer; |\lambda _{13}^{ct}$|⁠, transition from NED to death due to other causes.

Figure 1.

Structure of continuous-time MSMs. In the continuous-time MSM, all patients start in the NED health state and are at risk of either developing colorectal cancer (CRC) recurrences (Recur; transition |\lambda _{12}^{ct}$|⁠) or dying due to other causes (Death; transition |\lambda _{13}^{ct}$|⁠). Patients who develop recurrences are at risk of dying due to colorectal cancer (transition |\lambda _{23}^{ct}$|⁠). Transitions from NED to Recur health state (i.e., |\lambda _{12}^{ct}$|⁠) are not directly observed in SEER data and are estimated with the MSM. Transitions from Recur to death health state (i.e., |\lambda _{23}^{ct}$|⁠) are informed with SEER data on patients with de novo metastatic colorectal cancer, which are assumed to represent outcomes of patients with recurrent colorectal cancer based on the study published by Hassett et al. (13). c, cohort of patients diagnosed in either 1975–1984 or 1994–2003; Death, death state that includes death due to colorectal cancer and other causes; NED, no evidence of disease health state; Recur, colorectal cancer recurrence health state; t, piecewise-exponential time-dependent transition; |\lambda _{12}^{ct}$|⁠, transition from NED to Recur health state; |\lambda _{23}^{ct}$|⁠, transition from Recur to death due to colorectal cancer; |\lambda _{13}^{ct}$|⁠, transition from NED to death due to other causes.

Close modal

When information about the observed events is available in the data (i.e., the exact transition times are known), the likelihood is estimated with a transition intensity matrix Λ (see Supplementary Method Description in Supplementary Materials). If the observations of the Markov process are at arbitrary times (i.e., the exact transition times are unknown), the likelihood is estimated in terms of the transition probability matrix P(u) = exp(), which is a matrix exponential of the scaled transition intensity matrix [see Cox and Miller (37) for detailed explanation; ref. 36]. When considering time-homogenous process, the (i, j) entry of P(q, u + q) represents the probability of state j at a future time u + q, conditioning on i being the state at time q (36). Given the constant transition intensity matrix Λ in the time interval (q, u + q) in time-homogenous process, the transition probability matrix equals to P(q, u + q) = P(u). In our analysis, we fit time-varying MSMs where the rates change with time. To fit this type of MSM, time dependency was incorporated in the model in form of piecewise-constant time-dependent covariates. A detailed description on the construction of the MSMs and recurrence rates estimation is provided in the Supplementary Method Description in Supplementary Materials.

Our MSMs accounted for time-varying rates assuming time to transition follows a piecewise-exponential distribution (i.e., recurrence rates are constant over prespecified time intervals). We chose the cutoff point for defining the time intervals based on Pearson-type goodness-of-fit tests and visual inspection of the model predictions (i.e., expected outcomes) compared with the outcomes using SEER survival data (i.e., observed outcomes). For comparison purposes, the cutoff point was held constant across all colorectal cancer stages, patient age groups, cancer locations, and diagnosis periods. We reported estimated recurrence rates occurring 6 months or later after diagnosis because recurrences and cancer deaths in the first 6 months after diagnosis would mainly represent upstaging and surgical and treatment mortality.

To evaluate changes in recurrence in patients diagnosed with colorectal cancer in 1994–2003 compared with 1975–1984, we estimated hazard ratios (HR) and 95% confidence intervals (CI) applying Taylor expansions (38, 39). We performed secondary analyses for patient samples defined by age group 20–39 and 40–49, colon cancer location (right colon, including cecum, ascending colon, transverse colon and hepatic flexure, and left colon, including descending colon, sigmoid colon and splenic flexure), race, and sex.

We retrieved all data using SEER*Stat from the NCI (35). All statistical analyses were performed in the statistical computing language R, and the MSMs were estimated using the “msm” R package (36, 40, 41). This study was reviewed by the Norwegian Regional Committees for Medical and Health Research Ethics (REK) and determined to be exempt from REK approval.

Figure 2 presents a comparison of the relative survival curves of patients diagnosed in 1975–1984 and 1994–2003 stratified by patient age group and colorectal cancer stage and location (for stage II). In line with previous evidence (1), we found that the difference in relative survival between patients diagnosed in 1975–1984 and 1994–2003 was statistically significant across cancer stage, location (for stage II), and age groups.

Figure 2.

Estimates of relative survival with 95% CIs among patients ages 20–49, 50–64, and 65–79 years diagnosed with stage II colon, stage II rectum, or stage III colorectal cancer, respectively, in 1975–1984 and 1994–2003.

Figure 2.

Estimates of relative survival with 95% CIs among patients ages 20–49, 50–64, and 65–79 years diagnosed with stage II colon, stage II rectum, or stage III colorectal cancer, respectively, in 1975–1984 and 1994–2003.

Close modal

Using a Pearson-type goodness-of-fit test and visual inspection of the model predictions compared with SEER survival outcomes, we determined the cutoff point for the time at which rates changed for all individuals. The MSM with best fit in terms of the Pearson-type goodness-of-fit test had the cutoff point at year 5 after diagnosis. This gave piecewise-constant rates in periods: 6 months to 5 years and 5 to 10 years after diagnosis. The selected time at which rates changed was in line with patterns of trial-based recurrence rates over 8 to 10 years after diagnosis (6, 7).

Recurrence in patients diagnosed in 1975–1984

In patients diagnosed with stage II colon cancer in 1975–1984, the estimated annual recurrence rates varied between 0.040 (95% CI, 0.021–0.075; patients ages 65–79 in 5–10 years after diagnosis) and 0.060 (95% CI, 0.055–0.066; patients ages 50–64 in 6 months–5 years after diagnosis; Table 2 and Fig. 3). Recurrence rates in patients with stage II rectal cancer were higher and varied between 0.063 (95% CI, 0.044–0.089; patients ages 50–64 in 5–10 years after diagnosis) and 0.105 (95% CI, 0.084–0.131; patients ages 20–49 in 6 months–5 years after diagnosis). The annual recurrence rates in all patients with stage II colorectal cancer (combined colon and rectum) are provided in Supplementary Table S3. The 10-year cumulative risk of recurrence (95% CI), conditional on being recurrence-free at 6 months, in patients with stage II colon cancer was 37.1% (27.1%–51.3%), 37.5% (32.3%–43.8%), and 36.5% (28.9%–47.9%) for ages 20–49, 50–64, and 65–79, respectively. The 10-year cumulative risk in patients with stage II rectal cancer was 59.2% (44.6%–76.3%), 52.2% (45.3%–59.9%), and 53.3% (43.2%–66.8%) for ages 20–49, 50–64, and 65–79, respectively.

Table 2.

Population-based annual recurrence rates in patients with colorectal cancer stage II colon, stage II rectum, and stage III diagnosed in 1975–1984 and 1994–2003 by time since diagnosis.

Time since diagnosis
6 months to 5 years5 years to 10 years
Recurrence rate (95% CI)Recurrence rate (95% CI)
Diagnosis in 1975–1984 
Stage II 
Age 20–49 
 Colon 0.054 (0.045–0.065) 0.044 (0.023–0.085) 
 Rectum 0.105 (0.084–0.131) 0.085 (0.042–0.169) 
Age 50–64 
 Colon 0.060 (0.055–0.066) 0.040 (0.028–0.056) 
 Rectum 0.094 (0.085–0.104) 0.063 (0.044–0.089) 
Age 65–79 
 Colon 0.057 (0.053–0.062) 0.040 (0.021–0.075) 
 Rectum 0.095 (0.086–0.105) 0.067 (0.035–0.126) 
Stage III 
 Age 20–49 0.146 (0.125–0.172) 0.021 (0.004–0.112) 
 Age 50–64 0.177 (0.165–0.189) 0.016 (0.005–0.049) 
 Age 65–79 0.157 (0.146–0.168) 0.084 (0.051–0.137) 
Diagnosis in 1994–2003 
Stage II 
Age 20–49 
 Colon 0.035 (0.027–0.046) 0.024 (0.007–0.081) 
 Rectum 0.045 (0.033–0.062) 0.031 (0.009–0.106) 
Age 50–64 
 Colon 0.033 (0.028–0.039) 0.032 (0.014–0.076) 
 Rectum 0.056 (0.046–0.068) 0.055 (0.023–0.129) 
Age 65–79 
 Colon 0.040 (0.035–0.045) 0.038 (0.012–0.118) 
 Rectum 0.064 (0.055–0.075) 0.060 (0.019–0.191) 
Stage III 
 Age 20–49 0.099 (0.083–0.117) 0.012 (0.001–0.237) 
 Age 50–64 0.110 (0.100–0.121) 0.016 (0.002–0.107) 
 Age 65–79 0.094 (0.085–0.104) 0.066 (0.026–0.168) 
Time since diagnosis
6 months to 5 years5 years to 10 years
Recurrence rate (95% CI)Recurrence rate (95% CI)
Diagnosis in 1975–1984 
Stage II 
Age 20–49 
 Colon 0.054 (0.045–0.065) 0.044 (0.023–0.085) 
 Rectum 0.105 (0.084–0.131) 0.085 (0.042–0.169) 
Age 50–64 
 Colon 0.060 (0.055–0.066) 0.040 (0.028–0.056) 
 Rectum 0.094 (0.085–0.104) 0.063 (0.044–0.089) 
Age 65–79 
 Colon 0.057 (0.053–0.062) 0.040 (0.021–0.075) 
 Rectum 0.095 (0.086–0.105) 0.067 (0.035–0.126) 
Stage III 
 Age 20–49 0.146 (0.125–0.172) 0.021 (0.004–0.112) 
 Age 50–64 0.177 (0.165–0.189) 0.016 (0.005–0.049) 
 Age 65–79 0.157 (0.146–0.168) 0.084 (0.051–0.137) 
Diagnosis in 1994–2003 
Stage II 
Age 20–49 
 Colon 0.035 (0.027–0.046) 0.024 (0.007–0.081) 
 Rectum 0.045 (0.033–0.062) 0.031 (0.009–0.106) 
Age 50–64 
 Colon 0.033 (0.028–0.039) 0.032 (0.014–0.076) 
 Rectum 0.056 (0.046–0.068) 0.055 (0.023–0.129) 
Age 65–79 
 Colon 0.040 (0.035–0.045) 0.038 (0.012–0.118) 
 Rectum 0.064 (0.055–0.075) 0.060 (0.019–0.191) 
Stage III 
 Age 20–49 0.099 (0.083–0.117) 0.012 (0.001–0.237) 
 Age 50–64 0.110 (0.100–0.121) 0.016 (0.002–0.107) 
 Age 65–79 0.094 (0.085–0.104) 0.066 (0.026–0.168) 
Figure 3.

Population-based estimates of recurrence rates, with 95% CIs, of colorectal cancer by time since diagnosis, age, and cancer stage and location.

Figure 3.

Population-based estimates of recurrence rates, with 95% CIs, of colorectal cancer by time since diagnosis, age, and cancer stage and location.

Close modal

In patients diagnosed with colorectal cancer stage III in 1975–1984, the annual recurrence rates were highest between 6 months and 5 years after diagnosis ranging between 0.146 (95% CI, 0.125–0.172; patients ages 20–49) and 0.177 (95% CI, 0.165–0.189; patients ages 50–64; Table 2 and Fig. 3). Recurrence rates 5–10 years after diagnosis were substantially lower, especially in patients ages <65 ranging between 0.016 and 0.021. In patients with colorectal cancer stage III, the 10-year cumulative risk of recurrence (95% CI) was 53.5% (44.1%–73.6%), 58.4% (53.7%–66.6%), and 67.5% (59.8%–76.4%) for ages 20–49, 50–64, and 65–79, respectively.

Recurrence in patients diagnosed in 1994–2003

In patients diagnosed with stage II colon cancer in 1994–2003, the annual recurrence rates ranged between 0.024 (95% CI, 0.007–0.081; patients ages 20–49 in 5–10 years after diagnosis) and 0.040 (95% CI, 0.035–0.045; patients ages 65–79 in 6 months–5 years after diagnosis; Table 2 and Fig. 3). Recurrence rates in patients with stage II rectal cancer were higher and varied between 0.031 (95% CI, 0.009–0.106; patients ages 20–49 in 5–10 years after diagnosis) and 0.064 (95% CI, 0.055–0.075; patients ages 65–79 in 6 months–5 years after diagnosis). The annual recurrence rates in all patients with stage II colorectal cancer (combined colon and rectum) are provided in Supplementary Table S3. The 10-year cumulative risk of recurrence (95% CI) in patients with stage II colon cancer was 24.3% (14.5%–45.8%), 26.8% (17.8%–42.6%), and 30.8% (19.6%–54.9%) for ages 20–49, 50–64, and 65–79, respectively. The 10-year cumulative risk in patients with stage II rectal cancer 29.9% (17.4%–55.3%), 40.9% (27.7%–61.5%), and 44.6% (29.0%–72.5%) for ages 20–49, 50–64, and 65–79, respectively.

In patients diagnosed with colorectal cancer stage III in 1994–2003, the annual recurrence rates 6 months to 5 years after diagnosis ranged between 0.094 (95% CI, 0.085–0.104; patients ages 65–79) and 0.110 (95% CI, 0.100–0.121; patients ages 50–64; Table 2 and Fig. 3). Recurrence rates 5 to 10 years after diagnosis were lower, especially in patients ages <65 ranging between 0.012 and 0.016. In patients with colorectal cancer stage III, the 10-year cumulative risk of recurrence (95% CI) was 39.7% (31.5%–82.0%), 43.6% (36.8%–66.0%), and 52.8% (39.9%–72.9%) for ages 20–49, 50–64, and 65–79, respectively.

Changes in recurrence in patients diagnosed in 1994–2003 compared with 1975–1984

The estimated population-based recurrence rates decreased in patients with colorectal cancer diagnosis in 1994–2003 compared with 1975–1984 across all patient age groups and disease stages (Fig. 4; Supplementary Table S4). In 6 months to 5 years after diagnosis, we found a statistically significant improvement in recurrence rates with HRs ranging between 0.43 (95% CI, 0.36–0.51; stage II rectum ages 20–49) and 0.70 (95% CI, 0.65–0.76; stage II colon ages 65–79). In 5 to 10 years after diagnosis, our findings indicated reduction in recurrence rates in all of the patient subgroups studied, but none of the results were statistically significant except for stage II rectum ages 20 to 49 (HR, 0.36; 95% CI, 0.21–0.63).

Figure 4.

Estimated HRs of recurrence rates with their 95% CIs in patients with colorectal cancer by post-diagnosis time, age group, and cancer stage and location for patients diagnosed in 1994–2003 vs. 1975–1984. The results are presented on log scale.

Figure 4.

Estimated HRs of recurrence rates with their 95% CIs in patients with colorectal cancer by post-diagnosis time, age group, and cancer stage and location for patients diagnosed in 1994–2003 vs. 1975–1984. The results are presented on log scale.

Close modal

Secondary analyses

In our secondary analyses, we further stratified patients by age group 20–39 and 40–49, colon cancer location (right colon cancer and left colon cancer), race, and sex. These further stratifications often greatly decreased the sample size, leading to high uncertainty in the estimates making generalized statements challenging. Still, our results indicated higher annual recurrence rates 6 months to 5 years after diagnosis in patients ages 40 to 49 than in patients ages 20 to 39 (see Supplementary Table S5; Supplementary Fig. S1). Among patients with colon cancer, the recurrence rates were higher in patients with left colon cancer than in patients with right colon cancer (Supplementary Table S6; Supplementary Fig. S2). In patients diagnosed with colorectal cancer in 1994–2003, the annual recurrence rates 6 months to 5 years after diagnosis were generally higher in nonwhite race groups than in white race groups (Supplementary Table S7; Supplementary Figs. S3 and S4). Finally, our secondary analyses indicated that the annual recurrence rates 6 months to 5 years after diagnosis were generally higher in male than in female patients (see Supplementary Table S8; Supplementary Figs. S5 and S6).

We used statistical multistate survival modeling techniques to estimate population-based annual colorectal cancer recurrence rates, which are not directly observed in the SEER Program, and evaluated the potential population-level impact of improvements in colorectal cancer care on colorectal cancer recurrence over two diagnosis time periods. Our results indicate that patients diagnosed in 1994–2003 experience lower recurrence rates compared with those diagnosed in 1975–1984 when colorectal cancer care was based solely on surgical resection of the tumor, and most notably for 6 months to 5 years after diagnosis. In this post-diagnosis period, the decrease in recurrence rates was statistically significant in stage, location, and age groups. We found no statistically significant decrease in recurrence rates 5 to 10 years after diagnosis for most of the groups.

Recurrence rates among colorectal cancer patients have previously been estimated from RCT data. However, trial-based results may not reflect population estimates because clinical trials select younger, healthier patients. Direct comparisons of the population-based recurrence rates estimated in our study with trial-based estimates are challenging because the trial-based rates represent a wide range of diagnosis years and treatments (6, 7, 42). For example, 64% of patients used for the trial-based estimates were diagnosed between 1984 and 1994—years not included in our analysis (6, 7). Despite these differences, we compared the 10-year cumulative risks derived from the reported trial-based study (6) and our population-based estimates for the two diagnosis periods, and found that, as expected, the trial-based estimates are lower. To derive a single cumulative risk from our age-specific risks, we used the age distribution from the trials. For patients with colorectal cancer stage II, the 10-year cumulative risk from the trial data was 22.0%, which is lower than our estimate for the more recent diagnosis period (30.8%) and the earlier diagnosis period (44.4%). For stage III, the 10-year cumulative risk from the trial data was 42.7%, which is lower than our estimate for the more recent diagnosis period (46.6%) and the earlier diagnosis period (61.1%).

Evidence on distant metastatic recurrence plays an important role in improving disease management. However, collecting population-based information on recurrent disease is challenging because it requires frequent surveillance as well as detailed and systematized registering. Although a few countries have national registries that record cancer recurrences (43–46), their data on recurrent events are often limited in time frame and their completeness (47). A few studies used algorithms to identify recurrent events from health claims data (48–51), but they may provide imprecise results in terms of timing of the recurrence (52). In our study, we proposed the application of the multistate survival modeling approach. This approach has been previously used to analyze progression of diseases other than cancer with completely missing data for particular events or health states (9–12, 36, 53). As presented in our study, the MSM can be applied to infer cancer recurrence from a population-based cancer registry. Our study focused on colorectal cancer, but this approach can be used for any cancer type with solid tumors diagnosed in any time period as long as the registry includes a sufficiently large sample of the concerned population (33).

Our study has several limitations. To obtain estimates of population-based colorectal cancer recurrence rates, which are not reported by the SEER Program, we assumed that patients who experience colorectal cancer recurrence have similar survival to patients with de novo metastatic colorectal cancer. Previous evidence supports the plausibility of this assumption (13). However, we acknowledge that if time from colorectal cancer recurrence to death was longer than time from de novo metastatic colorectal cancer to death, this assumption would increase the time from recurrence to death in our analysis and would underestimate recurrence rates.

In this study, we used data collected for colorectal cancer patients from nine SEER registries and assumed that the population included in these registries was representative for all colorectal cancer patients in the United States. However, the selection of patients included in the SEER registries was not random in order to oversample some of the minority groups (54, 55). Consequently, this might have resulted in potential differences between the population included in the SEER registries and the rest of the US population.

Our continuous-time MSMs estimated piecewise-constant rates between diagnosis and recurrence, whereas recurrence rates in clinical practice could vary continuously with time since diagnosis. Thus, our analysis provides an approximation of overall recurrence rates of colorectal cancer. To better reflect clinical practice, we allowed the estimated piecewise-constant transition rates to change over the 10 years after diagnosis in our MSMs. Furthermore, the structure of our MSMs required the assumption of a piecewise-constant mortality rates due to other causes in 6 months to 5 years and 5 to 10 years after diagnosis for each age group and diagnosis period.

An analysis of secular trends in recurrence rates of colorectal cancer requires consistent colorectal cancer staging. In our analysis, we used the fifth edition of the AJCC staging system, which is the most commonly used staging system in colorectal cancer in clinical practice (30, 56). Because the AJCC staging was not recorded in SEER before 1988, we constructed staging information for all diagnosed cases in period 1975–1984 using an algorithm previously developed by the Cancer Intervention and Surveillance Modeling Network (CISNET) colorectal cancer group (31). Although the constructed AJCC staging for colon cancer is straightforward, constructing AJCC staging for rectal cancer (i.e., rectal and rectosigmoid cancer) is more challenging. For example, many patients with rectal cancer were treated with neoadjuvant chemotherapy and thus their disease severity at diagnosis might differ from the registered pathologic stage (57).

In our analysis, we used the 10 last years before the calendar year 2004 to represent the diagnosis period with moderate colorectal cancer care improvements (i.e., 1994–2003). More recent years could not be used because the SEER Program implemented a collaborative staging approach in 2004 that led to a stage definitions discontinuity with less than 6-year follow-up after this major change (16). Still, the 1990s was a decade with moderate improvements in colorectal cancer care including improvements in surgical procedures (18, 20, 21), the use of adjuvant therapy (22) and post- and preoperative radiotherapy (25–28), colorectal cancer screening (58–60), and more intensive post-treatment colorectal cancer surveillance (61, 62). Consequently, we assumed that data from 1994 to 2003 provided sufficient evidence to approximate the potential impact of these moderate improvements in colorectal cancer care on recurrent disease.

In conclusion, our study estimated population-based recurrence rates of colorectal cancer in the United States using data from a large population-based cancer registry. Comparing colorectal cancer recurrence rates before and after the start of substantial changes in colorectal cancer care in the 1990s, we found reductions in disease recurrence possibly as a result of improvements in colorectal cancer care. The reductions were largest and statistically significant between month 6 and year 5 after diagnosis. Our estimates can be used in decision-analytic models to facilitate evaluation of the effectiveness and cost-effectiveness of interventions that aim to prevent, detect, and treat colorectal cancer.

N. Kunst reports grants from the Research Council of Norway (grant numbers 276146 and 304034) during the conduct of the study, personal fees from Thermo Fisher Scientific outside the submitted work, as well as employment with LINK Medical Research, which together with the Research Council of Norway funds N. Kunst's PhD position. E. Aas reports funding for a project related to cancer end-of-life care (SAFE) from the Norwegian Cancer Association, and from the NordForsk (Nordic Funded Research) that could use cases from cancer treatment. In addition, Dr. Aas provided a report for the Directorate of Health in Norway on costs and cost-effectiveness of screening for colorectal cancer. D. Schrag reports personal fees from JAMA (editorship) and Pfizer (symposium speaker), and grants from AACR (research funding) outside the submitted work. K.M. Kuntz reports grants from NIH during the conduct of the study. No potential conflicts of interest were disclosed by the other authors.

The funding agencies had no role in the design of the study, interpretation of results, or writing of the manuscript. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The funding agreement ensured the authors' independence in designing the study, interpreting the data, writing, and publishing the report.

N. Kunst: Conceptualization, formal analysis, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. F. Alarid-Escudero: Conceptualization, validation, investigation, methodology, writing–review and editing. E. Aas: Validation, investigation, writing–review and editing. V.M.H. Coupé: Validation, investigation, writing–review and editing. D. Schrag: Validation, investigation, methodology, writing–review and editing. K.M. Kuntz: Conceptualization, supervision, validation, investigation, methodology, writing–review and editing.

The authors thank Dr. Christopher Jackson for his comments and discussions about the method application. N. Kunst was funded by the Research Council of Norway and LINK Medical Research (grant numbers 276146 and 304034). N. Kuntz and F. Alarid-Escudero were supported by a grant from the NCI (award U01- CA-199335) as part of the Cancer Intervention and Surveillance Modeling Network (CISNET).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
Rutter
CM
,
Johnson
EA
,
Feuer
EJ
,
Knudsen
AB
,
Kuntz
KM
,
Schrag
D
. 
Secular trends in colon and rectal cancer relative survival
.
J Natl Cancer Inst
2013
;
105
:
1806
13
.
2.
Murphy
CC
,
Harlan
LC
,
Lund
JL
,
Lynch
CF
,
Geiger
AM
. 
Patterns of colorectal cancer care in the United States: 1990–2010
.
J Natl Cancer Inst
2015
;
107
:
djv198
.
3.
Kahi
CJ
,
Boland
RC
,
Dominitz
JA
,
Giardiello
FM
,
Johnson
DA
,
Kaltenbach
T
, et al
Colonoscopy surveillance after colorectal cancer resection: recommendations of the US Multi-Society Task Force on colorectal cancer
.
Am J Gastroenterol
2016
;
111
:
337
46
.
4.
Kjeldsen
BJ
,
Kronborg
O
,
Fenger
C
,
Jørgensen
OD
. 
The pattern of recurrent colorectal cancer in a prospective randomised study and the characteristics of diagnostic tests
.
Int J Colorectal Dis
1997
;
12
:
329
34
.
5.
Minsky
BD
. 
Adjuvant radiation therapy for colon cancer
.
Cancer Treat Rev
1995
;
21
:
407
14
.
6.
Sargent
DJ
,
Patiyil
S
,
Yothers
G
,
Haller
DG
,
Gray
R
,
Benedetti
J
, et al
End points for colon cancer adjuvant trials: observations and recommendations based on individual patient data from 20,898 patients enrolled onto 18 randomized trials from the ACCENT Group
.
J Clin Oncol
2007
;
25
:
4569
74
.
7.
Sargent
D
,
Sobrero
A
,
Grothey
A
,
O'Connell
MJ
,
Buyse
M
,
Andre
T
, et al
Evidence for cure by adjuvant therapy in colon cancer: observations based on individual patient data from 20,898 patients on 18 randomized trials
.
J Clin Oncol
2009
;
27
:
872
7
.
8.
Mol
L
,
Koopman
M
,
van Gils
CWM
,
Ottevanger
PB
,
Punt
CJA
. 
Comparison of treatment outcome in metastatic colorectal cancer patients included in a clinical trial versus daily practice in the Netherlands
.
Acta Oncol
2013
;
52
:
950
5
.
9.
Jack
CR
,
Therneau
TM
,
Wiste
HJ
,
Weigand
SD
,
Knopman
DS
,
Lowe
VJ
, et al
Transition rates between amyloid and neurodegeneration biomarker states and to dementia: a population-based, longitudinal cohort study
.
Lancet Neurol
2016
;
15
:
56
64
.
10.
Gangnon
RE
,
Lee
KE
,
Klein
BE
,
Iyengar
SK
,
Sivakumaran
TA
, et al
Effect of the Y402H variant in the complement factor H gene on the incidence and progression of age-related macular degeneration: results from multistate models applied to the Beaver Dam Eye Study
.
Arch Ophthalmol
2012
;
130
:
1169
76
.
11.
Looker
HC
,
Nyangoma
SO
,
Cromie
DT
,
Olson
JA
,
Leese
GP
,
Philip
S
, et al
Predicted impact of extending the screening interval for diabetic retinopathy: the Scottish Diabetic Retinopathy Screening programme
.
Diabetologia
2013
;
56
:
1716
25
.
12.
Van der Gaag
N
,
Bijwaard
G
,
De Beer
J
,
Bonneux
L
. 
A multistate model to project elderly disability in case of limited data
.
Demographic Res
2015
;
32
:
75
106
.
13.
Hassett
MJ
,
Uno
H
,
Cronin
AM
,
Carroll
NM
,
Hornbrook
MC
,
Ritzwoller
DP
. 
Comparing survival after recurrent vs de novo stage IV advanced breast, lung, and colorectal cancer
.
JNCI Cancer Spectr
2018
;
2
:
pky024
.
14.
Forjaz de Lacerda
G
,
Howlader
N
,
Mariotto
AB
. 
Differences in cancer survival with relative versus cause-specific approaches: an update using more accurate life tables
.
Cancer Epidemiol Biomarkers Prev
2019
;
28
:
1544
51
.
15.
Conlon
ASC
,
Taylor
JMG
,
Sargent
DJ
. 
Multi-state models for colon cancer recurrence and death with a cured fraction
.
Stat Med
2014
;
33
:
1750
66
.
16.
Johnson
C
.
SEER Program Coding and Staging Manual
.
Bethesda (MD)
:
NCI
; 
2004
.
17.
Cancer Incidence—Surveillance, Epidemiology, and End Results (SEER) Registries Research Data [database on the Internet]
.
Bethesda (MD)
:
NCI [cited 2019 Apr 10]
.
Available from
: https://seer.cancer.gov/data-software/documentation/seerstat/nov2018/.
18.
Jamison
RL
,
Donohue
JH
,
Nagorney
DM
,
Harmsen
WS
,
Ilstrup
DM
, et al
Hepatic resection for metastatic colorectal cancer results in cure for some patients
.
Arch Surg
1997
;
132
:
505
10
.
19.
The Clinical Outcomes of Surgical Therapy Study Group
. 
A comparison of laparoscopically assisted and open colectomy for colon cancer
.
N Engl J Med
2004
;
350
:
2050
9
.
20.
Phillips
EH
,
Franklin
M
,
Carroll
BJ
,
Fallas
MJ
,
Ramos
R
,
Rosenthal
D
. 
Laparoscopic colectomy
.
Ann Surg
1992
;
216
:
703
7
.
21.
Clinical Outcomes of Surgical Therapy Study Group
,
Nelson
H
,
Sargent
DJ
,
Wieand
HS
,
Fleshman
J
,
Anvari
M
, et al
A comparison of laparoscopically assisted and open colectomy for colon cancer
.
N Engl J Med
2004
;
350
:
2050
9
.
22.
Adjuvant therapy for patients with colon and rectum cancer
.
Consens Statement
1990
;
8
:
1
25
.
23.
de Gramont
A
,
de Gramont
A
,
Chibaudel
B
,
Larsen
AK
,
Tournigand
C
,
André
T
. 
The evolution of adjuvant therapy in the treatment of early-stage colon cancer
.
Clin Colorectal Cancer
2011
;
10
:
218
26
.
24.
Goldberg
RM
,
Sargent
DJ
,
Morton
RF
,
Fuchs
CS
,
Ramanathan
RK
,
Williamson
SK
, et al
A randomized controlled trial of fluorouracil plus leucovorin, irinotecan, and oxaliplatin combinations in patients with previously untreated metastatic colorectal cancer
.
J Clin Oncol
2004
;
22
:
23
30
.
25.
Fisher
B
,
Wolmark
N
,
Rockette
H
,
Redmond
C
,
Deutsch
M
,
Wickerham
DL
, et al
Postoperative adjuvant chemotherapy or radiation therapy for rectal cancer: results from NSABP protocol R-01
.
J Natl Cancer Inst
1988
;
80
:
21
9
.
26.
Thomas
PR
,
Lindblad
AS
. 
Adjuvant postoperative radiotherapy and chemotherapy in rectal carcinoma: a review of the Gastrointestinal Tumor Study Group experience
.
Radiother Oncol
1988
;
13
:
245
52
.
27.
Swedish Rectal Cancer Trial
,
Cedermark
B
,
Dahlberg
M
,
Glimelius
B
,
Påhlman
L
,
Rutqvist
LE
, et al
Improved survival with preoperative radiotherapy in resectable rectal cancer
.
N Engl J Med
1997
;
336
:
980
7
.
28.
Kapiteijn
E
,
Marijnen
CAM
,
Nagtegaal
ID
,
Putter
H
,
Steup
WH
,
Wiggers
T
, et al
Preoperative radiotherapy combined with total mesorectal excision for resectable rectal cancer
.
N Engl J Med
2001
;
345
:
638
46
.
29.
Simmonds
PD
,
Best
L
,
George
S
,
Baughan
C
,
Buchanan
R
,
Davis
C
, et al
Surgery for colorectal cancer in elderly patients: a systematic review
.
Lancet North Am Ed
2000
;
356
:
968
74
.
30.
Fleming
I
,
Cooper
JS
,
Henson
E
,
Hutter
RVP
,
Kennedy
BJ
,
Murphy
GP
, et al
AJCC Cancer Staging Manual
. 5th ed.
Philadelphia
:
Lippincott-Raven
; 
1997
.
31.
AJCC Staging: Staging Colon Cancer Patients Using the TNM System from 1975 to the Present; [about 16 screens]. [cited 2019 Aug 6]
.
Available from
: http://www.mskcc.org/mskcc/html/84761.cfm.
32.
NCCN Clinical Practice Guidelines in Oncology [database on the Internet]
.
Plymouth Meeting (PA)
:
National Comprehensive Cancer Network [cited 2020 Feb 20]
.
Available from
: https://www.nccn.org/professionals/physician_gls.
33.
Jackson
C
. 
Multi-state modelling with R: the msm package. 2019 [cited 2020 May 17]
.
Available from
: https://cran.r-project.org/web/packages/msm/vignettes/msm-manual.pdf.
34.
Andersen
PK
,
Abildstrom
SZ
,
Rosthoj
S
. 
Competing risks as a multi-state model
.
Stat Methods Med Res
2002
;
11
:
203
15
.
35.
SEER*Stat software [database on the Internet]
.
Bethesda (MD)
:
NCI [cited 2019 Apr 7].
Available from
: https://seer.cancer.gov/seerstat/.
36.
Jackson
C
. 
Multi-state models for panel data: the msm Package for R
.
J Stat Softw
2011
;
38
:
1
28
.
37.
Cox
DR
,
Miller
HD
.
Theory of stochastic processes
.
London
:
Chapman and Hall
; 
1965
.
38.
Stuart
A
,
Ord
K
.
Kendall's advanced theory of statistics
. 6th ed.
London
:
Arnold
; 
1998
.
39.
Elandt-Johnson
RC
,
Johnson
NL
.
Survival models and data analysis
.
New York
:
John Wiley & Sons
; 
1980
.
40.
R Core Team [homepage on the Internet]
.
R: A language and environment for statistical computing
.
R Foundation for Statistical Computing
; 
2017
[cited 2019 Apr 7]. Available from
: https://www.R-project.org.
41.
Jackson
C
. 
Package ‘msm’: Multi-State Markov and hidden Markov models in continuous time. 2019 [cited 2019 Nov 19]
.
Available from
: https://cran.rstudio.com/web/packages/msm/msm.pdf.
42.
Shah
MA
,
Renfro
LA
,
Allegra
CJ
,
André
T
,
de Gramont
A
,
Schmoll
H-J
, et al
Impact of patient factors on recurrence risk and time dependency of oxaliplatin benefit in patients with colon cancer: analysis from modern-era adjuvant studies in the adjuvant colon cancer end points (ACCENT) database
.
J Clin Oncol
2016
;
34
:
843
53
.
43.
van den Hurk
CJG
,
Eckel
R
,
van de Poll-Franse
LV
,
Coebergh
JWW
,
Nortier
JWR
,
Hölzel
D
, et al
Unfavourable pattern of metastases in M0 breast cancer patients during 1978–2008: a population-based analysis of the Munich Cancer Registry
.
Breast Cancer Res Treat
2011
;
128
:
795
805
.
44.
Lord
SJ
,
Marinovich
ML
,
Patterson
JA
,
Wilcken
N
,
Kiely
BE
,
Gebski
V
, et al
Incidence of metastatic breast cancer in an Australian population-based cohort of women with non-metastatic breast cancer at diagnosis
.
Med J Aust
2012
;
196
:
688
92
.
45.
Holleczek
B
,
Stegmaier
C
,
Radosa
JC
,
Solomayer
EF
,
Brenner
H
. 
Risk of loco-regional recurrence and distant metastases of patients with invasive breast cancer up to ten years after diagnosis–results from a registry-based study from Germany
.
BMC Cancer
2019
;
19
:
520
.
46.
Snyder
RA
,
Hu
CY
,
Cuddy
A
,
Francescatti
AB
,
Schumacher
JR
,
Van Loon
K
, et al
Association between intensity of posttreatment surveillance testing and detection of recurrence in patients with colorectal cancer
.
JAMA
2018
;
319
:
2104
2115
.
47.
In
H
,
Bilimoria
KY
,
Stewart
AK
,
Wroblewski
KE
,
Posner
MC
,
Talamonti
MS
, et al
Cancer recurrence: an important but missing variable in national cancer registries
.
Ann Surg Oncol
2014
;
21
:
1520
9
.
48.
Hassett
MJ
,
Uno
H
,
Cronin
AM
,
Carroll
NM
,
Hornbrook
MC
,
Ritzwoller
D
. 
Detecting lung and colorectal cancer recurrence using structured clinical/administrative data to enable outcomes research and population health management
.
Med Care
2017
;
55
:
e88
98
.
49.
Hassett
MJ
,
Ritzwoller
DP
,
Taback
N
,
Carroll
N
,
Cronin
AM
,
Ting
GV
, et al
Validating billing/encounter codes as indicators of lung, colorectal, breast, and prostate cancer recurrence using 2 large contemporary cohorts
.
Med Care
2014
;
52
:
e65
73
.
50.
Chubak
J
,
Yu
O
,
Pocobelli
G
,
Lamerato
L
,
Webster
J
,
Prout
MN
, et al
Administrative data algorithms to identify second breast cancer events following early-stage invasive breast cancer
.
J Natl Cancer Inst
2012
;
104
:
931
40
.
51.
Lash
TL
,
Riis
AH
,
Ostenfeld
EB
,
Erichsen
R
,
Vyberg
M
,
Thorlacius-Ussing
O
. 
A validated algorithm to ascertain colorectal cancer recurrence using registry resources in Denmark
.
Int J Cancer
2015
;
136
:
2210
5
.
52.
Mariotto
AB
,
Zou
Z
,
Zhang
F
,
Howlader
N
,
Kurian
AW
,
Etzioni
R
. 
Can we use survival data from cancer registries to learn about disease recurrence? The case of breast cancer
.
Cancer Epidemiol Biomarkers Prev
2018
;
27
:
1332
41
.
53.
Lou
W
,
Wan
L
,
Abner
EL
,
Fardo
DW
,
Dodge
HH
,
Kryscio
RJ
. 
Multi-state models and missing covariate data: expectation-maximization algorithm for likelihood estimation
.
Biostat Epidemiol
2017
;
1
:
20
35
.
54.
Nattinger
AB
,
McAuliffe
TL
,
Schapira
MM
. 
Generalizability of the Surveillance, Epidemiology, and End Results registry population: factors relevant to epidemiologic and health care research
.
J Clin Epidemiol
1997
;
50
:
939
45
.
55.
Surveillance, Epidemiology, and End Results Program
[homepage on the Internet].
Bethesda (MD)
:
NCI [cited 2020 May 14]
.
Available from
: https://seer.cancer.gov/registries/.
56.
Colorectal Cancer Stages; [about 13 screens]. [cited 2019 Aug 7]
.
Available from
: https://www.cancer.org/cancer/colon-rectal-cancer/detection-diagnosis-staging/staged.html.
57.
Minsky
BD
. 
Progress in the treatment of locally advanced clinically resectable rectal cancer
.
Clin Colorectal Cancer
2011
;
10
:
227
37
.
58.
Milestones in Cancer Research and Discovery; [about 19 screens]. [cited 2019 Aug 5]
.
Available from
: https://www.cancer.gov/research/progress/250-years-milestones.
59.
Anderson
WF
,
Guyton
KZ
,
Hiatt
RA
,
Flowers
CR
,
Guerra
CE
,
LaMonte
SJ
, et al
Colorectal cancer screening for persons at average risk
.
J Natl Cancer Inst
2002
;
94
:
1126
33
.
60.
Towler
B
,
Irwig
L
,
Glasziou
P
,
Kewenter
J
,
Weller
D
,
Silagy
C
. 
A systematic review of the effects of screening for colorectal cancer using the faecal occult blood test, hemoccult
.
BMJ
1998
;
317
:
559
65
.
61.
Colorectal Cancer: Types of Treatment; [about 17 screens]. [2019 Jul 7]
.
Available from
: https://www.cancer.net/cancer-types/colorectal-cancer/types-treatment?sectionTitle=Treatment.
62.
Barclay
RL
,
Vicari
JJ
,
Doughty
AS
,
Johanson
JF
,
Greenlaw
RL
. 
Colonoscopic withdrawal times and adenoma detection during screening colonoscopy
.
N Engl J Med
2006
;
355
:
2533
41
.

Supplementary data