Projecting Benefits and Harms of Novel Cancer Screening Biomarkers: A Study of PCA3 and Prostate Cancer

This article is featured in Highlights of This Issue, p. 635

Research to develop biomarkers for the early detection of cancers has expanded rapidly since the identification over 20 years ago of the now widely used cancer antigen 125 (CA-125) for ovarian cancer and prostate-specific antigen (PSA) for prostate cancer (1). Continuing advances in molecular technology promise to widen the realm of potential biomarker candidates (2–4). The Early Detection Research Network (EDRN) was established by the National Cancer Institute (NCI) to facilitate collaboration between the increasingly numerous research groups studying new cancer biomarkers. EDRN researchers are currently studying hundreds of biomarkers spanning a wide range of cancers (5, 6).

One of the promising biomarkers under study by EDRN and others is prostate cancer antigen 3 (PCA3), a gene whose messenger RNA is overexpressed in prostate cancer tissue. Several studies have found that PCA3 increases the diagnostic accuracy of prostate cancer risk prediction models and informs the necessity for repeat biopsies (7–14). A recent EDRN validation study concluded that using PCA3 in conjunction with PSA can improve the performance of PSA-based detection (15). Given the accumulation of promising findings, there is growing interest in considering PCA3 as a screening tool.

As with all cancer biomarkers, advancing PCA3 from a promising biomarker to a clinically useful screening tool will require additional phases of research. Biomarker development generally progresses through several stages (16). Initial exploratory studies are followed by the development and validation of clinical assays for the most promising biomarkers, using specimens from individuals known to have cancer and those known to be cancer-free to determine the diagnostic properties of the biomarker. Later phases of biomarker development focus on determining screening costs in the prospective setting and benefit in terms of disease-specific mortality reduction. These phases are costly, take years to complete, and are still in the early stages for PCA3 (17).

If downstream outcomes were known earlier, these could be used to prioritize among candidate biomarkers and guide research. In this article, we show how simulation modeling can link early-phase biomarker data with information on cancer natural history to project downstream population outcomes for cancer biomarkers with measured diagnostic properties. We use the case of PCA3 as a biomarker for prostate cancer. We extend an existing model of PSA growth and prostate cancer natural history (18) to incorporate PCA3 distributions based on the EDRN validation study (15). We then examine a range of screening strategies involving triggers for biopsy that depend on both PCA3 and PSA levels. We evaluate performance of the candidate screening strategies in terms of test results and, ultimately, in terms of overdiagnosis and lives saved. In our development, we also address general principles regarding modeling the population impact of a new biomarker for early detection of cancer.

Data on the joint distribution of PCA3, PSA, age, and prostate cancer status were obtained from a multicenter EDRN validation study (15). Participants were men age 18 and over scheduled for biopsy between December 2009 and June 2011 for reasons including elevated or rising PSA, abnormal digital rectal exam (DRE), and/or low percent free PSA. PCA3 was measured in urine specimens collected following a DRE and recorded as the conventional score of the ratio of PCA3 mRNA to PSA mRNA multiplied by 1,000. Cancer status and Gleason grade were determined from the prostate biopsy.

We used a model of PSA growth and prostate cancer developed as part of the NCI's Cancer Intervention and Surveillance Modeling Network (CISNET) consortium (http://cisnet.cancer.gov). This model links PSA growth with tumor development and progression (18). PSA growth is slow for healthy men, faster for men following onset of a low-to-moderate grade (Gleason score 7 and below) preclinical cancer, and faster still following onset of high-grade (Gleason score 8–10) cancer. Cancer cases with faster PSA growth have shorter times to metastasis and nonscreen diagnosis. The rates of PSA growth are based on data from the Prostate Cancer Prevention Trial (19, 20), which screened 9,459 men in the control group annually for up to 7 years. The natural history parameters specifying transition rates between disease states are estimated so that the model replicates prostate cancer incidence in the Surveillance, Epidemiology, and End Results (SEER) program (http://seer.cancer.gov) from 1980 to 2000 (Supplementary Table S1). This interval covers years both before and after the introduction of PSA screening in the United States; data on the change in incidence due to screening are needed to inform the transition rates in the natural history model. The calibrated model is used to simulate life histories representative of the U.S. male population. These life histories include age and grade at onset of a biopsy-detectable preclinical tumor, grade-specific PSA trajectory, and age and stage at clinical diagnosis in the absence of screening.

To extend the simulated life histories to include individual PCA3 levels, we required information on (i) the distribution of PCA3 among prostate cancer cases and noncases, (ii) the correlation between PCA3 and PSA, (iii) any association between PCA3 and disease characteristic like grade, and, ideally, (iv) knowledge of how PCA3 grows over a man's lifetime.

Consistent with other studies (9, 21–23), the EDRN validation study indicates that PCA3 varies by cancer status but is only weakly correlated with PSA and age (15). We thus identified distributions for PCA3 for cancer cases and noncases that matched the observed EDRN data but that were independent of PSA and age. Of possible lognormal and exponential distributions, exponential distributions matching the medians in each of the cancer case and noncase groups most closely approximated the observed sensitivity and specificity of PCA3 at selected cutoffs (Table 1).

In the absence of longitudinal data on PCA3, we initially assumed that PCA3 discretely elevates at the time of preclinical disease onset and then stays constant. The pre- and post-onset values of PCA3 were drawn from the distributions of biopsy-negative and biopsy-positive subjects in the EDRN study, respectively.

We simulated two populations with extended life histories that include PCA3 levels at all time points of interest. The first, which we call our validation population, represents a population similar to men in the EDRN study: we simulated 1 million men ages 27 to 86 in 1996 to 2000 and sampled 10,000 of those referred for biopsy according to the joint distribution of cancer status, PSA, and age at biopsy observed in the EDRN trial. The years 1996 to 2000 reflect the most recent years to which the natural history model is calibrated. We evaluated our integration of PCA3 into the natural history model by comparing the sensitivity and specificity of several combination PSA-PCA3 screening strategies in the validation population with the values observed in the EDRN data. We then studied population-level screening strategies in a simulated U.S. population cohort of 10,000 men age 50 in 2000, which we call our projected population.

We simulated screening every 2 years in the projected population between ages 50 and 74 under 11 different combination test strategies. We acknowledge there is uncertainty about the most appropriate screening interval, but we do not anticipate that our results will be sensitive to the particular screening interval chosen. Our nomenclature for the strategies specifies the joint cutoffs for PSA and PCA3 that define positive screening tests. The PSA-only strategy is our “base case” and uses the standard cutoff of 4.0 ng/mL for biopsy referral. We labeled this strategy PSA(4)+PCA3(0) because it does not include a PCA3 cutoff for biopsy referral. Based on consultation with EDRN investigators, we evaluated two types of PSA-PCA3 combination strategies designed to improve specificity and reduce overdiagnosis. In the first type, men are referred to biopsy only if their PSA is above 4.0 ng/mL and their PCA3 is above a certain threshold, in an effort to “rule out” false-positive tests arising from men with elevated PSA. We examined several PCA3 thresholds between 20 and 40, and these strategies are labeled PSA(4)+PCA3(threshold). The second type is similar except that it additionally refers men to biopsy if their PSA is greater than 10.0 ng/mL, regardless of their PCA3. We examined the same thresholds between 20 and 40 and label these strategies PSA(4,10)+PCA3(threshold,0).

In all screenings, we assumed perfect compliance with biopsy referral and perfect sensitivity of the biopsy to detect existing tumors.

The natural history model assumes that all nonmetastatic diagnosed cases elect treatment according to age-, stage-, and grade-specific distributions observed among SEER cases (in 2004, overall 43% chose radiation, 36% chose surgery, and 21% chose conservative management; ref. 18). In the absence of screening, prostate cancer survival is based on age- and grade-specific survival from untreated cases reported in SEER in 1983 to 1986, just before the adoption of the PSA test for screening, improved by an HR of 0.62 for cases who elect surgery or radiotherapy (Supplementary Table S1; ref. 24). Other-cause mortality is independently generated using U.S. life tables. When the age at prostate cancer death precedes the age at other-cause death, the death is attributed to prostate cancer.

We modeled screening benefit with a cure mechanism, which posits cases that would have died of cancer in the absence of screening have a probability of being cured of cancer if detected early. Cure models in the literature typically specify the probability of being cured as constant or a function of the lead time (LT), the time by which diagnosis is advanced by screening (25). We chose the latter approach to reflect the intuition that earlier detection may confer more survival benefit. When LT = 0, screen detection occurs at the same time it would have in the absence of screening, so the probability of cure should be 0, i.e., survival should be equal to that observed in the absence of screening. As LT increases, we assume that the probability of cure increases but eventually plateaus, because there may be a limit to the marginal benefit of earlier detection. These properties are achieved by an exponential function of LT that has a single parameter |$a \,\gt\, 0$|⁠:

We chose a value for a that allowed us to approximate the prostate cancer mortality results from European Randomized study of Screening for Prostate Cancer (ERSPC; ref. 26). We screened our projected population three times, once every 4 years between ages 60 and 71, following the ERSPC's mean age at entry of 60.8, average inter-screen interval of 4 years, median follow-up of 11 years, and PSA cutoff of 3.0 ng/mL. In targeting the trial's mortality rate ratio of 0.71 (estimated in the absence of screening noncompliance and selection bias), we found that a value of a = 0.2 for our cure function allowed us to approximate the ERSPC trial. The resulting cure function predicts a 33% probability of cure when LT is 2 years, a 63% probability of cure when LT is 5 years, and an 86% probability of cure when LT is 10 years. We simulated a cure status for each individual according to the resulting cure function. For individuals that were cured, the date of death was set to their date of other-cause death. Men who were not cured retained their original survival in the absence of screening.

Recognizing that different patients will place different weights on the various harms and benefits associated with any screening program, we projected multiple screening and mortality outcomes for the projected population. Screening outcomes include true- and false-positive tests, sensitivity, and specificity. In men whose disease would never manifest clinically before their death from other causes, true-positive tests were considered overdiagnoses and are reported separately. We collected mortality outcomes until age 100. Among men whose disease would manifest clinically and who consequently may be saved by screening, we report the number of prostate cancer deaths and lives saved relative to the number under no screening. Finally, we report the number of “unnecessary biopsies” as the sum of false positives and overdiagnoses. All results reported are averages over 100 simulation runs.

We conducted two sensitivity analyses to examine alternative modeling assumptions. In the first, we investigated a PCA3 growth model in which PCA3 grows over time instead of staying at a constant level after onset. In the growth model, PCA3 grows from the pre-onset level at a constant annual rate once onset has occurred. We used the validation population to determine an annual growth rate for the PCA3 growth model that yielded a mean PCA3 among biopsy-positive cases similar to the mean PCA3 among cancer cases in the EDRN study.

In the second sensitivity analysis, we retained the constant PCA3 model but allowed PCA3 distributions to vary by grade. Based on literature suggesting a correlation between PCA3 and grade (9, 23, 27, 28), we assigned lower median PCA3 levels for low-grade (LG, Gleason 6) men and higher median levels for medium- to high-grade (M-HG, Gleason 7–10) men under several scenarios that preserved the observed overall PCA3 median observed in the EDRN study. To adapt the natural history model accordingly, we partitioned Gleason scores 7 and below in the simulated population into Gleason score 6 versus 7 based on PSA growth rates (Supplementary Table S1), then combined Gleason score 7 and 8–10.

Biopsies were performed on 859 men in the EDRN validation study, of which 38% were positive. Age ranged from 27 to 86, with 46% ages 55 to 64 and 33% ages 65 to 74. PSA levels ranged between 0.20 and 309 ng/mL. Mean and median PCA3 in the study population were each over two times higher in biopsy-positive subjects than in biopsy-negative subjects (Table 1). As the PCA3 cutoff increased from 20 to 60, specificity increased from 56% to 89% and sensitivity decreased from 78% to 42%. PCA3 levels were not significantly different for men with low Gleason grade (6) and moderate to high grade (7–10); median PCA3 values in each group were 50 and 49, respectively.

In the projected population of men age 50 in the year 2000, 36% have preclinical onset in their lifetime (cases), beginning on average at age 65. Of cases, 38%, or 14% of the total population, would progress to clinical disease in their lifetime in the absence of screening (clinical cases). Among clinical cases, 48% have onset before age 60, preclinical disease lasts on average 14 years, and mean age at prostate cancer death in the absence of screening is 84. The remaining 62% of cases are nonclinical cases. These natural history summary measures are consistent with reported incidence for the U.S. population and do not differ substantially from other independently developed models of prostate cancer natural history (29).

PCA3 distributions in the simulated populations reasonably approximate those observed in the EDRN study (Table 1). The simulated means closely reproduce the observed means, and the mean sensitivities and specificities at three preselected PCA3 cutoffs fall within 3% of the observed values. In addition, the sensitivity and specificities of several combination PSA-PCA3 strategies in the validation population reasonably reproduce those observed in the EDRN study (Table 1).

When we used the validation population to determine a reasonable annual growth rate for the PCA3 growth model, we found that an annual growth rate of 8% yielded a mean PCA3 among biopsy-positive cases in the validation population of 68.1, similar to the trial mean among cancer cases. The median of 38.1 was lower than the observed median of 49.6, but with the constant annual growth rate constraint, no value resulted in closer fits to both the observed mean and median. When the 8% growth rate was applied to the population cohort, the mean and median PCA3 values among cancer cases were 54.4 and 32.5 at age 60 and 74.4 and 39.2 at age 70.

The PSA-PCA3 combination strategy that most reduces overdiagnosis relative to the base case PSA-based strategy is PSA(4)+PCA3(40), which cuts unnecessary biopsies by over 70% (Table 2). However, it also reduces the number of lives saved from 173 to 99. The PSA(4,10) strategies, which allow men with PSA above 10.0 ng/mL to test positive regardless of PCA3, all reduce overdiagnoses and unnecessary biopsies less effectively than their PSA(4) counterparts, but save substantially more lives compared with the base case. For example, PSA(4,10)+PCA3(35,0) decreases overdiagnoses by 25% and unnecessary biopsies by 50% while preserving about 85% of lives saved compared with the standard PSA-based strategy, whereas PSA(4)+PCA3(35) reduces overdiagnoses by about 40% and unnecessary biopsies by 70% but only retains 60% of lives saved. Findings are very similar using the PCA3 growth model, with slightly greater overdiagnosis and slightly fewer lives saved for each strategy (data not shown).

The results are consistent with the principle that higher sensitivity generally translates to more lives saved and more overdiagnosed cases. Screening with PSA(4,10)+PCA3(35,0) detects 1,032 men with prostate cancer, whereas screening with PSA(4)+PCA3(35) only detects 747, for example. The additional cases detected are a mixture of cases who can be saved by screening and overdiagnoses, so both lives saved and overdiagnoses increase with higher sensitivity. Conversely, because the combination PSA-PCA3 strategies were selected to improve specificity and reduce overdiagnosis, they also reduce lives saved compared with the base case PSA-based strategy.

If PCA3 levels are not correlated with any measure of disease aggressiveness, increasing overall sensitivity amounts to increasing sensitivity similarly among both cases that would be detected in the absence of screening and overdiagnosed cases. Allowing PCA3 levels to be higher in M-HG than in LG cases (Table 3, select screening strategies), we find that as the correlation between PCA3 levels and grade strengthens, the sensitivity to detect M-HG cases increases, whereas the sensitivity to detect LG cases declines. As expected, this translates into a noticeable decline in the frequency of overdiagnosis, with little change in lives saved.

The recent recommendation against PSA screening by the US Preventive Services Task Force (30) underscores the potential for screening to induce harm as well as benefit and the need to identify screening biomarkers with favorable harm–benefit profiles as early in the development phase as possible. Early biomarker studies yield information about sensitivity and specificity of new biomarkers but not about the harm–benefit tradeoff. In this article, we use modeling to extend an EDRN validation study that found that adding PCA3 to PSA screening would reduce unnecessary prostate cancer biopsies (15). The EDRN study provided information on the operating characteristics of PSA and PCA3; by linking these markers with an existing model of PSA and prostate cancer progression, we project the outcomes of adding PCA3 to PSA screening.

We found that combination PSA-PCA3 strategies were able to substantially reduce overdiagnoses and false-positive tests compared with a base case PSA-based strategy. However, approaches that most sharply reduced harms also reduced benefit. We identified several strategies that substantially reduce harms while preserving the majority of lives saved relative to the standard PSA-only strategy. Given the current emphasis on reducing the harms of prostate cancer screening, these may be valuable candidates for the future. Our results also demonstrate that correlation with disease grade is a particularly desirable feature in screening biomarkers.

The EDRN validation study included a mixture of men presenting for initial biopsy and men returning for a repeat biopsy. Risk of cancer detection and PSA sensitivity tend to be lower in the repeat biopsy setting. We did not use this information in the model, but instead only used the PCA3 data for cancer cases and noncases. Policies could be tailored to the biopsy setting. A recent analysis using data from the same EDRN cohort investigated using PCA3 differently in these two settings, namely to rule in cancer (at PCA3>60) among men who had never previously been biopsied and to rule out cancer (at PCA3<20) among men returning for a repeat biopsy (15). Rather than considering initial and repeat biopsies separately, our goal was to use modeling to identify a single combination rule that improved specificity and reduced overdiagnosis in both settings while preserving survival benefits. The modeling framework developed here could also be used to study the downstream consequences of rules such as the ones considered in the recent EDRN study.

Our analysis is subject to several limitations. Our results hinge upon the relationship between the natural history of prostate cancer and the joint trajectory of PSA and PCA3 over time. Following current knowledge on PCA3, we explored only two simple PCA3 trajectories that had no explicit correlation with PSA. Future studies may reveal alternative longitudinal patterns for PCA3. When we correlated PCA3 with grade, we were able to distinguish the impact of different PCA3 levels for only two grade groups, low grade (Gleason 6) and moderate to high grade (Gleason 7–10). In addition, our method of incorporating PCA3 into prostate cancer screening represents only one possible approach. We used PCA3 in a logic rule with PSA and chose commonly cited thresholds for positive test status, but the literature on optimal cutoffs for PCA3 is not conclusive (17). Moreover, the EDRN study and others have shown that adding PCA3 as a predictor to regression models of prostate cancer risk improves diagnostic accuracy in terms of the area under the ROC curve (9, 11, 15, 28, 31, 32), indicating that there is a continuum of risk across levels of PCA3. We did not consider nonbiomarker risk factors in modeling screening; doing so could alter the diagnostic properties and outcomes of tests based on PSA and PCA3. Finally, our model for the survival benefit of detection does not allow for early detection to extend survival rather than cure individuals of cancer. Because we chose to evaluate screening based on number of lives saved rather than survival time, this choice did not substantively impact our results, but it is an important consideration in models of cancer screening. Despite uncertainty around some of the assumptions, our insights about the relative performance of screening strategies that incorporate PCA3 are likely to be useful for evaluating its clinical importance.

Although our particular model reflects specific choices relevant to PCA3, the general modeling process used in this article can serve as a prototype for modeling the impact of early detection biomarkers. We show that two submodels are necessary: a model of how the new biomarker evolves with the natural history of disease and a model for the survival implications of early detection. The first submodel requires data on the distribution of the new biomarker in cancer cases and noncases, an assumption of its longitudinal course, and its correlation with existing biomarkers and disease characteristics. The second submodel determines how early detection with a biomarker affects survival. We modeled the survival benefit using an approach that links benefit (in terms of probability of cure) to the timing of detection. If the same probability of cure was applied to all early diagnoses regardless of the timing of detection, the new biomarker would only detect additional cases.

Our work highlights the principle that simply improving sensitivity or specificity will not necessarily translate into improved population outcomes. In diseases like prostate cancer, increased sensitivity can have advantages and disadvantages, yielding more overdiagnoses, while at the same time reducing disease-specific deaths. Given that the process of advancing biomarkers from discovery to clinical approval can take many years, this type of modeling can support the process of developing and prioritizing early detection biomarkers in cancer.

No potential conflicts of interest were disclosed.

The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the NCI, the NIH, or the Centers for Disease Control.

Conception and design: J.K. Birnbaum, Z. Feng, R. Etzioni

Development of methodology: J.K. Birnbaum, Z. Feng, R. Gulati, R. Etzioni

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): Z. Feng, J. Fan, Y. Lotan, J.T. Wei, R. Etzioni

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): J.K. Birnbaum, Z. Feng, R. Gulati, Y. Lotan, J.T. Wei, R. Etzioni

Writing, review, and/or revision of the manuscript: J.K. Birnbaum, Z. Feng, R. Gulati, J. Fan, Y. Lotan, J.T. Wei, R. Etzioni

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): R. Gulati

Study supervision: R. Etzioni

The authors thank Jeffrey Katcher for his programming assistance and Kristi Schurman and Jaime Hicks for administrative support.

This research was supported by U01CA157224 from the NCI and the Centers for Disease Control (to R. Etzioni).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.