Molecular Profiles of Finasteride Effects on Prostate Carcinogenesis

The Prostate Cancer Prevention Trial (PCPT; ref. 1) and Reduction by Dutasteride of Prostate Cancer Events (REDUCE) trial (2–5) have shown that 5-α reductase inhibitors are the only agent class proven to be effective in preventing prostate cancer. These positive randomized controlled trials have important implications for understanding the biological potential of the tumor in cases of early prostate cancer as well as for the convergent management (treating and preventing microcancer) of men at a high risk of developing the disease.

Morphologically identical prostate tumors can progress at dramatically different rates, and patients in the same cancer risk category can respond very differently to identical treatment. These differences are due in part to the biological heterogeneity of prostate cancer. Some low-risk tumors remain indolent for more than a decade, whereas others rapidly turn deadly. Accurate characterization of the biological potential of the tumor is needed for personalizing prostate cancer treatment. Although the term “risk” commonly connotes the risk of primary prostate cancer, the term is commonly applied in cases of early prostate cancer because of the unusual characteristic of a relatively high frequency of indolent prostate cancer that does not pose a threat of clinical consequences in a patient's lifetime.

Categorizing prostate cancer into risk categories (low, intermediate, and high) depends on the available indicators. Prostate cancer patients are stratified into different risk groups and prescribed different treatments based on prostate-specific antigen level, Gleason score (GS), and clinical stage. Common management options for low- or intermediate-risk patients include watchful waiting, prostatectomy, and radiation therapy (6). No current constellation of clinicopathologic characteristics, however, reliably differentiates between morphologically similar prostate tumors with less- and more-aggressive biological potential.

Molecular signature studies have identified a long list of candidate genes that have potential utility as prognostic factors for prostate cancer, but the identified gene profiles have been inconsistent. These variances are due not only to technologic differences but also to a frequent focus on static states of prostate cancer, which may prevent an accurate determination of the biological potential of the tumor.

An unexpected result of the PCPT (1) provides a clue to how the understanding of prostate cancer heterogeneity can be improved. Finasteride reduced the 7-year period prevalence of prostate cancer by 25% (1) but paradoxically also caused an apparent increase in high-grade cases (GS, 7-10; 6.4% in the finasteride versus 5.1% in the placebo group; ref. 1). Follow-up analyses have refuted this apparent high-grade result (7, 8). Using missing-at-random assumption models to estimate the true rates of high-grade cancer (i.e., the rates that would have been detected had all men with a cancer diagnosis undergone radical prostatectomy), Pinsky et al. (7) calculated a 0.84 relative risk of GS 7 to 10 cancer associated with finasteride versus placebo [95% confidence interval (95% CI), 0.68-1.05]. Redman et al. (8) reported a similarly reduced relative risk of 0.73 (95% CI, 0.56-0.96) for GS 7 to 10 cancer for finasteride. These encouraging results were driven by a marked reduction in GS 7 cancers, and small number of GS >7 tumors made it impossible to firmly interpret the statistically nonsignificant finasteride relative risks for GS 8 to 10 tumors of 1.39 (95% CI, 0.78-2.5) in the Pinsky report (7) and 1.14 (95% CI, 0.96-1.35) in the Redman report (8).

Whether tumors that develop during finasteride treatment differ biologically from those developing in its absence is unknown. Follow-up survival end point studies are unlikely to settle this issue because of the small number of events, the need for decade-long follow-up (given the lead time introduced by intensive screening in the PCPT), and the widely varying treatment of men diagnosed with prostate cancer. We do know that androgen and androgen receptor (AR) signaling, which is integral to prostate cancer progression, are dynamically modulated by finasteride, resulting in decreased tumor heterogeneity. This androgen-mediated and AR signaling–mediated effect on tumor heterogeneity prompts the hypothesis that understanding the molecular response to a finasteride challenge holds the answers needed for risk stratification in early prostate cancer.

Prostate carcinogenesis and prostate cancer progression represent a continuum of genetic and phenotypic alterations (Fig. 1). It is postulated that these alterations occur in a stepwise fashion, thus providing the rationale for the convergent management of men at a high risk of developing the disease. The known molecular mechanisms associated with androgen ablation and 5-α reductase inhibitors will allow us to better understand how finasteride may segregate tumors of different biological potentials. Additionally, the molecular insight gained from the ongoing short-term intervention trial (see Conclusions section) may, by identifying predictive markers of response/resistance to 5-α reductase inhibitors, fulfill the promise of the PCPT and REDUCE trial in prostate cancer prevention and help devise an individualized care for patients with early prostate cancer.

Androgen ablation significantly suppresses AR signaling in the prostate via reducing dihydrotestosterone (DHT) and effectively treats androgen-dependent prostate cancer. Evidence from in vitro, animal, and clinical studies support the ability of androgen ablation to decrease cell proliferation and increase apoptosis (9–11).

Another way to reduce the amount of intraprostatic DHT available to ARs is via 5-α reductase inhibition with finasteride. Finasteride is a 4-azasteroid testosterone analogue and inhibitor of 5-α reductase type 2. Therefore, it blocks the conversion of testosterone to DHT, modulating androgens and androgen signaling pathways (12). Finasteride reduces the intraprostatic DHT level and simultaneously increases the testosterone level. Compared with DHT, however, testosterone is less potent in activating ARs and maintaining the normal activity of epithelial cells. Finasteride has significant beneficial effects on benign prostatic hyperplasia (13). Animal and clinical studies show that the most striking morphologic change following finasteride treatment is involution of the prostate (14, 15), and cellular and molecular evidence suggests that finasteride is associated with atrophy and apoptosis in the prostate (16–18), effects similar to but less dramatic than those of androgen ablation.

Another 4-azo testosterone analogue, dutasteride, can inhibit the activity of both type 1 and type 2 5-α reductases. As with finasteride, dutasteride significantly reduces DHT and increases testosterone (versus placebo; ref. 19). Clinical trials also indicated that the combined level of DHT and testosterone was 58% lower in dutasteride than placebo patients (19). Dutasteride similarly causes shrinkage of the prostate gland (20) and atrophy and apoptosis of benign prostatic epithelium and adenocarcinoma in patients with clinically organ-confined prostate cancer (19, 20). Two large multicenter clinical trials of dutasteride will advance our understanding of the effects of 5-α reductase inhibitors in preventing or inhibiting progression of clinically localized prostate cancer. First, just-reported results of the REDUCE trial indicated that dutasteride produced a 23% overall reduction in prostate cancer and no statistically significant increase in high-grade disease (versus placebo; refs. 2–4). The REDUCE trial involved a smaller sample size of men at a higher risk [a single, negative prostate biopsy and high prostate-specific antigen levels (2.5-10 ng/mL, men ages 50-60 years; 3.0-10 ng/mL, men ages >60 years)] than that of the PCPT, addressing clinically meaningful effects of 5-α reductase inhibitors in a convergent setting of treating (highly likely among these high-risk men) and preventing microcancer (2–5). The ongoing Reduction by Dutasteride of Clinical Progression Events in Expectant Management (REDEEM) trial will determine whether dutasteride can decrease the progression rate of low-risk prostate cancer (21), highlighting risk stratification both for developing or having been diagnosed with early-stage disease.

SRD5A3, another isoenzyme of 5-α reductase, recently was found to be overexpressed in hormone-refractory prostate cancer cells and tissues (22). SRD5A3 converts testosterone to DHT in vitro, and knocking down SRD5A3 expression in a hormone-refractory prostate cancer cell line by small interfering RNA strongly affected DHT production and cell viability (22). These data suggest that SRD5A3 may be important for prostate cancer progression (22). Much remains unknown about the contribution of the SRD5A3 gene to clinical prostate cancer development and progression and its influence on response to 5-α reductase inhibitor prevention.

Recent laboratory investigation shows that androgen and growth factors, such as platelet-derived growth factor, which are implicated in prostate cancer progression, regulate transcription of SRD5A1, SRD5A2, and SRD5A3 differentially in a cell type–specific manner (23). It is speculated that the effect of platelet-derived growth factor on the transcriptional regulation of the 5-α reductase isoenzymes may be mediated by the phosphoinositide 3-kinase (PI3K)/AKT and/or mitogen-activated protein kinase signaling pathways (discussed below in Altered Survival Pathways). Both pathways interact directly with AR in a ligand-independent manner (24, 25). AR signaling is regulated by both positive and negative feedback, implying changing transcriptional regulation of SRD5A isoenzymes as prostate cancer develops or progresses (23). Elucidation of the transcriptional control and signaling network of SRD5A1, SRD5A2, and SRD5A3 induction may provide a valuable marker profile for sensitivity and resistance to 5-α reductase inhibitors.

After long-term androgen ablation, prostate cancer becomes androgen independent and stops responding to androgen ablation. Molecular alterations underlying this independence may include modulation of the AR signaling pathway and altered survival pathways.

The AR signaling pathway is the major pathway that regulates cell growth and proliferation in the prostate (26, 27). Some cancer cells have found ways to adapt to modulation of androgen, and major reported potential adaptive alterations include an elevated concentration of AR protein due to either AR gene amplification or modification at the mRNA level, enhanced AR transactivation activity due to AR mutation, alternative binding of growth factors and cytokines to AR, and an altered ratio of AR coactivators to corepressors (26).

Although the mode of androgen modulation may be different, AR signaling changes in the prostate during finasteride treatment may be similar to those in castration-resistant cancer. Koivisto and colleagues (28) reported a high frequency of AR gene amplification and mutation in a small-scale clinical study involving patients diagnosed with prostate cancer during finasteride treatment for benign prostatic hyperplasia. The Arg726Leu mutation, which is located at the steroid-binding domain of AR protein, leads to AR transactivation by estradiol (28, 29); of note, finasteride also increases serum concentrations of estradiol (30). Therefore, cancer cells with either AR amplification or mutation can gain the advantage of growing in an androgen-modulated environment.

Another potential mechanism underlying prostate cancer treatment resistance and relapse is the presence of prostate cancer stem cells, originally found to be AR negative (31). These cells have significantly higher proliferation and self-renewal capability than do other cells in the tumor population. AR-negative stem cells may give rise to a continual supply of tumor cells even in the presence of treatment including castration (27). Sharifi et al. (32) reported the presence of AR-positive cells, however, among stem cells isolated from LNCaP cells, postulating that AR amplification may even happen among stem cells and lead to resistance to castration. Therefore, whether these stem cells can express AR or not remains a controversial topic.

Androgen ablation not only modulates AR signaling pathways but also alters AR-independent signaling pathways as well in some prostate tumors, indicating an imbalance between proapoptotic and antiapoptotic signals (33). An example of AR-independent signaling effects is PTEN inactivation (phosphatase and tensin homologue deleted on chromosome 10) and deregulation of Bcl-2. PTEN is a lipid and protein phosphatase that dephosphorylates phosphatidylinositol 3,4,5-trisphosphate to antagonize PI3K signaling. PTEN regulates PI3K/AKT and other pathways in promoting cell cycle arrest and apoptosis (34). Loss of one or two copies of PTEN occurs in ∼70% of prostate cancer patients (35). On one hand, loss of PTEN activity may directly affect AR signaling, inasmuch as PTEN inhibits AR transcriptional activity in prostate cancer cell lines (36). On the other hand, attenuated PTEN activity can inhibit apoptotic signaling and facilitate antiapoptotic signaling because overexpression of constitutively activated AKT can phosphorylate and thus inactivate caspase-9 and Bad and allow Bad to release Bcl-2 in promoting cell survival (37, 38). Bcl-2 overexpression has been strongly associated with the development of castration-resistant prostate cancer both in vitro and in vivo (39, 40). Another contributor to survival pathway up-regulation is deregulated AKT activation, which can reduce p27 expression leading to loss of cell cycle control (41). AKT seems to be at least one of the central regulators promoting cell survival and antagonizing apoptosis as prostate cancer cells adapt to the androgen-modulated environment generated by androgen ablation.

Besides Bcl-2, Bcl-xL and other antiapoptotic genes are up-regulated in androgen-independent prostate cancer cells but not in their androgen-dependent counterparts (42). Knockdown of Bcl-2 and Bcl-xL expression has synergistically delayed progression to androgen independence (42).

Other contributors to castration resistance could be activations of epidermal growth factor, insulin-like growth factor-I, and keratinocyte growth factor pathways. In such activations, the PI3K/AKT pathway again plays a key role (25). Downstream of the epidermal growth factor receptor, the mitogen-activated protein kinase pathway is also involved in androgen-independent growth of prostate cancer; this pathway can phosphorylate numerous transcription factors, including ARs, and stimulate the expression of downstream target genes, leading to the survival of prostate and other cancer cells (24). Through such antiapoptotic pathways, prostate cancer cells tilt the balance toward cell survival in the low-androgen environment.

In summary of the altered survival data, the PI3K/AKT, mitogen-activated protein kinase, Bcl-2, and other signaling pathways contribute to the selective or adaptive growth of cancer cells in the androgen-modulated environment created by androgen ablation. Whether finasteride treatment affects these signaling pathways in a similar way remains to be tested. In vitro studies showed that both androgen withdrawal and finasteride treatment can promote extracellular signal-regulated kinase phosphorylation without altering total extracellular signal-regulated kinase level (43). Finasteride also promotes the growth of human prostate cancer in a xenograft mouse model, whereas androgen supplementation suppressed tumor growth (44). More in vivo murine and other studies are needed to examine the influence of finasteride on these survival pathways.

As shown by Xu et al. (45), finasteride does not inhibit Dunning tumor growth but dutasteride does. This difference was due to SRD5A type 1 inhibition by dutasteride. Combining castration with dutasteride enhanced therapeutic efficacy. Although these studies show the importance of the presence of the SRD5A1 isoenzyme and the antitumor activity of dutasteride, this antitumor effect may also be due to other molecular changes induced by dutasteride treatment.

The epithelium-stroma interaction is an important factor in tumor progression. In normal prostate, cross-talk between epithelial cells and stromal cells plays an essential role in maintaining tissue homeostasis (46). In prostate, tumor cells can induce molecular responses in surrounding stromal cells (46). Reciprocally, activated stromal cells can provide adaptation cues to epithelial cells (46). The epithelial-mesenchymal transition has been proposed as an important means by which epithelial cells respond to signals sent from adjacent stroma (47). Induction of epithelial-mesenchymal transition leads to the reduced expression of E-cadherin and other epithelial markers. Concomitantly, mesenchymal proteins such as N-cadherin and fibronectin are up-regulated (47). As Scheel and colleagues (47) discuss extensively in a 2007 review, not all tumor cells are able to respond to the inductive signals from surrounding stromal cells. Inherent genetic factors and accumulated mutations help determine which tumor cells will respond to the induction signals. Epithelial-mesenchymal transition has been proposed as one mechanism contributing to the formation of castration-resistant prostate cancers after androgen ablation (48). Further investigation of whether the DHT-poor environment created by finasteride affects the cross-talk between epithelial and stromal cells and whether the disruption of this cross-talk could contribute to the development of finasteride resistance would be useful.

Finasteride induction of molecular changes in animal prostates suggests the possibility of using finasteride to stratify cancer patients into responders and nonresponders. Molecular changes associated with 10- and 28-day finasteride treatments have been found in RNA and protein profiling analyses of rat prostates (49, 50). Profound changes at the protein level could be detected even with a finasteride dose as low as 1 mg/kg/d (50).

In men with benign prostatic hyperplasia, Rittmaster et al. (16) showed that epithelial terminal deoxynucleotidyl transferase–mediated dUTP nick end labeling assay counts and tissue transglutaminase immunostaining indicated apoptosis in tissues of men taking 5 mg finasteride daily for 6 to 18 days. In a short-term intervention trial in prostate cancer patients challenged with finasteride for at least 30 days before prostatectomy, decreased expression of caspase-7 and insulin-like growth factor binding protein 3 by immunohistochemical staining was observed (51). These molecular effects need to be validated because of study limitations—small sample size and analysis of only a few apoptotic and antiapoptotic factors. Based on the evidence from these and animal studies, we suspect that other molecular changes could be detected following short-term finasteride treatment.

Growing evidence indicates that molecular markers can be successfully used as prognostic factors for classifying early cancer risks and assessing treatment effects. For example, estrogen and progesterone receptor status and HER2 and Ki67 expression have been used clinically for stratifying primary breast cancer according to prognosis (52). Estrogen receptor status is also valuable for predicting the tamoxifen treatment effect, for which other predictive markers have also been investigated (53). These studies will contribute to further selecting subpopulations of high-risk women for whom tamoxifen prevention is appropriate. Like treatment or prevention in any setting, finasteride may not be appropriate for every man at either normal risk (as in the PCPT) or high risk (as in the REDUCE trial) of prostate cancer because of the heterogeneity of prostate cancer and its unclear natural history. The high-throughput technology involved in determining the biological potential of early prostate cancers in a short-term clinical trial of finasteride is discussed later in Conclusions.

Not surprisingly, finasteride prevented prostate cancer in some but not all PCPT men. Molecular studies will advance our understanding of this sensitivity and resistance to finasteride chemoprevention. Identifying responders after a short-term intervention could substantially reduce resources required to conduct large clinical trials. Moreover, if nonresponders could be identified early, then they could be directed to other chemopreventive agents. This strategy applied to early prostate cancer cases could help stratify by prognosis based on the biological potential of tumors, again permitting a more rational approach to therapy.

Whether dutasteride will be effective in men with established low-volume disease and a low GS remains to be seen in future results of the REDEEM trial. A preoperative 4-month dutasteride trial discussed later in Conclusions suggests that important molecular changes may not be detectable at the end of longer-term treatment, as in the REDEEM protocol. Regardless of long-term molecular indicators in the REDEEM trial, however, it will add to the results of the REDUCE trial in defining the best clinical setting(s) for the use of 5-α reductase inhibitors in preventing and treating prostate cancer.

Initially limited largely by concerns over high-grade disease, the effect of the PCPT has continued to be affected by a lack of predictive markers of response for determining who will benefit from finasteride. Results from the PCPT prompted the American Society of Clinical Oncology and the American Urological Association to recommend jointly that asymptomatic men with a prostate-specific antigen of 3.0 ng/mL or lower consider using a 5-α reductase inhibitor, such as finasteride or dutasteride, to help prevent prostate cancer (54, 55). This recommendation needs to be carefully balanced with potential side effects of the drug, such as impotence, although risk is low, especially if the drug is to be taken long term (54, 55). In addition, finasteride did not apparently prevent high-grade tumors nor apparently did the dual 5-α reductase inhibitor dutasteride, which reportedly did not cause a statistically significant increase of high-grade disease in the REDUCE trial (4). Although early prostate carcinogenesis steps are exquisitely sensitive to androgen, modulating the AR signaling pathway alone may not be enough to prevent prostate cancer in all men, especially considering the complexity of prostate carcinogenesis and disease progression. An early adaptive molecular response to 5-α reductase inhibitors may level out the underlying genetic defects (tissue/tumor heterogeneity), giving rise to a signature of biological potential that predicts cancer progression.

Of particular importance, this research will provide molecular characterization and case stratification that will allow early prostate cancer patients to forgo with confidence a Pyrrhic victory over prostate cancer that secures freedom from cancer and unwarranted therapeutic costs and risks of impotence and incontinence. Whether molecular changes accompanying 5-α reductase treatment may be clinically informative is not a philosophical musing but rather an empirical question that can be tested in short-term and long-term intervention trials.

A preoperative model for testing the mechanism(s) of action of a drug in tissue has been useful in identifying therapeutics with the potential to modulate the continuum of genetic and phenotypic alterations in prostate carcinogenesis and prostate cancer progression (56, 57). We currently are studying the molecular effects of short-term finasteride in patients with clinically organ-confined prostate cancer. Two hundred patients with GS ≤7 prostate cancer, a prostate-specific antigen level <10 ng/mL, and stage T1c or T2 disease will be randomly assigned to take either finasteride or placebo for 4 to 6 weeks before prostatectomy. We will compare multiple high-throughput analyses results between finasteride and placebo patients to determine which molecular changes in prostate tissue can be associated with finasteride and/or the biological potential of the prostate tumors.

Using high-throughput technologies to determine this biological potential is a systems biology approach, which is especially helpful in identifying and analyzing numerous molecular changes. The recently developed reverse-phase protein array technology has made it possible to examine protein changes globally, and global genetic changes are analyzed via cDNA microarray. With such high-throughput technology, we can analyze genome-wide molecular changes at the DNA, RNA, and protein levels following finasteride treatment. Integrated within a systems biology approach, these analyses can give a wide perspective and are expected to advance the understanding of molecular mechanisms involved in the ability of finasteride to suppress prostate cancer initiation or progression. Because androgens and androgen signaling are central to prostate carcinogenesis and progression, androgen modulation by a short term of any 5-α reductase inhibitor may clarify the biological potential of tumors, potentially making these agents useful for categorizing prostate cancer aggressivity. After further validation, predictive biomarkers identified in these analyses could be used clinically to determine which tumors need immediate therapy and which are more appropriate for active surveillance and possibly later intervention with curative intent. Although a molecular signature could be ascribed to inherent differences in tumors, serial intrapatient evaluations would identify somatic molecular transformations attributable to the drug.

A similarly designed, albeit longer, 4-month test of the effects of dutasteride (58), near relative of finasteride, found no significant changes in apoptosis and proliferation associated with dutasteride, although the intervention led to a >90% reduction in serum and intraprostatic DHT levels. The authors did not assess changes in apoptosis or proliferation at time points before 4 months and speculated that early such changes may have been missed because of the relatively long treatment duration of the study (58).

The relatively short-term intervention of preoperative finasteride in organ-confined prostate cancer patients is designed to provide more information about early changes at the molecular level. Identifying early molecular responses to short-term finasteride and validating them in archived tissue of the PCPT and other cohorts with prostate cancer and related mortality data could allow stratifying early prostate cancer patients by risk. Notwithstanding the previous 4-month preoperative dutasteride study, the difficulty in convincing organ-confined prostate cancer patients to delay prostatectomy for as long as 4 months determined our short study duration and eliminated comparing earlier with later molecular results. Analyses of molecular effects of androgen ablation and 5-α reductase inhibitors in clinical tumor specimens not only will allow us to better understand how finasteride may segregate tumors of different biological potential but may also allow identifying predictive markers of response/resistance toward personalizing the use of 5-α reductase inhibitors in prostate cancer prevention.

In conclusion, the short-term preoperative intervention schema provides a valuable opportunity for advancing our understanding of the clinical implications of molecular events in prostate carcinogenesis. Future directions of this work include designing adjuvant chemoprevention, therapy trials in cases where aggressive biological potential of tumor has been identified molecularly. 5-α reductase inhibitors raise questions about their molecular effects on prostate carcinogenesis that are more practical than academic, given the proven efficacy of this agent class in the PCPT and REDUCE trial. The importance of these questions is increased further by recent disappointing results of the Selenium and Vitamin E Cancer Prevention Trial, finding that the two (previously) most promising other agents in this setting, selenium and vitamin E, did not reduce prostate cancer risk (59).

No potential conflicts of interest were disclosed.

We thank Drs. Howard Parnes, Christopher Logothetis, and Roger Rittmaster for their careful reviews and valuable comments on the manuscript.