Molecular Pathways and Mechanisms of TGFβ in Cancer Therapy

TGFβ is a cornerstone of homeostasis. It uses many mechanisms to control cellular development, tissue maintenance, and regeneration in a variety of tissues (1). It is classified as a major tumor suppressor based on its ability to regulate the cell cycle and cellular proliferation. Paradoxically, TGFβ can convert from being a tumor suppressor to being a tumor promoter (2). TGFβ’s action as a tumor suppressor is commonly ascribed to the exquisite sensitivity of epithelial cells to TGFβ-mediated G₁ arrest and differentiation (3). To become cancer, initiated cells must overcome this barrier; hence, almost all carcinomas are resistant to TGFβ growth suppression (4). Genetic alterations in mediators of TGFβ signaling occur in about a third of The Cancer Genome Atlas (TCGA) specimens; in particular, mutations of mothers against decapentaplegic homolog 4 (SMAD4) and TGFβ type II receptor (TGFBRII) are frequent in pancreatic, colorectal, and head and neck cancers (5). Nonetheless, many TGFβ transcriptional responses remain intact even after cancer cells have escaped TGFβ’s suppression of proliferation. In a study of more than 500 breast cancers, 92% were positive for nuclear, phosphorylated SMAD-2, indicating that activation of the TGFβ pathway is commonly maintained (6).

TGFβ activity is important in the construction of the tumor-enhancing microenvironment and tumor cells’ immune evasion that together promote the development of clinically evident cancer (7–10). TGFβ can act in a variety of ways to promote tumor progression. Exuberant production and activation of TGFβ by malignant cells suppress the host's antitumor immune response, enhance the production of extracellular matrix, and augment angiogenesis (11).

Loss of response to TGFβ as a growth inhibitor and increased expression of TGFβ activity have been associated with progression in most cancers, including breast, gastric, endometrial, ovarian, colorectal, and cervical cancers, as well as glioblastoma (GBM) and melanoma (12). Ultimately, the mechanisms by which TGFβ signaling and activity are corrupted give cancer-specific properties. Cancer cells that maintain the ability to signal via TGFβ take advantage of programs that engender plasticity such as epithelial-to-mesenchymal transition and stem-like self-renewal, enable construction and remodeling of the tumor microenvironment, and locally suppress immune recognition. Taken together, these mechanisms thwart effective cancer therapy. The redirection of TGFβ biology from tumor suppressor to tumor promoter during carcinogenesis is the topic of recent comprehensive reviews (7, 9, 13, 14). This overview of TGFβ inhibitors in clinical trials focuses on whether TGFβ’s lesser-known role in DNA damage repair provides an exploitable vulnerability for cancer therapy.

To understand TGFβ biology and targeting, we must understand TGFβ's secretion as a latent complex that is targeted to the extracellular matrix. A variety of latency-associated peptides (LAP) release TGFβ from its latent state. LAPs are encoded in each of the three mammalian TGFβ genes. The latent complex consists of a highly glycosylated disulfide-bonded LAP homodimer noncovalently associated with the approximately 24 kD TGFβ homodimer that is structurally characterized as a disulfide knot (15). LAP serves as a chaperone necessary for folding and has the signal sequence for secretion. Most LAP is covalently linked to latent TGFβ binding proteins that serve multiple functions to sequester the complex in the extracellular matrix.

TGFβ is activated when it is released from these complexes. Upon activation, the ligand-binding TGFβ receptor I causes heterodimerization with the type II receptor. Both receptors are threonine kinases that initiate a signaling cascade via phosphorylation of receptor-mediated SMADs. TGFβ receptor III (betaglycan) is not a kinase and is thought to facilitate signaling, particularly from TGFβ2 in certain cells.

The mechanisms of TGFβ activation can be used to target specific cell types or contexts. Activation can be controlled in a cell- and milieu-specific manner by binding proteins that include GARP (16) and LRCC33 on immune cells (17). The LAPs of TGFβ1 and 3 contain RGD sites for integrin-mediated activation by exerting contractile forces to unfold LAP and release active TGFβ (15, 18, 19). In contrast, latent TGFβ1 can be broadly and efficiently activated by extracellular oxidation of a LAP methionine that affects the molecular arrangement of the complex, giving it the ability to sense and signal oxidative stress (20, 21), as seen after exposure to ionizing radiation (22). Compared with normal tissues in which TGFβ activity is tightly controlled, cancers employ all these mechanisms so that active TGFβ is often abundant in the tumor microenvironment.

The mechanics of TGFβ regulation provide multiple means to abrogate its activity (Fig. 1). In brief, agents have been designed to block ligand, activation, or signaling. The effectiveness of each depends on knowing when and where TGFβ is activated and the dominant TGFβ-regulated mechanism that inhibits tumor eradication (see below).

The complexity of TGFβ signaling and its pleiotropic effects have been extensively reviewed (7–9, 13, 23). Briefly, TGFβ activation results in canonical TGFβ signaling initiated by the ligand binding to ubiquitous TGFβ receptors, which are serine and/or threonine kinases that phosphorylate SMAD2 and/or 3 to activate complexing with SMAD4, the mediator of transcription via SMAD-binding elements in target genes. One of the more rapid responses is SMAD7 induction, whose feedback inhibits receptor signaling, among other regulators, limiting the duration of TGFβ signaling. In concert with finely tuned activation, signaling feedback limits TGFβ activity in normal tissues, but dysregulated signaling in tumors can lead to plasticity, motility, and immunosuppression (1).

Cancers may indirectly escape TGFβ growth regulation while maintaining tumor-permissive functions. During carcinogenesis, malignant cells may escape TGFβ’s control of proliferation by reactivating c-Myc (24) or activating Ras (25). Human papillomavirus (HPV) targets TGFβ signaling components to allow squamous epithelial cells to proliferate, which increases infection. HPV protein E5 decreases TGFβ signaling (26), E6 renders cells resistant to TGFβ-mediated growth control by interacting with and degrading the TIP-2/GIPC (27), and E7 interacts with SMAD2, 3, and 4 to significantly impede SMAD4-mediated transcriptional activity (28).

Overexpression of TGFβ in preclinical models confers resistance to a range of chemotherapies that was only evident in vivo and was reversed by administering decorin, a protein that naturally blocks TGFβ (29). Comparison of phosphorylated SMAD2 in paired prechemotherapy and postchemotherapy cervical tumor samples mirrored the effect of TGFβ treatment to stimulate SMAD2/3 phosphorylation, cell migration, and markers related to epithelial–mesenchymal transition and cancer stem cells (30). These effects could all be abrogated by TGFβ inhibitors, confirming that chemotherapy stimulates TGFβ1 expression and activation. Multiple mechanisms, such as angiogenesis, hypoxia, and metabolism, are implicated in this phenomenon. Notably, cancer patients have significantly higher than normal levels of circulating TGFβ that may reflect tumor burden or response to therapy (31, 32).

This dichotomy is evident when TCGA is interrogated with a chronic TGFβ gene signature (33). Because the signature is composed of TGFβ gene targets (in contrast to pathway members), only cancers in which TGFβ activity is high and cells are competent to transduce signal will score high. The robust expression of this signature across cancer types shows that TGFβ activity and signaling competency are indeed high in the majority of cancers (34). However, there is a subset of cancers, represented in each tissue, in which expression of target genes is very low, either because the ligand abundance is low or cells are incompetent for signaling. This is an important distinction because the use of TGFβ inhibitors in the context of loss of signaling versus low ligand abundance should have different consequences, which has implications for patient stratification (see below).

TGFβ plays a major role in DNA damage response (DDR), as first demonstrated in 1996 when Glick and colleagues used a stringent genome amplification assay to show that Tgfb1 null murine keratinocytes were profoundly unstable (35). Consistent with these studies in mouse cells, TGFβ inhibition was shown to impair DDR and increase genomic instability in a nonmalignant human MCF-10A cell line (36, 37). Some type of DNA repair deficit is required to generate genetic diversity during carcinogenesis, but compared with proliferation, the knowledge that TGFβ dysregulation provides an avenue to genomic instability is generally understudied.

However, faulty DNA repair is a hallmark of cancer, and specific repair defects can provide the basis for response to specific therapies (38), hence the recognition of TGFβ’s role in genomic integrity prompted the question of whether TGFβ regulation of DDR is evident in the response to genotoxic cancer therapies. The translational potential of these findings was shown in a variety of mouse and human cancer cell lines in which blocking TGFβ increased sensitivity to radiation in clonogenic assays and tumor control (33, 39–42). TGFβ blockade compromises ataxia telangiectasia-mutated (ATM) kinase activity, which is necessary for DNA repair by homologous recombination (HR) and non-homologous end-joining (NHEJ). TGFβ suppresses ATM kinase by inhibiting miR-182, which degrades FOXO3 (33); FOXO3 promotes ATM autophosphorylation and kinase activity (43). TGFβ regulation of miR-182 positively regulates BRCA1, another key player in HR (33, 44). TGFβ is also implicated in nucleotide excision repair (45) and is coupled to mismatch repair in colorectal cancer (46). Cancer cells in which TGFβ signaling is partially maintained have more effective DNA repair, and hence a mechanism of therapy resistance, whereas cancer cells that are TGFβ incompetent because of mutations or downregulation of a key component have exploitable DDR vulnerabilities.

TGFβ is directly implicated in DDR following exposure to ionizing radiation, which activates TGFβ (47, 48). TGFβ inhibition in preclinical GBM models improves tumor response to standard-of-care chemoradiation (41, 48–50). Huber and colleagues reported that a small-molecule inhibitor of TGFβ receptor kinase improved response of preclinical GBM tumors to combination treatment with radiation and the oral alkylating agent temozolomide (49, 50). Glioma-initiating cells produce more TGFβ, which confers relative resistance by potentiating an effective molecular DDR and increasing cancer stem cell self-renewal. Blocking TGFβ increased glioma-initiating cells’ sensitivity to radiation nearly 3-fold (41, 50).

HPV-positive head and neck squamous cell carcinoma (HNSCC) is remarkably responsive to cisplatin and radiotherapy compared with HPV-negative HNSCC (51). Although this difference has been attributed to RB and p53, HPV-positive HNSCC primary tumors, patient-derived xenografts, and cell lines are unable to phosphorylate SMAD2/3 in response to TGFβ. In line with TGFβ control of BRCA1 levels and ATM kinase activity, HPV-positive HNSCC exhibits decreased HR and NHEJ in response to DNA damage. Blocking TGFβ signaling in HPV-negative cells phenocopies the DDR deficiencies of HPV-positive HNSCC cells, which increases sensitivity to cisplatin, PARP inhibition, and radiation (33, 34).

Cancer cells in which HR or NHEJ is defective use a backup mechanism described as alternative end-joining (alt-EJ; ref. 52). alt-EJ is highly error-prone because it relies on microhomologies at processed ends, which leads to deletions and insertions (52, 53). Cancer cells using alt-EJ are more sensitive to genotoxic chemotherapy or radiotherapy (54, 55). TGFβ inhibition decreases HR and NHEJ and increases repair by alt-EJ by suppressing the expression of POLQ, LIG1, and PARP1, which are required for alt-EJ (34). Hence, TGFβ not only promotes DNA repair but actively inhibits error-prone alt-EJ (Fig. 2). This observation suggests that cancers that maintain this TGFβ-directed biology would be less responsive to DNA-damaging therapies.

This idea was tested using transcriptomic analysis of the chronic TGFβ target signatures described above and a gene signature curated from genes identified in a functional alt-EJ screen (54). In keeping with their functional relationship, TGFβ and alt-EJ signatures are significantly correlated with their respective biological readouts, SMAD2 and/or 3 phosphorylation and unrepaired DNA damage, and both signatures and readouts are anticorrelated (56). These signatures are significantly anticorrelated across almost all solid cancers (34). The highly significant signature anticorrelation among cancer cell line transcriptomes indicates that the relationship is cell intrinsic. Given that cell lines are grown in TGFβ-rich serum, thereby removing abundance as a signal-limiting factor, the anticorrelation of low TGFβ target expression and high alt-EJ genes indicates loss of TGFβ signaling competency.

Consistent with functional alt-EJ, cancers in which low expression of the TGFβ signature is anticorrelated with high expression of alt-EJ genes have more mutations, more genome alterations, and an indel mutational signature pathognomonic of microhomology-mediated repair (34). Use of alt-EJ is predicted to increase sensitivity to genotoxic agents. Consistently, patients with cancers in which transcriptomic evidence of low TGFβ signaling is anticorrelated with high alt-EJ expression, regardless of tumor type, fare better in response to DNA-damaging therapy than those in which TGFβ signaling is high. As evident when TGFβ signaling is truncated in HPV-positive HNSCC, patients in which TGFβ signaling is defective experience significantly better overall survival (OS) in response to chemotherapy and/or radiotherapy compared with those who are TGFβ signaling competent.

In addition to the SMAD4 and TGFBRII mutations, TGFβ signaling may be abrogated by other means. For example, MED12, a component of the mediator transcription regulation complex, negatively regulates TGFβ receptor II through physical interaction, and its loss confers chemoresistance in BRCA-mutant breast cancer (57). Resistance to cisplatin and PARP inhibitors is associated with compromised HR and replication fork stability in MED12-deficient cells (58). Alternatively, chemotherapy-induced TGFβ activity in bone marrow is a mechanism of PARP resistance by facilitating DNA repair activity in leukemia cells (59). Hence, in cancers that maintain signaling, increasing TGFβ activity compels effective DNA repair by positively regulating HR and NHEJ and suppressing alt-EJ, which makes them resistant to genotoxic therapy. But cancers in which this control by TGFβ is lost are susceptible to chemoradiation and vulnerable to drugs that capitalize on defective DDR, which includes PARP inhibitors. Thus, compromised TGFβ signaling creates specific DNA damage deficits that can be exploited in combination with the current repertoire of genotoxic therapy.

Many TGFβ inhibitors have been developed for clinical investigation (9, 14, 60). Multiple means of inhibition have been or are currently in trials, including small-molecule inhibitors of type 1 receptor kinase, neutralizing antibodies, TGFβ traps, and antibodies that block integrin-mediated activation or stabilize LAP to prevent activation (Fig. 1). Use of these agents as monotherapies is limited but combining them with other treatment modalities is of considerable interest. Given the durable response to immune-targeted monotherapy observed in 25%–40% of patients, considerable effort has focused on identifying who will likely respond and why. TGFβ signaling provides multifaceted mechanisms of immune evasion via generation of immunosuppressive stromal fibroblasts (61, 62), myeloid cells (63), regulatory T cells (23), and mediating cell interactions (64, 65). Blocking these immunosuppressive mechanisms is the main goal of TGFβ inhibition (7, 8, 66). A signature of TGFβ-treated fibroblasts is also associated with resistance to immunotherapy; it is thought to represent a mechanism restricting T-cell infiltration (61). Consistent with detriment, an unbiased analysis of breast cancer patients treated with a combination of chemotherapy and immunotherapy found elevation of the TGFβ pathway in the tumors of patients who had residual disease compared to those who experience a pathological complete response (67).

Data from phase I clinical trials of the first small-molecule inhibitor of TGFβ signaling (68) and the first neutralizing antibody (69) were reported in 2014. Results from completed trials have been published for fresolimumab, PF-03446962 (anti-ALK1 receptor monoclonal), bintrafusp alfa (a bispecific anti-PD-L1 and TGFβ trap), and galunisertib, a small-molecule receptor kinase inhibitor (60). While these clinical studies are early-phase monotherapy trials with limited numbers of patients in different disease settings and different lines of prior therapy, all were well tolerated and showed some benefit in some indications (Fig. 3).

Fifteen trials have been completed with galunisertib, a small molecule, in various disease settings, including advanced metastatic disease, GBM, and pancreatic cancer (70). Both pancreatic cancer and GBM produce abundant TGFβ that drives a tumor-permissive microenvironment. As discussed above, preclinical GBM treated with radiation and temozolomide showed improved response (41, 48–50). In a randomized phase II clinical trial for which OS was the primary endpoint, patients with unresectable pancreatic cancer treated with galunisertib and gemcitabine had improved OS compared with gemcitabine alone (71). However, patients with recurrent GBM (NCT01582269) treated with galunisertib and lomustine failed to demonstrate improved OS relative to placebo and lomustine (72). A next-generation compound, LY3200882, was well tolerated as monotherapy and in combination with gemcitabine and nab-paclitaxel in treatment-naïve patients with advanced pancreatic cancer. Six of 12 patients achieved a partial response and 3 demonstrated stable disease, for an overall 75% disease control rate with the combination of LY3200882, gemcitabine, and nab-paclitaxel (73). Studies of this drug were discontinued in 2020 by the manufacturer.

Of seven studies testing fresolimumab (GC10008), the humanized form of a murine monoclonal that neutralizes all three TGFβ isoforms, only two have reported results. Trials were terminated before most patients were enrolled when the manufacturer discontinued further development of the antibody for oncology indications. The immunoregulatory effects of fresolimumab in 13 patients with relapsed malignant pleural mesothelioma (NCT01112293) suggested that patients who produced antitumor antibodies benefited, as evidenced by a doubling of the median OS (15 vs. 7.5 months, P < 0.03) compared with those who did not (74). A feasibility study of the combination of focal irradiation and fresolimumab in 23 patients with metastatic breast cancer randomized to receive high- or low-dose fresolimumab reported median OS doubled (16 vs. 7.8 months, P = 0.039) in those treated with a high dose (10 mg/kg) compared with those receiving a low dose (1 mg/kg). The high-dose combination also elicited more circulating CD8 central memory T cells (75).

Bintrafusp alfa, a novel bifunctional agent consisting of a PD-L1 antibody and TGFβ trap, was developed to target the nonredundant immune-related actions of the TGFβ pathway and PD-L1 signaling, supported by evidence that TGFβ may reduce the efficacy of, or even lead to resistance to, anti-PD-L1 therapies (76). Phase I second-line bintrafusp alfa in patients with non–small cell lung cancer (NSCLC) previously treated with platinum-based agents showed promising efficacy and manageable tolerability (77). A phase III study in which bintrafusp alfa was directly compared with anti-PD-1 pembrolizumab showed the therapy was unlikely to further improve progression-free survival in the first-line setting of stage IV NSCLC with high PD-L1 expression (NCT03631706). Another study determined that bintrafusp alfa was associated with an objective response rate of just 10% in first-line treatment for patients with locally advanced or metastatic biliary tract cancer in combination with cisplatin and gemcitabine (NCT03833661; NCT04066491). The confirmed objective response rate in 59 patients with advanced, pretreated, checkpoint inhibitor-naïve, HPV-associated cancers in phase I (NCT02517398) and phase II trials (NCT03427411), was 30.5%; 5 patients had complete responses and 8 had stable disease (78). Trials are underway for several other indications, including thymoma and metastatic breast cancer.

Recognition that TGFβ is a key moderator of both DNA repair and immunosuppression provides a rationale for combinations with genotoxic therapy. Radiotherapy can achieve both control and cure through the use of technically advanced modalities that specifically generate DNA damage in the tumor. A phase I trial combining fresolimumab and radiation in metastatic breast cancer was designed to detect out-of-field (abscopal) radiation effects and immune monitoring indicative of antitumor immunity (72). One to three lesions of highly distributed disease were irradiated with three fractions of 8 Gy in patients receiving fresolimumab (NCT01401062). Although evidence of abscopal responses was rare (1/27 patients), patients receiving 10 mg/kg fresolimumab had a significantly lower risk of death compared with 1 mg/kg (HR, 2.73 with 95% confidence interval: 1.02–7.30; P = 0.039). The median survival time doubled in women treated with a high dose of fresolimumab who also had a favorable systemic immune response. Likewise, results of an investigator-initiated, single-arm, phase II study of galunisertib and radiotherapy in previously untreated, locally advanced rectal adenocarcinoma are encouraging (76). Patient objective response was evaluated 5 to 9 weeks after oral galunisertib before and during fluorouracil-based or oral capecitabine and fractionated radiotherapy (NCT02688712). The regimen was well tolerated and resulted in a 32% complete response rate compared with historical response rates ranging from 8% to 13% for chemoradiotherapy alone. Consistent with an on-target effect, phospho-SMAD2 decreased in tumors after treatment with galunisertib.

These completed trials support the safety of TGFβ inhibition over the course of a few months; moreover, a few responsive patients were safely treated for years. Yet to date, no TGFβ-targeting agents have FDA approval for cancer treatment. Given the plethora of detrimental biological mechanisms by which TGFβ promotes cancer, the conundrum is why these trials have not achieved a clear signal of benefit. The reasons for this are complex. Target access, patient selection, drug efficacy, complex and dynamic biology, and compensatory pathways could all contribute, as has been discussed elsewhere (14, 60).

The growing body of evidence that TGFβ orchestrates a response to DNA damage opens a new perspective on what might be achieved by TGFβ inhibition. Faulty DDR is a hallmark of cancer, in which the specific deficit is often the basis for response to a specific type of therapy (38). Therapeutic control is thus determined by the degree and type of DNA damage inflicted and the cellular capacity to repair that damage. As is evident from the concerted effort to develop specific inhibitors of DNA repair (79), compromised DNA repair is a high-value target. The active enforcement of DNA repair by TGFβ is concordant with its role as a tumor suppressor, but control of the DDR also underlies the riddle of why tumors maintain TGFβ signaling even though it is an extremely potent inhibitor of proliferation. Cancers that maintain signaling are resistant to genotoxic therapy, as is evident in studies across a range of preclinical cancer models that show that TGFβ inhibition increases response to radiotherapy (41, 42, 48–50, 80, 81). Hence, using TGFβ inhibitors in conjunction with chemoradiation (82) could potentially move TGFβ inhibition to the frontline of cancer therapy.

Some clinical trials in immuno-oncology have sought to exploit DNA damage as a means to stimulate an immune response (83). One thesis is that radiation would act as an “in situ vaccination” in which immunogenic antigen release upon cell death would stimulate antitumor immunity (84). Preclinical data suggest that radiation potentiates preexisting immunity (85). However, therapy-induced TGFβ activity and hence potent immunosuppression could thwart potential synergy between radiotherapy and immunotherapy.

TGFβ regulation of DNA repair competency, together with its role in immunosuppression, suggests that compromised DDR upon TGFβ signaling inhibition in combination with genotoxic therapies, particularly radiotherapy, would lead to increased cell killing and thus increased antigen release that could promote an immune response. The association of response to immunotherapy in patients with colon cancer whose cancers exhibit mismatch repair or high microsatellite instability (86) promoted a basket trial based on these phenotypes (87). By analogy, one might anticipate that the association of the low TGFβ and high alt-EJ signature with greater genome alterations (88) might also associate with response to immunotherapy. Indeed, the combination of radiation and TGFβ inhibition synergizes with checkpoint inhibitors (81, 89). Bintrafusp alfa, the bispecific anti-PD-L1 and TGFβ trap, also effectively synergized with radiotherapy in multiple therapy-resistant murine tumor models with poor immune infiltration and protection from radiation lung toxicity (90). Hence, the rationale for dual targeting of TGFβ and immune checkpoint inhibitors, either in combination or with new bifunctional agents, is compelling.

The challenge for effective deployment of any of these agents is to determine the dominant mechanism for which to select appropriate indications among diverse patient populations based on biomarkers to stratify and monitor patients. TGFβ gene expression signatures that reveal its biological effects in the stroma (61) or pathway components (66) or signaling competency (34, 56) are associated with response to cancer therapies, which offers a means to select those patients whose cancers are modulated by TGFβ. Benefit will be realized when the rationale for the regimen, patient population, and biomarker are aligned. The most compelling example of which is the high rate of pathologic complete response of patients with colorectal cancer classified as deficient mismatch repair to immune checkpoint inhibitors (91). Aiming for this level of precision is necessary to realize the unequivocal rationale for TGFβ inhibition in cancer.

M.H. Barcellos-Hoff reports grants and non-financial support from Genentech; personal fees and non-financial support from Innovation Pathways, Inc.; personal fees from EMD Serono and Scholar Rock; grants from Varian, Inc.; other support from Bicara; and grants, personal fees, and non-financial support from Genzyme during the conduct of the study; in addition, M.H. Barcellos-Hoff has a patent for PCT/US2021/037078 pending and spoke at a commercial meeting organized by Hanson-Wade in 2022 and 2023. J.L. Gulley reports other support from EMD Serono outside the submitted work; in addition, J.L. Gulley has a patent 11440963 issued.

The authors thank Jim Gkantalis for Figure 2 design and Bonnie L. Casey for editing.

The authors of this article are supported by NIH awards R01NS109911 and R01CA239235 (MHBH), the UCSF Wen-Kun Fu Endowed Chair in Radiation Oncology (MHBH), and the Intramural Research Program of the Center for Cancer Research, NCI, NIH grant nos. ZIA BC 010945 and ZIA BC 010666 (J.L. Gulley).