Prostate Cancer Patients with Late Radiation Toxicity Exhibit Reduced Expression of Genes Involved in DNA Double-Strand Break Repair and Homologous Recombination

Radiotherapy is a widely used anticancer treatment, applied in approximately 50% of all cancer patients (1). Prostate cancer is well controlled by external beam radiotherapy, leading to high survival rates (2). However, the development of severe late side effects induced by radiotherapy remains a heavy burden and is observed in approximately 10% of patients (3), and only a small group of irradiated patients have no or only very few complications. Early identification of patients at high risk for late radiation toxicity may help selection for alternatives, such as surgery or brachytherapy.

Clinical factors like age, radiation dose, or volume can only partly explain the risk of late radiation toxicity (4, 5). In addition, previous abdominal surgery (6–8), diabetes mellitus (9), and cardiovascular disease (7, 10) have been incriminated as possible predictors, but there is a lack of confirmative evidence in independent studies. Therefore, it is suggested that patients without normal tissue complications (toxicity grade 0) have an intrinsically different radiation response than patients with severe complications (toxicity grade ≥ 3), and that there might be a genetic predisposition for the risk of severe late side effects (11). Several studies investigated genetic variations of genes involved in apoptosis (12, 13), DNA repair (14, 15) or fibrosis (16), and their role in the development of late radiation toxicity. Genome-wide association studies in large patient cohorts uncovered other possible genetic risk factors (17, 18). Apart from mutations or polymorphisms, also gene expression profiles have been explored as prognostic markers for late radiation toxicity (19, 20). However, overall results are conflicting (21), and no reproducible or reliable prognostic markers (22–24) associated with late radiation toxicity have been identified yet. In our previous retrospective study, we investigated the in vitro radiation response in lymphocytes of patients with prostate cancer, and we found a reduced induction of DNA repair genes in patients with severe late complications compared with those without any complications (25).

Furthermore, a number of more functional assays have been proposed to predict radiotherapy response in normal tissues, e.g., measuring clonogenic survival, levels of apoptosis (24, 26), or monitoring the induction and repair of individual DNA double-strand breaks (DSB; refs. 27–29). The immunofluorescent detection of γ-H2AX, a well-established biomarker for radiation-induced DNA-DSBs, is a widely used and relatively simple assay (30, 31). DSBs represent the most lethal DNA damage induced by radiation treatment and may lead, if unrepaired, to cell death. The two major pathways involved in the repair of DNA-DSBs are homologous recombination (HR) and nonhomologous end joining (NHEJ). The majority of DSB will be repaired via NHEJ as, in contrast to HR, this error-prone pathway does not need a sister chromatid to repair the breaks and can therefore be active throughout the whole cell cycle (32). In our previous retrospective study, more residual γ-H2AX foci were found in ex vivo–irradiated lymphocytes of radiosensitive patients compared with radioresistant patients, indicating less efficient repair of DNA-DSBs in patients with radiation toxicity (20).

The present study aims to validate the findings of our retrospective study (25), to assess the predictive value of DNA-DSB repair efficiency and the induction levels of HR DSB repair genes for late radiation toxicity in 200 men who were newly diagnosed with prostate cancer and referred for radiotherapy.

Between 2009 and 2013, we accrued 200 patients diagnosed with newly diagnosed locally advanced prostate cancer receiving curative external beam radiotherapy in combination with hormonal therapy at the Academic Medical Center (AMC) of the University of Amsterdam, with a follow-up of ≥2 years. We analyzed data from 198 patients with full information about prostate-specific antigen, age, T-classification, and comorbidities. After written informed consent, 40 mL whole blood was collected of all patients before start of treatment. Peripheral lymphocytes were isolated using Ficoll (Ficoll-Paque PLUS; GE Healthcare) gradient separation and stored in liquid nitrogen. Development of late toxicities was monitored over more than 2 years after treatment using the National Cancer Institute Common Terminology Criteria for Adverse Events (CTCAE) version 4.0. Toxicity grade was determined mainly focusing on late gastrointestinal (GI) and genitourinary (GU) toxicities.

Peripheral lymphocytes were thawed and ex vivo irradiated with 1 Gy γ-rays using a ¹³⁷Cs source with a dose rate of approximately 0.5 Gy/min. Induction and decay of radiation-induced γ-H2AX foci were measured in unstimulated G(0) cells. At 30 minutes and 24 hours after irradiation, lymphocytes were dropped on poly-D-lysine–coated slides and fixed in 4% paraformaldehyde. After 25 minutes, slides were washed with PBS and ready for immunostaining. The γ-H2AX foci immunostaining was performed as previously described (33).

The number of radiation-induced γ-H2AX foci was determined in fluorescent stack images using Image-Pro Plus software. Slices of 20 stacks with a 200-nm interval were obtained using a Leica-DM-RA-HC Upright Microscope. Stacks were deconvolved as 1 photomicrograph, and the number of foci per nucleus was scored (34). Decay ratios were calculated by dividing the number of γ-H2AX foci found at 30 minutes by the number of γ-H2AX foci found at 24 hours after irradiation. For every patient, the kinetics of γ-H2AX foci decay was determined in at least 100 cells.

In our previous retrospective study (25), foci decay ratios were determined in lymphocytes of 24 patients, 11 patients with grade 0 and 13 patients with grade 3 late radiation toxicity (see Supplementary Fig. S1). Receiver-operator characteristics (ROC) and AUC were computed in GraphPad-Prism 6 using the method of Hanley and McNeil and resulted in an AUC = 0.970 ± 0.03, 95% confidence interval (CI), 0.91–1.00, and P < 0.0001. A threshold ratio to separate both groups was determined at the highest sensitivity value with a 100% specificity. In this way, all grade 0 patients are correctly identified as negative for radiation toxicity as treatment regimens should not be adjusted in this group of patients. The threshold ratio was calculated as the mean of the lowest decay ratio in the grade 0 and the highest in the grade 3 group (excluding the decay ratio that was higher than the lowest grade 0 decay ratio). This resulted in a threshold decay ratio of 3.41, which separated both groups with 92.1% sensitivity and 100% specificity.

Similarly to the retrospective study, lymphocytes were cultured stimulated by phytohemagglutinin (concentration of 1 μg/mL). After 2 weeks, half of the cells were irradiated at room temperature with 2 Gy gamma rays from a ¹³⁷Cs source, dose rate of approximately 0.5 Gy/min, and the other half of the cells was left untreated. Isolation of RNA was executed 24 hours after radiation using the RNeasy Mini Kit (Qiagen) according to the manufacturer's protocol. RIN value and RNA quality of both treated as untreated RNA were assessed with a BD Bioanalyzer. Biotin-labeled cRNA probes were generated and RNA was hybridized to HT HG-U133+ PM GeneChip arrays (Affymetrix) according to the manufacturer's protocol. Scanning of the array was conducted by the MicroArray Department of the University of Amsterdam, and images were processed to obtain an intensity value for each oligonucleotide probe. Standardized microarray data quality control was performed using the R/Bioconductor package arrayQualityMetrics. Data were normalized and summarized to the probe set level using the robust multi-array average (35).

Genes that respond differently to irradiation between patient groups were determined using a linear model with a fixed main effect for grade, and nested interactions of grade with patient and radiation (2 Gy or 0 Gy). Significant differences in response between grade 0 and the other grades were determined using the appropriate contrasts and empirical Bayes moderated F- and t-statistics (R/Bioconductor package limma). Resulting P values were corrected for multiple testing using the Benjamini–Hochberg FDR adjustment. See Supplementary Methods for a detailed description of the microarray analysis and additional results. The microarray data have been deposited in NCBI Gene Expression Omnibus in a MIAME compliant format and are accessible under GEO series accession number GSE85570.

To validate the retrospective study, the HR gene set was used to determine differences in radiation response between patients of the four toxicity groups. Probesets for the HR gene set were chosen based on our retrospective study (25). Induction levels were assessed with a ROAST gene set test (limma package; ref. 36). ROAST P values were calculated using the nested interaction design described above for three possible alternative hypotheses using 50,000 rotations and default parameters. The alternative “up” tests whether the genes in a gene set tend to be upregulated, the alternative “down” tests whether the genes in a gene set tend to be downregulated, the alternative “mixed” tests whether the genes in the set tend to be differentially expressed, without regard for direction. The two-sided directional P value is reported.

Clinical data were analyzed in IBM SPSS statistics 22. Numbers of induced γ-H2AX foci and decay ratios were analyzed in GraphPad Prism version 5.0. Ordinal and categorical data were assessed with a cross tab χ² analysis. Continuous variables were first tested for normality using the Shapiro–Wilk test (P > 0.05), followed by either one-way ANOVA (normally distributed; post hoc analysis using Holm–Sidak test) or nonparametric Kruskal–Wallis (post hoc analysis using Dunn's multiple comparisons test). ROC, AUC, and CI were computed using the method of Hanley and McNeil. Significant P values are given: *, P < 0.05; **, P < 0.01; and ***, P < 0.001; ns, not statistically significant.

Out of 198 patients, 28 patients (14%) displayed grade ≥3 toxicity to the bladder and/or rectum at more than one time point within 2 years after radiotherapy. Grade 2 toxicity was recorded in the majority of the patients (109 patients, 55%), 47 (24%) patients developed grade 1 and 14 (7%) patients were recorded as grade 0. As described in the literature, the incidence of both extreme patient groups, either with no side effects at all (grade 0) or with severe late side effects (grade ≥3), is approximately 10% to 15%. See Supplementary Table S6 for a list of recorded grade ≥3 toxicity events.

General information, baseline characteristics, and medical history were recorded of all patients and correlated to the incidence and severity of normal tissue damage (Table 1). Interestingly, a higher dose of radiation did not influence the development of severe late side effects. Only 12 patients received 70 Gy instead of 77 Gy, and they were evenly distributed among patient groups. Furthermore, no significant differences in baseline characteristics or medical history were detected between grades 0, 1, 2, and 3, respectively. In conclusion, the four patient groups were well-matched, and no clinical prognostic events for the development of late radiation toxicity were observed.

The induction and decay of radiation-induced DNA-DSB breaks were monitored in ex vivo–irradiated lymphocytes by detecting γ-H2AX foci 30 minutes and 24 hours after radiation as depicted in Fig. 1A. Similarly to the retrospective study, radiosensitive patients appear to have a less efficient DNA-DSB damage response. Calculation of foci decay ratios revealed significant differences between grade ≥3 and grade 0 patients (P = 0.02). Furthermore, based on their foci decay ratios, patients with severe radiation toxicity (grade ≥3) can be discriminated from those without (grade 0) fairly accurately (AUC = 0.73; 95% CI, 0.55–0.92; Fig. 1C). A threshold foci decay ratio of 3.41 was determined in the retrospective patient cohort, in which only patients with either grade 0 or grade 3 toxicity were included (Materials & Methods; Supplementary Fig. S1; ref. 25). Based on this threshold, 82% (23 of 28) of grade ≥3 patients could be correctly classified and 64% (9 of 14) of grade 0 patients (Fig. 1D). Although linear regression analysis confirmed a negative correlation between foci decay ratios and the severity of normal tissue damage (R² = 0.03; P = 0.01; Fig. 1B), an overlap between the foci decay ratios of all four toxicity grades can be observed. Using the threshold decay ratio of 3.41, 36.2% of patients with grade 1 toxicity and 29.4% of those with grade 2 toxicity were erroneously classified as patients with grade 0 toxicity.

Gene expression levels of our previous established gene set for the DNA-DSB repair pathway HR were examined for all patients. Figure 2A shows the differences in radiation response for the HR genes between the two groups. In the volcano plot, the HR gene set is highlighted; results of individual HR genes are shown in Supplementary Table S4. Concordant with the retrospective study, the HR repair gene set was significantly downregulated after radiation (2 Gy vs. 0 Gy) in patients with grade 3 toxicity compared to patients with grade 0 toxicity [P = 0.008 (ROAST gene-set test); Fig. 2C]. Although HR induction levels distinguished radiosensitive patients from radioresistant patients (AUC = 0.75; 95% CI, 0.60–0.89; Fig. 2B), reliable classification based on gene expression levels was not possible. A linear support vector machine classifier fit on the HR induction levels of the retrospective cohort correctly classified only 13 of the 28 patients developing grade 3 toxicity in the prospective cohort (sensitivity 46%; Supplementary Methods, Section 5).

Radiotherapy aims to precisely target the tumor and spare the normal healthy tissue (37). However, during external beam therapy, the surrounding normal tissue is unavoidably exposed to radiation. The development of normal tissue complications is therefore a limiting factor for radiotherapy. In concordance with our retrospective results (25), our present study prospectively confirms a significant correlation between the severity of late radiation toxicity in prostate cancer patients and the ability of the normal lymphocytes to repair DNA-DSBs. Both the results of gene expression profiling and the γ-H2AX foci decay assay of in vitro–irradiated lymphocytes from patients show that inefficient repair of radiation-induced DSB is highly associated with the development of severe late normal tissue damage.

We measured the efficiency of DNA-DSB repair by the ratio of (a) the number of γ-H2AX foci 24 hours after in vitro irradiation relative to (b) the number of γ-H2AX foci 30 minutes after. Whether the disappearance of foci corresponds to the actual repair of a DSB is still subject to discussion (38). Several studies have shown that residual, persisting γ-H2AX foci are associated with DNA-repair deficiencies and radio-sensitivity (27, 30), which suggests that the γ-H2AX foci assay can be used to study DNA-DSB repair. In this study, we show that foci decay ratios correlate well with late radiation toxicity. Significantly lower foci decay ratios were found in patients with severe late radiation toxicity (grade ≥3) compared with patients who developed no or only mild complications (grade 0; P = 0.02). In addition, less efficient DNA damage response is reflected by a reduced transcriptional responsiveness of HR genes to in vitro irradiation. Gene expression levels of HR set are overall downregulated in radioresistant grade 0 patients compared with radiosensitive patients (grade ≥3; P = 0.008). A higher grade of toxicity seems to correlate with a lower activity of HR genes. However, we were not successful to establish a reliable classification model based on induction of HR expression levels.

None of the patient characteristics that have been described as prognostic marker for late radiation toxicity were predictive in the present prospective study. We could not confirm the observations by Peeters and colleagues (39) that the risk of radiation-related late rectal bleeding increased with a higher radiation dose. In addition, no association was found between previous abdominal surgery, diabetes mellitus or cardiovascular disease, and a higher risk for radiation toxicity. This may be due to the fact that systems to score toxicity in patients differ considerably. Valdagni and colleagues (6, 7, 40) used the patient-administered RTOG/EORTC questionnaire to score rectal and intestinal toxicities, whereas we used the clinician-based CTCAE system. Similar classification of patient status is very important for meaningful comparison of different studies and results. Most clinical trials monitor the grade of toxicity in patients by the CTCAE system; however, no standardized implementation of the system exists and interpretation of symptoms varies among clinicians. It has been reported that patients can effectively report toxicity using patient-reported outcomes (41, 42), but the results correlate more poorly with physical performance and somatic changes than clinician-based scoring systems (43).

Over the years, significant progress has been made in radiotherapy and tumor control. The increase of cure rates in cancer patients also increases the importance of minimizing the radiation impact to normal tissues. Prevention and reduction of late normal tissue toxicity can lead to an improved quality of life after treatment. Patients at high risk to develop late radiation toxicity after external beam radiotherapy may alternatively be treated by surgery or brachytherapy. In combination with hyperthermia, patients can be treated with lower radiation dose while maintaining tumor control (44). On the other hand, patients with a low risk of late toxicity may tolerate a higher radiation dose with a higher tumor control probability. Especially for other types of cancer, with a worse survival rate than prostate cancer, patients with a low risk of late toxicity may benefit from dose escalation. The major uncertainty lies in patients with an intermediate risk profile, in whom we found considerable misclassification in either direction.

In conclusion, this study confirmed that there are marked differences between prostate cancer patients in their DNA damage response in normal tissues. A less efficient in vitro DNA-DSB repair correlates with severe late radiation toxicity. Therefore, it is worthwhile to further investigate the DNA damage response or other DSB markers to elucidate the mechanisms that underlie the differences in radiation response. The ability to separate patients suffering from severe late radiation toxicities (grade ≥3) from those without toxicities (grade 0) by γ-H2AX foci decay ratio measurement is already promising. Because the γ-H2AX assay is a robust and highly reproducible technique, it should be possible to develop a standardized method for clinical use. However, due to the large overlap with patients experiencing milder toxicities, results need to be further validated before the assays can be used for decision making in clinical practise.

No potential conflicts of interest were disclosed.

Conception and design: B. van Oorschot, J.P. Medema, L.J.A. Stalpers, P.D. Moerland, N.A.P. Franken

Development of methodology: B. van Oorschot, L.J.A. Stalpers, N.A.P. Franken

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): B. van Oorschot, L. Uitterhoeve, N.A.P. Franken

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): B. van Oorschot, L. Uitterhoeve, I. Oomen, J.P. Medema, P.D. Moerland, N.A.P. Franken

Writing, review, and/or revision of the manuscript: B. van Oorschot, J.P. Medema, H. Vrieling, L.J.A. Stalpers, P.D. Moerland, N.A.P. Franken

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): B. van Oorschot, R. ten Cate, N.A.P. Franken

Study supervision: L.J.A. Stalpers, N.A.P. Franken

Other (writing the project for the Dutch Cancer Foundation #2008-4019): N.A.P. Franken

The authors thank the participating patients and nurses of the AMC Radiotherapy Department. They are grateful to the Maurits and Anna de Kock and the Nijbakker Morra foundations for sponsoring laboratory equipment.

This work was supported by the Dutch Cancer Foundation (nos. UVA 2008-4019) and the Stichting Vanderes.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.