The Impact of T2/FLAIR Evaluation per RANO Criteria on Response Assessment of Recurrent Glioblastoma Patients Treated with Bevacizumab

Glioblastoma is the most common primary brain tumor in adults with an incidence of 3.2 per 100,000 in the United States (1). Despite advances in treatment with the combination of surgery, radiotherapy, and chemotherapy, median survival for glioblastoma remains approximately 15 months from diagnosis (2). Recently, preliminary results from interim analysis of the EF-14 trial evaluating Tumor Treating Fields (TTF) in newly diagnosed glioblastoma patients showed an increase of median overall survival by 3 months (3). While there are immense efforts to develop new therapeutic agents, accurate and reliable demonstration of their effectiveness in clinical trials remains a challenge.

Although overall survival (OS) is generally considered the “gold-standard endpoint,” evaluation of therapeutic efficacies in phase II and III trials have also relied upon radiographic endpoints, including objective response rates (ORRs) and progression-free survival (PFS; ref. 4). Several studies have indicated that ORRs and PFS correlate with OS (5–7). In clinical trials, ORRs and PFS are particularly important because they are not confounded by salvage therapies and other variables that may affect OS and because they can be assessed relatively rapidly (8, 9).

To provide useful surrogate endpoints for clinical trials, response assessment criteria should have high reproducibility in determining response or progression, generalizability, ease of implementation, and high correlation with important established endpoints such as OS. Since 1990, the main response criteria used in clinical trials in neuro-oncology has been the Macdonald criteria, which uses two-dimensional measurements of contrast enhancement, in addition to corticosteroid use and clinical status (10). The Macdonald criteria have significant limitations, including failure to assess nonenhancing tumor, inability to account for pseudoprogression and pseudoresponse, and the lack of assessment of nonmeasurable disease. These deficiencies limit their applicability for determination of ORRs and PFS endpoints in clinical trials (11). Widespread adoption of bevacizumab, approved by the FDA in 2009 for recurrent glioblastoma, has highlighted the phenomenon of nonenhancing tumor progression and pseudoresponse (12). In pseudoresponse, there is an improvement of contrast enhancement that is likely due to normalization of vascular permeability, but this does not necessarily reflect a true antitumor effect. Despite stable or continual decrease in enhancement, some patients have exhibited a simultaneous increase in the nonenhancing component of the tumor, as depicted on fluid-attenuated inversion recovery (FLAIR) or T2-weighted images (13).

To address several limitations of the Macdonald criteria, the Response Assessment in Neuro-Oncology (RANO) Working Group proposed newly updated response criteria for high-grade gliomas in 2010 (14). These criteria still use measurement of enhancing lesions as the basis of response, but also incorporate qualitative evaluation of T2/FLAIR abnormality (15). Of note, determination of radiographic progression relative to the gold standard provided by histopathologic confirmation has not been validated for either the Macdonald criteria or the RANO criteria in prospectively conducted clinical trials. Nonetheless, the value of adding T2/FLAIR assessment to the Macdonald criteria remains to be determined, although preliminary evidence of benefit has been suggested previously (16, 17).

In this study, we examined the radiographic data of patients with recurrent glioblastoma from the randomized phase II BRAIN (AVF3708g) trial which contributed to accelerated approval of bevacizumab by the FDA (18). We hypothesize that the assessment of T2/FLAIR abnormality using the RANO criteria results in differences in ORRs and PFS as compared with the Macdonald criteria. Furthermore, we explore whether OR and PFS as determined by the RANO criteria can provide early indication of subsequent OS using landmark analyses.

We retrospectively analyzed radiologic data from the randomized phase II BRAIN (AVF3708g) trial, which was designed to assess the efficacy of bevacizumab with or without irinotecan in patients with recurrent glioblastoma. One hundred and sixty-seven patients who had histologically confirmed glioblastoma at first or second relapse were enrolled in this study. All patients had failed the initial standard treatment, including concurrent radiotherapy and temozolomide, and were at least 8 weeks from the completion of radiotherapy. Bevacizumab was given at a dose of 10 mg/kg every 2 weeks. The dose of irinotecan was 340 mg/m² for patients taking enzyme-inducing antiepileptic drugs or 125 mg/m² if not taking enzyme-inducing antiepileptic drugs every 2 weeks. All patients were treated for 104 weeks or until disease progression or discontinuation. Selected clinical variables from the BRAIN trial are summarized in Table 1 (18). Four patients who experienced disease progression after randomization and did not receive study treatment were not included in the analysis. The baseline and follow-up MRIs performed every 6 weeks from the remaining 163 patients were evaluated until progression or until the trial lock-date for imaging. In this trial, new T2/FLAIR (nonenhancing) lesion was considered as progressive disease, while enlarging T2/FLAIR lesion alone would not qualify for progression. T2/FLAIR assessment was also not included in determining response. Data were acquired in compliance with Health Insurance Portability and Accountability Act regulations and with approval of the local institutional review boards. All patients provided written informed consent before study participation.

Imaging data for all patients was reviewed by a neuroradiologist (R.Y. Huang), designated as the primary reader, who was blinded to patient-specific clinical information. To determine interobserver variability, all studies from 163 patients were independently reviewed by secondary readers, including one neuroradiologist (W.B. Pope) and four neuro-oncologists (P.Y. Wen, D.A. Reardon, L. Nayak, and E.Q. Lee). The two radiologists in our study team (R.Y. Huang and W.B. Pope) were certified by the American Board of Radiology (ABR) in Diagnostic Radiology with an ABR Neuroradiology Certificate of Advanced Qualification. The MRI studies of each subject were revealed to readers in order of acquisition dates. T1-weighted images without and with contrast enhancement and T2/FLAIR images were displayed on picture archiving and communication system workstations. For each MRI study following treatment initiation, the sum of diameter product measurements of enhancing area and qualitative assessment of T2/FLAIR abnormality were recorded for every available scan. For the Macdonald criteria, OR was defined as a ≥50% decrease in the sum of diameter products compared with the baseline scan that was confirmed on follow-up MRI at least 4 weeks later. For the RANO criteria, the definition of OR also included absence of significantly increased T2/FLAIR abnormality. The readers determined progression status based on both enhancing and T2/FLAIR abnormalities (RANO criteria) as well as based solely on enhancing abnormality (Macdonald criteria). A 25% or greater increase in the sum of diameter products within contrast enhancing area and/or new lesion(s) was necessary for defining progression by Macdonald criteria. Quantitative change in enhancing area (as defined by the MacDonald), the appearance of a new lesion, or significant, qualitative increase of nonenhancing T2/FLAIR abnormality defined disease progression by the RANO criteria.

For each patient, PFS was calculated from the date of bevacizumab initiation to the date of disease progression as determined by the primary reader, or death, whichever occurred first. Patients who did not progress were censored using the last scan date. OS was calculated from the date of bevacizumab initiation to death. Patients who did not die were censored according to the last contact date per the clinical data provided by the study sponsor.

Interobserver agreement was determined for all pairs of readers (R.Y. Huang + one other reader) for all studies from all patients. Agreement was determined by the number of pairs that agreed (both readers determined the patient had an OR or both determined patient did not have an OR) divided by the number of paired reads. Likewise, agreement between the RANO and Macdonald criteria was determined by the number of times both methods done by the same reader indicated that a patient had an OR or both indicated the patient did not have an OR divided by the number of patient reads. A κ statistic was used to summarize the concordance between the readers and between the RANO and MacDonald criteria. A κ value of 0 indicates lack of concordance and a value of 1 indicates perfect concordance. Correlations between tumor size measurement by different readers and between progression times were summarized with the Spearman statistic.

In landmark analyses, evaluation by the primary reader was used. All patients who had died prior to the specified landmark time were excluded from the analysis. At the specified landmark time, patients were classified as an OR or not an OR if they were classified as a confirmed response at that time or not. Note that for confirmed responses, the time of response was determined to be the first scan at which the tumor exhibited a response. A Cox proportional hazards model was then used where the residual survival time was defined as time from specified landmark time to death or last follow-up. Concordance statistics (C-index) were then used to summarize the predictive effects of OR or PFS on OS.

Agreement of OR determination between the primary reader and the other reader for the Macdonald criteria was 75·6% (κ statistic = 0.515; P < 0.0001) and 75.0% (κ statistic = 0.499; P < 0.0001) under the RANO criteria. The Spearman rank correlations between the primary reader and other readers were 0.634 and 0.566 for the measurements of enhancing and T2/FLAIR abnormalities, respectively.

On the basis of evaluation by the primary reader, the ORRs were 64/163 [39.3%; 95% exact confidence interval (CI), 31.7%–47.2%] and 54/163 (33.1%; 95% exact CI, 26.0%–40.9%) by Macdonald and RANO criteria (Table 2 for specific classifications by Macdonald and RANO), respectively. These classification of patients as an objective response or not were significantly different between the two methods (χ² P < 0.0001). Agreement of ORRs between the two criteria was 93.9% (κ statistic = 0.868, P < 0.0001).

Stratifications of patients at 4 months by OR status as determined by the Macdonald and RANO criteria are listed in Table 3. The numbers of patients who were alive at the three landmarks were 155, 144, and 123, respectively. For both criteria, Cox proportional hazard models confirmed the association of OR and residual survival at each landmark (Landmark times at 2 month and 4 month shown in Fig. 1).

Eighty-seven patients progressed per Macdonald criteria by the last available scan date, and 112 patients progressed per RANO criteria. There were 37 patients who only had a progression by RANO (i.e., had a T2/FLAIR progression). There were 12 patients who first developed a T2/FLAIR progression and then subsequently developed a T1 progression, and 25 patients who developed progressive disease (PD) by RANO criteria but did not have PD by Macdonald criteria before imaging lock-date. Among the 37 patients who had progression first based on T2/FLAIR disease, 16 patients did stop receiving bevacizumab at or before PD by RANO criteria due to new T2/FLAIR disease or drug toxicity. Eight patients continued to receive bevacizumab even after PD defined by Macdonald. The median overall survival of the 25 patients who experienced T2/FLAIR progression and did not develop T1 progression was 10.8 months (range, 2.1–32.0 months). This compares to a median OS of 8.6 months (range, 1.4–24.7 months) for the remaining 87 patients. The mean time between T2/FLAIR progression and T1 contrast-enhancing disease progression is 2.0 months (median 1.4 months; range, 0.9–5.5 months).

The median PFS was 6.4 months using Macdonald criteria, versus 4.6 months by RANO criteria (Fig. 2, P = 0.011). There was relatively good correlation in PFS times between Macdonald and RANO criteria (r = 0.781). The correlation of the PFS times for each patient was determined for paired readers. The Spearman correlations of the PFS times were 0.543 and 0.645 for the Macdonald and RANO criteria, respectively.

Progression status as determined by the Macdonald and RANO criteria was used to stratify patient groups at 2, 4, and 6 months (data from 2-month and 4-month landmark times shown in Table 3). Kaplan–Meier curves were generated by groups, including those who had experienced disease progression by the landmark time and those who had not; Cox proportional hazard models confirmed the association between the progression status at the landmark time and OS (landmark at 4 months shown in Fig. 3). There were 4 patients in this study who went off treatment prior to treatment completion or PFS. Excluding these patients did not affect the conclusion of the above analyses.

In this study, we compared the RANO criteria to the Macdonald criteria using imaging data of 163 patients from the randomized phase II BRAIN (AVF3708g) trial evaluating bevacizumab or bevacizumab and irinotecan in patients with recurrent glioblastoma. Our analyses revealed a moderate and statistically significant difference of 1.8 months (28%) in median PFS between the two criteria (P = 0.011, Fig. 2). A prior, single-institution retrospective study by Perez-Larraya and colleagues assessing 78 patients with recurrent glioblastoma treated by bevacizumab did not observe a significant difference in PFS between the two criteria, although the use of RANO trended towards a shorter PFS (19). The value of earlier detection of nonenhancing radiographic progression needs to be evaluated in the context of residual survival benefit. While prior studies demonstrating that progression determined by the Macdonald criteria correlate with survival (6, 9, 20), our landmark analyses demonstrate that progression determined by RANO criteria at 2, 4, and 6 months following bevacizumab treatment also correlate with OS. Comparing the two criteria in the same analyses, however, the inclusion of T2/FLAIR evaluation in determining progression did not significantly improve or reduce the correlation of progression status with survival. Nonetheless, our assessment of the BRAIN data revealed that use of RANO criteria allows detection of at least 35% of patients who had nonenhancing tumor progression that would not be captured in the next sequential imaging relying on Macdonald criteria. While the benefit of identifying this subgroup of patients is not clear currently due to a lack of effective post-progression treatment options, earlier detection of progression may become clinically relevant with availability of more effective therapy in the future.

In determining progression, T2/FLAIR abnormality is evaluated qualitatively as described by the RANO Working group. Besides nonenhancing tumor, other processes, including radiation effects, edema, ischemic injury, infection, seizures, and postoperative gliosis, can prolong T2 relaxation time (14). Currently, there is no objective method to distinguish among these possibilities using standard MR imaging sequences. Thus, the assessment of T2/FLAIR abnormalities depends on expert readers to interpret whether any T2/FLAIR changes are “relevant” to tumor growth. Specifically, readers should inspect serial T2/FLAIR imaging for regions of increasing nodularity, blurring of gray–white junction, and mass effect while excluding regions of infarct, vasogenic edema, and radiation leukoencephalopathy. While this type of morphologic assessment remains subjective, Nowosielski and colleagues recently demonstrated the association of “circumscribed” morphology of T2/FLAIR changes and survival outcome of patients receiving antiangiogenic therapy (21). Furthermore, the interobserver agreement for progression was similar for the RANO criteria as compared with the Macdonald criteria, indicating that the subjective nature of qualitative T2/FLAIR assessment was likely not a major source of variability. In contrast, quantitative measurements of T2/FLAIR abnormalities using the same method of measuring enhancing area resulted in a lower degree of interobserver agreement (r = 0.566), again underscoring challenges of incorporating objective criteria in assessment of T2/FLAIR abnormality.

Evaluation of OR revealed a statistically significant reduction in the ORR as a result of including T2/FLAIR assessment in the RANO criteria. This is not surprising as the RANO criteria are more restrictive than the Macdonald criteria in assessing response. The ORRs determined by the readers in our study were similar to the previous reported ORRs of 28 to 38% based on assessment of the same imaging data using Macdonald criteria in the BRAIN trial (18). Reported ORRs from other phase II clinical trials of patients with recurrent glioblastoma treated with bevacizumab using the Macdonald criteria have been widely variable, ranging from 35% to 63% (5, 22, 23). These differences were likely due to inter-reader variability in determining OR, as evident by rather modest inter-reader agreement. One possible factor that may contribute to higher inter-reader variability is the difficulty in accurately measuring the area of enhancement that is often faint, particularly following antiangiogenic treatment, and this can be accompanied by a varying degree of T1 shortening obscuring the truly enhancing area. Recent analysis of the same BRAIN trial data using T1 subtraction maps resulted in more consistent measurements of enhancing volume as well as improved prediction of OS and PFS (24). These data suggest that the use of T1 subtraction maps may be a promising method to reduce variability of OR assessment.

Despite the high variability of ORRs, there is an association of OR status and OS as demonstrated in our landmark analysis for both criteria. Recently, in a study of imaging data from a phase II trial (RTOG0625) of bevacizumab with irinotecan or temozolomide in recurrent glioblastoma, Boxerman and colleagues demonstrated that early progression at 8 weeks posttreatment based on both two-dimensional and three-dimensional measurements of enhancement, rather than response, was a significant prognostic marker for OS (16). This result suggests that the association of OR and OS observed in our study could be due to including early progressors in the nonresponder group, rather than directly result from the OR status as determined by Macdonald or RANO criteria.

Our study was limited by availability of imaging data up to the progression scan based on modified Macdonald criteria or to a prespecified lock date. Response assessment in our study was based only on radiologic data because readers were blinded to clinical information; nonetheless, we believe it is unlikely that this significantly affected our comparison of response criteria as results from the BRAIN trial indicated that there were no instances of clinical progression preceding radiologic progression. The partially available steroid dosage data (summarized in the appendix) also suggested no significant impact of this factor on response assessment by either criteria. In addition, variables involved in image acquisition and processing were not standardized for the BRAIN trial. Finally, our study compares outcome assessment in the context of VEGF blockade with bevacizumab therapy and cannot be extrapolated to other types of antitumor therapy.

In conclusion, the inclusion of T2/FLAIR assessment in RANO criteria resulted in earlier detection of progression among a subset of patients and as a result, statistically significant reductions in median PFS and ORRs. The use of RANO criteria allows detection of at least 35% of patients who had nonenhancing tumor progression that would not be captured in the next sequential imaging relying on Macdonald criteria. Despite these differences, PFS and OR determined by either RANO or Macdonald criteria similarly correlated with survival. Earlier detection of nonenhancing tumor progression may become clinically important with availability of effective postprogression therapies in the future.

B.M. Ellingson reports receiving a commercial research grant from Roche/Genentech and is a consultant/advisory board member for MedQIA and Roche/Genentech. D.A. Reardon reports receiving speakers bureau honoraria from Bristol-Meyers Squibb, Cavion, Genentech/Roche, Merck, Novocure, and Regeneron and is a consultant/advisory board member for Bristol-Meyers Squibb, Cavion, Genentech/Roche, Merck, and Regeneron. W.B. Pope is a consultant/advisory board member for Roche. T.F. Cloughesy is a consultant/advisory board member for and has provided expert testimony for Roche regarding first line avastin in Glioblastoma. P.Y. Wen reports receiving other commercial research grants from AbbVie, Agios, Angiochem, AstraZeneca, Cubist, Exelis, Genentech/Roche, GlaxosmithKline, Karyopharm, Merck, Novartis, Sanofi-Aventis, and Vascular Biogenics; speakers bureau honoraria from Merck; and is a consultant/advisory board member for AbbVie, Celldex, Genentech/Roche, Novartis, Novocure, SigmaTau, Midatech, Momenta, and Vascular Biogenics. No potential conflicts of interest were disclosed by the other authors.

Conception and design: R.Y. Huang, R. Rahman, B.M. Ellingson, L. Nayak, D.A. Reardon, W.B. Pope, T.F. Cloughesy, P.Y. Wen

Development of methodology: R.Y. Huang, R. Rahman, W.B. Pope, T.F. Cloughesy, P.Y. Wen

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): R.Y. Huang, R. Rahman, B.M. Ellingson, E.Q. Lee, L.E. Abrey, D.A. Reardon, W.B. Pope, T.F. Cloughesy, P.Y. Wen

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): R.Y. Huang, K.V. Ballman, S. Felten, K. Anderson, B.M. Ellingson, L. Nayak, E.Q. Lee, E. Galanis, D.A. Reardon, P.Y. Wen

Writing, review, and/or revision of the manuscript: R.Y. Huang, R. Rahman, K.V. Ballman, S. Felten, K. Anderson, B.M. Ellingson, L. Nayak, E.Q. Lee, L.E. Abrey, D.A. Reardon, W.B. Pope, T.F. Cloughesy, P.Y. Wen

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): R.Y. Huang, R. Rahman, S. Felten, L.E. Abrey, P.Y. Wen

Study supervision: R.Y. Huang, D.A. Reardon

Dr. R.Y. Huang was supported by ARRS/ASNR scholar award (foundation grant).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.