Real-World Evidence in Support of Oncology Product Registration: A Systematic Review of New Drug Application and Biologics License Application Approvals from 2015–2020

Regulatory real-world evidence (RWE) has garnered great interest since the passing of the 21st Century Cures Act in 2016, which aimed to accelerate medical product development by supporting approval of new indications for current therapies or satisfying post-approval requirements (1–3). RWE is especially important for new therapies that target rare, serious illnesses with high unmet needs, such as orphan drugs where single-arm trials are often conducted due to lack of standard of care. Additionally, RWE can play a role in contextualizing or serving as external controls for comparison with an investigational therapy to support accelerated approval, as confirmatory trials for regular approval, or to support a new indication. Using historical control data to support clinical trials is of interest in oncology research, especially given the possibility of decreasing time to approval and increasing availability of therapies.

Pharmaceutical companies have been working with the FDA to use RWE to support new drug applications (NDAs) and biologics license applications (BLAs). Guidance documents are available that may provide insight into RWE studies (2, 4–8). Specifically, FDA's framework for their RWE program defines RWE as clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of real-world data (RWD). RWD is then defined as data relating to patient health status and/or the delivery of healthcare routinely collected from sources such as electronic health records, claims and billing data, product and disease registries, patient-generated data, and data gathered from other sources, such as mobile devices (5). However, there lacks a central document regarding best practices and key elements that must be considered for effective RWE in FDA submissions. This study sought to understand the role of RWE in FDA's oncology therapy decision-making. In this study, we systematically reviewed and identified key components of RWE that supported the efficacy of oncology therapies in new and supplemental applications, summarized FDA feedback, and synthesized information to suggest best practices for regulatory applications with RWE.

NDA and BLA approvals from 2015 to 2020 were systematically reviewed to identify oncology products from FDA.gov sources: Center for Drug Evaluation and Research (CDER) Calendar Year Approvals (9), CDER Drug Approvals by Month (10), and Center for Biologics Evaluation and Research (CBER) Biological Approvals by Year (11). The corresponding FDA review documents (as of January 15, 2021) were screened for RWE. Texts were extracted into a consolidated database using automated text extraction, and searched for RWE keywords (e.g., “retrospective,” “observational,” “real world,” “chart review,” “claims,” “electronic medical record,” “natural history”). Supplemental (sNDA and sBLA) approvals were similarly identified, with additional sources: CDER Efficacy Supplement Approvals (12) and Hematology/Oncology Approvals and Safety Notifications (13).

Once drug approvals including RWE were identified, the following were extracted from the FDA review documents: pivotal trial-related information (study design, sample size, primary endpoint, and results), application-related information [indication, mechanism of action, investigational new drug (IND) submission date, approval date, time between IND and approval, designation and review types, and whether an advisory committee was held], and RWE-related information [data source, study design, sample size, correspondence to pivotal trial eligibility criteria, primary endpoint and results, statistical analysis, use of RWE for statistical comparison or contextualization (descriptive/qualitative review) of the pivotal trial results, use of RWE in the approval, date of first RWE discussion with the FDA, and corresponding FDA comments].

In total, 133 approvals for oncology NDAs and BLAs were identified (Fig. 1). Among these, 11 (8.3%) included RWE in support of efficacy with an average time from IND submission to approval of 5.7 years (Table 1). Median time from submission to approval was 5.5 years, with a range of 2.8 to 8.4 years. In total, 573 oncology sNDAs and sBLAs were identified, and among these were 249 for new oncology indications, 2 (0.8%) of which included RWE in support of efficacy (Fig. 2). The 11 NDAs/BLAs that included RWE were for avelumab, axicabtagene ciloleucel, entrectinib, erdafitinib, polatuzumab vedotin-piiq, selinexor, avapritinib, capmatinib, tafasitamab, and tazemetostat (NDAs 211723 and 213400), and the 2 sNDAs/sBLAs were for blinatumomab and palbociclib.

All 13 products with supporting RWE for efficacy received approvals by the FDA after the 21st Century Cures Act was enacted. In total, 11 applications received orphan drug designation, 12 received priority review, 9 were designated as breakthrough therapies, 6 received fast track designation, and 3 received all four mentioned designations. Nine received accelerated approval, whereas 4 received full (or regular) approvals. All original full approvals shared the following combinatory characteristics: designation of orphan disease, breakthrough therapy, and priority review; no advisory committee meeting held (i.e., no uncertainty in efficacy or safety for the FDA to call for an advisory committee meeting); and high level of primary efficacy in the pivotal trial [we observed all overall response rate (ORR) >70%]. Note that 3 of the 4 full approvals were not designated under fast track (Table 2; Supplementary Tables S1 and S2). Eight applications used RWE for contextualization, three for statistical comparison with clinical trial results, and two used RWE for both. For nonsupplemental applications, the pivotal trials for 10 therapies were single-drug, single-arm clinical trials with one single-drug, two dosing arm clinical trial.

Key variables for RWE efficacy approvals reviewed in this study are summarized in Table 2. Indications were commonly rare diseases with high unmet need (i.e., no available therapies or currently ineffective treatments). The most common RWD source was chart review from clinical sites. ORR was the most common primary endpoint across real-world studies, matching with the primary efficacy endpoint in the pivotal trial. RWE generated for only three therapies (avelumab, blinatumomab, and tafasitamab) used a real-world population with matched eligibility criteria to the trial. Sample sizes for the RWE studies ranged from 14 to 908. For the 11 new approvals (not supplemental), sample sizes for the RWE studies were substantially larger than the pivotal trial's [>50% in four (36%), about the same in three (27%), and substantially smaller (<50%) in four (36%) cases]. All RWE studies were retrospective.

FDA's comments pertained to sources of bias inherent in the use of historical controls and RWD, such as selection bias and residual or unobserved confounding (especially missing data on key covariates), different outcome assessment methods and frequency of measures as compared with trials, lack of comparability between external controls and trial populations, misclassification of outcomes, and insufficient statistical methods for adjustment of differences between comparator groups. Additional details for each approval are available in Supplementary Tables S1 and S2.

The FDA commonly focused on several areas of RWE in their reviews. The FDA noted that early engagement is crucial to confirm appropriate RWD sources and whether the RWE study should be designed as a natural history study for contextualization or as a historical/external control study for comparison. The FDA recommended a prespecified study protocol for transparency. The FDA emphasized the selection of a RWE population to be comparable to the pivotal trial population, by matching on trial inclusion and exclusion criteria to the extent possible and adjusting for the remaining imbalance in baseline characteristics with propensity score weighting methodology, such as inverse probability treatment weighting (IPTW). Regarding ORR, FDA had major critiques on real-world tumor assessment versus Response Evaluation Criteria in Solid Tumors (RECIST) criteria with respect to the difference in frequency of scans and the “baseline” scan used to define response. Concordance analysis between real-world tumor response and trial response is encouraged. Lastly, methods to minimize confounding, including appropriate index date and reduction in missing values, need to be described. The impact of unmeasured confounding should be evaluated through quantitative bias analysis. Despite these concerns, RWE may be used in successful approvals and even speed timelines. We found that median time from IND submission to approval of applications with RWE was 5.5 years, in contrast to 6.8 years for drugs in any expedited program (priority review, accelerated approval, fast track, or breakthrough therapy designation) and 8.3 years for drugs not in any expedited program based on a review of new cancer drugs approved from 2012 to 2017 (14). This suggests that inclusion of RWE may help play a role in accelerating the drug development process, specifically highlighting the unmet need.

The extent of FDA's critiques on RWE studies appears to be inversely correlated with the level of the primary efficacy achieved in the pivotal trial in our observation. For pivotal trials with a high ORR, often coupled with long duration of response (DOR) and low ORR in the RWE, we observed fewer critiques from the FDA on the quality of the RWE. Unsurprisingly, the magnitude of primary efficacy observed in the pivotal trial is also associated with being granted full (regular) approval rather than a contingency (accelerated) approval for NDAs. For example, high ORR and long DOR are typically seen in the full approvals; these had ORR >70% and median DOR was at least nine months. However, long DOR alone is not sufficient for full approval. In fact, we observed that low ORR and long DOR reflect a profile for accelerated approvals. For accelerated approvals, there was a mix: ORR ranged from 13% to <70%, and median DOR ranged from 4 months to not reached. The high efficacy as observed by an ORR >70% based on the limited number of full approvals in this review should not be interpreted as the determinant of a full approval. For example, whether the mechanism of the investigational drug acts on a driver mutation for a particular cancer versus on a passenger mutation could play a role in the interpretation of the efficacy and safety results. Other factors observed with full approvals include having all designations of orphan disease, breakthrough therapy, and priority review (but not necessarily fast track designation), and no need for the FDA to call for an advisory committee meeting.

The FDA emphasized that care must be exercised in selecting appropriate RWD, while ensuring adequate sample size and developing thoughtful study designs. Although chart review was the most common source for RWE, the FDA also commented that data from chart review studies could be limited and subject to selection bias and confounding [avelumab, blinatumomab, tazemetostat (NDA 211723), and tazemetostat (NDA 213400)]. Additionally, although the FDA acknowledged the quality of Flatiron data in the palbociclib review, it also noted criticisms on lack of generalizability to a wider population and incompleteness in other applications (erdafitinib and entrectinib). For instance, Flatiron data used for an RWE study to support entrectinib were unlikely to be generalizable due to the low rate of ROS1 testing in clinical practice and the high proportion of community-treated patients in this database. In the erdafitinib review, RWD were often incomplete, and missing data or lack of information on important prognostic factors would not allow for the evaluation of comparability between RWE and trial populations. FDA guidance on the use of electronic health records recommends developing a plan to handle missing data, although such successful plans have not been identified in these examples (4).

Importantly, RWE may be able to demonstrate unmet therapeutic needs that the investigational therapy can address. RWE results from erdafitinib, blinatumomab, capmatinib, and avapritinib applications demonstrated the burden of disease and poor prognosis with real-world standard of care in their respective populations. Avapritinib's application in particular demonstrated the importance of unmet need in the disease area coupled with infeasibility of establishing valid comparative clinical evidence through randomized controlled trials. The poor prognosis of this population shown through RWD (ORR of 0%–5%) contextualized with the ORR of 84% from the phase I trial helped demonstrate the clinical benefit of avapritinib. Strong trial evidence and indisputable demonstration of unmet need, even when considering RWD limitations, established a clear need for effective therapy that avapritinib could fulfill. This contributed to the granting of full (not accelerated) approval of avapritinib. Ultimately, the strength of trial data appears to be a key determinant of the extent to which RWE is needed and considered for FDA regulatory decision-making. Similarly, axicabtagene ciloleucel, and capmatinib single-arm trials demonstrated such benefit in a population with unmet therapeutic need as contextualized with RWE and received full approval.

The FDA has issued guidance highlighting many limitations of RWE regarding study design and analysis; however, some of these issues might have been corrected before submission if applicants had communicated with the FDA earlier and produced a priori protocols for FDA review. The FDA emphasized that “transparency about study design and analysis before execution is critical for ensuring confidence in the results” and encourages applicants to register observational studies on ClinicalTrials.gov (5). For example, the FDA criticized the entrectinib, selinexor, and both tazemetostat (NDAs 211723 and 213400) applications because a protocol for the RWD analysis was not submitted prior to the conduct of the study. Additionally, four reviews indicated that there was no discussion between the applicant and FDA regarding inclusion of RWE in the application package prior to submission of results. The FDA appeared most receptive to early notice as seen in the avapritinib application, whereby the natural history study was discussed fewer than two years after the IND submission at a Type B End-of-Phase 1 meeting.

There were several common weaknesses in RWE study design. Most cases did not apply the same trial inclusion/exclusion criteria to the RWE study, and the FDA critiqued this for entrectinib, erdafitinib, selinexor, tazemetostat (NDA 211723), and blinatumomab. Study design issues with regard to follow-up periods and index dates were noted, including less regular follow-up assessments for palbociclib, lack of comparability in terms of duration of follow-up for blinatumomab, lack of comparability on outcome measures for tafasitamab and tazemetostat (NDA 211723), and index date definition leading to immortal time bias for selinexor. Endpoints with definitions inconsistent with the pivotal trial were noted, especially with regard to tumor response, because RECIST criteria cannot be applied in real-world settings as they are in trials. Although some of these issues are a function of differences between clinical trial assessments and assessments done in a real-world setting, a priori alignment with the FDA on how these can be addressed within the conduct of the RWE studies would have helped strengthen findings.

In cases when RWE was directly compared with the pivotal trial, key concerns regarding statistical analyses were identified. Residual confounding, despite IPTW, was a key issue that the FDA noted, in addition to other assumptions regarding propensity score–weighted analysis, for the examples of entrectinib, erdafitinib, selinexor, tafasitamab, and blinatumomab. In the tafasitamab review, the FDA noted that the choice of covariates to include in the matching may not be fully sufficient, and the distribution of the observable covariates in the matched cohorts was not well balanced, even commenting upon the joint and marginal distributions. In some cases, the FDA agreed with the methods used to address limitations inherent to the data, such as the delayed entry model used in the erdafitinib submission to address left censoring. In order to address the FDA's concerns regarding statistical comparisons, especially residual or unobserved confounding due to incomplete RWD either from unobserved variables or missing values for observed variables, methods to evaluate the impact of these biases should be incorporated. This type of analysis can be approached and addressed in a number of ways through careful incorporation of statistical methods (15). For example, for unmeasured confounding, this may include array approach analysis and quantitative bias analysis. For missing values, this may include multiple imputation and tipping point analyses.

Although the FDA did have criticisms on the adequacy of the RWE that was generated in the drug registration submissions in this review, it does appear that FDA wants to see RWE included in the application. Additionally, the FDA has acknowledged that observational studies of external comparators can provide important information and may provide supportive information for NDA/BLA submissions and has encouraged pharmaceutical manufacturers to explore sources of RWD and their utility in drug development programs. Therefore, despite the criticisms and limitations from the FDA review, inclusion of RWE appears to be encouraged although with appropriate study design planning and discussion.

This study is limited to publicly available information on oncology applications approved by the FDA. Data on applications that were submitted and not approved could not be reviewed as they are not publicly available. Given that the 21st Century Cures Act went into effect recently, a limited number of examples are available from which to identify learnings; this study attempted to understand the information that is currently available and disseminate this knowledge for future RWE planning considerations. Lastly, further characterization of accelerated approvals, for example, based on evaluation of intermediate versus clinical endpoints, was beyond the scope of this article, as this would have implications on both clinical trial and RWE designs.

This study found that real-world studies used as external controls complemented efficacy data from single-arm trials in successful oncology product approvals. Key attributes identified include early engagement, a priori protocol development, and robust research design. High efficacy of the investigational agent in the pivotal trial, coupled with low efficacy of standard of care in RWE, and regulatory characteristics of the application may play a role in full approvals.

B. Arondekar reports personal fees and other support from Pfizer Inc. during the conduct of the study. M. Duh reports grants from Pfizer during the conduct of the study. R.H. Bhak reports other support from Pfizer during the conduct of the study. M. DerSarkissian reports other support from Pfizer during the conduct of the study. L. Huynh reports grants from Pfizer during the conduct of the study; grants from Takeda, GlaxoSmithKline, and Merck outside the submitted work. M. Wu reports other support from Analysis Group during the conduct of the study; other support from Analysis Group outside the submitted work. B. Wornson reports other support from Pfizer during the conduct of the study; other support from Pfizer outside the submitted work. A. Niyazov reports other support from Pfizer Inc outside the submitted work. G.D. Demetri reports nonfinancial support from Pfizer during the conduct of the study; grants, personal fees, and nonfinancial support from Bayer, Pfizer Novartis Epizyme, Roche/Genentech, AbbVie, GlaxoSmithKline, Janssen, Pharmamar, grants and personal fees from Daiichi-Sankyo, LOXO Oncology/Lilly, EMD-Serono, personal fees from ICON PLC, MEDSCAPE, Mirati, Synlogic, WCG/Arsenal Capital, MJ Hennessey/OncLive, C4 Therapeutics, McCann Health, RAIN Therapeutics, personal fees and other support from Blueprint Medicines, G1 Therapeutics, Caris Life Sciences, Cell Carta, Ikena Oncology, Kojin Therapeutics, Relay Therapeutics, other support from Erasca Pharmaceuticals, Bessor Pharmaceuticals, and Champions Biotechnology outside the submitted work; in addition, G.D. Demetri has a patent for Imatinib for GIST issued, licensed, and with royalties paid from Novartis; and is a member, Board of Directors of Blueprint Medicines, past member of Board of Directors for Translate BIO (ended Sept 2021) and Merrimack Pharmaceuticals (ended Sept 2019). No disclosures were reported by the other authors.

We would like to thank Eric Davis formerly of Analysis Group, Inc., for his analytic support. Medical writing assistance was provided by Loraine Georgy, PhD, an employee of Analysis Group, Inc., a consulting company that has received research funding from Pfizer, Inc.