Letting the GENIE Out of Its Bottle: Examining the Potential of Real-World Clinicogenomic Data

We would like to commend the authors of the article “Characteristics and outcome of AKT1^E17K-mutant breast cancer defined through AACR Project GENIE, a clinicogenomic registry” (1) in this issue of Cancer Discovery, for a well-conceived and well-designed study using a new and exciting data source, the international genomics registry, and data-sharing consortium Genomics Evidence Neoplasia Information Exchange (GENIE), launched by the American Association for Cancer Research (AACR). In its latest release (https://www.aacr.org/Research/Research/PAGES/AACR-PROJECT-GENIE-DATA.ASPX, accessed on January 24, 2020), this publicly available clinicogenomic database included more than 70,000 patients with cancer treated at nineteen institutions worldwide. This study by Smyth and colleagues utilizes this large and standardized database to expand our knowledge around the complicated biology of metastatic breast cancer, specifically the natural history of AKT1^E17K-mutant breast cancer, by piloting a distributed data curation process to complement detailed genomic data on a rare molecularly defined cohort.

The emergence of genetic profiling as a tool to identify unique molecularly defined diseases has led to more precise prognostication, and to the development of exquisitely targeted therapies and personalized treatment decisions in certain tumor types. Yet, the clinical implications of the genetic signature of most solid tumors remain unknown. The availability of clinically annotated genomic information will be increasingly crucial as new molecular diagnoses emerge and new therapeutics are developed.

Propelled by a growing number of known actionable targets, by technological advances enabling faster and cheaper genomic analyses, and by evolving reimbursement policies, the use of next-generation sequencing (NGS) for molecular tumor characterization is expanding. NGS provides information for potentially hundreds of known genetic variants in a single panel, and can now effectively be conducted on tissue or circulating tumor DNA as a “liquid biopsy.” On the heels of this escalating adoption, a number of research teams across the healthcare sector have partnered to develop platforms that combine large-scale NGS data and clinical data processing advancements, enabling the development of expansive real-world research databases, a necessary prerequisite to study rare cohorts.

Genomic profiling is uncovering the heterogeneity of metastatic breast cancers and identifying a growing number of clinically actionable targets for small subsets of patients. As a result, numerous biomarker tests and associated therapies are now recommended as part of the 2020 National Comprehensive Cancer Network Clinical Practice Guidelines for breast cancer (2), such as BRCA mutations and PARP inhibitors, PIK3CA mutations and alpelisib, MSI-H/dMMR and pembrolizumab, NTRK mutations and TRK inhibitors, or ERBB2 mutations and neratinib.

In this issue of Cancer Discovery, Smyth and colleagues use clinically annotated genomic data from GENIE to examine the natural history of metastatic breast tumors harboring an AKT1^E17K mutation, a known oncogenic driver occurring in many cancers at a low prevalence. Using a retrospective case–control observational study design, investigators observed no significant differences in baseline clinicopathologic features or overall survival between AKT1-mutant and AKT–wild-type cases. They did note that mTOR inhibitor therapy duration was longer in patients with AKT1-mutant tumors than in AKT1–wild-type cases, an observation previously unrecognized in pivotal clinical trials due to the rarity of this alteration. The clinical relevance of these findings will require deeper investigation into their biological mechanisms and into their potential translation into outcomes. However, the main conclusion from this pilot study extended beyond the predictive significance of AKT1 mutations by demonstrating the capability of the AACR Project GENIE to generate an analytic cohort of patients with metastatic breast cancer with a rare oncogenic driver.

Real-world clinical data have been used to inform trial designs, for post-marketing commitments, and in comparative effectiveness research. As we look forward, real-world clinicogenomic data can be utilized in these same ways, but, as Smyth and colleagues note, can also potentially be used to generate comparative evidence for single-arm, noncomparative studies, potentially functioning as “synthetic” or external control arms, or as part of hybrid control arms in rare molecularly defined subsets. The benefits of this strategy could reverberate throughout the clinical research landscape, accelerating clinical development of new drugs, possibly improving the efficiency of the regulatory process, and ultimately facilitating more rapid access to novel and effective therapies for patients with cancer. The question therefore emerges: Did the authors assemble a synthetic control arm, and if not, what will it take to get us there?

Randomized controlled trials are the gold standard for assessing the efficacy of treatment, as they represent the optimal clinical research approach to minimize systemic bias and the effect of unmeasured confounders. Modern clinical trial design allows us to draw causal inference confidently, that is, to conclude that the treatment in question is the cause of the outcome assessed in the study. In the absence of randomization, using real-world data for that same purpose requires methodologic designs and analytic approaches (in sampling procedures, data curation, identification of exposures, covariates, and outcomes) that apply extreme rigor to address the issues that randomization mitigates in clinical trials, namely potential biases and confounders. Addressing these challenges is the goal of ongoing initiatives spearheaded by the FDA (3) in partnership with academic, clinical practitioner, and industry researchers, where frameworks for the generation of high-quality real-world evidence are being defined and tested (https://healthpolicy.duke.edu/real-world-evidence-collaborative, accessed on January 24, 2020).

For real-world genomic cohorts, these challenges become particularly specific. Rigorous patient selection and indexing face unique and additional considerations. By definition, eligible patients have undergone genomic testing as part of their care. However, unlike clinical trials where genomic testing occurs on or prior to study entry, in the real world, genomic testing may occur at any point in the patient's disease course. The potential variation in testing patterns was evident in the study by Smyth and colleagues, where the overall time to sequencing from metastatic diagnosis spanned from 5 months prior to metastatic diagnosis to more than 16 years after it, although statistically significant differences were not seen between the AKT1-mutant and wild-type cases. Because patients are not considered “at risk” until their testing event, the potential selection bias incurred by the testing event itself must be carefully considered. This may mean using analytic techniques, such as left truncation used by Smyth and colleagues, to account for delayed entry to the cohort, or use of a landmark design in which patients are considered eligible for study inclusion only after the testing event has occurred. Moreover, for diseases in the midst of evolving genomic testing patterns, such as metastatic breast cancer, the potential for phenotypic drift within the cohort as well as changes in the standard of care over time should be considered. For instance, patient characteristics and timing of genomic testing are likely to shift with the emergence of PIK3CA as an actionable therapeutic target. Restricting the cohort of interest to those patients tested within a specific window of time—for instance, contemporaneous to the enrollment of a trial of interest—is one possible approach to reducing the heterogeneity of treatment and testing characteristics across real-world and trial cohorts.

Testing patterns are not the only heterogeneous feature of genomically defined cohorts. In the absence of a central lab, as in AACR Project GENIE, the type and quality of testing, the gene panel used, and the reporting practices can vary widely, across providers or even over time within the same laboratory. For instance, despite of the availability of models for a common genomic nomenclature, such as OncoKB, NIH's Genomic Data Commons, or Fast Healthcare Interoperability Resources (FHIR), reporting heterogeneity for gene names and alterations is prevalent among commercial vendors (4), and harmonization remains a necessary step to make genomic data analyzable and to ensure optimal cohort selection when aligning real-world and trial datasets. Beyond cohort selection, the deployment of imaging-based endpoints to evaluate clinical outcomes matching those used in clinical trials (i.e., RECIST-based criteria to define progression) is not yet feasible in real-world data (5). Focusing on objective endpoints such as mortality or discontinuation of therapy, as was done by Smyth and colleagues, is one approach to addressing this issue, but further work is needed to generate comparative evidence for RECIST-based trial outcomes from the real world. Collaborative work examining the correlation of endpoints derived from electronic health record data with more traditional, RECIST-based endpoints is ongoing (https://www.focr.org/publications/8th-annual-blueprint-breakthrough-validating-real-world-endpoints-evolving-regulatory, accessed on January 24, 2020).

This early pilot study using AACR Project GENIE illustrates that the opportunities for generating meaningful evidence from real-world sources is advancing and benefiting from partnerships formed across academia, industry, and regulatory agencies. As the breadth of genomic testing increases and the capture of associated clinical data improves, novel data sources are likely to transform not only the way we think about the biology and natural history of cancer, but also the discovery and development of new therapies.

E. Castellanos is associate medical director at Flatiron Health and has ownership interest in Flatiron Health and Roche. S.S. Baxi is medical director at Flatiron Health, a subsidiary of Roche, and has ownership interest in the same. No other potential conflicts of interest were disclosed.