Developing Precision Medicine in a Global World

The concept of personalized medicine is based on the fundamental assumption that one or more molecular aberrations can be discovered as either etiologic or sustaining a given malignancy. Identifying the aberration in a given tumor will lead to therapies that are more selective (and therefore hopefully less toxic) than classical chemotherapeutics. The tests that identify molecular aberrations that are themselves targets of a specific therapeutic are among the tests considered companion diagnostics. As such, the tests have predictive value—in a clinical trial, the test ensures selection of a population of patients more likely to have a response to treatment, and in any individual patient increases the likelihood of success. Companion diagnostics are to be differentiated from prognostic tests—that infer outcome for a patient. For example, lactate dehydrogenase levels are prognostic in lymphoma—placing patients in a higher risk group. But they do not predict activity of any individual agent. Predictive and prognostic markers must also be distinguished from pharmacodynamic markers, tests that can show an impact on a particular oncologic pathway by a drug, and from pharmacologic markers, usually polymorphic variant testing, to determine susceptibility to drug toxicity. In general, pharmacodynamic markers have not yet joined the armamentarium of clinically useful tests in oncology.

Although the term companion diagnostics has been newly applied to a test “co-developed” to identify populations of patients who may benefit from a particular drug, the concept is not new. Nor is individualized selection of therapy for a patient based on a pathology report. What distinguishes the two in U.S. Food and Drug Administration (FDA) parlance is that the companion diagnostic provides a result that is “essential for the safe and effective use of a corresponding therapeutic product” (italics added; Text Box 1). In addition to identifying patients who are likely to benefit, the FDA definition of a common diagnostic also encompasses those tests that would identify patients who are at risk of adverse reactions, as well as tests that monitor response to treatment for the purpose of adjusting that treatment. FDA has taken the approach that the essential diagnostic should be “developed contemporaneously”; there is recognition that this may not always be possible.

The development of HER2 testing to accompany trastuzumab is considered the first example of a companion diagnostic from the standpoint of co-development. However, lessons learned in the development of tests for estrogen receptor (ER) positivity in breast cancer certainly informed development of HER2 testing (1, 2). Radioisotope-based receptor-binding assays were first described in 1973 for ER and were later superseded by immunohistochemical assays (3). Fourteen tests for ER assays can be found in the FDA Medical Devices listings. The first test for ER was approved in 1981, and the most recent approval was in 2013, based on data suggesting that the test was “substantially equivalent.”

The FDA listing of approved companion diagnostics (reproduced in Table 1) includes 19 approvals, of which ten cover testing for HER2, three for EGF receptor (EGFR), two for B-RAF, and one each for ALK, KIT, RAS, and iron concentration. Examining the list of approved diagnostics for HER2, the complexity of the regulatory process is immediately understood. Furthermore, this list of FDA-approved companion diagnostics is dwarfed by the list of tests commercially available to clinicians for understanding the biology of a given tumor. These tests are not approved by the FDA, but are developed by laboratories that are Clinical Laboratory Improvement Amendments (CLIA) certified, and thus subject to regulation with regard to analytical validation. These tests, termed laboratory-developed tests (LDT), in some cases are also companion diagnostics (although without FDA approval); in others, they are prognostic or diagnostic tests that point the clinician toward one treatment scheme or another. The Tufts Medical Center Evidence-based Practice Center identified 112 gene-based tests alone for solid and hematologic tumors, 50 introduced between 2006 and 2001 (Table 2), and approximately 79 tests in development (4). These tests have entered the clinical use in the United States using the same regulatory pathways followed for immunohistochemical assays performed in commercial and academic pathology laboratories in the course of cancer diagnosis. As described by Parkinson and colleagues, diagnostic tests in cancer need to have analytical validity, clinical validity, and clinical utility (5).

One of the commercially available LDTs in Table 2 is that for ERCC1, a DNA repair protein. ERCC1 is one of a family of proteins with a critical role in at least four different pathways of DNA repair, including nucleotide excision repair needed for the removal of platinum adducts. High levels of ERCC1 in patient tumors have been associated with resistance to cisplatin, and some studies have suggested that cisplatin should not be used in treating tumors that express high levels of ERCC1, to avoid toxicity without benefit (6, 7). Ongoing clinical trials are testing whether chemotherapy choice can be altered on the basis of ERCC1 levels, with cisplatin omitted in the ERCC1-high cohorts. One prospective trial reported improved response rates with genotyping, without a survival advantage (8). However, a recent study casting doubt on the accuracy of the antibody used for ERCC1 detection (9) calls the analytical validity into question, despite its performance in a CLIA-approved laboratory. Both clinical validation (whether the test accurately predicts cisplatin resistance) and proof of clinical utility (whether knowing the result will allow selection of therapy) are as yet unproven, but remain under active investigation (10, 9).

Although many of the tests listed in Table 2 do not have the potential to become companion diagnostics, any discovery and validation of a new oncogenic target offers the potential for (i) a drug and (ii) a test. One example is the new discovery of activating mutations in MYD88, a Toll-like receptor, and interleukin-1 receptor adaptor protein, in Waldenstrom macroglobulinemia (11). Although therapies targeting MYD88 are not yet in the clinic, the identification of activating mutation has made the protein a target for drug development, and a companion diagnostic will be required. As shown in Table 3, data reported in 2012–2013 offer an unacceptably wide range of sensitivity, with a reported prevalence ranging from 70% to 100% of samples from patients with Waldenstrom macroglobulinemia (12). Any one of these tests could be commercialized and analytically validated by a CLIA-certified laboratory without proving the test accurately reflects the presence of the mutation in Waldenstrom macroglobulinemia (i.e., clinical validity) or that the test will affect outcome of treatment with a therapeutic (i.e., clinical utility). The development of a drug to target the L265P mutation of the MYD88 gene will require a test that is not only proven to be analytically and clinically valid through the FDA approval process for companion diagnostics, but also demonstrate clinically utility to secure favorable reimbursement and optimal utilization.

One question that stands out in the discussion of diagnostics is the degree of regulation that is actually necessary. The papers in this CCR Focus section are aimed at understanding the current regulatory and reimbursement environment in the United States and in Europe, and the articles clearly show that companion diagnostics are under increasing scrutiny and regulatory control in both regions. Certainly, a breast cancer specimen incorrectly labeled HER2 negative could result in significant harm to a patient, but all tests are imperfect, and the availability of trastuzumab with an imperfect companion diagnostic has greatly benefited patients with breast cancer in general. The difficulty in settling on the most accurate test for HER2 points to the need for careful analytical validation and clinical validation (1, 13, 14). There exists a large body of evidence about laboratory errors. Some of the errors are “preanalytic,” that is, due to the quality of sample obtained or how it was handled; some “postanalytic,” due to reporting or interpretation. For pathology samples, where secondary review is considered some indicator of error, discrepancies have ranged from 1% to 15% (15). Quality assurance processes have been put into place to mitigate errors, but every new test introduces new opportunity for error. A study in 2007 reported that 20% of HER2 assays yielded incorrect results, with a major problem caused by variable tissue fixation (13).

Companion diagnostics originate from preclinical hypotheses involving predictive biomarkers. One of the difficulties in development of companion diagnostics is that a decision to invest in a companion diagnostic must typically be made before the predictive value of the biomarker, or the efficacy of the experimental therapeutic in a population defined by the diagnostic, is known. Initially, there must be conversion of an assay from a laboratory-based test (developed from cell lines or other preclinical models) to an assay that can be used on clinical specimens, such as surgical specimens or biopsies. The complexity of this transfer is often underappreciated by both the basic scientists who identified the potential predictive biomarker and the clinicians involved in designing the clinical studies that will use the predictive marker. Furthermore, the statistical rigor that would be applied to the evaluation of the biomarker in a clinical trial may not have been present in preclinical hypothesis testing, leading to a false-positive conclusion that a particular biomarker is predictive for drug efficacy.

As noted earlier, the FDA guidance states that “a companion diagnostic device can be in vitro diagnostic device or an imaging tool that provides information that is essential for the safe and effective use of a corresponding therapeutic product.” Although interpretation of “essential for the safe and effective use of a corresponding therapeutic product” depends on the context of the patient population, analytical validity (the ability of the assay to accurately measure the biomarker) and clinical validity (the ability of the assay to accurately measure a clinical outcome of interest) do not depend on this context. Both analytical validity and clinical validity are typically assessed by 2 × 2 table analyses to identify sensitivity and specificity (and positive and negative predictive value).

Determination of whether an assay provides “information that is essential for the safe and effective use of a corresponding therapeutic product” relates to an assay's clinical utility. Clinical utility is discussed in detail in Parkinson and colleagues (5), and is often defined as an assessment of the use of the assay to improve patient outcomes (relative to the state of not using the assay). Because quantitative data are typically not available for this evaluation, clinical utility is often measured in a qualitative, subjective manner. For example, for a new drug that is much more effective than previous treatments and with a safety profile that is no worse than previous treatments, it can be argued that a companion diagnostic that has a high negative predictive value (i.e., minimizes false negatives) could be considered to have acceptable clinical utility, even if the positive predictive value of the assay is not “high.” To provide an example, based on original data from a registration trial that used an immunohistochemical assay to measure HER2 protein expression in breast cancer tissue, the negative and positive predictive value of the test for computed tomography-based response were 79% and 44%, respectively, for patients receiving the combination of traztuzumab and paclitaxel (from Herceptin label; http://www.accessdata.fda.gov/drugsatfda_docs/label/2000/trasgen020900lb.htm). Note that because only patients whose breast cancer tissue was scored as 2+ or 3+ were enrolled in this study (with +3 defined as “test positive”), 79% likely underestimates the true negative predictive value of the assay.

For assays with nonbinary outputs, the assay cutoff is another variable that affects clinical utility, and may be decided before registration studies are initiated, thus with some risk that performance of the assay cutoff in large populations may be poorly understood. For example, a cutoff selected on the basis of response rates might yield very different negative and positive predictive value related to overall survival, with overall survival data often not available before initiation of registration studies. Similarly, from a clinical utility perspective, a cutoff that is selected on the basis of a single-arm study of a new drug may perform quite differently in a randomized setting, where potential prognostic effects of the assay for the control arm (i.e., a standard-of-care treatment) may become apparent.

FDA and the European Medicines Agency approval pathways for companion diagnostics are discussed in detail in articles in this CCR Focus section (16–18). Similar to therapeutics, companion diagnostics are also the subject of health technology assessments of cost effectiveness (19). In cases of low prevalence of test-positive patients, the cumulative cost of screen failures can be considerable, adding to the overall cost of use of the therapeutic. Indeed, it is interesting to note that “personalized” medicine was originally envisioned as an approach to reduce cost and toxicity for patients, allowing ineffective treatments to be avoided. However, the reality is that there is can be an increase in cost at both the industry level, related to development of the companion diagnostic, and at the health care delivery level, where diagnostic tests are used to determine the most appropriate therapy. Additional complexity for patients and physicians can exist in the case where multiple different assays, perhaps with different cutoffs, are approved for drugs that have the same target but were developed by different companies.

As noted in Dr. Mansfield's contribution, the concept of coapproval of therapeutic product and companion diagnostics has been an important part of FDA's effort to promote implementation of personalized or precision medicine (17). The message to be gleaned is the recognition that laboratory testing and molecular diagnostic testing, in particular, is complex, and although a co-development model has merit, it cannot accommodate all clinical situations in the same way. The European experience as described by Byron and colleagues and Pignatti and colleagues acknowledges that a broad range of companion diagnostic tests for a specific drug is currently allowed, without the need for concurrent development with the drug, and without a requirement for a specific, regulatory authority-approved test in the drug label (18, 19). Although that route may present some challenges in terms of regulatory oversight, it utilizes all the laboratory testing capabilities that are available to bring precision medicine to the individual patient.

Recently, the classical paradigm of “one biomarker-one test” has been changing, in part due to advances in technology, as well as to obviate the need for multiple biopsies to obtain sufficient tissue for multiple tests. When multiple tests are needed on one specimen, there may not be sufficient tissue available to perform all the tests. This is an issue most pressing with carcinoma of the lung, where multiple genetic tests may ultimately need to be performed to detect different actionable oncogenic mutations. This problem may be solved, in part, by the advent of next-generation DNA sequencing (NGS) platforms. One example of use of this technology is the Lung Master Protocol effort, which is a collaboration involving Friends of Cancer Research, FDA, National Cancer Institute's National Clinical Trials Network/Southwest Oncology Group, patient advocacy organizations, several pharmaceutical companies, as well as a diagnostic company (17).

In this project, a master protocol will govern how multiple drugs will be tested as potential treatments for squamous cell carcinoma of the lung. Each arm of the study will test a different drug that has been determined to target a unique genetic alteration(s). The use of NGS will help identify which patient is a molecular match to each arm. Using a common screening platform will reduce the amount of tissue that would be needed, as compared with if each substudy were being conducted as completely independent trials. The testing platform has previously demonstrated analytic validity and the trial results will help evaluate the clinical outcome produced by each drug/biomarker (20). In this case, one test will be able to provide the determination of which drug may be a beneficial option for patients based on the genetic characteristics of their tumor. Successful drug/biomarker pairs will be eligible for FDA approval and sponsors could pursue market authorization with the NGS platform as the companion diagnostic for the specific biomarker signature or perform the necessary bridging studies to validate the use of an alternative diagnostic tool. This will create a rapidly evolving infrastructure that can simultaneously examine the safety and efficacy of new drugs. This approach will have the ability to improve enrollment, enhance consistency, increase efficiency, and reduce costs.

Furthermore, the FDA recently approved the first diagnostics using NGS, setting the stage for use of similar diagnostics to detect multiple gene mutations in a single biopsy specimen. The approved devices include methods used to diagnose cystic fibrosis (by detecting DNA mutations in the cystic fibrosis transmembrane conductance regulator gene) as well as the Illumina MiSeqDx platform, which allows development of NGS tests for any part of a patient's genome (21). Notably, the variables that govern the accuracy of complex, multigene tests are numerous and will require the same or greater attention that many of the single gene tests required (examined in detail in ref. 22). The variables include overall quality of the tissue specimen, fraction of tumor cells in the sample, and depth of sequencing, among others.

Projects like the Lung Master Protocol and marketing approval of NGS instruments demonstrate the continued expansion of the role diagnostic tools play in guiding therapy. As the biology of patients and tumors continue to be further understood, identifiable subpopulations of patients that are best fit for some treatment regimens will likely become smaller as compared with treating broad types of cancers with conventional cytotoxic drugs. Collaborative development consortia, like the Lung Master Protocol, and novel technology will be essential to support new, precision drug development in the future. Although this approach will increase the likelihood that patients that match a targeted therapy will positively respond, developing a drug with a companion diagnostic continues to present several challenges, some of which are highlighted next.

Key issues that often arise in companion diagnostic development include cutoff selection (for diagnostics with nonbinary measurements) and “prescreening” that can occur as a result of the availability of nonapproved tests used by patients and physicians to gain information about tumor genotype and phenotype.

About the cutoff selection, in many cases, there may be a monotonic relationship between a biomarker expression level and clinical outcome after treatment with the matching therapeutic, such as tumor shrinkage (Fig. 1). In these cases, setting a high cutoff for the test will favor positive predictive value at the expense of negative predictive value. Conversely, setting a low cutoff will favor negative predictive value at the expense of positive predictive value. From a payer perspective, a high cutoff might be favored because this would restrict the use of a new therapeutic to a patient population that would have a large clinical benefit. On the other hand, from an individual patient perspective, a low cutoff might be favored because this would increase the likelihood that the patient would be able to receive the new treatment, even if the chances of response were low (assuming that the treatment was associated with an acceptable safety profile).

Another issue with companion diagnostic development is that advances in technology have allowed increasing availability of genetic and phenotypic information about a patient's cancer at affordable prices (21). This creates tension between the patient and physician desire for information that could be helpful in understanding unique aspects of a patient's cancer (even if that information is imperfect) versus regulatory concerns over harm that could result from decisions made from faulty tests. For example, a reporter from the New York Times had her DNA sequenced by three different companies, with discordant risk predictions identified for several diseases (23).

The availability of these kinds of tests can confound companion diagnostic development. For example, when patients are “prescreened” before being screened for enrollment into a study, using an assay that is different from the companion diagnostic, the study estimate of the effect of the new treatment can be biased (24). This bias is caused by discordance between the test used as a prescreen and the companion diagnostic test. For example, patients who would have scored positive on the companion diagnostic but not the prescreen test would not be entered into the study. FDA and industry alike discourage enrollment of prescreened patients in trials when possible, but with greater availability of direct-to-consumer genetic testing, this can be unavoidable.

Molecular diagnostic tests are recognized for reimbursement purposes by specific current procedural terminology (CPT) codes. Through 2012, the CPT codes used for molecular tests were based on methodology (e.g., nucleic acid extraction, DNA amplification, separation by electrophoretic methods each had specific CPT codes) and, in that, differed from other laboratory tests for which codes are analyte specific. This system was very useful, in that, it could accommodate various technical approaches that could be applied to measure molecular analytes at a time when technology was rapidly evolving. Payers, however, balked at the lack of transparency of those codes and the need to report multiple codes for single procedures. Consequently, a work group established by the American Medical Association (AMA) CPT Editorial Panel developed a replacement CPT coding system for molecular diagnostic tests that is exquisitely analyte specific and technology agnostic for more than one hundred of the most common molecular diagnostic tests [e.g., 81275 KRAS (v-Ki-ras2 Kirsten rat sarcoma viral oncogene), gene analysis, variants in codons 12 and 13]. Other less common molecular tests, nearly 600, have been similarly defined and assigned codes in groups (tier 2), acknowledging to some extent the differences in technical and interpretive work needed to achieve clinically useful information. A similar AMA-sponsored work group is currently developing CPT codes to recognize various clinically defined tests that require NGS strategies. Although it is conceivable that such NGS tests can replace extant assays, the challenge will be in determining whether such an approach offers sufficient benefit in terms of cost efficiency and clinical utility to warrant recognition (and reimbursement) as CPT-defined procedures.

Establishing reimbursement for the refined molecular codes has been problematic. Rather than following the usual process for determining procedure values through the AMA Relative Value Update Committee-mediated process or via the cross-walk process for laboratory tests on the Clinical Laboratory Fee Schedule, CMS opted to leave the responsibility for pricing the new codes, to the local Medicare administrative contractors, using a process known as “gap fill.” They were neither prepared to determine the pricing nor had the resources to make fair and informed pricing decisions or coverage policies. As a consequence, none of the tests were priced at the beginning of 2013 so claims were not paid. With completion of the gap fill process, some molecular tests are now being covered but many are not. None of the tier-2 codes have had pricing determined. Furthermore, the failure to price the codes has led many payers to misinterpret this as a reason for noncoverage. These problems with reimbursement rates and policies imperil the existence of many smaller molecular diagnostic laboratories.

The lauded transparency of the new molecular codes will be problematic for clinicians who try to use molecular diagnostic assays in situations outside those recognized for specific companion diagnostics, leading to denial of payment. For example, KRAS codon 12 and 13 mutation testing is indicated for consideration of treatment of stage 4 colorectal cancer with targeted therapies. Mutations in KRAS codons 61 and 146 are also thought to be relevant but testing for them would probably not be covered. Similarly, KRAS mutation status is not immediately relevant for non–small cell lung carcinoma in choosing a therapy, but it is very informative in that KRAS mutations are mutually exclusive of other significant driver mutations in EGFR, ALK, and ROS-1.

The companion diagnostic model has been adopted in the United States, the European Union, and elsewhere in the world as an approach to improve the efficacy of therapy administered to patients by improving the accuracy of the tests used to select particular tumors for therapy. Although some of these efforts have been successful, and a model for future drug development, it is also important to understand the weaknesses of the co-development model. Although many quickly point to the Herceptin/HercepTest combination as a model, the difficulties in establishing uniform HER2 testing in the United States and elsewhere are notable, and although the notion that HER2 protein “overexpression” seems a reasonable marker for tumors that might benefit from the drug, the eventual recognition that another “biomarker,” HER2 gene amplification might be a better test, should give us pause (14).

In 2004, a similar approach was applied to selecting patients whose tumors might respond EGFR monoclonal antibody therapy (erbitux). An immunohistochemical assay (DakoCytomation EGFR pharmDx; Table 1) was approved as the companion diagnostic to detect overexpression of EGFR. In 2008, reports emerged that KRAS codon 12 and 13 mutation predicted resistance to EGFR inhibitor, and in 2009, FDA issued a labeling change for the drug requiring a molecular determination of KRAS mutation status. An FDA-approved “companion diagnostic” test for KRAS was not available until July 2012, with the difference in time frame relative to the labeling change reinforcing the point that development works most efficiently if trials for both diagnostic and drug are carried out simultaneously. It is not always possible to know a priori what the best biomarker is. They are chosen, hopefully after extensive debate, based on the best knowledge and understanding, but always with an awareness that there will be new discoveries over the horizon.

The questions begging to be addressed and that are discussed in this CCR Focus section include the following:

How are companion diagnostic tests best developed (before being approved and commercialized) and how do they evolve?

Who performs clinical testing when there is no FDA-approved companion diagnostic for a biomarker?

How is clinical utility best determined?

How can regulatory processes be harmonized to prevent duplication of effort but still allow innovation?

How should reimbursement considerations impact the development of new companion diagnostics?

E.H. Rubin is an employee of Merck & Co. No potential conflicts of interest were disclosed by the other authors.

Conception and design: E.H. Rubin, S.E. Bates

Writing, review, and/or revision of the manuscript: E.H. Rubin, J.D. Allen, J.A. Nowak, S.E. Bates

The authors thank Mark Raffeld for advice and review of the article and Robert Robey for editorial assistance.