The development of clinically useful molecular diagnostics requires validation of clinical assay performance and achievement of clinical qualification in clinical trials. As discussed elsewhere in this Focus section on molecular diagnostics, validation of assay performance must be rigorous, especially when the assay will be used to guide treatment decisions. Here we review some of the problems associated with assay development, especially for academic investigators. These include lack of expertise and resources for analytical validation, lack of experience in designing projects for a specific clinical use, lack of specimens from appropriate patient groups, and lack of access to Clinical Laboratory Improvement Amendments–certified laboratories. In addition, financial support for assay validation has lagged behind financial support for marker discovery or drug development, even though the molecular diagnostic may be considered necessary for the successful use of the companion therapeutic. The National Cancer Institute supports a large number of clinical trials and a significant effort in drug development. In order to address some of these barriers for predictive and prognostic assays that will be used in clinical trials to select patients for a particular treatment, stratify patients into molecularly defined subgroups, or choose between treatments for molecularly defined tumors, the National Cancer Institute has begun a pilot program designed to lessen barriers to the development of validated prognostic and predictive assays. Clin Cancer Res; 18(6); 1531–9. ©2012 AACR.

Despite enthusiasm on the part of patients, providers, and clinicians regarding the use of predictive and prognostic biomarkers, to date few assays have met the standards needed to convince the clinical community that they can be used to guide treatment decisions that will improve outcomes (1–5). In 2008, the President's Office of Science and Technology Policy (6) recommended that the National Institutes of Health “develop a funding program for academic/industry collaborative projects addressing biomarker standardization, statistical methods, and other aspects of study design necessary for validating the clinical utility of molecular diagnostics based on genomic correlations with disease characteristics,” and that the U.S. Food and Drug Administration (FDA) develop “standards for study design and product performance with regard to regulatory review of new diagnostic products” as well as a “regulatory approach to codevelopment of diagnostics and therapeutics.” These statements were strongly supported by a combined effort of the FDA, American Association for Cancer Research, and National Cancer Institute (NCI), known as the Cancer Biomarkers Collaborative [CBC (7)]. The CBC supported >27 recommendations covering biospecimens, reference standards, analytical performance, standardization and harmonization, bioinformatics, collaboration and data sharing, regulatory issues, policy, and education. But what are the major issues that prevent wider use of molecular diagnostics in cancer patients? Recent reviews have defined several issues related to the complexity and heterogeneity of cancer, lack of appropriate tumor specimens, bias inherent in the assay platform or analysis, study design issues, analysis and interpretation of results, lack of appropriate controls or standards applicable to complex assays, and technical validation of assays (8). Elsewhere in this Focus issue, Poste and colleagues (9) and others (10, 11) describe aspects of this problem by defining the different types of markers and their uses in clinical trials, as well as differences between the identification of markers during discovery and clinical assay development. In addition, Schilsky and colleagues (10) and others (11, 12) describe the use of integral markers that are essential for the performance of a trial, such as identification of patients as candidates for a therapeutic, risk stratification, modification of the dose of a therapeutic, or some other aspect of medical decision-making. The use of such integral markers requires validation of the assay's analytical performance so that the assay can be performed in a laboratory accredited under the Clinical Laboratory Improvement Amendments (CLIA) of 1988. In this article we review the background of analytical validation of a clinical assay, discuss why a complete validation of most molecular diagnostic assays is currently not possible, and describe a new NCI pilot program that is designed to help predictive and prognostic assays move from research to clinical readiness.

The development of a prognostic or predictive assay that can guide treatment for cancer patients is usually an iterative process that begins with the discovery of a molecular characteristic or signature and preliminary correlation with a clinical outcome. Subsequent steps involve optimization of the assay (including technical or analytical validation of the assay), clinical validation (clinical qualification) or evidence that the result of the analytically validated assay correlates with the clinical outcome of interest, and assessment of clinical utility for the intended use [i.e., does use of the assay result in a better outcome than standard methods or treatment? (Fig. 1)].

Figure 1.

Development of a prognostic or predictive assay for use in a clinical trial in which clinical utility will be evaluated. The figure depicts a potential path from discovery to clinical qualification of an assay.

Figure 1.

Development of a prognostic or predictive assay for use in a clinical trial in which clinical utility will be evaluated. The figure depicts a potential path from discovery to clinical qualification of an assay.

Close modal

The pharmaceutical industry has incorporated markers into its workflow for drug development (11, 12). However, at a workshop in 2003 (13), members of the American Association of Pharmaceutical Scientists, the Ligand Binding Assay Bioanalytical Focus Group, and the U.S. Clinical Ligand Society suggested that analytical validation of assays for markers should not follow the same process employed for bioanalytic assays commonly used by industry. Instead, the first step in assay development should be to define the intended clinical use for the result of the assay. For predictive and prognostic assays, most clinicians consider the intended use to reflect how the marker will assist medical decision-making in a specific clinical situation. The workshop participants suggested that, for the clinical laboratory scientist, a “useful starting point in determining the direction of a biomarker assay validation is to consider the assay to be used and the data type that the assay will generate” (13). Their report defined several different types of assays depending on the type of data obtained. Definitive quantitative assays are those in which the result is a continuous number expressed using a definitive, approved, or certified reference standard. An example is human insulin. Relative quantitative assays are those in which the output is a continuous number but the reference standard is not well characterized or fully representative of the endogenous biomarker. This category constitutes the great majority of clinical biomarkers for which the reference standard is not approved or certified by a regulatory body but is added to the matrix of the sample in which the clinical assay is to be performed. This “spike in” can be affected by matrix components that interfere with detection of the analyte, and hence lead to only relative accuracy. Quasi-quantitative assays are defined as those that also have an uncertified reference standard but are expressed in relation to a baseline characteristic of the sample. A fourth type of assay, which is very common in clinical practice, is the qualitative assay, in which the results are presented in categorical terms–either as numbers (ordinal) or in non-numeric (nominal) form. For example, in the immunohistochemistry (IHC) assay, results are usually expressed as low, medium, or high, or as 1+ to 3+. These assays generally lack calibrators but may have standards for the different categorical values that are usually not certified by a regulatory body. This report (13) formed the foundation upon which the development of fit-for-purpose markers was initially based.

Subsequently, Lee and colleagues (14) expanded this report to describe in greater detail what is meant by “fit-for-purpose,” which generally means ensuring that the assay performance meets reasonable expectations for its intended use. Lee and colleagues (14) outlined the steps necessary for an iterative approach for assay improvement that focuses on the intended use of the assay, and method development and validation intended to meet that purpose. They focused on assays for biomarkers as a component of drug development, as opposed to the development of a clinical diagnostic. As a result, the intended use becomes critical and requires one to focus on developing an assay that is validated within the matrix expected for the intended use, as well as considerations of analyte stability rather than the stability of the reagents used in the assay. Another major difference is that a diagnostic performed for medical decision-making must be performed in a CLIA-certified laboratory, following standards provided by the Clinical Laboratory Standards Institute. Lee and colleagues (14) and others (15–19) also set out guidelines for the number of replicates needed to validate the performance of molecular diagnostic assays, as well as such considerations as the linearity of assay response, dynamic range, limits of detection, analyte stability within the intended matrix, and intra- and interlaboratory coefficient of variability (CV; Box 1). Although several of these authors prescribed how many replicates should be performed and the limits of allowable variability, these recommendations may need to be altered according to the type of assay and marker to be developed. In a previous Focus section, Chau and colleagues (16) recommended that validation of quantitative assays should include “at least five different concentrations of VS [validation samples] analyzed in duplicate on at least six different runs during the prestudy validation because quantitative biomarker assays often exhibit nonlinear calibration curves.” The reproducibility of an assay is generally measured by the % CV, which is defined as the standard deviation divided by the mean of the assay result expressed as a percent. Chau and colleagues (16) suggested that the % CV should be <25%; however, many clinical laboratory scientists will only accept much lower % CVs. The reproducibility of qualitative assays such as IHC is a distinct problem in that it is difficult to measure variation. For IHC assays, reproducibility is generally measured in terms of the κ statistic (22) and percent agreement (23) among different observers. The ideal level of agreement or concordance in such qualitative assays is unclear, although a level of agreement of ≥85% is considered to be acceptable.

Box 1. Typical questions about the analytical performance of molecular diagnostic assays to be considered in assay development
  1. What is the dynamic range of the assay (units)?

  2. What are the usable limits of detection for the assay (units)?

  3. Is the assay linear† in the usable range?

  4. How stable is the analyte within its matrix (<7 days, 7–14 days, 15–30 days or longer, and under what storage conditions)?

  5. What is the accuracy for detecting the analyte or alterations (mutations) in the analyte within the matrix?

  6. What are the sensitivity and specificity of the assay?

  7. What is the intralaboratory reproducibility (% CV)?

  8. What is the interlaboratory reproducibility (% CV, same specimens)?

  • This information is adapted from the templates for IHC and ISH available at the Cancer Diagnosis Program (20) website. The information in this box is not intended to provide complete information for analyzing the analytical performance of all clinical assays; it only lists a few characteristics that need to be considered during assay development.

  • †Linearity is often important in enzyme-linked immunosorbent assays and other assays in which a standard curve is made and the amount of analyte is calculated from that standard curve (21).

An example illustrates this problem. The Eastern Cooperative Oncology Group (ECOG) led a randomized phase III stage II colon carcinoma adjuvant trial in which the 18q loss of heterozygosity (LOH) assay was an integral marker used to assign adjuvant therapy. The assay was based on the original description of 18q LOH by Jen and colleagues (24), who suggested that loss of 18q/DCC is associated with a poor prognosis (25) that might benefit from adjuvant therapy. Because the assay was not validated before initiation of the trial, the Cancer Diagnosis Program performed an interlaboratory validation study (26) that involved the reference laboratories for ECOG, the North Central Cooperative Treatment Group, and the Cancer and Leukemia Group B, all of which participated in this and other related phase III trials in colon carcinoma. All 3 laboratories used a single standard operating procedure, common Taqman primers and probes for the PCR assays, and similar ABI Prism equipment for this assay, which was performed in formalin-fixed paraffin-embedded (FFPE) diagnostic samples. All of these elements are considered to ensure concordance among observers. LOH is reported as a categorical variable, and interobserver analysis was performed with DNA extracts from 50 pairs of tumor (colon carcinomas) and normal tissue, tested by the 3 laboratories. The interobserver agreement was only 73% among the 3 laboratories, which is surprisingly low given the common DNA extracts as a starting point. Of interest, a second phase of concordance testing was performed when one of the collaborators, Dr. Thibodeau, suggested that the concordance could be improved by the individual reference pathologists using their own macrodissection techniques to prepare the DNA extracts. When this was done in a second set of >100 samples, the interlaboratory agreement improved to 92% (26). This experience suggests that even when almost all of the potential variables are standardized, there may still be variables that can affect the results, e.g., individual laboratory optimization of tumor content to increase enrichment of tumor DNA to improve detection of LOH.

Analytical validation of an assay involves assessments of its accuracy, precision, specificity (ability to detect the analyte in a complex matrix), and sensitivity. It is often difficult to validate novel biomarker assays due to the lack of a gold standard assay, and there may not be any approved reference standard that can be used to assist validation of the assay method. In addition, Chau and colleagues (16) remarked that only the definitive quantitative assay can be truly accurate because only with this type of method can the exact amount of the endogenous marker be measured, because there is a gold standard, certified quality control that is suited for the matrix of the clinical assay. All other categories of assays only estimate the accuracy of a marker with standards provided by the assay developer, but not at a standard that is approved by either the U.S. Pharmacopeia (USP) or the FDA (27). A reference material is defined by the National Institute of Standards and Technology [NIST (28)] as a material that is homogeneous and stable enough to be fit for use in a measurement process. A certified reference material is a type of “reference material characterized by a metrologically valid procedure for one or more specified properties, accompanied by a certificate that provides the value of the specified property, its associated uncertainty, and a statement of metrological traceability”. A NIST standard reference material is a “certified reference material issued by NIST that also meets additional NIST-specific certification criteria and is issued with a certificate or certificate of analysis that reports the results of its characterizations and provides information regarding the appropriate use(s) of the material.” The USP and the National Formulary list certified reference materials that meet the standards of the International Organization for Standardization (ISO) for reference materials in their ISO 17025 (29) and ISO Guide 34 (30) for reference material producers. Standard or certified reference materials are critical for interlaboratory harmonization efforts because they can serve as a common reference standard. Clearly, complex endogenous genes, proteins, and lipids can undergo many alterations during malignant transformation, and present a far greater challenge to the development of certified reference materials than the development of certified reference materials for analytes such as sodium, other electrolytes, and even enzymes for which clinical chemistry tests are routine and accepted. However, until this challenge is met, it will be difficult for both the clinical laboratory and the clinician to accept the accuracy of any laboratory-developed test that is not approved or cleared by the FDA or accepted by major healthcare insurance payers.

The Centers for Disease Control and Prevention (CDC), through its initiation and support of the GeT-RM project (31), has made reference materials publicly available through the Coriell Repository to support genetic tests. In addition to a number of reference materials for cystic fibrosis, Huntington disease, fragile X syndrome, and several genetic conditions with relatively high prevalence in the Ashkenazi Jewish population (32–35), it has also provided information for several pharmacogenetic markers, including members of the CYP450 gene family, VKORC1, and UGT1A1 (36). The GeT-RM is now providing reference materials for both genomic and somatic mutations that occur in hematologic and solid malignancies (37, 38).

An example illustrates the importance of certified reference materials. HER-2 testing in breast and other cancers can identify patients who might benefit from treatment that targets the gene (39). HER-2 expression is identified by IHC for protein, or FISH for gene amplification. However, studies have shown considerable discordance in assay results even between CLIA-certified laboratories testing the same samples (40, 41). The College of American Pathology/American Society of Clinical Oncology issued guidelines in 2007 (42) that cover all aspects of pre- and postanalytic procedures for the qualitative IHC and quasi-quantitative FISH assays, and recommended proficiency testing twice a year with standard controls contained in each assay run. Because the NIST does not provide a standard reference material for HER-2 or Her-2 in its catalog (43), and the USP does not have a HER-2 or Her-2 analyte in its compendium (44), there is no certified traceable standard for the IHC and FISH assays. As a result, the accuracy of the tests can only be estimated, although the precision of the assay can be measured. Similar situations exist with other FDA-approved or -cleared or commonly accepted diagnostics, such as ALK translocation. This will become an increasing problem as clinical trials successfully demonstrate the clinical utility of an integral marker, because demands for analyzing the marker in community hospitals in the same manner as it was tested in the trial will also increase. Unfortunately, it is very difficult to ensure that clinical laboratories will be able to provide the assay as it was performed in a clinical trial without standard or certified reference materials that trace back to specimens in the trial. The uncertainty raised in payers, clinicians, and patients makes this an important issue. In the future, with appropriate support from government agencies and the pharmaceutical industry, it may be possible to ensure that samples from pivotal trials are preserved to develop standard or certified reference materials for subsequent assay development.

Another major problem for scientists in academia or small business is the lack of funding to develop and validate clinical predictive and prognostic tests. The funding that supports the development of new clinical and biological discoveries vastly exceeds that which supports clinical assay development and validation. In fiscal year 2011, 0.8% of the total NCI budget was spent on development of clinically useful assays, as compared with 5.3% for identification of biomarkers (45). The actual difference between discovery and clinical assay development support may be considerably greater because it is difficult to determine how much of the molecular diagnostics portfolio is designated for development of assays within a CLIA-certified laboratory. The other major source of support for development and validation of clinical assays is diagnostics companies. These companies support the development of diagnostics for use in clinical cancer care, and/or collaborate with the pharmaceutical industry to develop companion diagnostics. There has been a recent increase in this activity (46–49), but this source of support is generally not open to academic investigators or even many small biotechnology firms.

Several larger academic cancer centers (e.g., the MD Anderson Cancer Center, Memorial Sloan Kettering Cancer Center, Dana-Farber Cancer Institute with the Massachusetts General Hospital and the Brigham and Women's Hospital, Vanderbilt-Ingram Cancer Center, Moffitt Cancer Center, and the Washington University Genomic Center) recently began to genotype patients who come to their institutions as a means of identifying patients who may be candidates for trials involving targeted therapy (50). These centers have resources to bring promising assays from the research laboratory to the CLIA-certified laboratory. The costs for these efforts are shared by the institution and the sponsors of the drugs that are under development. However, investigators at other institutions or small companies do not have access to these resources and may find it a difficult and bewildering process to convert their biologically important discovery into a clinically useful diagnostic. These investigators may have obtained biologically important findings, but may not be able to develop them into clinically useful tools unless they can turn to a clinical assay developer.

In response to the need for technically validated assays to be incorporated into NCI-supported clinical trials as integral assays, the NCI developed the Clinical Assay Development Program (CADP) and the Molecular Characterization Clinical Assay Development Laboratory (MC-CADL), The CADP is designed to overcome some of the hurdles faced by researchers during the analytic and clinical validation of a biomarker assay (see Table 1 for relevant definitions of terms). Integral assays are those that must be performed on every patient entered in the trial before the trial can proceed (10). These assays are used for patient selection (eligibility), stratification, and treatment assignment. It is especially important to perform analytical and clinical validation studies before using an assay in a clinical trial (6–10). Whether an Investigational Device Exemption (IDE) from the FDA is needed depends on the risk the assay poses to the patient as a result of its planned clinical use [see Meshinchi and colleagues (51) in this Focus issue]. Preparation for use of the assay in a CLIA-certified laboratory as a laboratory-developed test or for submission for an IDE will require careful assessments of accuracy, precision, specificity, and sensitivity. The goals of the CADP are to (i) provide the needed services to evaluate, optimize, and validate the analytical performance and clinical validity of promising assays that will be used in a clinical trial to demonstrate clinical utility; (ii) educate and engage researchers and others in a process of assay validation that leads to robust, clinically useful assays to guide cancer treatment; and (iii) develop and evaluate standard operating procedures, controls, and calibrators for clinical molecular characterization of malignancies. CADP is not a grants program; rather, it is a resource for investigators who want assistance in converting their research assay into an assay that is performed in a CLIA-certified laboratory. Investigators apply to the program through an electronic submission process described on the CADP website (52). Investigators with markers that have a defined clinical use and biologic rationale, and a research assay that functions in human tissues but whose performance needs to be improved and validated for use in a clinical laboratory are excellent candidates for the CADP. Assays that are further along in development but may need to be transferred to a CLIA environment, or may require additional refinement or a platform transfer, are also suitable for CADP. If approved, investigators will be provided access to the NCI's full suite of assay development and validation resources. Appropriate attention to the validation of these assays will increase the probability for success of the clinical trials in which they are employed, resulting in improved patient management.

Table 1.

Definitions

TermMeaning
Analytical validity How accurately an assay detects the analyte of interest 
Clinical validity How well the test relates to the clinical outcome of interest 
Clinical utility Whether the results of the test provide information that can contribute to and improve current optimal management of the patient's disease 
Integral marker (or assay) A test that must be performed before the trial can proceed; integral studies are inherent to the design of the trial from the onset and must be performed in real time for the conduct of the trial 
Integrated marker (or assay) A test that is performed on all patients or a predefined subset of patients in a clinical trial, but will not be used to direct treatment for that clinical trial 
TermMeaning
Analytical validity How accurately an assay detects the analyte of interest 
Clinical validity How well the test relates to the clinical outcome of interest 
Clinical utility Whether the results of the test provide information that can contribute to and improve current optimal management of the patient's disease 
Integral marker (or assay) A test that must be performed before the trial can proceed; integral studies are inherent to the design of the trial from the onset and must be performed in real time for the conduct of the trial 
Integrated marker (or assay) A test that is performed on all patients or a predefined subset of patients in a clinical trial, but will not be used to direct treatment for that clinical trial 

The terms analytical validity, clinical validity, and clinical utility are defined as in ref. 10.

Clinical Assay Development Network

The Clinical Assay Development Network (CADN) comprises 5 academic and 3 commercial (or reference) CLIA laboratories that are designed to optimize and analytically validate complex assays. These laboratories were selected competitively for their proven ability to convert research assays to clinical assays. Once an assay is selected for development by the CADP, each CADN laboratory bids according to its ability to develop that assay. After the choice of laboratory (or laboratories) is made, the investigator/assay submitter works with the CADP management team and members of the CADN laboratory to develop the assay, which is then returned to the assay submitter. The CADN is intended to be very interactive with the original investigator and may continue to collaborate with the investigator once the assay has been developed and validated. This may be very beneficial to the investigator who wants to use the assay in clinical trials but lacks the resources to do so directly in his or her institution. Available assay technologies in the CADN include IHC, FISH, chromogenic in situ hybridization, reverse transcriptase PCR, genomic sequencing, and LC/MS. Each assay to be developed will be assigned a project management team, which includes the assay submitter. The project management team is responsible for devising a timeline, milestones, and go/no-go decision points for the project.

Specimen Retrieval System

The Specimen Retrieval System (SRS) is designed to provide needed specimens for analytic and clinical validation studies depending on the assay to be supported. It is anticipated that most tissue-based assays will use FFPE tissues, because they are the predominant form of diagnostic material currently available in U.S. hospitals (53, 54). The CADP has contracted with the Health Maintenance Organization, Kaiser Permanente Northwest, to provide such samples. This organization was selected because it participates in the Cancer Research Network of the Health Maintenance Organization Research Network and can provide FFPE tissues from pathology archives that use standard handling and storage procedures. Tissues are obtained from patients who have undergone surgery for a malignant disease, after appropriate Institutional Review Board approval has been obtained. In typical surgeries, more FFPE blocks are created than are needed by institutions for diagnosis. The excess blocks are deidentified and provided to the CADP, and the providing institutions retain the diagnostic blocks.

The specimens are accompanied by clinical data that is routinely abstracted by Tumor Registries [part of the quality management system of the Commission on Cancer (55)]. These clinical data are derived from pathology reports, clinic visits, oncology visits, and pharmacy records. These records are deidentified by scrubber software derived from an earlier NCI-supported program for the retrieval of FFPE specimens from diagnostic archives for research purposes (56). This software has been modified by investigators from Harvard working with clinicians at Kaiser Permanente Northwest (57) and is now >99% effective in removing all personal health identifiers and information. Records are still manually checked so that when specimens are forwarded to the CADP, the specimen and the patient's clinical history are deidentified. This enables the CADP to access specimens that fit the clinical context for assay development, as well as the clinical outcome information that is necessary to begin the process of qualifying assay results by correlating assay results with clinical outcome. The CADP may also contract with other sites as necessary for similar deidentified tissues (e.g., frozen tissue, blood, and plasma) if they are required for a particular assay.

Other services

The CADP provides advice on assay development, including potential development strategies, and focuses on an appropriate and feasible intended use for the assay, through the involvement of the personnel in the Cancer Diagnosis Program. Statistical advice is also available through the NCI's Biomedical Research Branch. In addition, the potential applicant may be referred to other potential collaborators (e.g., other academic investigators, Cooperative Groups, and other tissue resources) with which they can negotiate their own collaboration.

Procedures for access to the CADP

Academic researchers, commercial entities, and government researchers may apply electronically to the CADP for needed validation services (50). Applicants are asked to describe the potential clinical relevance of the proposed assay; must stipulate a single, well-defined intended clinical use; and must have a prototype assay that has been tested in relevant tumor tissue, as well as an estimate of the prevalence of the abnormality the assay will detect. Detailed data on additional previous work (if available) and relevant literature may be submitted as appendices. Assays that have progressed further toward validation are also encouraged. Applications are reviewed by a Special Emphasis Panel that is constituted under the Federal Advisory Committee Act and is composed of experts external to the NCI. Members of this committee include academic experts in disease, molecular biology, and diagnostics; members of industry with similar expertise; and patient advocates. Applications are evaluated on the basis of scientific rationale, feasibility, potential clinical impact, and plans for assessment of clinical utility (see Table 2).

Table 2.

Evaluation criteria for CADP applications

Scientific merit (30%) Sound hypothesis that associates marker(s) for drug effects or state of disease with a known clinical outcome and is supported by current state of the field. Marker is expected to have clinical utility. 
Feasibility (30%) Highly technically feasible approach; assay platform and specimen requirements should allow straightforward implementation for clinical use. 
Impact/clinical need (30%) If successful, the proposed diagnostic is likely to change clinical practice and have a major impact on outcome of treatment. 
Path to clinical implementation (10%) The path to clinical implementation is clear. 
Scientific merit (30%) Sound hypothesis that associates marker(s) for drug effects or state of disease with a known clinical outcome and is supported by current state of the field. Marker is expected to have clinical utility. 
Feasibility (30%) Highly technically feasible approach; assay platform and specimen requirements should allow straightforward implementation for clinical use. 
Impact/clinical need (30%) If successful, the proposed diagnostic is likely to change clinical practice and have a major impact on outcome of treatment. 
Path to clinical implementation (10%) The path to clinical implementation is clear. 

Applications are then evaluated internally to ensure that the necessary resources can indeed be obtained by the NCI for the proposed assay or test. Unsuccessful applicants may resubmit the same project once and/or receive additional input regarding their efforts from CADP staff. Applications are accepted 3 times a year (see Fig. 2).

Figure 2.

Overview of the workflow of the CADP. The PI (assay submitter) with a research assay applies to the CADP (http://cadp.cancer.gov). If the assay is selected, the PI and the assay are assigned to one or more CADN laboratories (small colored circles), which then work with the PI (submitter) and the CADP management team. Specimens are provided as needed and considered fit-for-purpose by the SRS. If appropriate, the MC-CADL may assist with the development of the assay. After the assay has been developed and its analytical performance has been validated, the clinical assay is returned to the submitter from the CADP and is considered ready for use in clinical trials. It should be noted that it is possible for the CADN laboratories to assist in the performance of the assay in clinical trials, but this arrangement would be made by the assay submitter, not through the CADP. PI, principal investigator.

Figure 2.

Overview of the workflow of the CADP. The PI (assay submitter) with a research assay applies to the CADP (http://cadp.cancer.gov). If the assay is selected, the PI and the assay are assigned to one or more CADN laboratories (small colored circles), which then work with the PI (submitter) and the CADP management team. Specimens are provided as needed and considered fit-for-purpose by the SRS. If appropriate, the MC-CADL may assist with the development of the assay. After the assay has been developed and its analytical performance has been validated, the clinical assay is returned to the submitter from the CADP and is considered ready for use in clinical trials. It should be noted that it is possible for the CADN laboratories to assist in the performance of the assay in clinical trials, but this arrangement would be made by the assay submitter, not through the CADP. PI, principal investigator.

Close modal

The MC-CADL is operated by SAIC-Frederick under contract to the NCI. Established to provide a means of translating genome discoveries into clinical applications, the MC-CADL will focus on newer genomic technologies such as whole transcriptome gene expression and next-generation sequencing. Of importance, the MC-CADL will apply these technologies using well-defined, robust protocols, and appropriate standards and controls to ensure reproducible results. The goal is to discern the molecular properties of a cancer that would explain a response (or lack of response) to the investigational targeted therapy. It will assist with early phases of assay development and transition to clinical laboratory readiness and support NCI-sponsored clinical trials within and outside the institute by developing standards and calibrators that can be used to compare assays among different laboratories or across platforms and can be shared with others. In this capacity, the MC-CADL will also work with the FDA, NIST, and other regulatory bodies that may develop standard and certified reference materials.

The MC-CADL will use FFPE specimens because they are the dominant form of tissues available throughout the country, and will facilitate adoption of developed assays. It will collaborate with the NCI's Office of Biorepositories and Biospecimen Research [OBBR (58)] and the Cancer Genome Atlas to study characteristics of specimens that affect the molecular characterization of malignancies, particularly in massively parallel sequencing platforms. As part of the CADP, successful efforts by the MC-CADL can be transferred into the CADN and/or the MC-CADL can serve as a reference laboratory for the CADN and the investigator.

Many challenges remain in the effort to incorporate molecular diagnostics into routine clinical practice. In addition to the performance of the assay, the association of the result with a clinical endpoint, and the demonstration that such an association will be clinically useful, there are the other concerns highlighted by the Office of Science and Technology Policy (6) and the CBC (7) that involve lack of appropriate biospecimens, standardization and harmonization, bioinformatics, collaboration and data sharing, regulatory issues, policy, and education. It is difficult to address all of these issues at once. However, the CADP addresses several of them because it enables independent academic and small-company investigators to develop clinically rigorous assays that then can enter clinical trials for clinical qualification and ultimately assessment of clinical utility. In addition to providing assistance with assay validation, members of the CADN may be able to assist investigators with supporting clinical assays in clinical trials. A major unmet need recognized by the Office of Science and Technology Policy (6) and the CBC (7) is the need for reference standards as a means to assess the accuracy of many molecular diagnostic assays. The MC-CADL is also beginning to address the reference issue through the creation of resources that may be able to be certified and distributed as standard or certified reference materials. It is hoped that as we move forward in the next few years, certified reference materials will become available so that investigators can actually measure the accuracy of tests, and correlation of results across different laboratories will be facilitated.

No potential conflicts of interest were disclosed.

1.
Hayes
DF
,
Bast
RC
,
Desch
CE
,
Fritsche
H
 Jr
,
Kemeny
NE
,
Jessup
JM
, et al
Tumor marker utility grading system: a framework to evaluate clinical utility of tumor markers
.
J Natl Cancer Inst
1996
;
88
:
1456
66
.
2.
Locker
GY
,
Hamilton
S
,
Harris
J
,
Jessup
JM
,
Kemeny
N
,
Macdonald
JS
, et al
American Society of Clinical Oncology Tumor Markers Expert Panel
. 
ASCO 2006 update of recommendations for the use of tumor markers in gastrointestinal cancer
.
J Clin Oncol
2006
;
24
:
5313
27
.
3.
Harris
L
,
Fritsche
H
,
Mennel
R
,
Norton
L
,
Ravdin
P
,
Taube
S
, et al
American Society of Clinical Oncology 2007 update of recommendations for the use of tumor markers in breast cancer
.
J Clin Oncol
2007
;
25
:
5287
312
.
4.
Allegra
CJ
,
Jessup
JM
,
Somerfield
MR
,
Hamilton
SR
,
Hammond
EH
,
Hayes
DF
, et al
American Society of Clinical Oncology provisional clinical opinion: testing for KRAS gene mutations in patients with metastatic colorectal carcinoma to predict response to anti-epidermal growth factor receptor monoclonal antibody therapy
.
J Clin Oncol
2009
;
27
:
2091
6
.
5.
Keedy
VL
,
Temin
S
,
Somerfield
MR
,
Beasley
MB
,
Johnson
DH
,
McShane
LM
, et al
American Society of Clinical Oncology provisional clinical opinion: epidermal growth factor receptor (EGFR) Mutation testing for patients with advanced non-small-cell lung cancer considering first-line EGFR tyrosine kinase inhibitor therapy
.
J Clin Oncol
2011
;
29
:
2121
7
.
6.
The White House [homepage on the Internet]
. 
Priorities for Personalized Medicine
.
Report of the President's Council of Advisors on Science and Technology
.
[September 2008; cited 2011 Dec 12]. Available from:
http://www.whitehouse.gov/files/documents/ostp/PCAST/pcast_report_v2.pdf
7.
Khleif
SN
,
Doroshow
JH
,
Hait
WN
.
AACR-FDA-NCI Cancer Biomarkers Collaborative
. 
AACR-FDA-NCI Cancer Biomarkers Collaborative consensus report: advancing the use of biomarkers in cancer drug development
.
Clin Cancer Res
2010
;
16
:
3299
318
.
8.
Coyle
VM
,
Johnston
PG
. 
Genomic markers for decision making: what is preventing us from using markers?
Nat Rev Clin Oncol
2010
;
7
:
90
7
.
9.
Poste
G
,
Carbone
DP
,
Parkinson
DR
,
Verweij
J
,
Hewitt
SM
,
Jessup
JM
. 
Leveling the playing field: bringing development of biomarkers and molecular diagnostics up to the standards for drug development
.
Clin Cancer Res
2012
;
18
:
1515
23
.
10.
Schilsky
RL
,
Doroshow
JH
,
LeBlanc
M
,
Conley
BA
. 
Development and use of integral assays in clinical trials
.
Clin Cancer Res
2012
;
18
:
1540
6
.
11.
Carden
CP
,
Sarker
D
,
Postel-Vinay
S
,
Yap
TA
,
Attard
G
,
Banerji
U
, et al
Can molecular biomarker-based patient selection in Phase I trials accelerate anticancer drug development?
Drug Discov Today
2010
;
15
:
88
97
.
12.
Valentin
MA
,
Ma
S
,
Zhao
A
,
Legay
F
,
Avrameas
A
. 
Validation of immunoassay for protein biomarkers: bioanalytical study plan implementation to support pre-clinical and clinical studies
.
J Pharm Biomed Anal
2011
;
55
:
869
77
.
13.
Lee
JW
,
Weiner
RS
,
Sailstad
JM
,
Bowsher
RR
,
Knuth
DW
,
O'Brien
PJ
et al 
Method validation and measurement of biomarkers in nonclinical and clinical samples in drug development: a conference report
.
Pharm Res
2005
;
22
:
499
511
.
14.
Lee
JW
,
Devanarayan
V
,
Barrett
YC
,
Weiner
R
,
Allinson
J
,
Fountain
S
, et al
Fit-for-purpose method development and validation for successful biomarker measurement
.
Pharm Res
2006
;
23
:
312
28
.
15.
Cummings
J
,
Raynaud
F
,
Jones
L
,
Sugar
R
,
Dive
C
. 
Fit-for-purpose biomarker method validation for application in clinical trials of anticancer drugs
.
Br J Cancer
2010
;
103
:
1313
7
.
16.
Chau
CH
,
Rixe
O
,
McLeod
H
,
Figg
WD
. 
Validation of analytic methods for biomarkers used in drug development
.
Clin Cancer Res
2008
;
14
:
5967
76
.
17.
Wagner
JA
,
Williams
SA
,
Webster
CJ
. 
Biomarkers and surrogate end points for fit-for-purpose development and regulatory evaluation of new drugs
.
Clin Pharmacol Ther
2007
;
81
:
104
7
.
18.
Williams
SA
,
Slavin
DE
,
Wagner
JA
,
Webster
CJ
. 
A cost-effectiveness approach to the qualification and acceptance of biomarkers
.
Nat Rev Drug Discov
2006
;
5
:
897
902
.
19.
Lathia
CD
,
Amakye
D
,
Dai
W
,
Girman
C
,
Madani
S
,
Mayne
J
, et al
The value, qualification, and regulatory use of surrogate end points in drug development
.
Clin Pharmacol Ther
2009
;
86
:
32
43
.
20.
Cancer Diagnosis Program [homepage on the internet] [cited 2012 Feb 24]
. .
21.
Clinical Laboratory Standards Institute [homepage on the Internet]. Evaluation of the linearity of quantitative measurement procedures: a statistical approach; approved guideline. 2003; Vol 23, no. 16 [cited 2012 Feb 24]
.
Available from
: http://www.clsi.org/source/orders/free/ep6-a.pdf
22.
Mengel
M
,
von Wasielewski
R
,
Wiese
B
,
Rüdiger
T
,
Müller-Hermelink
HK
,
Kreipe
H
. 
Inter-laboratory and inter-observer reproducibility of immunohistochemical assessment of the Ki-67 labelling index in a large multi-centre trial
.
J Pathol
2002
;
198
:
292
9
.
23.
Garg
K
,
Broaddus
RR
,
Soslow
RA
,
Urbauer
DL
,
Levine
DA
,
Djordjevic
B
. 
Pathologic scoring of PTEN immunohistochemistry in endometrial carcinoma is highly reproducible
.
Int J Gynecol Pathol
2012
;
31
:
48
56
.
24.
Jen
J
,
Kim
H
,
Piantadosi
S
,
Liu
ZF
,
Levitt
RC
,
Sistonen
P
, et al
Allelic loss of chromosome 18q and prognosis in colorectal cancer
.
N Engl J Med
1994
;
331
:
213
21
.
25.
Shibata
D
,
Reale
MA
,
Lavin
P
,
Silverman
M
,
Fearon
ER
,
Steele
G
 Jr
, et al
The DCC protein and prognosis in colorectal cancer
.
N Engl J Med
1996
;
335
:
1727
32
.
26.
Jessup
JM
,
Dobbin
K
,
Hamilton
S
,
Thibodeau
S
,
Redston
M
,
Taube
S
, et al
Interlaboratory assay reproducibility study for loss of heterozygosity on chromosome 18 (18q LOH) in colon cancer
.
J Clin Oncol
27
:
15s
, 
2009
(
suppl; abstr 4052
).
27.
Food and Drug Administration [homepage on the Internet]
. 
FDA Device Advice [cited 2011 Sep 10]
.
Available from:
http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/default.htm
28.
National Institute of Standards and Technology [homepage on the Internet]
. 
Definitions [cited 2011 Sep 10]
.
Available from:
http://www.nist.gov/srm/definitions.cfm
29.
International Organization for Standardization [homepage on the Internet]
. 
ISO/IEC 17025:2005 [cited 2011 Sep 10]
.
Available from:
http://www.iso.org/iso/Catalogue_detail?csnumber=39883
30.
International Organization for Standardization [homepage on the Internet]
. 
ISO Guide 34:2009 [cited 2011 Sep 10]
.
Available from:
http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=50174
31.
Centers for Disease Control and Prevention [homepage on the Internet]
. 
Genetic Testing Reference Materials Coordination Program (GeT-RM)—Home. DLS - Laboratory Practice Evaluation and Genomics Branch [cited 2012 Feb 24].
Available from:
http://wwwn.cdc.gov/dls/genetics/rmmaterials
32.
Pratt
VM
,
Caggana
M
,
Bridges
C
,
Buller
AM
,
DiAntonio
L
,
Highsmith
WE
, et al
Development of genomic reference materials for cystic fibrosis genetic testing
.
J Mol Diagn
2009
;
11
:
186
93
.
33.
Kalman
L
,
Johnson
MA
,
Beck
J
,
Berry-Kravis
E
,
Buller
A
,
Casey
B
, et al
Development of genomic reference materials for Huntington disease genetic testing
.
Genet Med
2007
;
9
:
719
23
.
34.
Amos
Wilson J
,
Pratt
VM
,
Phansalkar
A
,
Muralidharan
K
,
Highsmith
WE
 Jr
,
Beck
JC
, et al
Fragile Xperts Working Group of the Association for Molecular Pathology Clinical Practice Committee
. 
Consensus characterization of 16 FMR1 reference materials: a consortium study
.
J Mol Diagn
2008
;
10
:
2
12
.
35.
Kalman
L
,
Wilson
JA
,
Buller
A
,
Dixon
J
,
Edelmann
L
,
Geller
L
, et al
Development of genomic DNA reference materials for genetic testing of disorders common in people of Ashkenazi Jewish descent
.
J Mol Diagn
2009
;
11
:
530
6
.
36.
Pratt
VM
,
Zehnbauer
B
,
Wilson
JA
,
Baak
R
,
Babic
N
,
Bettinotti
M
, et al
Characterization of 107 genomic DNA reference materials for CYP2D6, CYP2C19, CYP2C9, VKORC1, and UGT1A1: a GeT-RM and Association for Molecular Pathology collaborative project
.
J Mol Diagn
2010
;
12
:
835
46
.
37.
Barker
SD
,
Bale
S
,
Booker
J
,
Buller
A
,
Das
S
,
Friedman
K
, et al
Development and characterization of reference materials for MTHFR, SERPINA1, RET, BRCA1, and BRCA2 genetic testing
.
J Mol Diagn
2009
;
11
:
553
61
.
38.
Centers for Disease Control and Prevention [homepage on the Internet]
. 
Genetic Testing Reference Materials Coordination Program (GeT-RM) - Home [cited 2012 Feb 24]
.
Available from:
http://www.cdc.gov/dls/genetics/rmmaterials/default.aspx
39.
Slamon
DJ
,
Leyland-Jones
B
,
Shak
S
,
Fuchs
H
,
Paton
V
,
Bajamonde
A
, et al
Use of chemotherapy plus a monoclonal antibody against HER2 for metastatic breast cancer that overexpresses HER2
.
N Engl J Med
2001
;
344
:
783
92
.
40.
Paik
S
,
Bryant
J
,
Tan-Chiu
E
,
Romond
E
,
Hiller
W
,
Park
K
, et al
Real-world performance of HER2 testing—National Surgical Adjuvant Breast and Bowel Project experience
.
J Natl Cancer Inst
2002
;
94
:
852
4
.
41.
Roche
PC
,
Suman
VJ
,
Jenkins
RB
,
Davidson
NE
,
Martino
S
,
Kaufman
PA
, et al
Concordance between local and central laboratory HER2 testing in the breast intergroup trial N9831
.
J Natl Cancer Inst
2002
;
94
:
855
7
.
42.
Wolff
AC
,
Hammond
ME
,
Schwartz
JN
,
Hagerty
KL
,
Allred
DC
,
Cote
RJ
, et al
American Society of Clinical Oncology/College of American Pathologists
. 
American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer
.
Arch Pathol Lab Med
2007
;
131
:
18
43
.
43.
National Institute of Standards and Technology [homepage on the Internet]
. 
Standard Reference Materials Catalog [cited 2012 Feb 24]
.
Available from:
http://www.nist.gov/srm/upload/SRM-2011-Cat-v2_ All.pdf
44.
U.S. Pharmacopeial Convention [homepage on the Internet]
. 
Standards [cited 2012 Feb 24]
.
Available from:
http://www.usp.org/referenceStandards/
45.
This is based on a Query, View and Report (QVR
, http://era.nih.gov/nih_and_grantor_agencies/other/query_view_and_report.cfm
) search on 12/12/2011 for active grants on QVR using the terms ‘assay development’ and ‘molecular diagnostics’ compared to grants found with ‘biomarkers’ alone.
46.
Wilson
C
,
Schulz
S
,
Waldman
SA
. 
Biomarker development, commercialization, and regulation: individualization of medicine lost in translation
.
Clin Pharmacol Ther
2007
;
81
:
153
5
.
47.
Oakman
C
,
Bessi
S
,
Zafarana
E
,
Galardi
F
,
Biganzoli
L
,
Di Leo
A
. 
Recent advances in systemic therapy: new diagnostics and biological predictors of outcome in early breast cancer
.
Breast Cancer Res
2009
;
11
:
205
.
48.
Food and Drug Administration [homepage on the Internet]
. 
Device approvals and clearances, recently approved devices [cited 2012 Feb 24]
.
Available from:
http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/DeviceApprovalsandClearances/Recently-ApprovedDevices/default.htm
49.
Food and Drug Administration [homepage on the Internet]
. 
In vitro diagnostics medical devices home page [cited 2012 Feb 24]
.
Available from:
http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedures/InVitroDiagnostics/default.htm.
50.
Kaiser
J
. 
Cancer research. Looking for a target on every tumor
.
Science
2009
;
326
:
218
20
.
51.
Meshinchi
S
,
Hunger
SP
,
Aplenc
R
,
Adamson
PC
,
Jessup
JM
. 
Lessons learned from the Investigational Device Exemption review of Children's Oncology Group Trial AAML1031
.
Clin Cancer Res
2012
;
18
:
1547
54
.
52.
National Cancer Institute [homepage on the Internet]
. 
Clinical Assay Development Program (CADP) [cited 2012 Feb 24]
.
Available from:
http://cadp.cancer.gov/
53.
Signoretti
S
,
Bratslavsky
G
,
Waldman
FM
,
Reuter
VE
,
Haaga
J
,
Merino
M
, et al
Tissue-based research in kidney cancer: current challenges and future directions
.
Clin Cancer Res
2008
;
14
:
3699
705
.
54.
Lim
MD
,
Dickherber
A
,
Compton
CC
. 
Before you analyze a human specimen, think quality, variability, and bias
.
Anal Chem
2011
;
83
:
8
13
.
55.
Stewart
AK
,
McNamara
E
,
Gay
EG
,
Banasiak
J
,
Winchester
DP
. 
The Rapid Quality Reporting System—a new quality of care tool for CoC-accredited cancer programs
.
J Registry Manag
2011
;
38
:
61
3
.
56.
Patel
AA
,
Gupta
D
,
Seligson
D
,
Hattab
EM
,
Balis
UJ
,
Ulbright
TM
, et al
Shared Pathology Informatics Network
. 
Availability and quality of paraffin blocks identified in pathology archives: a multi-institutional study by the Shared Pathology Informatics Network (SPIN)
.
BMC Cancer
2007
;
7
:
37
.
57.
Glass
A
,
Taube
S
,
McMurry
A
,
Eddy
C
,
La Chance
PA
,
McShane
L
, et al
C-C4-02: Using a natural language processor to remove all elements of personal health information (PHI) to de-identify clinical annotations for the Specimen Retrieval System (SRS)
.
Clin Med Res
2011
;
9
:
170
.
58.
National Cancer Institute [homepage on the Internet]
. 
Office of Biorepositories and Biospecimen Research [cited 2012 Feb 24]
.
Available from:
http://biospecimens.cancer.gov/default.asp