Purpose:

Immunoprofiling to identify biomarkers and integration with clinical trial outcomes are critical to improving immunotherapy approaches for patients with cancer. However, the translational potential of individual studies is often limited by small sample size of trials and the complexity of immuno-oncology biomarkers. Variability in assay performance further limits comparison and interpretation of data across studies and laboratories.

Experimental Design:

To enable a systematic approach to biomarker identification and correlation with clinical outcome across trials, the Cancer Immune Monitoring and Analysis Centers and Cancer Immunologic Data Commons (CIMAC-CIDC) Network was established through support of the Cancer MoonshotSM Initiative of the National Cancer Institute (NCI) and the Partnership for Accelerating Cancer Therapies (PACT) with industry partners via the Foundation for the NIH.

Results:

The CIMAC-CIDC Network is composed of four academic centers with multidisciplinary expertise in cancer immunotherapy that perform validated and harmonized assays for immunoprofiling and conduct correlative analyses. A data coordinating center (CIDC) provides the computational expertise and informatics platforms for the storage, integration, and analysis of biomarker and clinical data.

Conclusions:

This overview highlights strategies for assay harmonization to enable cross-trial and cross-site data analysis and describes key elements for establishing a network to enhance immuno-oncology biomarker development. These include an operational infrastructure, validation and harmonization of core immunoprofiling assays, platforms for data ingestion and integration, and access to specimens from clinical trials. Published in the same volume are reports of harmonization for core analyses: whole-exome sequencing, RNA sequencing, cytometry by time of flight, and IHC/immunofluorescence.

Translational Relevance

The Cancer Immune Monitoring and Analysis Centers and Cancer Immunologic Data Commons (CIMAC-CIDC) is a network of laboratories and a bioinformatics center established to perform biomarker analysis and correlation with clinical outcome data from immunotherapy trials. The specific goal for the network is to perform comprehensive immunoprofiling of specimens from trials, using assays that span genomics, transcriptomics, and phenotyping analysis of the tumor, tumor microenvironment, and periphery. Identification of biomarkers to optimize immunotherapies for patients with cancer requires analytically validated and harmonized assays across multiple laboratories, allowing cross-site and cross-trial analyses. Therefore, harmonization of assay protocols, a key requirement for reducing data variability and allowing interpretation and integration of assay data across trials and laboratories, plays an important part in the network's infrastructure. A centralized database for integration of clinical and assay data will facilitate the identification of biomarkers to optimize immunotherapy approaches and management of patients with cancer.

Despite recent advances, the benefits of immunotherapy are limited to a minority of patients with cancer. While thousands of clinical trials (1) have been underway to test novel approaches and combination strategies, only a few investigational regimens have shown added benefit and received regulatory approvals. A major impediment to furthering the success of immunotherapy is inadequate understanding of the complex interplay between tumor and immune system, and the diverse mechanisms of resistance to therapy in individual patients. Immune profiling of tumor and tumor microenvironment (TME) and correlation with clinical outcome have the potential to enhance understanding of biology, identify biomarkers of response and toxicities, and reveal mechanisms of resistance that might be actionable. Especially, analyses of clinical samples longitudinally (at baseline, on treatment, and at progression) are valuable for elucidating the mechanisms of drug action and monitoring changes in tumor and TME through treatment (2, 3). The integration of stringently validated biomarkers in immunotherapy trials could accelerate therapeutic development and optimization of clinical outcome.

Findings in early- and late-phase clinical trials have led to identification of candidate predictive markers for response to anti-PD-1/PD-L1 monotherapy, such as PD-L1 expression (4–6), CD8+ T-cell density (7), tumor mutational burden (TMB; refs. 8, 9), neoantigen prediction (10), transcriptomic profiles (11), T-cell receptor (TCR) clonality (7), and microsatellite instability (MSI) status (12). Biomarkers associated with poor outcomes have also been identified; examples include tumor loss of antigen presentation machinery (13, 14), activation of Wnt/β-catenin signaling (15), and cyclin-dependent kinase 5 expression that may dampen the ability of T cells to reject tumors (16). Furthermore, biomarkers to predict immune-related adverse events (irAE) are also of high interest, particularly for irAEs with life-threatening consequences.

However, to date, different biomarkers have been investigated with variable success. In particular, no biomarkers have been identified for selection of combination regimens. Determining the predictive accuracy of biomarkers for immunotherapy must involve a comprehensive approach that encompasses the complexity of tumor biology and the host immune system. While multi-omics technologies are widely available to support objectives of biomarker discovery, variability in assay methodology, assay data reporting, and specimen collection and processing procedures prevents comparison and interpretation of data across individual laboratories and clinical trials (17).

Elements critical to a systematic effort in biomarker development across different sites and studies must include: multidisciplinary expertise and capacity for complex tumor and immune profiling, assay platforms that not only are analytically validated, but also demonstrate comparable assay performance across laboratories, appropriate clinical study design and sufficient sample sizes to make statistically valid conclusions, and a database for biomarker and clinical data integration with bioinformatics tools for correlative analysis within and across trials (Fig. 1).

Figure 1.

Strategies to enhance the value of biomarker discovery through deep tumor immune profiling in individual patients, biomarker analysis with clinical correlation within trials, and integrative analysis across trials.

Figure 1.

Strategies to enhance the value of biomarker discovery through deep tumor immune profiling in individual patients, biomarker analysis with clinical correlation within trials, and integrative analysis across trials.

Close modal

To facilitate the process of immune biomarker identification and comparison between different NCI-sponsored immunotherapy trials and laboratories, the NCI established the Cancer Immune Monitoring and Analysis Centers and Cancer Immunologic Data Commons (CIMAC-CIDC) Network (https://cimac-network.org/; “the Network”). The Network is composed of four multidisciplinary academic centers (the CIMACs), with capacity for state-of-the-art immunoprofiling assays, and a data coordination center (CIDC) that provides a database and informatics platform for analysis and integration of clinical and biomarker data across trials.

The CIMAC-CIDC Network was launched in September 2017 through the Cancer MoonshotSM Initiative supported by the NCI. As the Network was being established, in parallel, it formed a collaboration with the Partnership for Accelerating Cancer Therapies (PACT), another Cancer MoonshotSM project, and it became the public-sector side of the public–private partnership (PPP) overseen by the Foundation for the NIH. This collaboration of the Network and PACT, launched in February 2018 (18), allowed for exchange of ideas between the industry partners from 12 leading biopharmaceutical companies, the FDA, NCI, and the academic partners in the Network. The Network and PACT agreed that validation and harmonization of biomarkers are essential for the future of immunotherapy development. In particular, the industry partners, through the PPP, provide major financial support for the bioinformatics needs of the Network for optimization of data collection methodologies, data integration, and building a database of biomarker and clinical data at the CIDC. In addition, they support development of novel biomarker assays and correlative studies in immuno-oncology clinical trials sponsored by NCI, industry, academic centers, and other organizations.

This overview describes components required for the establishment of the Network. Also published in this volume are separate articles that summarize the harmonization efforts on key assay platforms, including those for whole-exome sequencing (WES) and RNA sequencing (RNA-seq; ref. 19), mass cytometry by time of flight (CyTOF; ref. 20), and singleplex and multiplex IHC/immunofluorescence (IHC/IF; ref. 21).

The CIMAC-CIDC Network infrastructure

The four CIMACs are located at Dana-Farber Cancer Institute (Boston, MA), the Icahn School of Medicine at Mount Sinai (New York, NY), the University of Texas MD Anderson Cancer Center (Houston, TX), and Stanford University (Stanford, CA). The CIDC is hosted at Dana-Farber Cancer Institute (Boston, MA). Each CIMAC encompasses a multidisciplinary group of investigators with basic, translational, clinical, and computational expertise required for conducting complex correlative analyses.

The operational structure of the Network is depicted in Fig. 2. Clinical trial teams of the NCI- and PACT-solicited trials seeking to collaborate with the Network apply through an established process that includes evaluation of scientific merit and feasibility of biomarker studies in the context of the clinical trial. For the selected trials, the CIMAC-CIDC investigators partner with the clinical trial team to design a biomarker plan and conduct immunoprofiling assays and correlative analyses. For correlative studies sponsored by the private sector PACT funds, the industry members also form a working group that helps to advise and refine the study design with the trial team. Blood and tissue specimens from the trials are collected at or transferred to designated central biorepositories for pathology quality control, processing, and distribution to the CIMACs. Guidelines and template agreements were developed for data access and sharing, specimen transfers, and intellectual property stipulations (https://cimac-network.org/documents/).

Figure 2.

Organizational structure of the CIMAC-CIDC Network. Clinical trial teams supported by NCI or selected by PACT collaborate with the Network on the design and execution of correlative studies using specimens and data from clinical trials of immunotherapy. Assay data from CIMACs and clinical data from the trials are transferred to CIDC. CIMACs, trial investigators, and CIDC jointly perform integrative analyses of biomarker and clinical data using the CIDC bioinformatics platform.

Figure 2.

Organizational structure of the CIMAC-CIDC Network. Clinical trial teams supported by NCI or selected by PACT collaborate with the Network on the design and execution of correlative studies using specimens and data from clinical trials of immunotherapy. Assay data from CIMACs and clinical data from the trials are transferred to CIDC. CIMACs, trial investigators, and CIDC jointly perform integrative analyses of biomarker and clinical data using the CIDC bioinformatics platform.

Close modal

The four CIMACs provide a wide range of validated and harmonized assay platforms for comprehensive genomic, phenotypic, and functional characterizations for analysis of specimens from immunotherapy trials. Raw assay data generated by CIMAC laboratories are transferred to the CIDC. A set of clinical data elements is extracted from the clinical trial database and also transferred to the CIDC. CIDC facilitates the Network activities through optimizing data collection methodologies and providing the central database, investigator access, and bioinformatics tools for integrative data analysis of biomarker and clinical data, both within and across trials, to function as an immunoprofiling data coordination center.

Currently, more than 30 clinical trials from various NCI trial networks, academic sites, and industry sponsors have been selected for collaboration with the CIMAC-CIDC Network. These trials range from phase I/pilot studies to randomized phase II and III trials and involve a variety of clinical settings, including pediatric malignancies, rare tumors, patients with preexisting autoimmune disorders, as well as patients with common solid tumors and hematologic malignancies. Therapeutic strategies being tested in the trials include monotherapy with immunotherapy agents and combinations with other immune modulators, targeted agents, or chemotherapy/radiation. In addition to baseline tumor tissue and longitudinal blood sample collections, many early-phase trials also incorporate on-treatment and at-progression biopsies.

Overall, the Network has set out the following scientific and strategic goals:

  • (i) Establish evidence-based translational research approaches for analytical validation of assays and biomarkers.

  • (ii) Conduct both hypothesis-driven and hypothesis-generating correlative studies in immuno-oncology trials, with the goal of identifying candidate biomarkers and their associations with clinical outcomes.

  • (iii) Establish the administrative infrastructure of the Network, including development of a CIMAC-CIDC Human Material Transfer Agreement involving multiple stakeholders (CIMAC-CIDC, NCI, clinical trial organizations and sponsors, and investigators), CIMAC-CIDC guidelines to guide the Network operations, and a study proposal intake process (documents found at: https://cimac-network.org/documents/).

  • (iv) Develop and implement a specimen collection and processing “umbrella” protocol supporting the immunoprofiling assays of the CIMACs, including standardization of preanalytical conditions (found at: https://cimac-network.org/documents/).

  • (v) Analytically validate all tier 1 and tier 2 assays (Table 1) to be performed by CIMACs, conduct harmonization of the key tier 1 assay platforms across different CIMACs, and establish reference standards for longitudinal monitoring of assay performance.

  • (vi) Support translational efforts in the immuno-oncology scientific community by providing access to a set of protocols for harmonized and validated assays that can be implemented by both academic and industrial laboratories.

  • (vii) Establish the CIDC, including the bioinformatics platforms for within- and cross-trial analysis and integration of biomarker and clinical data.

  • (viii) Facilitate broad data sharing with the larger research community by transferring data and findings from the CIDC to the NCI Cancer Research Data Commons (CRDC), including data from the industry-sponsored trials.

Table 1.

Tier 1, 2, and 3 assays in the CIMAC-CIDC Network.

Tier 1 assays (planned for all or most trials)Tier 2 assays (planned for selected trials)Tier 3 assays (highly novel and exploratory)
  • CyTOF

  • Olink immunoassay

  • WES

  • RNA-seq

  • nCounter® (NanoString)

  • mIHC/IF

  • Singleplex IHC (sIHC)

 
  • CyTOF Phosphoflow

  • Grand Serology ELISA

  • ctDNA

  • ATAC-seq

  • scTCR-seq

  • TCR-seq

  • Microbiome analysis (16S sequencing, shotgun metagenomics)

  • MIBI

 
  • ELISPOT

  • HLA tetramers

  • scRNA-seq

  • CITE-Seq

 
Tier 1 assays (planned for all or most trials)Tier 2 assays (planned for selected trials)Tier 3 assays (highly novel and exploratory)
  • CyTOF

  • Olink immunoassay

  • WES

  • RNA-seq

  • nCounter® (NanoString)

  • mIHC/IF

  • Singleplex IHC (sIHC)

 
  • CyTOF Phosphoflow

  • Grand Serology ELISA

  • ctDNA

  • ATAC-seq

  • scTCR-seq

  • TCR-seq

  • Microbiome analysis (16S sequencing, shotgun metagenomics)

  • MIBI

 
  • ELISPOT

  • HLA tetramers

  • scRNA-seq

  • CITE-Seq

 

Abbreviations: ATAC-seq, assay for transposase-accessible chromatin sequencing; CITE-Seq, cellular indexing of transcriptomes and epitopes by sequencing; ctDNA, circulating tumor DNA; CyTOF, cytometry by time of flight; HLA, human leukocyte antigen; MIBI, multiplexed ion beam imaging; mIHC/IF, multiplex immunohistochemistry/immunofluorescence; RNA-seq, RNA sequencing; scRNA-seq, single-cell RNA sequencing; scTCR-seq, single-cell TCR sequencing; TCR-seq, TCR sequencing; TCR, T-cell receptor; WES, whole-exome sequencing.

Selection and prioritization of biomarker modules for clinical trials

The CIMACs selected a variety of platforms to provide comprehensive tumor and immune profiling for characterization of antitumor immune responses. These assays are categorized by tiers on the basis of scientific priority and technical robustness for implementation in clinical trials (Table 1).

Selection of assay platforms

From a biological perspective, molecular profiling should encompass components essential to antitumor immune response, including tumor intrinsic factors (e.g., immunogenicity and oncogenic pathways), host factors, and immune cell subsets in the TME and periphery. A guiding principle for selection of CIMAC assays was to prioritize platforms that provide the most comprehensive and unbiased analysis.

CyTOF mass cytometry was selected for assessing function and phenotypes of immune cell subsets, such as T cells, B cells, natural killer cells, macrophages, and myeloid-derived suppressor cells, as it was considered advantageous over flow cytometry for the higher number of biomarkers detected by antibody panels, with little or no spillover between detector channels (22). Olink was chosen as the core assay for profiling of cytokines, chemokines, and growth factors, which are essential to immune response and intercellular communication. Olink was selected after comparisons with several other platforms for multiplex soluble analyte measurement. The advantages of Olink include a dedicated panel for immuno-oncology, high number of measurable analytes per sample (more than 90), high dynamic range of detection due to its proximity extension assay, low volume requirement, internal calibration controls, and good reproducibility. WES was prioritized over use of targeted gene panel sequencing, to provide genomic correlates, such as TMB calculated as total number of single-nucleotide variants (9), immunogenic neoepitopes resulting from novel mutations (23), germline mutations and polymorphic variants (24), and MSI status (12). Whole-transcriptome profiling (RNA-seq) and TCR sequencing assays were chosen for their utility in measuring complex, dynamic physiologic states and their ability to provide a wide range of information in a single readout, including tumor gene expression, neoantigen load, T-cell infiltrate, TCR clonality, HLA haplotype, and other signatures relevant to response or resistance to immunotherapy (25). The NanoString platform was identified as an alternative approach for transcriptional analysis. Although it covers a targeted panel of genes, it is robust, sensitive, and applicable especially in cases of low-quality RNA from formalin-fixed, paraffin-embedded (FFPE) samples. Tissue imaging via multiplex IHC/IF (mIHC/mIF) was included as a core assay for its ability to probe multiple cellular markers simultaneously and provide information on spatial organization of cellular targets in relation to tumor cells, stroma, vasculature, and immune cell subsets (26, 27). Although not considered a “tier 1” assay, microbiome analysis was harmonized between two CIMACs. Aspects harmonized included stool sample collection, aliquoting, DNA extraction, 16S rRNA sequencing, and computational considerations.

Prioritization of assays for implementation in clinical trials

An organizing principle for biomarker prioritization was the concept of assay “tiers” based on level of comprehensive and unbiased features, as well as the envisioned scope of assay application across clinical trials. To generate data that could be integrated across multiple trials, a core set of assays defined as “Tier 1” was chosen to be applied in all or most trials and require harmonization to ensure comparability of the data across CIMAC sites (Table 1).

“Tier 2” assays are used in selected trials and do not require harmonization. They could be available at a single CIMAC site and require less throughput than tier 1 assays. For example, the assay for transposase-accessible chromatin sequencing (ATACseq), which measures epigenetic changes in sorted specific immune cell populations, is designated as a tier 2 assay (28). ATAC-seq analysis can be implemented in trials studying drugs that target DNA methylation or that work via other type of epigenetic reprogramming (29, 30).

“Tier 3” assays are considered novel and may be relevant to specific treatment questions in selected trials. Examples include use of single-cell genome or single-cell transcriptome assays to provide a “deep dive” into immune system complexity. Such assays can reveal cell population differences, cellular evolutionary relationships, and clonal heterogeneity within the tumor (31).

Tier-based categorization can change over time as tier 2 assays prove sufficiently robust to become tier 1. For example, the circulating tumor DNA (ctDNA) assay, which yields information on tumor burden dynamics in cancer progression and may potentially circumvent the need for repeated tumor biopsies (32), has emerged as a biomarker of interest in multiple clinical trials and thus could be reclassified from tier 2 to tier 1. With further development, tier 3 assays could potentially be promoted to tier 2 status.

Considerations for specimen collection and preanalytics

How specimens are collected and processed for preservation can have a large impact on the quality of assay data and correlative analyses. The CIMAC-CIDC Network includes more than 30 clinical studies led by multiple different trial groups, across a projected collection of thousands of specimens from more than 3,000 patients. Allocation of limited specimen material for various assays needed for comprehensive profiling across CIMAC sites frequently poses a logistical challenge.

Efforts to achieve the goals set for the CIMAC-CIDC Network require that specimens meet a high standard of quality to ensure robust and comparable profiling. To guide investigators through sample distribution options among a variety of assays and provide standardized methods for specimen collection and handling, the Network and NCI developed the Specimen Collection “Umbrella” protocol (found at: https://cimac-network.org/documents/). The Umbrella protocol addresses various steps in the “sample flow,” from tissue or blood sample acquisition at trial sites, to immediate processing and storage at biorepositories, to subsequent processing and downstream distribution to the CIMAC laboratories. An overview of the Umbrella protocol is provided in Table 2.

Table 2.

Preanalytical elements in the CIMAC specimen collection “Umbrella” protocol.

Specimen typeCollection and processing at siteImmediate processing at biobankProcessing at biobank for distribution to CIMAC laboratoriesIntended assay use at CIMAC
Tissue biopsies 
  • De novo core needle biopsy

  • Endoscopic/punch biopsy

  • De novo surgical resection

 
  • 1–2 cores or 1 segment (FFPE)

 
  • Embed fixed tissue

  • Store blocks

 
  • Unstained slides + H&E

  • DNA/RNA extraction

 
  • Fresh frozen samples:

    • WES tumor/normal, RNA-seq, TCR sequencing

 
 
  • 1–2 cores flash frozen or 1 segment flash frozen

 
  • Store frozen

 
  • DNA/RNA extraction

 
 
  • Archival FFPE material

 
  • FFPE blocks or unstained slides

  • Core punches

 
  • Store blocks or vacuum-seal slides

  • Refrigerate punches

 
  • Unstained slides + H&E

  • DNA/RNA extraction

 
  • FFPE samples:

    • IF, IHC, MIBI, WES/germline, RNA-seq, TCR sequencing

 
Blood 
  • Sodium heparin green-top tubes

 
  • 30 mL draw

 
  • Isolate plasma and PBMCs

  • Smart tubes

 
  • Ship smart tube, plasma, or PBMCs

  • DNA (TCR sequencing)

 
  • Plasma (Olink, ELISA)

  • PBMCs (CyTOF, TCR sequencing)

 
  • Streck cell-free DNA tubes

 
  • 10 mL draw

 
  • Isolate plasma and freeze aliquots

 
  • Ship plasma aliquots

 
  • cfDNA

 
  • K2-EDTA purple-top tubes

 
  • 2 mL draw (solid tumor germline)

  • 5–10 mL draw (hematologic germline)

  • 2 mL draw (TCR sequencing)

 
  • Freeze germline aliquots

  • 2 mL aliquots (TCR sequencing)

 
  • Extract and ship DNA aliquots

 
  • Germline WES, TCR sequencing

 
Bone marrow, CSF, stool 
  • Bone marrow aspirates or Cerebrospinal fluid

 
  • Custom volume in K2-EDTA tubes

 
  • Supernatant

  • Cell fraction

 
  • Ship aliquots

  • Unstained slides + H&E

  • DNA/RNA extraction

 
  • CyTOF, Olink, IF, IHC, MIBI, RNA-seq

 
  • Stool samples

 
  • Self-collection (ship ambient or frozen)

 
  • 2 mL aliquots (DNA stabilizer)

  • Frozen stool

 
  • Ship frozen aliquots

 
  • 16S rRNA

  • Shotgun metagenomics

 
Specimen typeCollection and processing at siteImmediate processing at biobankProcessing at biobank for distribution to CIMAC laboratoriesIntended assay use at CIMAC
Tissue biopsies 
  • De novo core needle biopsy

  • Endoscopic/punch biopsy

  • De novo surgical resection

 
  • 1–2 cores or 1 segment (FFPE)

 
  • Embed fixed tissue

  • Store blocks

 
  • Unstained slides + H&E

  • DNA/RNA extraction

 
  • Fresh frozen samples:

    • WES tumor/normal, RNA-seq, TCR sequencing

 
 
  • 1–2 cores flash frozen or 1 segment flash frozen

 
  • Store frozen

 
  • DNA/RNA extraction

 
 
  • Archival FFPE material

 
  • FFPE blocks or unstained slides

  • Core punches

 
  • Store blocks or vacuum-seal slides

  • Refrigerate punches

 
  • Unstained slides + H&E

  • DNA/RNA extraction

 
  • FFPE samples:

    • IF, IHC, MIBI, WES/germline, RNA-seq, TCR sequencing

 
Blood 
  • Sodium heparin green-top tubes

 
  • 30 mL draw

 
  • Isolate plasma and PBMCs

  • Smart tubes

 
  • Ship smart tube, plasma, or PBMCs

  • DNA (TCR sequencing)

 
  • Plasma (Olink, ELISA)

  • PBMCs (CyTOF, TCR sequencing)

 
  • Streck cell-free DNA tubes

 
  • 10 mL draw

 
  • Isolate plasma and freeze aliquots

 
  • Ship plasma aliquots

 
  • cfDNA

 
  • K2-EDTA purple-top tubes

 
  • 2 mL draw (solid tumor germline)

  • 5–10 mL draw (hematologic germline)

  • 2 mL draw (TCR sequencing)

 
  • Freeze germline aliquots

  • 2 mL aliquots (TCR sequencing)

 
  • Extract and ship DNA aliquots

 
  • Germline WES, TCR sequencing

 
Bone marrow, CSF, stool 
  • Bone marrow aspirates or Cerebrospinal fluid

 
  • Custom volume in K2-EDTA tubes

 
  • Supernatant

  • Cell fraction

 
  • Ship aliquots

  • Unstained slides + H&E

  • DNA/RNA extraction

 
  • CyTOF, Olink, IF, IHC, MIBI, RNA-seq

 
  • Stool samples

 
  • Self-collection (ship ambient or frozen)

 
  • 2 mL aliquots (DNA stabilizer)

  • Frozen stool

 
  • Ship frozen aliquots

 
  • 16S rRNA

  • Shotgun metagenomics

 

Abbreviations: cfDNA, cell-free DNA; H&E, hematoxylin and eosin.

The Umbrella protocol was developed using an iterative approach. Optimizing performance of CIMAC assays and defining specimen preanalytical requirements were instrumental in the assay validation and harmonization efforts of the CIMAC-CIDC, described later in this overview. Existing biorepository standard operating procedures (SOP) that had been well validated with clinical trial samples were employed or adapted as far as possible. Where feasible, novel approaches were considered to support the need for flexibility and maximize use of limited tissue.

Since its development, the Umbrella protocol has been incorporated into several trials selected for collaboration with CIMAC-CIDC. Potentially, the Umbrella protocol could have broader applicability beyond CIMAC-CIDC, as a consensus guidance for prospective immunotherapy trials to ensure high-quality specimen collections for downstream analysis.

Principles for assay validation and harmonization

To enable robust and systematic biomarker analysis across the CIMACs and across clinical trials, objective quality control measures are required for all assays to be performed in the Network. These measures include each assay's analytical validity, including its prespecified level of variability and reproducibility, concordance between laboratories, as well as acceptance criteria appropriate to its intended use (Table 3).

Table 3.

Analytical performance metrics evaluated for CIMAC assays.

  • Analytes

  • Technical platform(s)

  • Reagents, controls, and calibrators

  • Quality control parameters for specimens/analytes (e.g., cell viability, RNA/protein quality/integrity)

  • Critical preanalytical variables

  • Analytical performance characteristics for each assay:

    • ▪ Current status and results of studies defining the sensitivity, specificity, accuracy, precision, reproducibility, reportable range, reference ranges/intervals (normal values), turnaround time, and failure rate of the assay.

    • ▪ Use of positive and negative controls, calibrators, and reference standards.

    • ▪ Number of samples in the reproducibility study.

    • ▪ How run-to-run variation (CV) was assessed and handled.

    • ▪ How interlaboratory variability in the measurements was assessed and how these sources of variation were minimized to maintain performance at all sites within acceptable limits and to prevent drift or bias in the assay.

    • Scoring procedures and type of data to be acquired:

      • Quantitative/continuously distributed

      • Semiquantitative/ordered categorical

      • Qualitative/nonordered categorical

 
  • Analytes

  • Technical platform(s)

  • Reagents, controls, and calibrators

  • Quality control parameters for specimens/analytes (e.g., cell viability, RNA/protein quality/integrity)

  • Critical preanalytical variables

  • Analytical performance characteristics for each assay:

    • ▪ Current status and results of studies defining the sensitivity, specificity, accuracy, precision, reproducibility, reportable range, reference ranges/intervals (normal values), turnaround time, and failure rate of the assay.

    • ▪ Use of positive and negative controls, calibrators, and reference standards.

    • ▪ Number of samples in the reproducibility study.

    • ▪ How run-to-run variation (CV) was assessed and handled.

    • ▪ How interlaboratory variability in the measurements was assessed and how these sources of variation were minimized to maintain performance at all sites within acceptable limits and to prevent drift or bias in the assay.

    • Scoring procedures and type of data to be acquired:

      • Quantitative/continuously distributed

      • Semiquantitative/ordered categorical

      • Qualitative/nonordered categorical

 

Note: Adapted from “Study Checklist for CTEP-Supported Early-Phase Trials with Biomarker Assays,” found at http://ctep.cancer.gov/protocolDevelopment/ancillary_correlatives.htm.

Analytical validation

An analytically validated assay should accurately and reliably measure the analyte of interest in specimens representative of the population of interest. Analytical validity is built on the concept of a total test, including preanalytical, analytical, and interpretative/postanalytical phases of assay development (Table 3; ref. 33). Analytical validation should demonstrate how robustly and reliably the test meets predefined performance standards of reproducibility, specificity, sensitivity, and dynamic range. Regression analysis by an appropriate linear or nonlinear method should be performed comparing measured with expected biomarker assay performance across the quantification range. The general acceptance criteria for the correlation coefficient (r) should be predetermined on the basis of the context of use (34).

The level of analytical validation required for CIMAC assays is set at the level of evidence claimed for research use–only assays, which are usually applied for sample characterization and hypothesis generation, but cannot be used in clinical decision-making and, therefore, are not required to be performed in a Clinical Laboratory Improvement Amendments (CLIA)–certified laboratory. For the purpose of CIMAC-CIDC study, the assays went beyond the typical validation required for routine research-based assays, as the analytical validation process and consensus SOPs were aimed at informing future standardization and harmonization guidelines for these assays. Each tier 1 and tier 2 assay required a qualification document demonstrating analytical validation of the assay, including sensitivity, specificity, intra- and interassay precision, accuracy, linearity, reproducibility, and robustness/ruggedness (Table 3).

Assay harmonization for CIMAC-CIDC studies

To allow comparisons of biomarker data across trials, concordance of tier 1 assay performance was established across the different CIMAC sites performing a given assay, following development of consensus protocols. To enable comparison and integration of assay data across different studies and sites, harmonization of laboratory-specific protocols and development of consensus SOPs are recommended. During this process, each participating laboratory evaluates and compares the validity of reagents, standards, methodologies, protocols, and data reporting specific to each laboratory. Development of consensus protocols enables data comparison and interpretation supporting biomarker development across different clinical trial sites (Fig. 3; ref. 17).

Figure 3.

Cross-site assay harmonization, an iterative process ensuring reproducibility and robustness of assays to overcome methodologic and data variability across different sites. After Fig. 1 in van der Burg SH and colleagues 2011 (17).

Figure 3.

Cross-site assay harmonization, an iterative process ensuring reproducibility and robustness of assays to overcome methodologic and data variability across different sites. After Fig. 1 in van der Burg SH and colleagues 2011 (17).

Close modal

Across the CIMACs, the principles of harmonization have been applied and successfully completed for CIMAC tier 1 assays assessing genomics (WES), transcriptomics (RNA-seq), and phenotypic characterization of tumor (mIHC/mIF) and peripheral blood mononuclear cell (PBMC) subtypes (CyTOF). Olink, also a tier 1 assay, is validated and performed at a single CIMAC. The results of the harmonization for individual assays are described in reports also published in this volume (19–21). All assays met the predefined acceptance criteria for concordance that had been established for each assay, demonstrating a high level of comparability of results between participating laboratories. The CIMAC assay SOPs are available on the CIMAC-CIDC website at https://cimac-network.org/assays/.

Statistical evaluation of assay performance

Most biomarkers are measured as continuous variables, but some are categorical in nature. Repeatability is defined as the agreement of repeated measurements within one laboratory under similar conditions. On the other hand, harmonization is defined as the agreement of repeated measurements across different laboratories under various conditions. Because a large number of different biomarkers will be analyzed in a wide variety of settings, no single statistical method and no single criterion can be applied to analyze the agreement of measurements for all conditions. However, several statistical methods can be applied to evaluate the agreement or concordance between measurements within and across laboratories:

  • (i) Pearson and Spearman correlation coefficients are calculated to examine the agreement of two measurements of a continuous variable; a scatter plot can also be generated. The Pearson correlation coefficient is more efficient when the data are Gaussian distributed. The Spearman correlation coefficient is more robust when the data deviate from the Gaussian distribution; it is less influenced by outliers. Although there is no uniformly accepted criterion of “acceptable” agreement, a correlation of greater than 0.7, 0.8, and 0.9 can be considered as having adequate, good, and excellent correlation, respectively.

  • (ii) Coefficient of variation (CV) is defined as the SD of a measurement divided by its mean. By definition, it is a measure of variation on the scale of the mean. Hence, CV is a unitless measurement and is useful for quantifying the variation or precision of measurements. Both intra- and interlaboratory CVs can be calculated. Similarly, there is no uniformly accepted criterion regarding the magnitude of an “acceptable” CV. However, a CV of less than 0.3, 0.2, and 0.1 can be considered as having adequate, good, and excellent precision between the measurements, respectively.

  • (iii) Variance component model under the one-way ANOVA can be constructed to model the variability both within and between laboratories at different sites. The total variability can be broken down into between-site variability, between-subject variability, and within-subject random error (i.e., measurement error). The relative magnitude of the different variabilities can be calculated by forming the percent of variability that is explained. Intraclass correlation coefficient (ICC) can also be calculated as the proportion of the total variance contributed by between-site variance. ICC can be generalized to allow for covariate effects. Small portions of variability due to site and random error indicate good harmonization.

  • (iv) Linear mixed effect model can be applied when biomarkers are measured over time within the same individual. Typically, the subject is considered as a random effect, and time a fixed effect. When multiple sites are involved, site can be added as a fixed or a random effect. When data are skewed, transformation can be applied to biomarkers to make the transformed values more Gaussian distributed. Other covariates can be added as well. The contribution of various components to the biomarker value can be dissected and evaluated.

Reference materials for longitudinal assay performance monitoring

To extend the full benefits of the CIMAC-CIDC validation and harmonization efforts, a long-term plan was put into place to monitor assay performance over the duration of the project. For several assays, standard reference materials were generated in batches for quality control assessment of assay performance within and across different CIMACs over time (Table 4). Control materials will also be used in “bridging” studies to compare assay performance following a transition to a different platform or modification of an assay.

Table 4.

Longitudinal reference standards used by CIMAC-CIDC.

Reference materialDescriptionTotal amount availableFrequency of testing
CyTOF BioLegend PBMCs 1:1 mixture of dual-labeled activated and resting PBMC Veri-cells 200–350 vials (1 × 106 cells/vial) Spiked into every clinical sample at 10% volume 
Olink Various 
  • Pooled plasma from healthy donors

  • Randox cytokine cocktail

  • Olink experimental controls

 
More than 1,000 aliquots available Used in every run 
ELISA grand serology Healthy plasma pools 
  • Healthy donor plasma pools as negative control and titer calculations

  • Positive plasma pools from patients with reactivity to several antigens

 
More than 300 mL of plasma available (3–15 μL per assay) Used as appropriate 
IHC/IF, MIBI CHTN Master TMA Master TMA containing normal, neoplastic, and tumor tissue Sequential sections will be distributed (four TMA blocks) Twice per year 
WES HapMap cell line pool 
  • Two pools of 10 HapMap cell lines containing different allele fractions

  • Two cell lines as germline controls

 
Cell line pellets embedded into FFPE blocks, extracted DNA distributed to each site Twice per year, Used as analysis pipelines develop over time 
Microbiome Various 
  • Healthy donor fecal samples (RefA)

  • ZymoBIOMICS Microbial Community Standard (RefB)

  • DNA library of gut-relevant microbes of known abundance (RefC)

 
  • RefA: 60 ready-to-use aliquots from ∼100 mg of material

  • RefB: commercially available

 
Used to control biases and batch effects for extractions, library preparation, and sequencing runs 
Reference materialDescriptionTotal amount availableFrequency of testing
CyTOF BioLegend PBMCs 1:1 mixture of dual-labeled activated and resting PBMC Veri-cells 200–350 vials (1 × 106 cells/vial) Spiked into every clinical sample at 10% volume 
Olink Various 
  • Pooled plasma from healthy donors

  • Randox cytokine cocktail

  • Olink experimental controls

 
More than 1,000 aliquots available Used in every run 
ELISA grand serology Healthy plasma pools 
  • Healthy donor plasma pools as negative control and titer calculations

  • Positive plasma pools from patients with reactivity to several antigens

 
More than 300 mL of plasma available (3–15 μL per assay) Used as appropriate 
IHC/IF, MIBI CHTN Master TMA Master TMA containing normal, neoplastic, and tumor tissue Sequential sections will be distributed (four TMA blocks) Twice per year 
WES HapMap cell line pool 
  • Two pools of 10 HapMap cell lines containing different allele fractions

  • Two cell lines as germline controls

 
Cell line pellets embedded into FFPE blocks, extracted DNA distributed to each site Twice per year, Used as analysis pipelines develop over time 
Microbiome Various 
  • Healthy donor fecal samples (RefA)

  • ZymoBIOMICS Microbial Community Standard (RefB)

  • DNA library of gut-relevant microbes of known abundance (RefC)

 
  • RefA: 60 ready-to-use aliquots from ∼100 mg of material

  • RefB: commercially available

 
Used to control biases and batch effects for extractions, library preparation, and sequencing runs 

Abbreviations: CHTN, Cooperative Human Tissue Network; TMA, tissue microarray.

Cancer Immunologic Data Commons (CIDC)

The Dana-Farber Cancer Institute (Boston, MA) maintains and hosts the CIDC for the CIMAC-CIDC Network. The CIDC provides bioinformatics methods and the computational expertise and resources to facilitate the analysis of immuno-oncology trial data for the Network (Fig. 4). CIDC receives clinical data from various sources, including NCI trial network trials, investigator-initiated trials, and industry trials, for integrative analysis of the assay and clinical data. While the integrity of the provided data is maintained, the data are mapped to a clinical data model and current standards. In conjunction with the CIMACs, CIDC has developed data standards and software for recording molecular, clinical, and metadata generated by the Network. As these data standards evolve, the CIDC will work with the NCI Center for Cancer Data Harmonization to ensure data are available for rapid sharing via the NCI CRDC, as well as facilitate future cross-trial analyses.

Figure 4.

Network data coordination: CIDC provides bioinformatics services and functionality on the basis of data received from CIMAC laboratories and clinical trial organizations.

Figure 4.

Network data coordination: CIDC provides bioinformatics services and functionality on the basis of data received from CIMAC laboratories and clinical trial organizations.

Close modal

CIDC software

The CIDC software platform is a Google cloud–based system designed to facilitate the ingestion of molecular, clinical, and metadata generated by CIMAC laboratories and participating clinical trial centers. The software system abides by stringent security controls under the Risk Management Framework published by the National Institute of Standards and Technology. The main components of the system are: a high-performance data transfer tool for CIMACs to transfer assay data and metadata to the cloud, centralized storage of all metadata in a managed database and files in cloud storage buckets, and a web-based data portal for browsing and downloading of data files associated with the clinical trials. Access management is implemented using a role-based methodology ensuring access is tightly controlled by the Network's data access and sharing policies. The prototype CIDC data portal is operational and is in the process of ingesting assay data generated from clinical trial samples and different assay types.

CIDC bioinformatics

The CIDC worked with the CIMACs to establish the experimental and computational pipelines and to identify the relevant pre- and postanalytical metadata to be collected with the assay data. Using a unified workflow management system, the CIDC has established several uniform bioinformatics processing pipelines. These include pipelines for processing WES, bulk RNA-seq, ATAC-seq, and TCR sequencing data. The pipelines also provide self-contained, comprehensive HTML reports, and use conda and bioconda (35) to ensure reproducibility and portability.

The WES processing pipeline follows Gene Analysis Toolkit best practices (36) implemented in Sentieon for identifying germline/somatic mutations, indels, and copy-number variations. The pipeline also includes mutation interpretation, tumor purity, and clonality analysis features. In addition, HLA typing and neoantigen prediction are incorporated specifically for immunologic data.

The RNA-seq processing pipeline includes steps for preprocessing, quality control, conventional differential expression analysis, and downstream analysis (e.g., gene module and gene set enrichment analysis). Tailoring to immuno-oncology, it includes additional functions to estimate infiltrating immune cells, evaluate immunotherapy response prediction biomarkers, predict MSI status, infer infiltrating immune repertoires, and identify microbiota and their classifications. In addition, the CIDC has engaged in efforts to harmonize genomics data from three different experimental platforms (MD Anderson Cancer Center, Houston, TX, Frederick National Laboratory for Cancer Research, Frederick, MD, and the Dana-Farber Cancer Institute, Boston, MA) and is starting to use the pipelines to process trial samples. In the coming year, the CIDC aims to improve the neoantigen prediction function by integrating WES and RNA-seq data and incorporating the newest immunopeptidome data.

For TCR sequencing, the CIDC team built an interactive web application that generates HTML reports for users to visualize immune repertoire information for each sample, cluster the samples, and compare samples between different groups. The CIDC has also finished developing the ATAC-seq data processing pipeline based on a previous chromatin immunoprecipitation sequencing pipeline (37). Finally, for CyTOF data, the CIDC licensed the Astrolabe platform, which uses an automated gating strategy to determine cell populations (38). All the bioinformatics pipelines developed and adopted are accompanied by documentation, software versions, analysis parameters, and reference data, and are tested regularly when necessary updates are made.

Assay harmonization was identified as an important objective for both the CIMAC-CIDC Network and the PACT PPP, to generate highly concordant and interpretable datasets across multiple laboratories and studies and to facilitate development of a database of biomarker and clinical data for secondary analyses. Harmonized assays reduce variability and enhance reproducibility of individual laboratory protocols and comparability of data across different laboratories and studies. In this regard, the primary objective of the Network activity has been achieved. In the second phase, the harmonized assays are being implemented for specimen analysis and correlation of assay data with clinical outcome variables. We hope the availability of CIMAC assay protocols and the publication of the data from CIMAC-CIDC Network harmonization projects will increase awareness in the immuno-oncology community of the importance of harmonization principles in successful biomarker identification, qualification, and implementation. It is the hope that these publicly available protocols will be adopted in academic and industry trials, allowing for uniform biomarker data generation enabling cross-trial analysis. Ultimately, the clinical utility of immune assays and optimization of immunotherapies based on biomarker data will depend on implementation of assay harmonization principles across the immuno-oncology community.

H.T. Maecker reports grants from NIH during the conduct of the study. S. Gnjatic reports grants from NCI during the conduct of the study; grants from Bristol-Myers Squibb, Genentech, Immune Design, Agenus, Janssen R&D, Pfizer, Takeda, and Regeneron and personal fees from Merck, Neon Therapeutics, and OncoMed outside the submitted work; and a patent for multiplex immunohistochemistry issued and licensed to Caprion. S.J. Adam reports employment with the FNIH: the FNIH facilitates the private sector funding for PACT, which is a major public–private biomedical research partnership supported by stakeholder membership including government, industry, academia, and patient advocacy and other not-for-profit organizations. X.S. Liu reports grants from NIH during the conduct of the study; is a cofounder, board member, and consultant of GV20 Oncotherapy and its subsidiaries; is a SAB of 3DMedCare; is a consultant for Genentech; is a stockholder of BMY, TMO, WBA, ABT, ABBV, and JNJ; and receives research funding from Takeda and Sanofi. F.S. Hodi reports grants and personal fees from Bristol-Myers Squibb and personal fees from Merck, Genentech, EMD Serono, Sanofi, Novartis, Surface, Compass, Apricity, Aduro, Pionyr, Torque, Rheos, Kairos, Bicara, Pieris, Eisai, Checkpoint Therapeutics, Idera, Takeda, and Bioentre outside the submitted work, and has patents for MICA related disorders pending, licensed, and with royalties paid; vaccine compositions and methods for restoring NKG2D pathway function against cancers pending, licensed, and with royalties paid; therapeutic peptides issued; methods of using pembrolizumab and trebananib pending; compositions and methods for identification, assessment, prevention, and treatment of melanoma using PD-L1 isoforms pending; and angiopoiten-2 biomarkers predictive of anti-immune checkpoint response pending. C.J. Wu reports other from BioNTech outside the submitted work. I.I. Wistuba reports grants and personal fees from Genentech, Bayer, Bristol Myers Squibb, AstraZeneca/Medimmune, Pfizer, HTG Molecular, and Merck; personal fees from Roche, Asuragen, GlaxoSmithKline, Guardant Health, Oncocyte, Flame Inc, MSD, Adaptive, and Adaptimmune; and grants from DepArray, EMD Serono, Takeda, Amgen, Karus, Johnson & Johnson, Iovance, 4D, Novartis, and Akoya outside the submitted work. C. Bernatchez reports grants from NIH during the conduct of the study and grants from Iovance Biotherapeutics and other from Myst Therapeutics outside the submitted work. S.C. Bendall is a consultant for and receives equity and royalties from Ionpath Inc. B. Sanchez-Espiridion reports grants from NIH during the conduct of the study and outside the submitted work. J. Peterson-Klaus reports employment with the FNIH: the FNIH facilitates the private sector funding for PACT, which is a major public–private biomedical research partnership supported by stakeholder membership including government, industry, academia, patient advocacy, and other not-for-profit organizations. G. Bongers reports current employment with Janssen R&D, LLC. R.R. Jenq reports other from Seres Therapeutics, Kaleido Biosciences, Liscure Biosciences, Maat Pharma, and Prolacta Bioscience outside the submitted work, as well as a patent for US20170258854A1 with royalties paid from Seres Therapeutics. M. Thurin reports other from Foundation for the NIH during the conduct of the study. No disclosures were reported by the other authors.

H.X. Chen: Conceptualization, resources, data curation, supervision, funding acquisition, validation, investigation, writing–original draft, project administration, writing–review and editing. M. Song: Conceptualization, resources, supervision, funding acquisition, validation, investigation, methodology, writing–original draft, project administration, writing–review and editing. H.T. Maecker: Conceptualization, resources, data curation, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. S. Gnjatic: Conceptualization, resources, data curation, software, formal analysis, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. D. Patton: Conceptualization, resources, data curation, software, formal analysis, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. J.J. Lee: Conceptualization, resources, data curation, software, formal analysis, supervision, validation, investigation, visualization, methodology, writing–original draft, writing–review and editing. S.J. Adam: Conceptualization, resources, software, supervision, funding acquisition, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. R. Moravec: Resources, data curation, software, formal analysis, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. X.S. Liu: Conceptualization, resources, data curation, software, formal analysis, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. E. Cerami: Conceptualization, resources, data curation, software, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. J. Lindsay: Conceptualization, resources, data curation, software, formal analysis, supervision, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. M. Tang:Supervision, software, resources, investigation, visualization, methodology, writing–original draft. F.S. Hodi: Conceptualization, resources, data curation, software, formal analysis, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. C.J. Wu: Conceptualization, resources, data curation, formal analysis, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. I.I. Wistuba: Conceptualization, resources, data curation, software, formal analysis, supervision, funding acquisition, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. G. Al-Atrash: Resources, data curation, formal analysis, supervision, validation, investigation, methodology, project administration, writing–review and editing. C. Bernatchez: Resources, data curation, formal analysis, supervision, validation, investigation, methodology, project administration, writing–review and editing. S.C. Bendall: Supervision, funding acquisition, validation, investigation, methodology, project administration, writing–review and editing. S.M. Hewitt: Resources, formal analysis, validation, investigation, visualization, methodology, writing–review and editing. E. Sharon: Conceptualization, resources, supervision, project administration, writing–review and editing. H. Streicher: Conceptualization, resources, supervision, project administration, writing–review and editing. R.A. Enos: Resources, data curation, supervision, visualization, methodology, writing–original draft, project administration, writing–review and editing. M.D. Bowman: Resources, data curation, methodology, project administration, writing–review and editing. V.M. Tatard-Leitman: Resources, data curation, visualization, methodology, writing–original draft, project administration, writing–review and editing. B. Sanchez-Espiridion: Resources, data curation, software, formal analysis, supervision, validation, investigation, methodology, project administration, writing–review and editing. S. Ranasinghe: Resources, data curation, formal analysis, supervision, validation, investigation, methodology, project administration, writing–review and editing. M. Pichavant: Resources, data curation, supervision, validation, investigation, methodology, project administration, writing–review and editing. D.M. Del Valle: Resources, data curation, software, formal analysis, supervision, validation, investigation, methodology, project administration, writing–review and editing. J. Yu: Resources, data curation, software, formal analysis, supervision, validation, investigation, methodology, project administration, writing–review and editing. S. Janssens: Resources, data curation, software, formal analysis, validation, investigation, visualization, methodology, project administration, writing–review and editing. J. Peterson-Klaus: Resources, data curation, software, supervision, funding acquisition, validation, investigation, methodology, project administration, writing–review and editing. C. Rowe: Resources, data curation, software, methodology, project administration, writing–review and editing. G. Bongers: Resources, validation. R.R. Jenq: Resources, validation. C.-C. Chang: Resources, validation. J.S. Abrams: Conceptualization, resources, supervision, funding acquisition, methodology, project administration, writing–review and editing. M. Mooney: Conceptualization, resources, supervision, funding acquisition, project administration, writing–review and editing. J.H. Doroshow: Conceptualization, resources, supervision, funding acquisition, methodology, project administration, writing–review and editing. L.N. Harris: Conceptualization, resources, supervision, funding acquisition, methodology, project administration, writing–review and editing. M. Thurin: Conceptualization, resources, formal analysis, supervision, funding acquisition, validation, methodology, writing–original draft, project administration, writing–review and editing.

The authors thank Jason Cristofaro, Anna Amar, Sherry Ansher, and Jianqiao Zhang at NCI and David Wholley and Lynn Smelkinson at FNIH for their dedicated work on the agreements for the Network. They thank Irina Lubensky, Hala Makhlouf, and Chaz Stephens for their communications with the NCI biobanks; Melissa McKay-Daily, Nina Lukinova, and Tracy Lively for their roles in the incorporation of biomarker plans into clinical trial protocols and their overall guidance on the quality of biomarkers for clinical trials; Jeffrey Moscow and Percy Ivy for their guidance on matters related to the Experimental Therapeutics Clinical Trials Network (ETCTN); and Andrea Denicoff for overseeing the NCI contract support from the Emmes Company, LLC. They thank Jeremiah Faith and Jose Clemente for their work on microbiome harmonization. Importantly, the authors thank all of the CIMAC-CIDC and clinical collaborators for making the success of this important initiative possible. Special appreciation goes to the investigators at the NIH and the Frederick National Laboratory for Cancer Research, especially the Molecular Characterization Laboratory, who shared their expertise, including serving as the reviewers of the validation and harmonization reports for the assays. Scientific and financial support for the CIMAC-CIDC Network was provided through the NCI Cooperative Agreements: U24CA224319 (to S. Gnjatic, D.M. Del Valle, and G. Bongers of the Icahn School of Medicine at Mount Sinai CIMAC), U24CA224331 (to F.S. Hodi, C.J. Wu, and S. Ranasinghe of the Dana-Farber Cancer Institute CIMAC), U24CA224285 (to J.J. Lee, I.I. Wistuba, G. Al-Atrash, C. Bernatchez, B. Sanchez-Espiridion, R.R. Jenq, and C.-C. Chang of the MD Anderson Cancer Center CIMAC), U24CA224309 (to H.T. Maecker, S.C. Bendall, and M. Pichavant of the Stanford University CIMAC), and U24CA224316 (to X.S. Liu, E. Cerami, J. Lindsay, M. Tang, and J. Yu of the CIDC at Dana-Farber Cancer Institute). Additional support was made possible through the NCI CTIMS contract HHSN261201600002C (to R.A. Enos, M.D. Bowman, V.M. Tatard-Leitman, and S. Janssens of the Emmes Company, LLC). Scientific and financial support for the Partnership for Accelerating Cancer Therapies (PACT) public–private partnership (PPP) was made possible through funding support provided to the FNIH by AbbVie Inc., Amgen Inc., Boehringer-Ingelheim Pharma GmbH & Co. KG., Bristol-Myers Squibb, Celgene Corporation, Genentech Inc, Gilead, GlaxoSmithKline plc, Janssen Pharmaceutical Companies of Johnson & Johnson, Novartis Institutes for Biomedical Research, Pfizer Inc., and Sanofi.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
Xin Yu
J
,
Hubbard-Lucey
VM
,
Tang
J
. 
Immuno-oncology drug development goes global
.
Nat Rev Drug Discov
2019
;
18
:
899
900
.
2.
Roh
W
,
Chen
PL
,
Reuben
A
,
Spencer
CN
,
Prieto
PA
,
Miller
JP
, et al
Integrated molecular analysis of tumor biopsies on sequential CTLA-4 and PD-1 blockade reveals markers of response and resistance
.
Sci Transl Med
2017
;
9
:
eaah3560
.
3.
Sharma
P
,
Hu-Lieskovan
S
,
Wargo
JA
,
Ribas
A
. 
Primary, adaptive, and acquired resistance to cancer immunotherapy
.
Cell
2017
;
168
:
707
23
.
4.
Daud
AI
,
Wolchok
JD
,
Robert
C
,
Hwu
WJ
,
Weber
JS
,
Ribas
A
, et al
Programmed death-ligand 1 expression and response to the anti-programmed death 1 antibody pembrolizumab in melanoma
.
J Clin Oncol
2016
;
34
:
4102
9
.
5.
Garon
EB
,
Rizvi
NA
,
Hui
R
,
Leighl
N
,
Balmanoukian
AS
,
Eder
JP
, et al
Pembrolizumab for the treatment of non-small-cell lung cancer
.
N Engl J Med
2015
;
372
:
2018
28
.
6.
Hirsch
FR
,
McElhinny
A
,
Stanforth
D
,
Ranger-Moore
J
,
Jansson
M
,
Kulangara
K
, et al
PD-L1 immunohistochemistry assays for lung cancer: results from phase 1 of the blueprint PD-L1 IHC assay comparison project
.
J Thorac Oncol
2017
;
12
:
208
22
.
7.
Tumeh
PC
,
Harview
CL
,
Yearley
JH
,
Shintaku
IP
,
Taylor
EJ
,
Robert
L
, et al
PD-1 blockade induces responses by inhibiting adaptive immune resistance
.
Nature
2014
;
515
:
568
71
.
8.
Chalmers
ZR
,
Connelly
CF
,
Fabrizio
D
,
Gay
L
,
Ali
SM
,
Ennis
R
, et al
Analysis of 100,000 Human Cancer Genomes reveals the landscape of tumor mutational burden
.
Genome Med
2017
;
9
:
34
.
9.
Snyder
A
,
Makarov
V
,
Merghoub
T
,
Yuan
J
,
Zaretsky
JM
,
Desrichard
A
, et al
Genetic basis for clinical response to CTLA-4 blockade in melanoma
.
N Engl J Med
2014
;
371
:
2189
99
.
10.
McGranahan
N
,
Furness
AJ
,
Rosenthal
R
,
Ramskov
S
,
Lyngaa
R
,
Saini
SK
, et al
Clonal neoantigens elicit T cell immunoreactivity and sensitivity to immune checkpoint blockade
.
Science
2016
;
351
:
1463
9
.
11.
Ayers
M
,
Lunceford
J
,
Nebozhyn
M
,
Murphy
E
,
Loboda
A
,
Kaufman
DR
, et al
IFN-gamma-related mRNA profile predicts clinical response to PD-1 blockade
.
J Clin Invest
2017
;
127
:
2930
40
.
12.
Le
DT
,
Durham
JN
,
Smith
KN
,
Wang
H
,
Bartlett
BR
,
Aulakh
LK
, et al
Mismatch repair deficiency predicts response of solid tumors to PD-1 blockade
.
Science
2017
;
357
:
409
13
.
13.
Yoshihama
S
,
Roszik
J
,
Downs
I
,
Meissner
TB
,
Vijayan
S
,
Chapuy
B
, et al
NLRC5/MHC class I transactivator is a target for immune evasion in cancer
.
Proc Natl Acad Sci U S A
2016
;
113
:
5999
6004
.
14.
Zaretsky
JM
,
Garcia-Diaz
A
,
Shin
DS
,
Escuin-Ordinas
H
,
Hugo
W
,
Hu-Lieskovan
S
, et al
Mutations associated with acquired resistance to PD-1 blockade in melanoma
.
N Engl J Med
2016
;
375
:
819
29
.
15.
Spranger
S
,
Bao
R
,
Gajewski
TF
. 
Melanoma-intrinsic beta-catenin signalling prevents anti-tumour immunity
.
Nature
2015
;
523
:
231
5
.
16.
Dorand
RD
,
Nthale
J
,
Myers
JT
,
Barkauskas
DS
,
Avril
S
,
Chirieleison
SM
, et al
Cdk5 disruption attenuates tumor PD-L1 expression and promotes antitumor immunity
.
Science
2016
;
353
:
399
403
.
17.
van der Burg
SH
,
Kalos
M
,
Gouttefangeas
C
,
Janetzki
S
,
Ottensmeier
C
,
Welters
MJ
, et al
Harmonization of immune biomarker assays for clinical studies
.
Sci Transl Med
2011
;
3
:
108ps44
.
18.
Baker
RG
,
Hoos
AX
,
Adam
SJ
,
Wholley
D
,
Doroshow
JH
,
Lowy
DR
, et al
The Partnership for Accelerating Cancer Therapies
.
Cancer J
2018
;
24
:
111
4
.
19.
Zeng
Z
,
Fu
J
,
Cibulskis
C
,
Jhaveri
A
,
Gumbs
C
,
Das
B
, et al
Cross-site concordance evaluation of tumor DNA and RNA sequencing platforms for the CIMAC-CIDC network
.
Clin Cancer Res
2021
;
27
:
5049
61
.
20.
Sahaf
B
,
Pichavant
M
,
Lee
B
,
Duault
C
,
Thrash
E
,
Davila
M
, et al
Immune Profiling Mass Cytometry Assay Harmonization: Multicenter Experience from CIMAC-CIDC
.
Clin Cancer Res
2021
;
27
:
5062
71
.
21.
Akturk
G
,
Parra
E
,
Gjini
E
,
Lako
A
,
Lee
JJ
,
Neuberg
D
, et al
Multiplex tissue imaging harmonization: a multicenter experience from CIMAC-CIDC Immuno-Oncology Biomarkers Network
.
Clin Cancer Res
2021
;
27
:
5072
83
.
22.
Olsen
LR
,
Leipold
MD
,
Pedersen
CB
,
Maecker
HT
. 
The anatomy of single cell mass cytometry data
.
Cytometry A
2019
;
95
:
156
72
.
23.
Rizvi
NA
,
Hellmann
MD
,
Snyder
A
,
Kvistborg
P
,
Makarov
V
,
Havel
JJ
, et al
Cancer immunology. Mutational landscape determines sensitivity to PD-1 blockade in non-small cell lung cancer
.
Science
2015
;
348
:
124
8
.
24.
Queirolo
P
,
Morabito
A
,
Laurent
S
,
Lastraioli
S
,
Piccioli
P
,
Ascierto
PA
, et al
Association of CTLA-4 polymorphisms with improved overall survival in melanoma patients treated with CTLA-4 blockade: a pilot study
.
Cancer Invest
2013
;
31
:
336
45
.
25.
Hugo
W
,
Zaretsky
JM
,
Sun
L
,
Song
C
,
Moreno
BH
,
Hu-Lieskovan
S
, et al
Genomic and transcriptomic features of response to anti-PD-1 therapy in metastatic melanoma
.
Cell
2016
;
165
:
35
44
.
26.
Parra
ER
,
Uraoka
N
,
Jiang
M
,
Cook
P
,
Gibbons
D
,
Forget
MA
, et al
Validation of multiplex immunofluorescence panels using multispectral microscopy for immune-profiling of formalin-fixed and paraffin-embedded human tumor tissues
.
Sci Rep
2017
;
7
:
13380
.
27.
Remark
R
,
Merghoub
T
,
Grabe
N
,
Litjens
G
,
Damotte
D
,
Wolchok
JD
, et al
In-depth tissue profiling using multiplexed immunohistochemical consecutive staining on single slide
.
Sci Immunol
2016
;
1
:
aaf6925
.
28.
Buenrostro
JD
,
Giresi
PG
,
Zaba
LC
,
Chang
HY
,
Greenleaf
WJ
. 
Transposition of native chromatin for fast and sensitive epigenomic profiling of open chromatin, DNA-binding proteins and nucleosome position
.
Nat Methods
2013
;
10
:
1213
8
.
29.
Marwitz
S
,
Scheufele
S
,
Perner
S
,
Reck
M
,
Ammerpohl
O
,
Goldmann
T
. 
Epigenetic modifications of the immune-checkpoint genes CTLA4 and PDCD1 in non-small cell lung cancer results in increased expression
.
Clin Epigenetics
2017
;
9
:
51
.
30.
Seremet
T
,
Koch
A
,
Jansen
Y
,
Schreuer
M
,
Wilgenhof
S
,
Del Marmol
V
, et al
Molecular and epigenetic features of melanomas and tumor immune microenvironment linked to durable remission to ipilimumab-based immunotherapy in metastatic patients
.
J Transl Med
2016
;
14
:
232
.
31.
Sade-Feldman
M
,
Yizhak
K
,
Bjorgaard
SL
,
Ray
JP
,
de Boer
CG
,
Jenkins
RW
, et al
Defining T cell states associated with response to checkpoint immunotherapy in melanoma
.
Cell
2019
;
176
:
404
.
32.
Goldberg
SB
,
Narayan
A
,
Kole
AJ
,
Decker
RH
,
Teysir
J
,
Carriero
NJ
, et al
Early assessment of lung cancer immunotherapy response via circulating tumor DNA
.
Clin Cancer Res
2018
;
24
:
1872
80
.
33.
Masucci
GV
,
Cesano
A
,
Hawtin
R
,
Janetzki
S
,
Zhang
J
,
Kirsch
I
, et al
Validation of biomarkers to predict response to immunotherapy in cancer: volume I - pre-analytical and analytical validation
.
J Immunother Cancer
2016
;
4
:
76
.
34.
Piccoli SP
AB
,
Allinson
J
,
Arnold
M
,
Amur
S
,
Aubrecht
J
, et al
Points to consider document: scientific and regulatory considerations for the analytical validation of assays used in the qualification of biomarkers in biological matrices
.
Biomarker Assay Collaborative Evidentiary Considerations Writing Group, Critical Path Institute (C-Path)
, 
2019
.
Available from
: https://c-path.org/wp-content/uploads/2019/06/evidconsid-whitepaper-analyticalsectionv20190613-1.pdf.
35.
Gruning
B
,
Dale
R
,
Sjodin
A
,
Chapman
BA
,
Rowe
J
,
Tomkins-Tinch
CH
, et al
Bioconda: sustainable and comprehensive software distribution for the life sciences
.
Nat Methods
2018
;
15
:
475
6
.
36.
Van der Auwera
GA
,
Carneiro
MO
,
Hartl
C
,
Poplin
R
,
Del Angel
G
,
Levy-Moonshine
A
, et al
From FastQ data to high confidence variant calls: the Genome Analysis Toolkit best practices pipeline
.
Curr Protoc Bioinformatics
2013
;
43
:
11 0 1–0 33
.
37.
Qin
Q
,
Mei
S
,
Wu
Q
,
Sun
H
,
Li
L
,
Taing
L
, et al
ChiLin: a comprehensive ChIP-seq and DNase-seq quality control and analysis pipeline
.
BMC Bioinformatics
2016
;
17
:
404
.
38.
Amir
ED
,
Lee
B
,
Badoual
P
,
Gordon
M
,
Guo
XV
,
Merad
M
, et al
Development of a comprehensive antibody staining database using a standardized analytics pipeline
.
Front Immunol
2019
;
10
:
1315
.