Implementation of Validated Pharmacodynamic Assays in Multiple Laboratories: Challenges, Successes, and Limitations

The development of clinical biomarkers of pharmacodynamic (PD) activity at the level of target engagement was initiated at the National Cancer Institute (NCI) as part of the Division of Cancer Treatment and Diagnosis with the specific intent of providing an accurate, early indication of target engagement by experimental therapeutics in first-in-man clinical trials, focusing on tumor biopsy specimens as the preferred testing material (1). The initial project was to develop and validate an assay that could be objectively demonstrated to accurately report target engagement by veliparib. The assay readout was the quantity of enzyme product, PAR (polyadenosyl ribose polymer) in patient tissues, previously shown to decrease in tumors and peripheral blood mononuclear cells (PBMC) after veliparib treatment (2). The trial objectives included demonstration of achievement of a target plasma level of drug and the inhibition of PARP1 and 2 and in patient biopsy specimens after administration of a single dose of veliparib (3). An additional correlative effort of the project was to measure the effect of veliparib on PARP in circulating PBMCs on the day of drug administration (4). The success of the work in demonstrating target engagement resulted in some additional, and in some respects, unanticipated findings. From this early effort with veliparib evolved an NCI effort to develop a series of PD markers that could be exported to the NCI clinical trials network. The aim was to move the assay beyond a simple laboratory developed test to one that could be exported to other institutions. To achieve this goal, the assays had to be not only analytically validated, but also standardized so that results obtained in different institutions would report comparable analytical results. These assays are now available and accessible to the broader NCI extramural community (5). In the description of the NCI effort to develop PDs markers that follows, we will highlight challenges that were encountered along the way and are summarized in Text Box 1.

Readying PD assays for clinical implementation involves development and validation of clinically transferable standard operating procedures for the entire process from collection and stabilization of the specimen, sample preparation, assay performance through analysis, interpretation, and reporting of the data (Fig. 1). Our goal is to develop and optimize procedures that are sufficiently accurate and precise and that have a suitable dynamic range and sensitivity for the clinically available materials (biopsies, PBMCs, circulating tumor cells; CTC) and the anticipated drug response level. A critical assay performance parameter for PD studies is assay reproducibility. The total imprecision from all sources must be sufficiently low to make it possible for the desired drug effect measurement to reach statistical significance (1). Preclinical modeling not only validates the intended assay procedures, but also even more importantly, helps guide the design of the clinical trial by providing information about the dose necessary to achieve a reproducibly measurable effect on the PD marker and the window of time for tumor biopsy after drug administration that is most likely to yield a PD signal.

Substantial effort is expended to mirror the intended clinical sampling and assay procedures in fit-for-purpose preclinical studies; however, experience in our program has confirmed that it is also critical to perform a small-scale clinical feasibility study on human clinical samples and fine tune assay procedures when required before analyzing clinical trial specimens. Some of the challenges that we have faced with upon initial evaluation of clinical samples in ELISA-based assays are lower than expected levels of the PD analytes and higher than expected variation in baseline biomarker levels. One unexpected finding during the early implementation of the PAR IA measurement on core biopsies was that the protein yield from an 18-guage needle biopsy was significantly lower and more variable compared with the protein yield that had been obtained from core needle biopsies of human tumor xenograft models during fit-for-purpose studies. This required larger volumes of lysate to be loaded into the assay well to achieve the desired protein load for the assay measurement, requiring assay method modifications. A similar problem arose during early implementation of an immunoassay for total topoisomerase I (Top1). Baseline levels of total Top1 were found to be lower than expected in human biopsies as compared with xenograft models. In addition, the baseline level of this biomarker is highly variable (6). In the case of this assay, the lysis buffer and procedure were modified to allow for a larger volume of lysate to be loaded into the assay well without interference. A new procedure was quickly established and was shown to significantly improve assay sensitivity on clinical biopsies as part of the ongoing evaluation of indenoisoquinolines class of investigational Top1 inhibitors, and subsequently was demonstrated to measure a decrease of total Top1 in response to Top1 inhibitor therapy (7).

Adequate functional and analytical specifications and testing procedures must be established for all critical reagents and control materials to identify and mitigate reagent issues with lot-to-lot variability and outright inconsistency, and to manage changes in critical reagent suppliers as R&D companies are bought and sold, and launch and then discontinue these products. It is useful to identify suppliers that are also often necessary when supporting a clinically implemented assay over the long, multi-year course cycle of clinical drug development. Early in the implementation of the PAR IA, failed lots of two different antibody reagents complicated the support of ongoing analyses. In both cases, communication with the research supply company led to the identification of the process changes that were likely the root cause of the lot failures and resolution of the supply issues. Another hurdle that was faced was that the original supplier of the PAR polymer standard was acquired and the product was discontinued. Because no reference material or quantitative reference method was available for this heterogeneous analyte, an anion exchange high-performance liquid chromatography method that was originally reported for the analysis of PAR polymer in vivo was adapted to provide a qualitative analysis of the distribution of the polymer sizes and branching structure in a given PAR standard preparation (8). This method was used to confirm the comparability of the PAR polymer from an alternate vendor. In addition, a master lot of material for the PAR polymer has been established and working lots of standard are compared with this master lot to provide continuity of assigned PAR values over time.

It was an objective of the NCI program to standardize measurement methods for tissues and then disseminate that information through the NCI clinical trials network. This in turn raised the issue of assuring assay quality over time across laboratories. In our strategy, clinically proven PD assays are transferred to the cancer research community with a training and certification program provided at the Frederick National Laboratory for Cancer Research. The training program is designed to achieve operator proficiency via intensive classroom and hands on laboratory training in the standard operating procedures for sample preparation, immunoassay measurement, data analysis, and interpretation. Generation of relevant assay controls and calibrators that can be used at any of the network testing laboratories is also an important part of the strategy for achieving consistent interlaboratory performance of an assay. These controls and calibrators vary from assay to assay, but are supported for all assays that NCI supports through community training and open-access standard operating procedures (SOP; ref. 5). We have established mechanisms to share limited amounts of assay critical reagent, including controls and calibrators, with newly certified PD assay operators so that they can establish proficiency on an assay within their own laboratories.

As our laboratory proceeded through this process of increasing robustness and continued to generate usable data, Trevigen, the vendor of the raw materials for the validated immunoassay, elected to develop a commercial kit that has been successfully used by a number of investigators (9), and which has been offered by the company as part of a service supporting the development of PARP inhibitors by monitoring PAR in PBMCs from patients.

We next sought to develop an assay that would examine the downstream effects of PARP inhibition, DNA damage. Although the PAR immunoassay measures the product of the enzyme targeted by PARP inhibitors and is therefore a direct reporter of target engagement, the pharmacologic objective of PARP inhibition is to impede or prevent DNA repair. Assays to evaluate this are also needed to evaluate the efficacy combining PARP inhibitors in the clinic with drugs that cause DNA damage. Such an assay would also support the PD evaluation of other types of DNA-damaging agents. The Division of Cancer Treatment and Diagnosis initiated a program to develop the indenoisoquinoline class of Top1 inhibitors (NSC 724998, 725776, and 706744; ref. 10) resulting in accumulation of double-strand DNA (dsDNA) breaks. The assay implemented for this drug effect was measurement of pS139 H2Ax (called γ-H2AX in the literature) in formalin-fixed, paraffin-embedded biopsy tissues via immunofluorescence assay (IFA) using microscopy (11). Phosphorylation of histone H2AX to form γ-H2AX is well documented as a marker for DNA double-strand breaks and programmed cell death (11–13). Several immunologic assays have been reported for evaluation of γ-H2AX, all based on a specific antibody that recognizes phosphorylated H2AX protein.

The work was greatly helped by the existence of a commercial monoclonal antibody (JBW301) that was available in large quantities either conjugated to FITC or unconjugated, and which was highly reproducible across lots purchased from the vendor, as established by a series of internal quality control measures including specificity testing by Western blotting and on test tissues established as positive and negative for target expression. During assay development, direct modulation of γ-H2AX levels by the three indenoisoquinoline Top1 inhibitors in mouse target tissues included skin (hair follicles become positive), intestine (crypt cells become positive), as well as in both responsive and nonresponsive xenografts (13). A key outcome of this work was the demonstration of a strong correlation between tumor growth inhibition (measured xenograft volumes) and the amount of γ-H2Ax induced on day 1 of the treatment cycle as a function of dose, for both topotecan, a known Top1 inhibitor, and 724998 (now called LMP400 and in clinical trials at NCI). The analytical validation, fitness-for-purpose demonstration, and interlaboratory transfer of the tissue IFA and its application in the clinic demonstrated robustness of the methodology, but once again the need for assay modification arose after transfer, once testing on actual clinical specimens was initiated. Important elements that were necessary for the clinical implementation included a sampling strategy in which the area of the biopsy to be tested was bounded by use of hematoxylin and eosin-stained slides to demonstrate presence of tumor and exclude areas of tumor necrosis, then subsequently testing a series of seven 5-micron slides, separated from each other by three sections, across the biopsy. This approach also had the advantage of providing backup slides to use for restaining if required and for additional correlative marker assessment.

Tissue extraction assay methods have the advantage of reporting the average analyte concentration and distribution within a biopsy but have the disadvantage that the amount of signal specifically associated with tumor cells cannot be assessed. Tissue IFAs have the capability of reporting that data, but unless multiple slides are tested, actual distribution throughout the volume of the biopsy is not assessable. In addition, estimation of target concentration is problematic. We overcame these issues by quantifying the number of cells in the biopsy that became γ-H2AX positive after drug treatment and scaled the effect by use of a responsive xenograft tissue “reference standard” which we used to determine the fraction of cells exhibiting dsDNA breaks in a sensitive xenograft model at doses that caused tumor regression in mouse models, and at lower dose levels that did not. The ability to construct a response curve from these reference standards was enabled by the dynamic range of the assay: from 1% to 3% of cells positive in vehicle-treated controls to >30% positive in tumors of mice treated at the daily × 5 maximum tolerated dose of the test compound, which caused tumor regression.

Our preclinical experience with the robustness of γ-H2Ax as a PD biomarker suggested additional useful assay platforms. As a replacement for normal tissue surrogates (PBMCs and skin), we developed PD biomarker applications on the CellSearch CTC platform, already used clinically for patient monitoring (described by Yap and colleagues in this CCR Focus section; ref. 14) opening up the possibility of measuring drug effect on putative tumor cells in the blood instead of requiring a tumor biopsy.

We successfully developed this methodology and reported that active drugs known to induce DNA damage (topotecan and gemcitabine) also induced a strong γ-H2Ax signal in CTCs isolated from patients on clinical trials. In contrast, drugs that do not induce DNA damage (the kinase inhibitor fostamatinib and rapamycin; refs. 15–17) did not induce a γ-H2Ax signal in CTCs. A limitation of the approach was the necessity of testing CTCs 24 hours after drug administration due to limitations within the clinic and the uncertainty around CTC half-life in the circulation (estimated to be 1–2 hours; ref. 18). The concern was that the initially high blood concentration of drug during an intravenous infusion could cause the PD response in CTCs, rather than drug exposure inside the tumor lesions, so CTC sampling emphasized time points well after peak drug levels were reached. Although subsequent studies have not identified an immediate response in CTCs that could be associated with high plasma drug concentrations, the γ-H2Ax signal observed in the first clinical applications of the assay was not the punctate, focal pattern associated with the immediate response of the repair enzymes to DNA damage. Rather, the γ-H2Ax signal 24 hours after dosing was distributed throughout the nucleus (Fig. 2), and images also showed a number of γ-H2Ax–positive CTCs with the nuclear segmentation that is a hallmark of apoptosis (19–21).

The original IFAs with drug–response calibrators for use with tumor biopsies and the CellSearch CTC-based assay are now part of the NCI training program for clinical trial laboratories, and assay controls and validation methods are also taught and distributed. Veridex currently offers γ-H2AX testing for clients in its CLIA laboratory.

Assay selection for specific translational research is largely dependent on study intent, assay sensitivity and specificity, intra- and interassay variability, quantitative/dynamic range, and labor and equipment costs. Western blotting and Dot blotting assays are used for detecting γ-H2AX in cell lysates, whereas conventional or confocal microscopy-based IFAs and flow cytometry-based fluorescence-activated cell sorting are used for quantifying γ-H2AX signal from intact cells (2, 22), although the latter are much more costly and have lower throughput. With these limitations in mind, a third assay was developed and reported (23), using the same tissue extraction and the two-site enzyme immunoassay (EIA) technique developed for measuring PAR. This γ-H2AX immunoassay uses one unique pair of high-affinity anti-γ-H2AX antibodies as a sandwich bracketing the γ-H2AX antigen, rather than one antibody, which improves assay specificity compared with a single antibody detection step. A mouse monoclonal antibody to phosphorylated H2AX is immobilized onto each well of plate as capture antibody. A rabbit polyclonal antibody against total H2AX of both phosphorylated and nonphosphorylated protein is used for detection. Anti-rabbit horseradish peroxidase conjugate is subsequently used as the reporter in the presence of chemiluminescent substrates for γ-H2AX quantitation. Synthetic phosphorylated H2AX peptide standard at eight different concentrations from 1 to 128 pmol/L is used as quantitative calibrators, and well-characterized extracts of the cancer cell line MCF-7 at three known concentrations are used for assay controls. γ-H2AX quantitation is normalized on the basis of cell numbers or total protein in lysate. A detailed assay SOP is being posted on the NCI-DCTD website (5), and Trevigen has developed and commercialized a quantitative sandwich EIA to measure γ-H2AX in cell and tissue extracts.

Cancer cells develop enhanced survival and greater resistance to therapy by evading normal process of programmed cell death or apoptosis by a variety of mechanisms (24, 25). Apoptosis is a complex process and involves multiple prosurvival and proapoptotic proteins mainly belonging to a super-family of proteins known as Bcl2 proteins (26). The prosurvival members of Bcl-2 family proteins such as Bcl-2, Bcl-Xl, Mcl-1, Bcl-w, and A1 dynamically engage or sequester proapoptotic members of Bcl2 family such as BAX, BAK, BIM, Bad, Puma, Noxa, and Bid to regulate cellular commitment to apoptosis through mitochondrial outer membrane permeabilization (27). It seems that cancer cells dysregulate the delicate balance between pro- and antiapoptotic proteins by overexpression of prosurvival protein members such as Bcl2 in leukemia and colon cancers, by posttranslational modification of proapoptotic proteins as seen in some breast cancers, and by regulating proteasomal degradation or transcriptional regulation of proteins involved in apoptosis. Several first-in-class drugs are designed and developed with a specific goal of blunting these mechanisms and thus kill cancer cells (24, 28). These drugs selectively target proteins involved in apoptosis pathway and can be roughly grouped into four categories: (i) Bcl2 or BH3 mimetics (29–33); (ii) IAP inhibitors or Smac mimetics (34, 35); (iii) TRAIL agonist and activating antibodies (36–38); and (iv) proteasome inhibitors (39–41). As these newer drugs move to advanced stages of clinical development, they present unique challenges for PD monitoring and require effective biomarker implementation strategies. Several members of the Bcl2 family proteins have been explored as potential biomarkers, in preclinical and in a few clinical studies. Overexpression of Bcl-2 (or its family) measured by real-time PCR is the most often studied prognostic biomarker for leukemia, whereas immunohistochemistry determination of Ki-67 and cleaved caspases is the most explored PD biomarker for drugs targeting apoptosis (42, 43). In addition, circulating levels of Bcl2 proteins, FASl, TRAIL, surviving, and caspase fragments in blood and immunoblot measurement of Bcl2 family proteins and cleaved caspases in tumor tissues have been studied in some clinical trials. However, none of these biomarkers or their implementation strategies has proven to be effective in determining PD modulation for the drug classes named above (44, 45).

We developed a multiplex biomarker approach, on the Luminex (multiplexed sandwich immunoassay) platform, to study PD modulation by drugs targeting apoptosis (46). This approach involves quantitative measurements of multiple biomarkers covering the intrinsic apoptosis pathway, and not only provides estimates of individual target modulation, but also surveys the entire pathway using a single tumor biopsy, thus maximizing the amount of PD information that can be obtained from a single core needle biopsy.

The assays have been developed in collaboration with a commercial provider (RBM Myriad) to assure availability of robust reagents to the wider research community. A unique feature of the multiplex panel (Table 1) is its precise quantitative measurements of the homo- and heterodimers formed in vivo between the various prosurvival and proapoptotic proteins, intended to reveal how these oligomers are affected by drug (27, 28, 30). Currently, such assays can only be performed in a qualitative manner by immunoprecipitation techniques (47), which is not feasible in core needle biopsies. The immunoassays we developed utilize well-characterized recombinant fusion-proteins as calibrators that mimic the in vivo formed heterodimer of Bcl2 proteins. The calibrators also provide a means to compare results from different trials and patients. Finally, the apoptosis process occurs at the interface of the cytosol and mitochondrial membrane where several members of the Bcl2 family are localized. Therefore, central to our strategy for implementation of apoptosis biomarkers was subcellular fractionation of tissue lysates for PD measurements. This strategy also minimizes any in vitro activation of caspases that can occur if mitochondrial contents are mixed with that of cytosol. We implemented a simple protocol to prepare fractionated tissue lysates that can be used in most clinical laboratories. Although the sample processing and assay method are undergoing thorough validation in a clinical trial to study the PD of an apoptosis modulation, sufficient data have been generated from preclinical studies involving birinapant (Smac mimetic; ref. 46) and selumetinib (MAP–ERK kinase inhibitor; unpublished data) to demonstrate that definitive PD responses can be measured in fractionated extracts. The feasibility of fractionated extractions of 18 g needle biopsies has been demonstrated in biopsies collected from multiple patients (unpublished data), and the protein yields and fractionation's efficiency conform to acceptable limits.

Our experience with PD biomarkers of apoptosis is still evolving and success of this approach will be revealed by the ongoing clinical utility studies. We envision clinical implementation of the assay panel to be drug specific in at least some cases. For example, the best possible PD biomarker indicators of drug action leading to apoptosis, in a dose and time-dependent manner, can be identified in preclinical animal models. Then, that preferred PD biomarker of the apoptosis panel for that drug can be designated in the clinical protocol as the clinical correlate to be measured in cases where the amount of biopsy material available is limited—a focus that may justify use of just one of the extraction panels for the assay, thereby saving time, money, and patient material.

In summary, PD markers are critical for determining whether a given drug is modulating the target in an expected manner. We have recognized in the effort to develop validated PD markers studies that the process requires a meticulous, thorough approach that is unique for each analyte. Not every analyte will be amenable to PD marker development. Some of those developed successfully are included in Text Box 2. Numerous challenges must be faced during the development of PD assays, including baseline specimen heterogeneity in the studied population, intratumor heterogeneity, detection methodology, reagent supply and stability, availability of reference materials, and generation of standards and controls. We have been surprised to learn that the markers do not always change in the manner expected on the basis of preclinical models, and to learn that the levels in patient samples are often far lower than observed in those preclinical models. These challenges support a dedicated, noncommercial effort to development of assays for the cancer research community.

R. Kinders is a consultant/advisory board member for Trevigen. L. Wang is a consultant/advisory board member for Janssen Diagnostics, LLC. No potential conflicts of interest were disclosed by the other authors.

Conception and design: R. Kinders, A.K. Srivastava, J. (Jay) Ji, R.E. Parchment

Development of methodology: R. Kinders, K. Ferry-Galow, L. Wang, A.K. Srivastava, J. (Jay) Ji, R.E. Parchment

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): K. Ferry-Galow, L. Wang, A.K. Srivastava, J. (Jay) Ji

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): R. Kinders, K. Ferry-Galow, L. Wang, A.K. Srivastava, J. (Jay) Ji, R.E. Parchment

Writing, review, and/or revision of the manuscript: R. Kinders, K. Ferry-Galow, L. Wang, A.K. Srivastava, J. (Jay) Ji, R.E. Parchment

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): R. Kinders, L. Wang, J. (Jay) Ji

Study supervision: R. Kinders, A.K. Srivastava, J. (Jay) Ji, R.E. Parchment