Purpose: Define an immunologic response using the tetramer and enzyme-linked immunospot (ELISPOT) assays.

Experimental Design: Ten healthy subjects and 21 patients with melanoma (all HLA-A*0201) donated a total of 121 blood samples to determine the lower limit of detection (LLD), analytic coefficient of variation (aCV), and physiologic CV (pCV) of the tetramer and ELISPOT assays. The mean, SD, and reference change value (RCV) were calculated to define changes beyond the assay imprecision, and its application was tested in the monitoring of T-cell expansion after CTLA4 blockade with ticilimumab (CP-675,206).

Results: The LLD for the tetramer assay was 0.038% CD8+ cells and seven spots per 105 peripheral blood mononuclear cells for the ELISPOT assay. The aCV of the tetramer assay was <10% and was higher for the ELISPOT (24.69-36.32%). There was marked between-subject variability on baseline homeostatic values, which was correlated to prior antigen exposure. An immunologic response was defined as an increase beyond the mean + 3 SD in antigen-specific cells for subjects with baseline levels below the LLD, or beyond the assay RCV for baseline levels above the LLD. In four patients receiving ticilimumab, expansions of antigen-specific T cells beyond the assay variability were noted for EBV and MART1 antigens.

Conclusions: A combined approach of change from negative (below the LLD) to positive (above the LLD) and a percentage change beyond the assay variability using the RCV score can be computed to define which change in circulating antigen-specific T cells represents a response to immunotherapy.

Modern cellular immunologic assays, like the enzyme-linked immunospot (ELISPOT; refs. 14), the MHC tetramer binding assay (5, 6), and the intracellular cytokine flow cytometry assay (7, 8), allow for the enumeration of antigen-specific T lymphocytes at a single cell level (911). These assays have rapidly replaced older assays requiring extensive ex vivo manipulations, like the limiting dilution microcytotoxicity assay and the lymphoproliferative assay (10). However, few studies had been conducted to define key methodologic variables, like accuracy, precision, and reproducibility (10). These variables are critical to determine which level of T-cell expansion is beyond the baseline variability of the assay and therefore would represent a positive immune response.

The most commonly used approach to define a positive immune response is the detection of circulating antigen-specific T cells after immunotherapy beyond the mean and 2 or 3 SD from negative values, using results from a negative control antigen or a positive antigen in a negative population (10, 12). This approach is logical for values that are below a certain threshold of negativity at baseline, which would reflect the assay background. However, the baseline values of circulating antigen-specific T cells for many infectious disease and tumor antigens can be positive at baseline for a significant subset of patients (12, 13). In this case, the definition based on a fixed value (an increase beyond the mean + SD from negatives) would not be easily applicable. This approach could only be used if several baseline values were collected for each patient, allowing defining a mean and SD for their individual baseline values of circulating antigen-specific T cells, and then comparing it to its change after immunotherapy. This is not feasible in routine practice.

The calculation of the reference change value (RCV) may allow a more reliable definition of an immune response if baseline values are above the assay lower limit of detection (LLD). The RCV allows determining the change that must occur in an individual's serial test results before the change is considered significant by taking into account the main components of assay variability and a desired statistical significance. Its calculation has the goal to define the magnitude of change in results that would be beyond the assay imprecision. Since its original description by Harris and Brown in 1979 (14), different versions of the formula have been proposed (1519). We reasoned that when applied to immunologic monitoring, it would allow us to define positive and negative changes in circulating antigen-specific T cells after an immunotherapy intervention for subjects with baseline measurable values (above the assay LLD), thereby representing a true positive or negative immune response and not the assay noise.

Study eligibility and screening procedures for the methodology study. HLA-A*0201-positive healthy subjects and HLA-A*0201-positive patients with melanoma were eligible, provided that they were not receiving active therapy or having an intercurrent event (like an influenza infection or symptoms of common cold) for a minimum period of 30 days before the blood draws and during the blood draws. Eligible patients with melanoma included stage II to IV melanoma, a Karnofsky performance status of >70%, and adequate hematopoietic, hepatic, and renal function. To be eligible, tumors were required to express Melan-A/MART-1 (MART-1) by immunohistochemistry or reverse transcription-PCR. Exclusion criteria included prior therapy < 4 weeks before study entry, untreated central nervous system metastasis, pregnancy, or concurrent immunosuppressive conditions or therapy. All subjects signed a written informed consent approved by the University of California at Los Angeles Institutional Review Board (UCLA IRB 01-12-029). Screening procedures included complete blood counts; serum electrolyte determination; liver function tests; coagulation tests; serology determination of antibodies to HBV, HCV, cytomegalovirus (CMV), EBV, and HIV; and an autoimmunity panel testing, including anti-nuclear antibodies, rheumatoid factor, anti-neutrophilic cytoplasmic, anti-tyroglobin, anti-LK microsome, anti-Ro, anti-La, and anti-phospholipid.

Blood collection and cryopreservation. Subjects underwent three to four peripheral blood collections of 40 mL of whole blood: an initial one on study day 1, a second one between study days 2 and 7, a third one at weeks 2 to 4, and an optional fourth one between weeks 5 and 7. Additionally, two patients (study subjects 103 and 109) with a clearly defined population of antigen-positive cells underwent leukapheresis under a different protocol (UCLA IRB 93-07-289). Peripheral blood mononuclear cells (PBMC) were obtained by Ficoll-Hypaque (Amersham Pharmacia, Piscataway, NJ) centrifugation of whole blood or the leukapheresis product. PBMCs were cryopreserved in liquid nitrogen in RPMI (Life Technologies Bethesda Research Laboratories, Gaithersburg, MD) supplemented with 20% (all as v/v) heat-inactivated AB human serum (OmegaSci., Inc., Tarzana, CA) and 10% DMSO (Sigma, St. Louis, MO). PBMCs were cryopreserved at a concentration of 5 × 106/mL.

Subject enrollment and blood collection in the validation study. Four HLA-A*0201-positive patients with metastatic melanoma receiving the anti-CTLA4 monoclonal antibody ticilimumab (Pfizer, Inc., New London, CT) were included in the methodology validation part of the study. These four patients had been previously enrolled in a phase I single-dose escalation trial at the University of California at Los Angeles (IRB 01-09-054; ref. 20). The patients consented to donate blood for immunologic studies at redosing at 3 or 6 mg/kg under a separate redosing study (IRB 03-01-014). Two were redosed after being enrolled at the initial low dose cohorts, and two were enrolled after having clinical benefit to the initial dose.

Peptides and tetramers. The following peptides were used: AFP325-332 (GLSPNLNRFL), CMVpp65495-503 (NLVPMVATV), EBV BMLF1259-267 (GLCTLVAML), MART-126-35 (ELAGIGILTV), MART-127-35 (AAGIGILTV), and a HLA-A*0201-binding nonrelevant peptide (ref. 21; referred to as the negative peptide from here on). All tetramers were purchased from Beckman Coulter, Inc. (San Diego, CA), except a PE-labeled HLA-A*0201 MART127-35 tetramer, which was obtained from the National Institute of Allergy and Infectious Disease (Emory University, Atlanta, GA).

Tetramer-binding assay. Cryopreserved PBMC aliquots were thawed and diluted with RPMI supplemented with 10% human AB serum, 1% penicillin, streptomycin, amphotericine (PSA, OmegaSci), and DNase (0.002%, Sigma) for an hour at 37°C. PBMCs were then washed and resuspended at 1 to 3 × 106/100 μL in 100% adult bovine serum (OmegaSci). Only samples with viability of >90% by trypan blue exclusion were used. PBMCs were stained for 30 minutes at room temperature in the dark with 8 μL MHC tetramer, 8 μL FITC-conjugate anti-CD8 (SFI21Thy2D3), and 8 μL iMASC antibodies (a pre-prepared mixture with PC5-conjugated CD4 −13B8.2−, CD13 −IMMU103.44−, and CD19 −J4.119−, all from Beckman Coulter), which were used to gate out CD8 lymphocytes. Cells were washed in 3 mL of PBS and, immediately before flow cytometric analysis, 10 μL of 7-amino-actinomycin D was added to gate out dead cells. PBMCs were analyzed on a FACSCalibur (BD Biosciences, San Jose, CA) within 1 hour of staining, without adding any fixative. Stained cells were kept constantly at 4°C. A minimum of 30,000 CD8+ T cells were acquired, and up to 200,000 were preferred. Analysis was done both with CellQuest (Beckman Coulter) and FCS Express (DeNovo Software, Thornhill, Ontario, Canada) software. In addition to the routine daily FACSCalibur calibration, FLOW-SET Fluorospheres (Beckman Coulter) were used to set peak channels before every tetramer flow cytometry experiment.

For tetramer spiking experiments, CD8+ T cells were isolated by magnetic column separation (Miltenyi Biotec, Auburn, CA). CD8+ cells were stained with the CMVpp65 tetramer and sorted into tetramer-positive or tetramer-negative cells using a FACSAria (BD Biosciences). Immediately after sorting, tetramer-negative but CD8+ cells were mixed with the CD8+/CMVpp65 tetramer-positive cells at different rations and analyzed on a FACSCalibur. At least 80,000 CD8+ T cells were acquired.

IFN-γ ELISPOT assay. Multiscreen HA plates (Millipore, Bedford, MA) were coated overnight at 4°C with 4 μg/mL of purified antihuman IFN-γ monoclonal antibody (BD PharMingen, San Diego, CA) in a 0.1 mol/L sodium bicarbonate buffer (pH 8.2). Unbound antibody was removed by five washings with PBS, and nonspecific binding to plates was blocked with PBS/10% (v/v) heat-inactivated human AB serum (1 hour, 37°C). Different concentrations of thawed PBMCs and antigen-presenting cells (APC) were seeded in triplicate at a final volume of 200 μL/well in X-Vivo 10 media (Bio Whittaker, Walkersville, MD) supplemented with 10% (v/v) heat-inactivated human AB serum. The following APC were tested: T2, purchased from American Type Culture Collection (Rockville, MD); JY, provided by Dr. Martin Kast (Loyola University, Mayfield, IL); and HLA-A*0201-transfected K562 (K562/A*0201), provided by Prof. Wolfgang Herr and Dr. Cedrik M. Britten (Johannes Gutenberg-University of Mainz, Mainz, Germany; ref. 22). Plates were incubated for 24 hours at 37°C in 5% CO2 in a water-saturated atmosphere. After washings with ice-cold PBS/0.05% Tween 20 (Fisher Scientific, Chino, CA), plates were incubated overnight with 3 μg/mL of biotinylated mouse anti-human IFN-γ antibody (BD PharMingen) at 4°C. Avidin-peroxidase (1:2,000 dilution; Vector, Burlingame, CA) was added for 1 hour in the dark at room temperature, and spots were developed using 0.4 mg/mL of 3-amino-9-ethylcarbazole (Sigma) in N,N′-dimethylformamide, freshly diluted 1:25 in 0.05 mol/L sodium acetate (pH 5), and analyzed in a series 1 Immunospot Image Analyzer (Cellular Technology Ltd., Columbus, OH). The murine anti-HLA-A*02 antibody BB7.2 (BD PharMingen) was used to confirm surface expression of HLA-A*0201 on K562/A*0201 cells. K562/A*0201 cell cultures used in this study were free of Mycoplasma infection, tested using a Mycoplasma PCR ELISA kit (Roche Diagnostic, Indianapolis, IN).

Statistical analysis. The mean and SDs were calculated using Microsoft Excel. The data set was inspected for outliers, which were evaluated using Reed's criterion (18). We defined the LLD as the mean + 2 SDs for an HLA-A*0201-matched negative peptide not eliciting a detectable response by both assays in the majority of the study population. The coefficient of variation (CV) was calculated as CV = (SD / mean) × 100. Two CVs were calculated for each epitope and assay, the analytic CV (aCV) and the physiologic CV (pCV). The aCV defines the intra-assay variability, whereas the pCV defines the within-subject biological variability. We defined an acceptable aCV as an assay with ≤20% CV. The RCV to define a significant negative or positive change in baseline values above the LLD was calculated as RCV = 21/2 × Z × (aCV2 + pCV2)1/2, where a bidirectional Z score with a 95% or 99% probability of significance was used, and the median aCV and pCV derived from subjects with mean values above the assay LLD (17, 18, 23). The Z score was adjusted to the degrees of freedom corresponding to the mean minus one from the mean number of samples analyzed for each assay and condition.

Subject characteristics

Thirty-one HLA-A*0201-positive subjects were enrolled, including 10 healthy subjects and 21 patients with a diagnosis of melanoma. Self-referred healthy subjects were younger (median age of 34 versus 59 for patients with melanoma) and were predominantly female (90% versus 35% for patients with melanoma). Nine patients with melanoma had no evidence of disease, and 12 had measurable melanoma during blood collections. Eligibility criteria required no active oncologic treatment or intercurrent illness for at least 30 days before enrollment and during the entire blood collection time.

Tetramer assay

Assay optimization. Standardizing the conditions of blood procurement, processing, and assay methodology allows for minimizing preanalytic variation (18). Therefore, initial trial runs were done to establish the optimal conditions and reagents for the tetramer assay, with the goal of generating standard operating procedures. Using PBMCs from subjects with known populations of tetramer-positive cells, different tetramers sources (National Institute of Allergy and Infectious Disease or Beckman Coulter) and algorithms for gating tetramer/CD8 double-positive cells were tested. We noted marked variations in the absolute number of CD8+ cells within subjects (data not shown). To maintain consistency, eligible samples were required to allow the collection of a minimum of 30,000 CD8+ events, and up to 200,000 were collected if feasible. Results are presented as the percentage of tetramer-positive cells among total CD8+ cells throughout this study. Additional details on the preferred reagents and approach are described in Materials and Methods and in Fig. 1A.

Fig. 1.

Assay optimization studies. A, gating algorithm for the MHC tetramer assay. PBMCs cryopreserved from a subject with a known defined population of CMVpp65495-503 tetramer-positive cells were thawed and stained with CMVpp65495-503 tetramers. i, cells positive for iMASC (non-CD8 WBC) or 7-amino-actinomycin D (7-AAD, dead cells) were excluded and gating was focused on the R1 population. ii, from the R1 population, only cells with high surface expression of CD8+ were gated on. iii, from the R1 + R2 population, cells with a size consistent with lymphocytes were gated on in a forward and side scatter. iv, from the R1 + R2 + R3 population, we selected the CD8/CMVpp65495-503 tetramer double-positive population excluding double-positive cells with low CD8 expression. B, biologic limit of detection derived from a spike experiment. CD8+ cells from a subject with a clearly defined population of CMVpp65495-503-specific T cells were stained with the CMVpp65495-503 tetramer and sorted by flow cytometry. The CMVpp65495-503 tetramer-negative population was then spiked with different concentrations of cells from the CMVpp65495-503 tetramer-positive population. The resulting mixtures were restained with the CMVpp65495-503 tetramer and analyzed by flow cytometry. C, choice of antigen-presenting cells for the ELISPOT assay. PBMCs from a subject with known CMVpp65495-503 reactivity were restimulated with JY or T2 cells pulsed with the CMVpp65495-503 peptide and analyzed from IFN-γ-producing cells. In a second experiment, PBMCs from a subject with known reactivity to EBV BMLF1259-267 were restimulated with T2 or K562/A*0201 cells pulsed with the CMVpp65495-503 peptide and analyzed for IFN-γ-producing cells. D, calculations and significance of the RCV. Calculation of the aCV. The aCV was calculated by analyzing samples derived from a single blood draw that had been cryopreserved in multiple aliquots. The mean value and SD from the different aliquots were used to calculate the aCV for each blood draw, and the overall aCV was calculated using the median of the individual aCV. Calculation of the pCV. The pCV was calculated by analyzing samples derived from three or four blood draws taken from the same subject without intercurrent treatments. The mean value and SD from the different blood draws were used to calculate the pCV for each subject, and the overall pCV was calculated using the median of the individual pCV. Calculation of the RCV. The formula for the RCV incorporates a constant, a Z score that reflects the desired level of significance (95% or 99%), and the computed median aCV and pCV for each assay and antigen. E, significance of the RCV. The mean and SD for the MART-126-35 tetramer results in two extreme cases are used to illustrate the problem of using a single definition for a positive immune response. The tetramer results for patient 102, with a baseline number of circulating MART-126-35-specific T cells below the LLD (red), and for patient 109, with a baseline number of circulating MART-126-35-specific T cells above the LLD, are presented in a semilogarithmic plot. A fixed change (defined by the mean + 3 SD based on results of a negative antigen) would allow to define a response in patient 102, who has homeostatic set value below the assay LLD. However, only the RCV could define a response in patient 109, because adding the mean + 3 SD of results based on a negative antigen would not be considered a biologically significant change.

Fig. 1.

Assay optimization studies. A, gating algorithm for the MHC tetramer assay. PBMCs cryopreserved from a subject with a known defined population of CMVpp65495-503 tetramer-positive cells were thawed and stained with CMVpp65495-503 tetramers. i, cells positive for iMASC (non-CD8 WBC) or 7-amino-actinomycin D (7-AAD, dead cells) were excluded and gating was focused on the R1 population. ii, from the R1 population, only cells with high surface expression of CD8+ were gated on. iii, from the R1 + R2 population, cells with a size consistent with lymphocytes were gated on in a forward and side scatter. iv, from the R1 + R2 + R3 population, we selected the CD8/CMVpp65495-503 tetramer double-positive population excluding double-positive cells with low CD8 expression. B, biologic limit of detection derived from a spike experiment. CD8+ cells from a subject with a clearly defined population of CMVpp65495-503-specific T cells were stained with the CMVpp65495-503 tetramer and sorted by flow cytometry. The CMVpp65495-503 tetramer-negative population was then spiked with different concentrations of cells from the CMVpp65495-503 tetramer-positive population. The resulting mixtures were restained with the CMVpp65495-503 tetramer and analyzed by flow cytometry. C, choice of antigen-presenting cells for the ELISPOT assay. PBMCs from a subject with known CMVpp65495-503 reactivity were restimulated with JY or T2 cells pulsed with the CMVpp65495-503 peptide and analyzed from IFN-γ-producing cells. In a second experiment, PBMCs from a subject with known reactivity to EBV BMLF1259-267 were restimulated with T2 or K562/A*0201 cells pulsed with the CMVpp65495-503 peptide and analyzed for IFN-γ-producing cells. D, calculations and significance of the RCV. Calculation of the aCV. The aCV was calculated by analyzing samples derived from a single blood draw that had been cryopreserved in multiple aliquots. The mean value and SD from the different aliquots were used to calculate the aCV for each blood draw, and the overall aCV was calculated using the median of the individual aCV. Calculation of the pCV. The pCV was calculated by analyzing samples derived from three or four blood draws taken from the same subject without intercurrent treatments. The mean value and SD from the different blood draws were used to calculate the pCV for each subject, and the overall pCV was calculated using the median of the individual pCV. Calculation of the RCV. The formula for the RCV incorporates a constant, a Z score that reflects the desired level of significance (95% or 99%), and the computed median aCV and pCV for each assay and antigen. E, significance of the RCV. The mean and SD for the MART-126-35 tetramer results in two extreme cases are used to illustrate the problem of using a single definition for a positive immune response. The tetramer results for patient 102, with a baseline number of circulating MART-126-35-specific T cells below the LLD (red), and for patient 109, with a baseline number of circulating MART-126-35-specific T cells above the LLD, are presented in a semilogarithmic plot. A fixed change (defined by the mean + 3 SD based on results of a negative antigen) would allow to define a response in patient 102, who has homeostatic set value below the assay LLD. However, only the RCV could define a response in patient 109, because adding the mean + 3 SD of results based on a negative antigen would not be considered a biologically significant change.

Close modal

LLD. The LLD is usually estimated as the mean of a blank sample + 2 SD, which is sometimes described as the limit of absence (23). It describes the range of numbers that are possible when the correct result is 0. PBMC aliquots from cryopreserved samples were analyzed for reactivity to two candidate negative control epitopes, the AFP325-332 peptide (24) and an irrelevant HLA-A*0201-binding peptide (ref. 21; termed negative peptide). The negative tetramer resulted in low and tight tetramer binding (Table 1), with all 31 subjects having similar mean values (0.000-0.04%). For AFP325-332, the mean values had a wider range (0.00-0.22%). Four subjects (two healthy donors and two patients with melanoma) had a clearly evident AFP325-332-positive population in peripheral blood and were excluded from the computation of the LLD for this antigen. The LLD was defined as 38 CD8+ cells nonspecifically binding to the negative tetramer within a population of 105 CD8+ cells (0.038%, Table 1), which provides a stringent level to define the assay background noise.

Table 1.

LLD and aCV of the tetramer and ELISPOT assays

NegativeAFP325-332EBV BMLF1259-267*CMV pp65495-503*MART-126-35*MART-127-35*
Tetramer       
    No. samples 129 109 29 30 76 — 
    No. subjects 31 27  
    No. time points 121 109 17  
    Mean (SD) 0.017 (0.01) 0.040 (0.02) 3.25 (0.34) 0.37 (0.05) 3.88 (0.41) — 
    LLD 0.038 0.084 — — — — 
    aCV (%) — — 8.24 6.55 8.30 — 
ELISPOT       
    No. samples 310 328 121 79 81 61 
    No. subjects 27 29 
    No. time points 108 112 15 12 
    Mean (SD) 2 (3) 2 (2) 113 (41) 99 (25) 137 (37) 68 (20) 
    LLD — — — — 
    aCV — — 27.96 24.69 36.32 34.18 
NegativeAFP325-332EBV BMLF1259-267*CMV pp65495-503*MART-126-35*MART-127-35*
Tetramer       
    No. samples 129 109 29 30 76 — 
    No. subjects 31 27  
    No. time points 121 109 17  
    Mean (SD) 0.017 (0.01) 0.040 (0.02) 3.25 (0.34) 0.37 (0.05) 3.88 (0.41) — 
    LLD 0.038 0.084 — — — — 
    aCV (%) — — 8.24 6.55 8.30 — 
ELISPOT       
    No. samples 310 328 121 79 81 61 
    No. subjects 27 29 
    No. time points 108 112 15 12 
    Mean (SD) 2 (3) 2 (2) 113 (41) 99 (25) 137 (37) 68 (20) 
    LLD — — — — 
    aCV — — 27.96 24.69 36.32 34.18 
*

Calculated by only taking into account the samples with values above the LLD defined using the negative peptide.

We then compared this result with the biological limit of detection, which can be derived from a spiked sample whose concentration approximates the detection limit (23). CMVpp65495-503 tetramer-positive cells were sorted by flow cytometry and spiked into the CMVpp65495-503 tetramer-negative population. In these studies, five CMVpp65495-503 tetramer-positive cells could be detected in a background of 106 CD8+ but CMVpp65495-503 tetramer-negative cells (0.0005%; Fig. 1B). This apparent improvement in assay sensitivity is achieved by measuring high-intensity tetramer staining spiked events, as opposed to the determination of the LLD, which more adequately reflects the problem of nonspecific low-intensity tetramer staining representing the assay background.

Analytic variability. The analytic variability of the assay was determined using replicate cryopreserved aliquots derived from the same blood draw (either a peripheral blood draw with multiple aliquots or a leukapheresis) that were rerun at different time points (Fig. 1D). Only samples with CMVpp65495-503, EBV BMLF1259-267, and MART-126-35 tetramer values above the LLD were used (values below the LLD should not be considered “measurable”). aCV ranged from 6.55% to 8.30%, depending on the epitope tested (Table 1), indicating that the tetramer assay had an acceptable precision (below 20%).

Physiologic variability. The availability of three (in three subjects) or four (in 28 subjects) PBMC samples obtained over a period of time (3-9 weeks) without a concurrent treatment or new illness, for a total of 121 blood draws, allowed us to determine the physiologic (within person or biological) variability in baseline reactivity of multiple tetramer assays (Fig. 1D). Figure 2A, B, and C shows large differences among the set points of different individuals. In addition, results for subjects vary around their own homeostatic set point (depicted as the SE for the different assay results for each subject). To minimize the effect of this between-subject variability, we divided the population between subjects that may have been previously exposed to the antigen (and therefore should be more likely to have developed a T-cell response), using surrogate clinical and laboratory data. Circulating CMVpp65495-503 and EBV BMLF1259-267 tetramer-binding cells were significantly higher in seropositive than in seronegative subjects (Fig. 2A and B). For the MART-126-35 tetramer, healthy subjects had significantly lower numbers of tetramer-positive/CD8+ cells than patients with prior diagnosis of melanoma but currently no evidence of disease, whereas they were not significantly different from patients with active melanoma (Fig. 2C). Table 2A presents data on the physiologic variability for subjects with mean values above the LLD. Values below the LLD would be expected to vary widely due to the assay inaccuracy at that level. Above the LLD, the pCV of the tetramer assay ranged from 16.71% to 35.95%, which shows that the levels of circulating antigen-specific T cells can vary over time without any immunotherapy treatment (Table 2A).

Fig. 2.

Physiological variability of circulating antigen-specific cells. Cryopreserved PBMC samples collected from different time points were thawed and analyzed with tetramers after exclusion with iMASC antibodies or were restimulated in ELISPOT plates with K562/A*0201 pulsed with each peptide. Plates were incubated for 24 hours at 37°C in 5% CO2 and the number of IFN-γ-producing cells in the ELISPOT assay were quantitated using a series 1 Immunospot Image Analyzer. A, CMVpp65495-503 tetramer. B, EBV BMLF1259-267 tetramer. C, MART-126-35 tetramer. D, CMVpp65495-503 ELISPOT. E, EBV BMLF1259-267 ELISPOT. F, MART-127-35 ELISPOT.

Fig. 2.

Physiological variability of circulating antigen-specific cells. Cryopreserved PBMC samples collected from different time points were thawed and analyzed with tetramers after exclusion with iMASC antibodies or were restimulated in ELISPOT plates with K562/A*0201 pulsed with each peptide. Plates were incubated for 24 hours at 37°C in 5% CO2 and the number of IFN-γ-producing cells in the ELISPOT assay were quantitated using a series 1 Immunospot Image Analyzer. A, CMVpp65495-503 tetramer. B, EBV BMLF1259-267 tetramer. C, MART-126-35 tetramer. D, CMVpp65495-503 ELISPOT. E, EBV BMLF1259-267 ELISPOT. F, MART-127-35 ELISPOT.

Close modal
Table 2.

Physiologic variability of the tetramer and ELISPOT assays

Epitope
Clinical group
No. subjects
No. samples
Mean (SD)
pCV
A. Physiologic variability of the tetramer assay above the LLD
EBV BMLF1259-267 Seronegative 11 42 0.53 (0.21) 25.91 
 Seropositive 13 50 1.27 (0.34) 21.02 
CMVpp65495-503 Seronegative 14 53 0.10 (0.05) 35.95 
 Seropositive 10 39 0.97 (0.06) 16.71 
MART-126-35 Healthy Donors 27 0.29 (0.08) 29.46 
 Pts with Melanoma 15 55 0.76 (0.21) 27.40 
      
B. Physiologic variability of the ELISPOT assay above the LLD
 
     
EBV BMLF1259-267 Seronegative 33 32 (26) 100.13 
 Seropositive 82 48 (34) 60.47 
CMVpp65495-503 Seronegative 31 16 (16) 75.17 
 Seropositive 90 47 (22) 45.00 
MART-127-35
 
Healthy Donors — — 
 Pts with Melanoma 54 43 (25) 62.47 
Epitope
Clinical group
No. subjects
No. samples
Mean (SD)
pCV
A. Physiologic variability of the tetramer assay above the LLD
EBV BMLF1259-267 Seronegative 11 42 0.53 (0.21) 25.91 
 Seropositive 13 50 1.27 (0.34) 21.02 
CMVpp65495-503 Seronegative 14 53 0.10 (0.05) 35.95 
 Seropositive 10 39 0.97 (0.06) 16.71 
MART-126-35 Healthy Donors 27 0.29 (0.08) 29.46 
 Pts with Melanoma 15 55 0.76 (0.21) 27.40 
      
B. Physiologic variability of the ELISPOT assay above the LLD
 
     
EBV BMLF1259-267 Seronegative 33 32 (26) 100.13 
 Seropositive 82 48 (34) 60.47 
CMVpp65495-503 Seronegative 31 16 (16) 75.17 
 Seropositive 90 47 (22) 45.00 
MART-127-35
 
Healthy Donors — — 
 Pts with Melanoma 54 43 (25) 62.47 

Abbreviation: Pts, patients.

IFN-γ ELISPOT assay

Assay optimization. We based our assay methodology on our previously published experience (25, 26) and then tested several key conditions to attempt to decrease preanalytic variation. We compared three widely used surrogate APC: JY, T2, and K562/A*0201 cells (Fig. 1B). Higher background was noted with the use of JY and T2 cells. Therefore, K562/A*0201 cells were chosen for further testing. The standardized conditions of 1:10 APC to PBMC ratio, with APC pulsed with 10 μg of peptide, were defined in peptide and APC/PBMC ratio titration studies. Additional optimization studies concluded that 1 × 105 PBMCs per well and performing the restimulation in the plate were optimal. K562/A*0201 cells were periodically phenotyped, with an average of 96 ± 1% expression of HLA-A*0201, and were free of Mycoplasma contamination (data not shown).

LLD. Replicate aliquots from cryopreserved samples were analyzed for reactivity to the two candidate negative control epitopes (negative and AFP325-332) to determine the LLD (Table 1). Four of 31 subjects had clear baseline reactivity (mean values over the mean + 2 SDs from the rest of subjects) to the negative peptide by ELISPOT assay (the same subjects had no negative tetramer-binding cells). Two subjects had clear reactivity to AFP325-332. Results from these subjects were considered outliers compared with the overall population and were not used to calculate the assay LLD (18). In this assay, both the negative and AFP325-332 peptides provided a similar LLD, which was set at seven IFN-γ spots per 1 × 105 PBMCs.

Analytic variability. The within-run replication experiment, using samples with values above the LLD, showed a slightly higher aCV for the native self-melanosomal antigen MART-127-35 compared with the xenoantigens CMVpp65495-503 and EBV BMLF1259-267 (Table 1). Additionally, it showed that the ELISPOT assay has much higher analytic variation than the tetramer assay, even beyond the 20% CV that is routinely considered as an acceptable assay variability for analytic methods (23).

Physiologic variability.Figure 2D, E, and F depicts the results of testing all the available samples for IFN-γ-producing cells detected in the ELISPOT assay in response to K562/A*0201 cells pulsed with the CMVpp65495-503, EBV BMLF1259-267, and MART-127-35 peptides. As with the tetramer assay, subjects without prior exposure shown by a negative humoral response to CMV or EBV had low levels of T-cell responses in the ELISPOT assay, whereas subjects with prior exposure had higher mean values of IFN-γ-producing cells (Fig. 2D and E). For MART-127-35, again a higher reactivity was noted for patients who were with no evidence of disease at the time of blood donations compared with healthy subjects and patients with active melanoma (Fig. 2F).

The pCV above the LLD is described in Table 2B. It could not be calculated for the MART-127-35 peptide in the healthy donor population, because all subject had values below the LLD. Consistent with the high assay analytic variability, the pCV was high for all conditions and peptides, between 45% and 100.13% (Table 2B).

Definition of significant changes in circulating antigen-specific T cells after immunotherapy. Given the marked between-subject variability on homeostatic levels of circulating antigen-specific T cells (Fig. 2), we used our definition of the LLD as a threshold to separate subpopulations of subjects with baseline negative and positive values for each assay. We reasoned that patients with levels of circulating antigen-specific T cells below the LLD could be defined as responders to immunotherapy if their antigen-specific cells expanded beyond the mean + 2 SD (95% significance) or 3 SD (99% significance) from baseline values. For patients with baseline values above the LLD, we reasoned that the RCV would allow quantifying changes in response to immunotherapy beyond the assay variability. This algorithm is based on a preliminary analysis of the use of these two approaches in two extreme cases within our series (Fig. 1E).

Table 3A provides the values for EBV, CMV, and MART-1 to be used as cutoff points to define a positive immune response to immunotherapy in subjects with baseline levels of circulating antigen-specific T cells below the LLD for both assays. Calculations for the EBV BMLF1259-267 tetramer could not be done, because all samples were above the assay LLD. For each epitope and assay, the use of mean + 2 SD gave results very close to the LLD. Therefore, we choose to consider a significant change from below the LLD to equal or greater than the mean + 3 SD. Despite the high analytic variability of the ELISPOT assay above the LLD, there was little variability in background levels, with a tight SD. This is due to the low nonspecific reactivity with K562-A*0201 as APC (Fig. 1C), with many samples having values of 0 spot in the four time points. Using the mean + 3 SD approach for the ELISPOT assay, a change from below seven to nine spots would be considered a positive immune response. However, given the assay variability (pCV up to 100%), an increase from seven to nine spots cannot be considered biologically relevant. Therefore, the mean + 3 SD approach is valid for the tetramer assay, where there is a wide range of values below the assay LLD but not for the ELISPOT assay.

Table 3.

A. Calculation of significant antigen specific expansion for subjects with baseline levels of circulating antigen-specific T cells below the assay LLD
AssayEpitopeNo. subjectsNo. samplesMean + 2 SD (95%)Mean + 3 SD (99%)
Tetramer EBV BMLF1259-267 — — — —  
 CMVpp65495-503 24 0.064 0.066  
 MART-126-35 36 0.050 0.062  
ELISPOT EBV BMLF1259-267 16 188  
 CMVpp65495-503 19 224  
 MART-127-35 27 295  
       
B. RCV to calculate significant antigen specific expansion for subjects with baseline levels of circulating antigen-specific T cells above the assay LLD
 
      
Assay
 
Epitope
 
Degrees of freedom
 
95% Z score
 
99% Z score
 
95% RCV
 
99% RCV
 
Tetramer EBV BMLF1259-267 43 2.02 2.66 64.50 85.89 
 CMVpp65495-503 33 2.03 2.73 51.52 69.29 
 MART-126-35 66 1.99 2.65 80.57 107.29 
ELISPOT EBV BMLF1259-267 99 1.98 2.62 186.55 246.85 
 CMVpp65495-503 71 1.99 2.65 144.45 192.36 
 MART-127-35 63 1.99 2.66 200.41 267.89 
A. Calculation of significant antigen specific expansion for subjects with baseline levels of circulating antigen-specific T cells below the assay LLD
AssayEpitopeNo. subjectsNo. samplesMean + 2 SD (95%)Mean + 3 SD (99%)
Tetramer EBV BMLF1259-267 — — — —  
 CMVpp65495-503 24 0.064 0.066  
 MART-126-35 36 0.050 0.062  
ELISPOT EBV BMLF1259-267 16 188  
 CMVpp65495-503 19 224  
 MART-127-35 27 295  
       
B. RCV to calculate significant antigen specific expansion for subjects with baseline levels of circulating antigen-specific T cells above the assay LLD
 
      
Assay
 
Epitope
 
Degrees of freedom
 
95% Z score
 
99% Z score
 
95% RCV
 
99% RCV
 
Tetramer EBV BMLF1259-267 43 2.02 2.66 64.50 85.89 
 CMVpp65495-503 33 2.03 2.73 51.52 69.29 
 MART-126-35 66 1.99 2.65 80.57 107.29 
ELISPOT EBV BMLF1259-267 99 1.98 2.62 186.55 246.85 
 CMVpp65495-503 71 1.99 2.65 144.45 192.36 
 MART-127-35 63 1.99 2.66 200.41 267.89 

Calculation of the RCV to define a positive or negative change in circulating antigen-specific T cells beyond the assay variability.Table 3B provides the RCV for CMVpp65495-503, EBV BMLF1259-267, MART-127-35, or MART-126-35 epitopes for both assays. To minimize the effects of between-subject variability (not incorporated into the RCV formula being applied here), all aCV and pCV were taken from antigen preexposed subjects with values above the LLD for each assay (values from Tables 1 and 2). Thanks to its much lower analytic variability, the RCV was lower for the structural tetramer assay compared with the functional IFN-γ ELISPOT assay for all epitopes and study populations.

Definition of an immune response based on the tetramer and ELISPOT assays. Based on the results of this methodology study, we propose to use the following criteria to define an immune response. (a) Tetramer assay: In subjects with baseline levels below the LLD, we would consider an increase from the tetramer assay LLD of 0.038% to ≥0.062% (the mean + 3 SD) as being significant. In subjects with baseline values above the assay LLD, a significant change should at least reach the 95% RCV for each peptide. (b) ELISPOT assay: Because an increase from below seven to at least nine spots per 105 PBMCs is not considered biologically relevant given the assay imprecision, we propose to use the 95% RCV for both subjects with baseline levels below and above the LLD. For patients starting below the assay LLD, a significant positive change should be calculated using the assay LLD plus the 95% RCV for each peptide.

Application of the definition of immunologic response in patients receiving the anti-CTLA4 antibody ticilimumab. The definitions of immune response based on the quantitation of circulating antigen-specific T cells using the tetramer and ELISPOT assays were applied to the analysis of serial PBMC samples from four HLA-A*0201-positive patients with metastatic melanoma after dosing with ticilimumab (see Materials and Methods). This monoclonal antibody blocks a major negative regulatory pathway in T cells, which is expected to then permit their expansion and activation leading to antitumor immune responses (27). These four patients had received a prior dose of ticilimumab between 8 and 18 months before, and blood samples were collected during redosing (see Materials and Methods). All patients had at least one baseline PBMC sample before dosing, and at least two follow-up PBMC samples. Table 4 provides the results of the MHC tetramer and ELISPOT assays (not enough PBMCs were available on patient 120X to perform an ELISPOT assay).

Table 4.

Patient
Dose (mg/kg)
Epitope
Predose
Within 1 wk
Within 2-4 wks
Change 95%
Change 99%
A. Tetramer quantitation of antigen-specific T-cell expansion after administration of ticilimumab
113X Negative 0.03 0.01 0.01   
  AFP325-332 0.03 0.05 0.02   
  EBV BMLF1259-267 4.25 4.0 3.76   
  CMVpp65495-503 <0.01 0.01 0.02   
  MART-126-35 0.07 0.05 0.07   
119X Negative 0.02 0.01 0.01   
  AFP325-332 0.03 0.01 0.02   
  EBV BMLF1259-267 0.81 1.15 (↑ 42%) 1.39 (↑ 72%) 64.50% 85.89% 
  CMVpp65495-503 0.53 0.50 0.51   
  MART-126-35 0.02 0.03 0.04   
120X Negative 0.01 0.02 0.02   
  AFP325-332 3.35 4.22 3.25   
  EBV BMLF1259-267 0.46 0.38 0.45   
  CMVpp65495-503 0.01 0.04 0.02   
  MART-126-35 0.19 0.38 (↑ 100%) 0.22 80.57% 107.29% 
125X 10 Negative 0.02 <0.01 0.01   
  AFP325-332 — — —   
  EBV BMLF1259-267 0.74 1.46 (↑ 97%) 0.87 64.50% 85.89% 
  CMVpp65495-503 — — —   
  MART-126-35 0.05 0.04 0.02   
        
B. IFN-γ ELISPOT quantitation of antigen-specific T-cell expansion after administration of ticilimumab
 
       
113X Negative   
  AFP325-332   
  EBV BMLF1259-267 129 246 (↑ 91%) 247 (↑ 91%) 186.55% 246.85% 
  CMVpp65495-503 21 23 0 (↓ >100%) 144.45% 192.36% 
  MART-127-35   
  MART-126-35 11 30 (↑ 173%) 11 NA NA 
119X Negative 36 30 33   
  AFP325-332 10 11 14   
  EBV BMLF1259-267 25 21 —   
  CMVpp65495-503 24 21 24   
  MART-127-35 14 7 (↓ 50%) 9 (↓ 36%) 200.41% 267.89% 
  MART-126-35 21 5 (↓ 76%) 1 (↓ 95%) NA NA 
125X 10 Negative   
  AFP325-332   
  EBV BMLF1259-267 34 (↑ 467%) 14 (↑ 133%) 186.55% 246.85% 
  CMVpp65495-503   
  MART-127-35   
  MART-126-35   
Patient
Dose (mg/kg)
Epitope
Predose
Within 1 wk
Within 2-4 wks
Change 95%
Change 99%
A. Tetramer quantitation of antigen-specific T-cell expansion after administration of ticilimumab
113X Negative 0.03 0.01 0.01   
  AFP325-332 0.03 0.05 0.02   
  EBV BMLF1259-267 4.25 4.0 3.76   
  CMVpp65495-503 <0.01 0.01 0.02   
  MART-126-35 0.07 0.05 0.07   
119X Negative 0.02 0.01 0.01   
  AFP325-332 0.03 0.01 0.02   
  EBV BMLF1259-267 0.81 1.15 (↑ 42%) 1.39 (↑ 72%) 64.50% 85.89% 
  CMVpp65495-503 0.53 0.50 0.51   
  MART-126-35 0.02 0.03 0.04   
120X Negative 0.01 0.02 0.02   
  AFP325-332 3.35 4.22 3.25   
  EBV BMLF1259-267 0.46 0.38 0.45   
  CMVpp65495-503 0.01 0.04 0.02   
  MART-126-35 0.19 0.38 (↑ 100%) 0.22 80.57% 107.29% 
125X 10 Negative 0.02 <0.01 0.01   
  AFP325-332 — — —   
  EBV BMLF1259-267 0.74 1.46 (↑ 97%) 0.87 64.50% 85.89% 
  CMVpp65495-503 — — —   
  MART-126-35 0.05 0.04 0.02   
        
B. IFN-γ ELISPOT quantitation of antigen-specific T-cell expansion after administration of ticilimumab
 
       
113X Negative   
  AFP325-332   
  EBV BMLF1259-267 129 246 (↑ 91%) 247 (↑ 91%) 186.55% 246.85% 
  CMVpp65495-503 21 23 0 (↓ >100%) 144.45% 192.36% 
  MART-127-35   
  MART-126-35 11 30 (↑ 173%) 11 NA NA 
119X Negative 36 30 33   
  AFP325-332 10 11 14   
  EBV BMLF1259-267 25 21 —   
  CMVpp65495-503 24 21 24   
  MART-127-35 14 7 (↓ 50%) 9 (↓ 36%) 200.41% 267.89% 
  MART-126-35 21 5 (↓ 76%) 1 (↓ 95%) NA NA 
125X 10 Negative   
  AFP325-332   
  EBV BMLF1259-267 34 (↑ 467%) 14 (↑ 133%) 186.55% 246.85% 
  CMVpp65495-503   
  MART-127-35   
  MART-126-35   

Abbreviation: NA, not available.

There is no clear trend of generalized antigen-specific T-cell expansion after ticilimumab administration evaluated using our definitions of immune response (Table 4). Analyzed with the tetramer assay, two of four patients had an increase in EBV BMLF1259-267-specific T cells, whereas one of four had an increase in MART-126-35-specific T cells, at the 95% significance. None had expansion of T cells specific for CMVpp65495-503, AFP325-332, or negative control peptides. Additional testing in patient 125X failed to reveal increases in the number of circulating gp100208-216 or tyrosinase368-376 antigen-specific T cells (data not shown), two other immunodominant epitopes in melanoma (28).

Analyzed with the ELISPOT assay, one of three had an increase in EBV BMLF1259-267-reactive T cells at both the 95% and 99% significance level, and an additional patient had a marked increase that did not reach the 95% significance level. One patient had a marked decrease in IFN-γ-producing cells specific for CMVpp65495-503. None had expansion of T cells reactive to the AFP325-332 or negative control peptides. One of three had increase in the number of IFN-γ-producing cells specific for the heteroclitic MART-126-35 peptide but not the native MART-127-35 peptide (significance could not be assessed, because our definition was based on the MART-127-35 native peptide). One patient had a decrease in cells producing IFN-γ in response to these two MART-1-derived peptides, which did not reach significance.

Much progress has been made in the past 10 years in the analysis of antigen-specific T-cell responses. The pioneering work of many groups (16, 9, 11, 12, 29, 30) has resulted in assays already being used as surrogate end points for immune activation in experimental cancer immunotherapy trials (31). The tetramer and ELISPOT assays quantitate antigen-specific T cells using two different approaches; therefore, it is not surprising that results are not directly concordant. Cells found to have a TCR specific for one antigen by the tetramer may or may not produce a particular cytokine (in our case IFN-γ) detected in the ELISPOT assay. The available data suggest that approximately one third of infectious disease or tumor antigen-specific T cells detected using the tetramer assay produce IFN-γ upon antigenic stimulation (12). The use of a combined tetramer and IFN-γ intracellular cytokine flow cytometry assay may allow to more directly define the discrepancy between the two assay systems tested in the current study (12).

A critical issue in the implementation of these assays for the immune monitoring of vaccine and other immunotherapy approaches for human cancer is the definition of what is a positive immunologic response. Two consensus approaches for the ELISPOT assay were defined in an immune monitoring workshop organized by the Society of Biological Therapy (10). These were the number of spots in six wells exceeding by 10 the number of spots in six control wells, or the detection of a 2- or 3-fold increase over unstimulated control wells. No consensus definition on an immunologically positive response was proposed for the tetramer assay (10); some groups have defined a positive response as a doubling of the T-cell frequency observed after vaccine (32), whereas others propose increases beyond the mean + 2 or 3 SD from negative values (12). These approaches interpret change against fixed criteria (a population-based reference value, a locally agreed protocol, a value proposed by expert committees, or multiples of the upper reference limit), which has been described as the “clinical fixed limits” or cutoff point. Their advantage is that they are simple to implement and use and are believed to be in some way related to the clinical outcome based on good evidence or in the views of experts in the field (18). Definition of a positive immune response as an increase beyond the mean + 2 or 3 SD from negative values can adequately address the situation of starting at a negative value (at or below the LLD for a negative antigen). On the contrary, it is difficult to envision how this criterion can be used in the not infrequent situation that the prevaccination levels of antigen-specific T cells are above the LLD (Fig. 1E). We tested the approach of applying the RCV, which we reasoned would allow defining a positive or negative immune response based on a change in value beyond the assay variability. The RCV calculation is unreliable below the assay LLD, where the assay is most variable. In addition, a danger of RCV, compared with the “fixed cutoff” point, is that RCV is changeable depending on the experiment, sample size, and patient population. However, it allows defining if changes above the LLD are true immune responses or just the variability that would be encountered when performing repeated measurements without any intercurrent treatments or illnesses. The LLD, analytic variability, and between-subject or within-subject biological variability in our study are in the same range as the ones defined at other laboratories (11, 12, 3234).

We tested the performance of our standardized assays and definitions of immune response in four HLA-A*0201 patients receiving the CTLA4 blocking antibody ticilimumab. In this small group of patients, we were able to establish which changes from baseline levels of circulating antigen-specific T cells were beyond the assay imprecision. Although some values changed after dosing, there was no clear trend of antigen-specific T-cell expansion. Overall, these data do not specifically rule out an antitumor T-cell expansion by anti-CTLA4 antibodies, because the relevant cells may be restricted to different epitopes or may only be notable in peripheral blood transiently after the first antibody administration. These possibilities are currently being tested in a larger set of patients receiving ticilimumab under a separate protocol (UCLA IRB 03-01-059), where immune response will be analyzed based on the definitions derived from this methodology study.

In conclusion, the MHC tetramer and ELISPOT assays can be used for the ex vivo analysis of T-cell responses after having defined the assay LLD, between-subject variability in homeostatic set points, and analytic and physiologic variability. The ELISPOT assay has a much higher analytic imprecision compared with the tetramer assay, even after optimizing its performance and using standardized operating procedures in replicate samples. An immunologic response should be considered positive only when the pre- and post-immunotherapy determinations of circulating antigen-specific T cells expand significantly above the LLD of the assay or are beyond the RCV for the assay.

Grant support: Flow cytometry and enzyme-linked immunospot assays were done in the University of California at Los Angeles Jonsson Comprehensive Cancer Center and Center for AIDS Research Flow Cytometry Core Facility that is supported by NIH grants CA-16042 and AI-28697, Jonsson Comprehensive Cancer Center, University of California at Los Angeles AIDS Institute, and David Geffen School of Medicine at University of California at Los Angeles.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1
Czerkinsky C, Andersson G, Ekre HP, et al. Reverse ELISPOT assay for clonal analysis of cytokine production. I. Enumeration of γ-interferon-secreting cells.
J Immunol Methods
1988
;
110
:
29
–36.
2
Tanguay S, Killion JJ. Direct comparison of ELISPOT and ELISA-based assays for detection of individual cytokine-secreting cells.
Lymphokine Cytokine Res
1994
;
13
:
259
–63.
3
Herr W, Schneider J, Lohse AW, et al. Detection and quantification of blood-derived CD8+ T lymphocytes secreting tumor necrosis factor α in response to HLA-A2.1-binding melanoma and viral peptide antigens.
J Immunol Methods
1996
;
191
:
131
–42.
4
Schmittel A, Keilholz U, Scheibenbogen C. Evaluation of the interferon-γ ELISPOT-assay for quantification of peptide specific T lymphocytes from peripheral blood.
J Immunol Methods
1997
;
210
:
167
–74.
5
Altman JD, Moss PAH, Goulder PJR, et al. Phenotypic analysis of antigen-specific T lymphocytes.
Science
1996
;
274
:
94
–6.
6
Murali-Krishna K, Altman JD, Suresh M, et al. Counting antigen-specific CD8 T cells: a reevaluation of bystander activation during viral infection.
Immunity
1998
;
8
:
177
–87.
7
Labalette-Houache M, Torpier G, Capron A, et al. Improved permeabilization procedure for flow cytometric detection of internal antigens. Analysis of interleukin-2 production.
J Immunol Methods
1991
;
138
:
143
–53.
8
Chikanza IC, Corrigal V, Kingsley G, et al. Enumeration of interleukin-1 α and β producing cells by flow cytometry.
J Immunol Methods
1992
;
154
:
173
–8.
9
Clay TM, Hobeika AC, Mosca PJ, et al. Assays for monitoring cellular immune responses to active immunotherapy of cancer.
Clin Cancer Res
2001
;
7
:
1127
–35.
10
Keilholz U, Weber J, Finke JH, et al. Immunologic monitoring of cancer vaccine therapy: results of a workshop sponsored by the Society for Biological Therapy.
J Immunother
2002
;
25
:
97
–138.
11
Whiteside TL. Monitoring of antigen-specific cytolytic T lymphocytes in cancer patients receiving immunotherapy.
Clin Diagn Lab Immunol
2000
;
7
:
327
–32.
12
Hobeika AC, Morse MA, Osada T, et al. Enumerating antigen-specific T-cell responses in peripheral blood: a comparison of peptide MHC Tetramer, ELISpot, and intracellular cytokine analysis.
J Immunother
2005
;
28
:
63
–72.
13
Lee PP, Yee C, Savage PA, et al. Characterization of circulating T cells specific for tumor-associated antigens in melanoma patients.
Nat Med
1999
;
5
:
677
–85.
14
Harris EK, Brown SS. Temporal changes in the concentrations of serum constituents in healthy men. Distributions of within-person variances and their relevance to the interpretation of differences between successive measurements.
Ann Clin Biochem
1979
;
16
:
169
–76.
15
Harris EK, Yasaka T. On the calculation of a “reference change” for comparing two consecutive measurements.
Clin Chem
1983
;
29
:
25
–30.
16
Harris EK. Reference values for change: an addendum.
Clin Chem
1983
;
29
:
997
.
17
Queralto JM, Boyd JC, Harris EK. On the calculation of reference change values, with examples from a long-term study.
Clin Chem
1993
;
39
:
1398
–403.
18
Fraser CG. Biological variation: from principles to practice. Washington (DC): AACC Press; 2001.
19
Trape J, Botargues JM, Porta F, et al. Reference change value for α-fetoprotein and its application in early detection of hepatocellular carcinoma in patients with hepatic disease.
Clin Chem
2003
;
49
:
1209
–11.
20
Ribas A, Camacho LH, Lopez-Berestein G, et al. Antitumor activity in melanoma and anti-self responses after CTLA4 blockade with the monoclonal antibody CP-675.206. Journal of Clinical Oncology 2005;23:10.1200/JCO.205.01.109. [epub ahead of print].
21
Maecker HT, Moon J, Bhatia S, et al. Impact of cryopreservation on tetramer, cytokine flow, and ELISPOT. BMC Immunol 2005;6:17.
22
Britten CM, Meyer RG, Kreer T, et al. The use of HLA-A*0201-transfected K562 as standard antigen-presenting cells for CD8(+) T lymphocytes in IFN-γ ELISPOT assays.
J Immunol Methods
2002
;
259
:
95
–110.
23
Westgard JO. Westgard QC: tools, technology and training for healthcare laboratories. Available from: http://www.westgard.com 2005.
24
Butterfield LH, Meng WS, Koh A, et al. T cell responses to HLA-A*0201-restricted peptides derived from human α fetoprotein.
J Immunol
2001
;
166
:
5300
–8.
25
Butterfield LH, Ribas A, Dissette VB, et al. Determinant spreading associated with clinical response in dendritic cell-based immunotherapy for malignant melanoma.
Clin Cancer Res
2003
;
9
:
998
–1008.
26
Ribas A, Glaspy JA, Lee Y, et al. Role of dendritic cell phenotype, determinant spreading, and negative costimulatory blockade in dendritic cell-based melanoma immunotherapy.
J Immunother
2004
;
27
:
354
–67.
27
Chambers CA, Kuhns MS, Egen JG, et al. CTLA-4-mediated inhibition in regulation of T cell responses: mechanisms and manipulation in tumor immunotherapy.
Annu Rev Immunol
2001
;
19
:
565
–94.
28
Panelli MC, Riker A, Kammula U, et al. Expansion of tumor-T cell pairs from fine needle aspirates of melanoma metastases.
J Immunol
2000
;
164
:
495
–504.
29
Scheibenbogen C, Lee KH, Mayer S, et al. A sensitive ELISPOT assay for detection of CD8+ T lymphocytes specific for HLA class I-binding peptide epitopes derived from influenza proteins in the blood of healthy donors and melanoma patients.
Clin Cancer Res
1997
;
3
:
221
–6.
30
Arlen P, Tsang KY, Marshall JL, et al. The use of a rapid ELISPOT assay to analyze peptide-specific immune responses in carcinoma patients to peptide vs. recombinant poxvirus vaccines.
Cancer Immunol Immunother
2000
;
49
:
517
–29.
31
Ribas A, Butterfield LH, Glaspy JA, et al. Current developments in cancer vaccines and cellular immunotherapy.
J Clin Oncol
2003
;
21
:
2415
–32.
32
Speiser DE, Pittet MJ, Guillaume P, et al. Ex vivo analysis of human antigen-specific CD8+ T-cell responses: quality assessment of fluorescent HLA-A2 multimer and interferon-γ ELISPOT assays for patient immune monitoring.
J Immunother
2004
;
27
:
298
–308.
33
Hoffmann TK, Donnenberg VS, Friebe-Hoffmann U, et al. Competition of peptide-MHC class I tetrameric complexes with anti-CD3 provides evidence for specificity of peptide binding to the TCR complex.
Cytometry
2000
;
41
:
321
–8.
34
Asai T, Storkus WJ, Whiteside TL. Evaluation of the modified ELISPOT assay for γ interferon production in cancer patients receiving antitumor vaccines.
Clin Diagn Lab Immunol
2000
;
7
:
145
–54.