Reproducibility and Correlations of Multiplex Cytokine Levels in Asymptomatic Persons

Cytokines are humoral signaling molecules that bind to immune system cell membrane receptors at relatively low concentrations, generally acting over distances of a few cell diameters (1). Grossly elevated blood levels in states such as sepsis and cutaneous burns reflect the activation of the cytokine network associated with a concomitant deregulated immune response, but variations within physiologic ranges could also have significance. Notably, cytokine signaling operates in a pleiotropic (each cytokine acts on multiple molecular targets) and redundant (several cytokines respectively elicit the same cellular response) fashion. Thus, the blood levels of multiple cytokines collectively may reflect subtle states of immune dysfunction and/or immune-related disease.

To determine cytokine profiles in serum or plasma, one possible multiplex platform is the Luminex xMAP, a bead array coupled with discrete fluorescent molecules to detect multiple soluble analytes. Available Luminex assays theoretically have the capacity to measure up to 100 cytokines simultaneously in volumes of 25 to 50 μL of serum. In contrast, conventional individual measurement by ELISA requires 50 to 200 μL of serum per analyte. In addition, Luminex assays may have a greater dynamic range (∼1-10,000 pg/mL) than ELISAs (2-4).

Reproducibility data regarding Luminex cytokine assays are primarily limited to supraphysiologic levels, and their performance characteristics relevant to the lower levels expected in epidemiologic studies have not been defined (5-8). We therefore assessed a commercially available multiplex cytokine assay for the reproducibility of serum measurements, correspondence between serum and plasma, and correlations among serum levels in specimens from healthy donors.

We measured cytokine levels in serum samples taken from a total of 76 subjects: 27 persons from a population-based random sample of asymptomatic adults from the United States (9), 11 human T-cell lymphotropic virus-1 negative subjects from a survey conducted in Barbados (10), and 38 blood donors at the Department of Transfusion Medicine, NIH (Fig. 1). The samples did not go through additional freeze-thaw cycles before analysis. To assess the reproducibility of serum measurements, we ran two additional replicates for the U.S. and Barbados groups (n = 38); assays were run on three different days using blinded aliquots. To assess correlations of measurements in serum with plasma, we used matched samples of sodium heparinized and acid citrate dextrose (ACD) plasma from the NIH subjects. We also defibrinated heparinized (n = 19) and ACD (n = 20) plasma from the NIH donors to mimic serum preparations and assessed their correlations with serum.

Thirteen cytokines, interleukin (IL) 1β, IL2, IL4, IL5, IL6, IL7, IL8, IL10, IL12p70, IL13, IFNγ, granulocyte colony-stimulating factor (G-CSF), and tumor necrosis factor-α (TNFα) were measured using the MILLIPLEX MAP 13-plex Cytokine Kit (Millipore, Billerica, MA) at the BioPharma Service Laboratory (currently, Drug Discovery SBU; Millipore, St. Charles, MO). In preliminary studies, we evaluated three candidate commercial multiplex kits and chose the current kit as having the best coefficients of variation (CV) and level of detection; our pilot data agreed with a recent report of multiplex cytokine measurements in spiked samples which found the Linco kit to be the most sensitive assay among the same three kits we compared (11). Median fluorescence intensity, calculated from duplicates for each sample, was collected using the Luminex-100 system version 1.7 (Luminex). The StatLIA software package (ver. 3.2; Brendan Scientific, Inc.), incorporating a weighted five-parameter logistic curve-fitting method, was used to calculate sample cytokine concentrations. Because most of the serum and plasma cytokines showed a significant signal-to-noise ratio at the minimum standard concentration (3.2 pg/mL) specified by the manufacturer's instructions, we included an extra dilution to 0.64 pg/mL in the standard curves in an attempt to more accurately estimate concentrations at the lower ranges.

We considered 0.64 pg/mL, the lowest dilution on the standard curve, to be the lower limit of detection (LOD). Due to the left truncation imposed by the detection limit of the assay, we imputed cytokine concentrations for the measurements below the LOD using a maximum-likelihood estimation procedure (12). Simpler methods such as assuming a value of one-half of the LOD or omitting undetectable samples from analyses generate biased estimates of the reproducibility measures (13). Reproducibility statistics based on imputed data are approximately unbiased when less than half of the measurements are below the LOD (12). For the 10 cytokines with >50% missing data, we assessed the histograms of detected measurements. Based on visual inspection, we imputed the missing values for IL13 on the original measurement scale, and for the other nine cytokines, by assuming a log-normal distribution of the concentrations. After imputation, the normality of the imputed data was assessed using quantile-quantile plots and the Shapiro-Wilk test. In one individual, very high levels of three cytokines (IL4, TNFα, and G-CSF) were excluded as outliers.

Reproducibility statistics [coefficient of variance (CV) and intraclass correlation coefficient (ICC)] were estimated using log-transformed measurements for all cytokines except for IL13. For each analyte, we calculated the average percentage CV (SD/mean × 100) for the 38 serum samples (omitting one outlier for IL4, TNFα, and G-CSF) assayed as triplicates on 3 different days. To assess the effect of measurement error on comparisons between subjects, the ICC was calculated as the ratio of the variance between subjects to the total variance for a given analyte. The variance was partitioned into between-subject and within-subject components and modeled as

where x_ij denotes the cytokine levels for subjects i (i = 1,…,38) on the jth. day (j = 1, 2, 3). The overall mean concentration is denoted by μ, α_i = subject-specific effect and ε_ij = normally distributed error term. Parameters were estimated using the restricted maximum likelihood method (REML) incorporated in SAS 9.1.3 (SAS Institute; ref. 14).

We also estimated alternative CVs based solely on observations above the detection limits but did not perform similar calculations for ICCs because the mixed models require normally distributed data. The alternative CVs were smaller than those estimated with imputed values included and the more conservative larger estimates are presented in Table 1.

We computed nonparametric Spearman rank correlations of cytokine levels in serum with levels in matched samples of plasma or defibrinated plasma. We computed pairwise Spearman rank order correlations among the various cytokines using all 76 serum samples. We also used these data to perform principal components analysis, followed by varimax rotation, to transform the data to a new coordinate system such that the greatest variance by any projection of the data comes to lie on the first coordinate (called the first principal component, PC), the second greatest variance on the second coordinate, and so on (15). The number of PCs was determined by Cattell scree plots (16), using a cutoff of 1 for the sorted eigenvalues of the covariance matrix. The correlation between individual cytokines and each PC was assessed by factor loadings (15).

All statistical analyses were done with STATA 9.0 (StataCorp) except for the imputation procedure and REML procedure incorporated in SAS 9.0 (SAS Institute). All P values are two-sided, with P < 0.05 considered as statistically significant.

Of the 13 cytokines, three (IL2, IL5 and IFNγ) were undetectable in more than half of the serum samples. The percentages detectable (i.e., >0.64 pg/mL) for the other 10 cytokines ranged from 55% to 97%, with CVs ranging from 18% to 44% and ICCs ranging from 55% to 98% (Table 1). After imputation, the data for all cytokines had a log-normal distribution, except for IL13, which was normally distributed on the original scale (P > 0.05). ICC values were computed based on a single imputed data set; results were similar when imputations were repeated.

Levels in plasma were higher than in serum and ACD plasma was higher than heparin plasma for all cytokines except for IL8. Median levels were above the LOD for all plasma cytokines. We present the serum-plasma correlations for ACD anti-coagulated plasma and heparinized plasma in Table 2. Figure 2 plots the log-transformed cytokine levels (pg/mL) in serum versus heparinized plasma. The best agreements between serum and plasma were found for IL1β, IL2, IL4, IL6, IL8, and IL13. The relationships between serum and ACD anti-coagulated plasma measurements are similar (data not graphically shown). Only IL4, IL6, and IL8 levels had statistically significant correlations of both ACD and heparinized plasma levels with serum measurements; IL1β, IL2, and IL13 had significant correlations in heparinized (but not ACD) plasma with serum (Table 2). Defibrination did not substantially improve the correlations between plasma and serum levels (Table 2).

Pairwise correlations among the 13 serum cytokines were strongest between IL7 and IL12p70 (Spearman r = 0.85, P < 0.01). Five other cytokines had significant correlations with Spearman |r| > 0.7: IL1β and IL8, IL5 and IFNγ, IL12p70 and IL13, IL12p70 and G-CSF, and IFNγ and G-CSF (Table 3). Results were similar when we restricted analyses to levels above the LOD.

Principal components analysis identified two PCs, which together, accounted for 70% of the variance. The first PC explained 48% of the total variance; this component was mostly determined by IL7, IL10, IL12p70, IFNγ, and G-CSF based on factor loadings > 0.7 (Fig. 3). The second PC, mostly correlated to IL4 and IL6, explained a further 22% of the total variance.

Our study suggests the potential utility of multiplex cytokine assays for case-control studies. ICC assesses interindividual variability relative to total variability. The high ICCs of the majority of the assays indicate a good ability to distinguish between individuals at a single point in time. However, there will be some attenuation of the associations of the log relative risk of disease with the log cytokine levels. The downward bias on the relative risk estimates is expected to be moderate for ICCs >80% (17), which includes all of the cytokines except for IL4 and G-CSF.

In contrast, CVs reflect laboratory variation and by this criterion, the reproducibility of the Luminex cytokine assays at low or sub–pg/mL concentrations is less than optimal. Previous reliability studies were tested at high cytokine levels (e.g., blister fluids), which coincided with the linear ranges of the standard curves, which maximizes reliability. This study assessed assay performance at lower ranges in which the standard curves are sigmoidal and hence more prone to imprecision. Other potential reasons for the variability are random errors of the test instrument (e.g., bead heterogeneity and carry-over of beads from previous wells) and the presence of interfering substances in blood (18).

In general, plasma measurements did not reflect serum levels, even after defibrination of the plasma. The lower serum levels of many cytokines might be due to degradation during the clotting process. On the other hand, for IL8, the higher levels observed in serum may reflect ex vivo degranulation of granulocytes and platelets. In addition, protein matrix effects (nonspecific interference) and/or sample preparation artifacts could generate nonsystematic differences (2). Without knowing which method of blood processing better reflects the circulating blood milieu, we cannot determine which values more closely represent actual blood levels. Except for IL5, cytokine measurements in the two plasma preparations showed moderate to strong correlation (Spearman ρ range, 0.28-0.85; data not shown), with higher levels in ACD plasma than in heparinized plasma. Prior comparisons of Luminex immunoassays with ELISA measurements showed good correlation in serum (4, 19) but not in plasma (20). Given the importance of standardized reagents, platforms, and laboratory procedures for accurate measurements, our study design avoided other sources of variation such as differences in kit production lots, manufacturers, antibodies, and reference cytokines (21).

We examined interrelationships among the 13 cytokines because of the redundancy, synergism, and antagonism present in the cytokine network. Cytokines have been variously classified to subgroups by their three-dimensional protein structure (22) and by cytokine T helper-1 (Th1) and Th2 CD4⁺ T cell effector function (23, 24). The classic dichotomous Th1/Th2 balance was originally defined by secretory profiles of murine T-cell clones directed against humoral and cell-mediated immunity groups, respectively: Th1 (e.g., IL2, IL12p70, IFNγ, G-CSF, and TNFβ; ref. 25) and Th2 (e.g., IL4, IL5, IL6, and IL13; ref. 25). In our data, IL7 and IL12p70 were highly correlated, which is consistent with the roles of these cytokines in inducing antigen-specific cellular and humoral responses in T cells (26, 27). The pairwise correlations for IL12–G-CSF, G-CSF–IFNγ and IFNγ–IL5 involve three cytokines important in the innate immune response to infection (28, 29). The correlations of IL8 with IL1β and IL6 suggest generic patterns of coexpression of proinflammatory cytokines. To further elucidate these relationships, we used principal components analyses to group the 13 serum cytokine levels into clusters. Although the Th1/Th2 paradigm is less clearly established for humans, the first PC that we identified for healthy individuals includes three cytokines (IL12p70, IFNγ, and G-CSF) classically thought to belong to the Th1 group, and the second PC includes the Th2 cytokines, IL4 and IL6. These results suggest that our panel of 13 cytokines reflects archetypal Th1- and Th2-type responses and provide insight regarding the polarization of cytokine activity at normal levels. Moreover, the underlying factors seem to be estimable and can be used to represent divergent profiles in epidemiologic analyses. Apart from a desirable reduction in the number of variables, principal components analysis has potential for more robust assessment of immune status in accordance with the pleiotropic nature of cytokines. Although these assays measure blood biomarkers of immune response, our data do not assess if circulating levels of cytokines reflect physiologically relevant processes and/or biological effects. Our data also do not address the validity of cytokine measurements that may be affected by sample interference, within-subject temporal fluctuations, and/or other artifacts. Little is known about how cytokines interact with soluble receptors, carriers, binding proteins, antagonists, and other endogenous and exogenous factors in the sample matrix. Thus, one limitation of our study is the lack of a gold standard measurement. Comparisons using parallel assays in ELISAs and multiplex assays based on identical antibody (recognition and capture) pairs would aid the future evaluation of the validity of multiplex cytokine measurements. However, our study is the first report of the reproducibility and serum-plasma correlations of multiplex serum cytokine measurements in reference ranges. We used novel statistical modeling procedures to generate unbiased reproducibility measures.

In conclusion, our data supports the application of Luminex-based cytokine assays to epidemiologic studies. Despite the technical variability, healthy individuals have detectable differences in cytokine levels. Whether these differences in inflammation markers are associated with current or subsequent disease should be examined in further studies. With the low sample volume requirement, low assay cost per analyte, and potential for many analytes per sample, Luminex-based cytokine assays may help elucidate immunologic pathways in pathogenesis.

S. Ji: Millipore Corporation employee (current affiliation: Biolegend, San Diego, CA).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

We are grateful to Dr. Jay Lubin (NIH) and Dr. Elizabeth Breen (UCLA) for suggestions on the manuscript.