Translating Expression Profiling into a Clinically Feasible Test to Predict Neuroblastoma Outcome Free

Neuroblastoma is a common childhood tumor derived from the sympathetic nervous system. To avoid overtreatment of spontaneously regressing tumors and undertreatment in aggressive neuroblastomas, precise prediction of tumor behavior and risk assessment of each individual patient at diagnosis is a major goal of current neuroblastoma research. Similar to analyses in other cancer entities, several genome-wide mRNA expression profiling studies have identified reliable outcome predictors for neuroblastoma, but with little or no overlap in the decision-making genes (1–3). The comparability of microarray data remains a major concern in the interpretation of expression analyses. Today, numerous options are available to carry out whole-genome surveys of gene expression owing to a variety of technical platforms and a still increasing number of distributors. A comprehensive study of six microarray technologies recently addressed the overall consistency within each platform, as well as the correlation among replicates within and between technologies (4). The authors concluded that technological differences may influence the results of transcriptional profiling given high-quality arrays and the appropriate normalization, but that the primary factor determining variance is biological rather than technological. This is reassuring for the clinical usefulness of microarray data because bridging the gap between bench-top microarray analysis and clinical diagnostics is one of the major goals not only in current cancer research but also in molecular medicine in general. However, translation of expression profiling results into routine clinical diagnostics requires a less complex alternative technology for reliable outcome prediction. An essential first step is the implementation of standard operating procedures for target preparation, the hybridization process, and readout formats, as well as between-laboratory comparisons of microarray results (5, 6). The validation of results on technically independent platforms is one of the prerequisites for the transfer of results from molecular biology into a clinical setting (7). The validation by real-time PCR, which is currently the gold standard for measuring gene expression, is considered to be the final proof of array data. A successful approach to evaluate the results from different array systems by means of real-time PCR has previously been published (8). The authors compared data from several expression studies to build a model consisting of only six genes, the expression of which predicted the outcome of diffuse large B-cell lymphoma. Subsequently, it was pointed out that these data must also be evaluated carefully and that validity should be assessed using time-dependent receiver operating characteristic curves instead of risk group stratification (9). Because most gene classifiers identified to date as reliable outcome predictors for different cancer entities including neuroblastoma consist of >20 genes (1, 3, 10), validation of every gene by conventional real-time PCR is not suited to the clinical setting. Technologies appropriate for clinical applications must not only fulfill the requirement of high reproducibility but must also be convenient and efficient in their handling. With the introduction of TaqMan Low Density Arrays (Applied Biosystems, Foster City, CA), researchers can simultaneously assay the RNA expression levels of up to 384 genes on a single microfluidic card (MFC). In principle, this technology allows the rapid validation of differentially expressed genes in a genome-wide microarray study. To assess the potential of these MFCs as an alternative application for candidate gene analysis and diagnostics in a routine clinical setting, we here analyzed a clinically well-defined cohort of 63 primary neuroblastomas. Genes predictive for recurrence of disease according to a recently published microarray study (1), as well as established biological factors including MYCN and NTRK1/TrkA, were selected for the construction of a MFC (Table 1). Interassay reproducibility of data obtained from eight MFCs representing the 63 patients was subsequently evaluated by measuring one sample from each of these cards on a 9th card (“reference card”). Normalized real-time PCR values were also compared with expression data from Affymetrix oligonucleotide arrays from a subgroup of the same patients. Finally, the usefulness of MFC-derived results for outcome prediction and individual risk assessment of neuroblastoma patients was evaluated.

Patient cohort and RNA preparation from primary neuroblastomas. RNA preparation from neuroblastomas was done as previously described (1). Data collection, microarray data analysis, and the majority of the patient cohort have been documented before (1). Briefly, expression profiles from 68 neuroblastoma patients had been obtained using Affymetrix U95A chips. Expression patterns characteristic for biological and clinical features were obtained. Of these 68 samples, 47 were included in the present study, solely chosen on the basis of availability of RNA from the same tumor sample. For independent validation of the data, 16 additional patients with neuroblastoma were included. Written informed consent was obtained from patients or their parents. To avoid biasing the methods towards event-free survival, no patient was included in the study showing event-free survival, MYCN amplification or loss of 1p, and <2 years of follow-up. Mean and median follow-up were 1,443 and 1,401 days, respectively. Patient characteristics are listed in Table 2.

Construction of a neuroblastoma-specific MFC. A set of 39 genes obtained by prediction analysis of microarray (PAM) data was shown to be able to discriminate between neuroblastoma patients with recurrent tumors and those with no evidence of disease following initial therapy (1). For the setup of the MFC, we used the top-ranked genes of the PAM classifier and we also included known factors contributing to neuroblastoma biology (MYCN and NTRK1/TrkA), as well as four reference genes (GAPDH, HPRT1, SDHA, and UBC), as previously suggested (11). Genes and the corresponding assays on demand used for the setup of the MFC are listed in Table 1.

Statistical analysis of real-time PCR data. Real-time PCR data were preprocessed and stored in SDS 2.2 software (Applied Biosystems). Results from each run were analyzed separately using a software-defined baseline and a manual threshold between 0.1 and 0.5 to record the cycle thresholds (C_t). Data normalization was done with the geometric mean of four reference genes using qBase (Hellemans et al.)¹

High-level informatics for comparison of real-time PCR and Affymetrix microarray data. Statistical calculations were done using R.² The R implementation of significance analysis of microarray (SAM; ref. 12) was used with default parameters, for the two-class unpaired case, with t statistic. Support Vector Machine (SVM) calculations were carried out using the LIBSVM library (13), with the interface provided by R. A sigmoidal kernel was used, with parameters γ = 0.5, cos t = 1, ε = 0.1. The PAM (14) implementation in R was used with default values except for the threshold, which was set to zero. Decision trees were constructed using the implementation of recursive partitioning and regression trees in R. Standard parameters were used for the classification tree, with the minsplit parameter set to 5. Computerized Affected Sibling Pair Analyzer and Reporter (CASPAR; ref. 15) was used to predict survival times for the individual patient. In addition, an adaptation of CASPAR for classification problems was used to distinguish relapsed from cured patients. In essence, this adaptation uses a logistic regression model for class assignments instead of the Cox model used by standard CASPAR for survival analysis.

Assessment of mRNA expression using MFCs is reproducible. To evaluate the potential of MFCs as a useful alternative application for candidate gene analysis and prognosis prediction in a routine clinical setting, we constructed a neuroblastoma-specific MFC and reanalyzed neuroblastoma samples previously studied by Affymetrix U95A microarrays. Genes were selected for the setup of the neuroblastoma-specific MFC based on the PAM analysis discriminating between neuroblastoma patients with or without relapse of disease (1). We also added known biological factors including the MYCN oncogene and the neurotrophin receptor TrkA/NTRK1, as well as genes from chromosome 1p, a region frequently deleted in neuroblastoma. Additionally, four reference genes were also included on the MFC (for a list of genes and assays, see Table 1). Because multiplex real-time PCR may yield varying results due to methodologic difficulties including inconsistent amplification efficiency, we assessed reproducibility of the MFCs as a first step of quality control by evaluating one neuroblastoma sample from each card on an independent card (reference card). A total of 63 neuroblastomas were analyzed using eight cards, and one sample from each of these cards was randomly chosen for analyses on the reference card. In summary, the mean δC_t between the two independent runs for the same sample was 0.4, which is equivalent to a 1.3-fold difference or 32.2% difference assuming 100% amplification efficiency. The median difference was 0.29- or 1.22-fold (i.e., half of the sample gene pairs were within 21.9% difference). Analysis of the complete cumulative difference distribution indicated that 75% of the sample gene pairs were within 1.47-fold, 90% within 1.82-fold, and 95% within 2.15-fold (Fig. 1A). MFC reproducibility between independent runs seemed to be somewhat dependent on the target gene, in particular for low-abundance transcripts. For instance, we observed consistently higher δC_t values for the NME2 gene, which is expressed at low levels. On the whole, our data indicate a high level of intercard reproducibility.

Expression analysis of neuroblastoma-specific genes by MFCs and Affymetrix U95A microarrays. The MFC quantitative PCR results were compared with those obtained by Affymetrix U95A chips because there was considerable overlap of genes included on the MFC and the previously described Affymetrix-based PAM classifier discriminating patients with event-free survival or early relapse. Following normalization of the real-time PCR data, Pearson correlation coefficients between array and PCR were calculated for each gene. Correlation coefficients ranged from 0 (no correlation; COX6C) to 0.95 (high correlation; MYCN), indicating that the Affymetrix array and MFC quantitative PCR platform produced different results for the expression of some genes of interest (Fig. 1B). Univariate analysis of the individual genes was done to assess their ability to distinguish between patients with or without relapse. Application of t statistics to the results from the MFCs revealed that expression of two genes, NEDD8 and PSMD10, differed significantly between event-free survival and relapse patients, confirming our previous results from Affymetrix arrays (1). All but one of the remaining genes displayed the same direction of regulation, as was predicted by PAM analysis of the Affymetrix array results, but did not reach statistical significance. SAM analysis of MFC quantitative PCR data indicated that high CDC42, PTPRF, and NTRK1 expression was associated with event-free survival, whereas high LDHA and PSMD10 expression correlated with relapse of disease (Fig. 2A).

Prediction of clinical parameters and patient outcome by supervised analysis of real-time PCR results. We next assessed the correct assignment of the 63 patients (including 47 patient samples from our previously published microarray study) to their clinically defined groups using MFC quantitative PCR data. In the aforementioned analysis of primary neuroblastomas on the Affymetrix platform, a considerable percentage of samples failed to pass the inclusion quality criteria for the microarray study. Nevertheless, 16 of these samples had a RNA quality sufficient for real-time PCR analysis as judged by A₂₆₀/A₂₈₀ ratios and conventional PCR using reference genes (data not shown). These samples were included for MFC analysis as a test set, mimicking the clinical situation in which not every tumor sample is qualitatively suitable for microarray analysis. Training of the class prediction models was done using the MFC quantitative PCR data from the 47 patients already profiled using Affymetrix microarrays. Event-free survival of neuroblastoma patients was predicted with 75% classification accuracy (12 of 16) in the validation set using SVM and 81% classification accuracy (13 of 16) using PAM. Interestingly, PAM correctly predicted all relapses in this independent data set. CASPAR analysis and decision trees resulted in 69% classification accuracy (11 of 16). All analysis methods applied here misclassified the same three patients. Interestingly, predictions of all methods used PTPRF expression and one of the proteasome components (PSMB5, NEDD8, or PSMD10) and all but one method include NTRK1, CDC42, and LDHA as top-ranked genes (Table 3), which is also supported by SAM analysis (Fig. 2A). Because we included genes located on chromosome 1p on the MFC, we also tested whether it was possible to predict the 1p status of neuroblastoma patients using SAM and PAM analyses of MFC quantitative PCR data. Interestingly, we not only found a high correlation between 1p loss and expression of genes in the MYCN amplicon, which is a well-known phenomenon, but also a strong association of 1p loss and expression of proteasomal components (PSMD10 and, to a lesser extent, PSMB5 and NEDD8; Fig. 2B) using SAM analysis. The expression of NTRK1 and 1p genes was anticorrelated with 1p loss. PAM-based prediction of 1p status using a leave-one-out approach resulted in 88% prediction accuracy (Table 4). Taken together, these results show a high accuracy of MFC-based predictions for clinical features of neuroblastoma.

Individual risk prediction of relapsing neuroblastoma using PAM and CASPAR. To assess the individual risk of neuroblastoma patients for suffering a relapse, PAM was applied to the entire data set using leave-one-out cross-validation. This analysis method trains the classifier on all but one patient (i.e., n = 62), then uses this optimized classifier to predict the outcome of the remaining patient. The prediction accuracy of the PAM classifier was close to 80% (79.4%). Transforming these results for PAM predictions “relapse/no relapse” into a survival time analysis revealed that the class prediction was highly effective and significant (P < 0.0001; Fig. 3).

CASPAR was applied to the MFC quantitative PCR data to calculate cure or time to relapse for each patient individually using leave-one-out cross-validation. The graphical representation of the receiver operating characteristics as a measure of sensitivity and specificity of CASPAR predictions is summarized in Fig. 4A [area under the curve (AUC) plot]. An AUC value of 1 implies perfect prediction in terms of sensitivity and specificity. AUC values of ∼0.7 for time points >3 years were obtained based on the predictions by CASPAR using the data presented here.

This is comparable to the predictive power of other recently published microarray studies (15–18). As most of the neuroblastoma relapses occur within the first 3 years after diagnosis, we analyzed the power of CASPAR predictions for this time frame. For this purpose, patients were divided into two groups based on the predictions from CASPAR analysis: those patients with a predicted time to relapse of <3 years or those with predicted event-free survival or time to relapse of >3 years. The resulting Kaplan-Meier plot shows a highly significant separation of patients with event-free survival or relapse (Fig. 4B), which was confirmed by log-rank test (P = 0.0006). Thus, CASPAR-based analysis of MFC quantitative PCR data is a useful tool for predicting the individual patient risk of neuroblastoma relapse.

A major drawback for the clinical relevance of microarray analysis is the difficulty to compare results obtained on different platforms. Strategies to render these platforms compatible are urgently needed, and some approaches have already been described (4, 19–21). To date, attempts at prediction of outcome in neuroblastoma have relied on single expression profiling platforms, partly without addressing the need for independent validation procedures with features qualifying them for convenient use at the bedside (2, 3). As an independent technology for validation of microarray data derived from different platforms, quantitative real-time PCR has turned out to be the method of choice to obtain reliable information about gene expression. Approaches based on quantitative real-time PCR have also been used for neuroblastoma biomarker discovery, sometimes in large cohorts (22–26). All of these studies used single-tube assays lacking efficient handling procedures. However, technologies appropriate for clinical applications not only have to fulfill the requirement of high reproducibility and accuracy but must also be feasible and efficient in the clinical setting. In contrast to conventional real-time PCR analysis, this demand is met by the introduction of MFCs or TaqMan Low Density Arrays (Applied Biosystems). We therefore aimed to assess the value of MFCs as a useful tool for the clinical application of outcome predictors that were previously identified using microarray analyses. Thus, we compared the clinical significance of previously obtained results from a neuroblastoma microarray study to the significance of results obtained on the same primary tumor samples using the highly parallel quantitative real-time PCR format available as MFCs. To assess the reproducibility of the platform, we first analyzed intercard variability using a reference card loaded with eight randomly chosen tumor samples from the other MFCs used in this study. This approach confirmed that all PCR reactions have comparable amplification efficiencies and minimal interrun variability, resulting in equal C_t value recording and constant δC_t's between plates. We found a high correlation of gene expression, suggesting a high intercard reproducibility of quantitative PCR results, rendering this application capable, in principle, of addressing clinical samples in a reproducible manner. The consistency of results between different batches of MFCs, laboratories, and quantitative PCR machines must still be addressed in multisite studies because it has been pointed out that only stringent quality controls ensure reproducibility of microarray-based results between laboratories (27). It is also striking that, at least for some genes, there were considerable differences in correlation between gene expression in real-time PCR and Affymetrix chip analyses. A recent study pointed out that differences in microarray and real-time PCR results using MFCs may occur if genes are expressed at low levels or if the difference in gene expression between samples is low (28). Platform-dependent differences (hybridization versus amplification) and insufficient specificity of the array probes might also account for lack of correlation between results obtained on the Affymetrix and MFC platforms. Additionally, interrogation of different gene products resulting from alternative splicing must be taken into consideration. The latter might account for differences detected for TrkA/NTRK1 mRNA expression, displaying a low correlation coefficient between real-time PCR and array analysis. Recent reports indicate the existence of an alternative oncogenic splice variant, TrkAIII, present in aggressive neuroblastomas (29). This may explain why high TrkA expression is, in general, a hallmark of benign tumors but was shown to be elevated in some relapsing tumors by MFC quantitative PCR. Therefore, next-generation gene-based outcome prediction should rather be focusing on exon-specific expression rather than gene-specific expression.

Because some genes included on this first-generation MFC proved to be uninformative for prediction of neuroblastoma outcome, the gene list should also be revised. We previously reported accuracies of 80% and 85% in predicting early recurrence of neuroblastoma using SVM and PAM to analyze Affymetrix array data, respectively (1). Whereas SVM did slightly worse on the real-time PCR data, prediction accuracy obtained on MFC quantitative PCR data using PAM analysis was comparable to that on Affymetrix data. Generally, classification of neuroblastoma patients by high-level analysis according to their disease status as event-free survival or relapse worked with comparable efficiencies of 80% to 85% using either MFC quantitative PCR or Affymetrix array data. Interestingly, the same three patients were misclassified by the predictors generated from analysis of either Affymetrix array or MFC data. This might indicate that rather therapy-related than biology-based reasons account for the outcome of these particular patients. Alternatively, there might be more than one biological signature for relapsing neuroblastoma, which is not addressed by the gene set investigated in the present study. Because genes from the frequently deleted chromosomal region, 1p36, were also included in our analysis, we attempted to predict the chromosome 1p status (normal 1p or loss of 1p) based on gene expression. This was feasible in principle, as Wang et al. (30) have shown reduced expression of multiple genes on chromosome 1p in cases of neuroblastoma with 1p loss of heterozygosity. Although our MFC was not designed for the purpose of detecting 1p aberrations, prediction accuracy of 1p status (normal versus loss) was 88% by PAM analysis. We could further confirm the strong association between MYCN expression and 1p loss described earlier (31). Interestingly, overexpression of proteasome components was highly correlated with 1p loss of heterozygosity as well. Therefore, it is tempting to speculate that patients with 1p loss might especially profit from specific therapies involving proteasomal inhibitors such as Velcade. Finally, loss of 1p was also inversely correlated to NTRK1 expression. This relationship is intuitively comprehensible but has never been reported before, although the association between NTRK1 expression and favorable outcome, as well as the anticorrelation with MYCN amplification, has previously been documented (31, 32). These results show that expression of neuroblastoma-specific genes also reflects the genomics of neuroblastoma, at least in terms of 1p status.

Interestingly, none of the methods applied here uses MYCN expression for decision-making when clinical outcome is addressed. This further corroborates the notion that MYCN expression is not necessarily as informative as MYCN amplification (33). Additionally, it has been pointed out recently that high-level MYCN expression is associated with favorable outcome in neuroblastoma lacking MYCN amplification (34). Thus, determining MYCN expression levels cannot substitute for MYCN status analyses by fluorescence in situ hybridization or other methods.

To predict the individual risk of suffering a neuroblastoma relapse, we did PAM- and CASPAR-based analyses on the neuroblastoma MFC data. Kaplan-Meier analysis using the decisions of the respective methods revealed a highly significant discrimination of patient outcome predicted both by PAM (Fig. 3A) and CASPAR (Fig. 4B). Moreover, a powerful discrimination between patients with event-free survival or relapse was possible not only in the cohort but also in a time-dependent manner with AUC values >0.7 for t > 5 years after diagnosis (Fig. 4A and B), emphasizing the accuracy of the combination of real-time PCR and CASPAR analyses. Application of either microarray- or real-time PCR–based methods to clinical decision-making requires easy-to-perform and reliable patient-specific predictions. At present, molecular diagnostics based on gene expression profiling is entering the clinical setting in many prospective studies. However, the advantage of highly parallel real-time PCR methods such as MFCs over conventional microarrays is their efficiency and the ease of implementation in decentralized institutions.