Consensus on Molecular Subtypes of High-Grade Serous Ovarian Carcinoma

Ovarian carcinoma is a genomically complex disease, for which the accurate characterization of molecular subtypes is difficult but is anticipated to improve treatment and clinical outcome (1). Substantial effort has been devoted to characterize molecularly distinct subtypes of high-grade serous ovarian carcinoma (HGSOC; Table 1). Initial large-scale efforts to classify HGSOC of the ovary did not reveal any reproducible subtypes (2). Tothill and colleagues (3) reported four distinct HGSOC subtypes: (i) an immunoreactive expression subtype associated with infiltration of immune cells; (ii) a low stromal expression subtype with high levels of circulating CA125; (iii) a poor prognosis subtype displaying strong stromal response, correlating with extensive desmoplasia; and (iv) a mesenchymal subtype with high expression of N/P-cadherins. The Cancer Genome Atlas (TCGA) project also identified four subtypes characterized by (i) chemokine expression in the immunoreactive subtype; (ii) proliferation marker expression in the proliferative subtype; (iii) ovarian tumor marker expression in the differentiated subtype; and (iv) expression of markers suggestive of increased stromal components in the mesenchymal subtype, but did not report differences in patient survival (4). Further experimental characterization revealed an increased number of samples with infiltrating T lymphocytes for the immunoreactive subtype, whereas desmoplasia, associated with infiltrating stromal cells, was found more often for the mesenchymal subtype (5). Konecny and colleagues (6) independently evaluated the TCGA subtypes and also reported the presence of the four transcriptional subtypes using a de novo clustering and classification method.

However, robustness and clinical relevance of these subtypes remain controversial (7). The previous subtyping efforts have assessed prognostic significance in different patient cohorts and have taken different approaches to validate these subtypes in independent datasets. A recent review of HGSOC subtyping schemes highlighted the difficulty of comparing results of studies that used different subtyping algorithms, and that better general agreement on how molecular subtypes are defined would allow more widespread use of expression data in clinical trial design.(1)

Assessing the generalizability of subtyping algorithms is challenging as true subtype classifications remain unknown. This challenge is evident in the lack of published validation of the proposed HGSOC subtypes. Subsequent efforts have performed de novo clustering of new datasets and noted similarity in the clusters identified, but they have not reported quantitative measures such as classification accuracy or rate of concordance with previously published algorithms (8). In this article, we address these limitations by reimplementing three major subtyping methods (3, 5, 9) and assess between-classifier concordance and across-dataset robustness in a widely used database containing 1,770 HGSOC tumors (10), whose curation and data consistency has been independently validated (11). We show that each pair of subtype classifiers is significantly concordant and is virtually identical for tumors classified with high certainty. However, the subtypes do not meet established standards of robustness to reclustering (12), and only approximately one third of tumors are classified concordantly by all three subtype classifiers. Using this core set of tumors concordantly classified by each method, we develop consensusOV, a consensus classifier that has high concordance with the three classifiers, therefore providing a standardized classification scheme for clinical applications.

Analysis was carried out on datasets from the curatedOvarianData compendium; details of curation and of grading systems used by individual studies are described elsewhere (10). Datasets were additionally processed using the MetaGxOvarian package (Supplementary Information; ref. 13). Analysis was restricted to datasets featuring microarray-based whole-transcriptome studies of at least 40 patients with late-stage, high-grade, primary tumors of serous histology. This resulted in 15 microarray studies, providing data for 1,774 patients (Table 2). Duplicated samples identified by the doppelgangR package were removed (14). Survival analysis was performed for 13 of these datasets, which included 1,581 patients with annotated time to death or last time of follow-up.

Subtype classifiers were reimplemented in R (15) using original data as described by Konecny and colleagues (6), Verhaak and colleagues (5), and Helland and colleagues (9). These classifiers are based on nearest-centroids (6), subtype-specific single-sample GSEA (5), and subtype-specific linear coefficients (9), respectively. Implementations were validated by reproducing a result from each of the original publications (Supplemental File, Section “Reproduction of Published HGSOC Subtype Classifiers”).

Subtype calls from all included datasets were combined to generate a single Kaplan–Meier plot for each subtyping algorithm (stratified by subtype). HRs for overall survival between subtypes were estimated by Cox proportional hazards, and statistical significance was assessed by log-rank test using the survcomp R package (16). HRs were calculated using the lowest-risk subtype as the baseline group, and stratification by dataset was performed for HRs and significance testing.

Prediction strength (PS; ref. 12) is defined as a measure of the similarity between pairwise comemberships of a validation dataset from class labels assigned by (i) a clustering algorithm and (ii) a classification algorithm trained on a training dataset (Supplementary Fig. S1). The quantity is an established measure of cluster robustness with the following interpretation: a value of 0 or below indicates poor concordance, and a value of 1 indicates perfect concordance between models specified from training and validation data. Tibshirani and Walther (12), and subsequent applications of PS (17), have considered a value of at least 0.8 to be an evidence of robust clusters. PS was computed as implemented in the genefu Bioconductor package (18).

The tumors in each dataset were clustered de novo using our reproduced implementations of the algorithms of Konecny, TCGA/Verhaak, and Tothill (Supplemental File, Section “Reproduction of Subtype Clustering Methods”). Each dataset was also classified using implementation of the originally published subtype classifiers. This produced two sets of subtype labels for each sample in each validation dataset; these labels were used to compute PS.

For each pair of classifiers, subtypes were mapped on the basis of the observed concordance suggested in the original studies: Subtype C2 from Tothill corresponding to Immunoreactive in TCGA/Verhaak and C1_Immunoreactive-like in Konecny; C4 corresponding to Differentiated and C2_Differentiated-like; C5 corresponding to Proliferative and C3_Proliferative-like; and C1 corresponding to Mesenchymal and C4_Mesenchymal-like. Statistical significance of pairwise concordance was assessed by Pearson χ² test, and Cramer V was assessed to evaluate the strength of concordance. Two-way concordance was defined as the proportion of patients that were classified as the same mapped subtype across methods. Similarly, overall three-way concordance was defined as the proportion of tumors sharing the same mapped subtype across all three classifiers. Subtype-specific three-way concordance was defined as the number of tumors concordantly classified as that subtype by all three classifiers, divided by the number of tumors classified to that subtype by at least one method.

Each subtype classifier outputs for each patient a real-valued score for each subtype. Marginally classifiable tumors were identified on the basis of the difference between the top two subtype scores, denoted as the “margin” value. Thus, a higher margin indicates a more confident classification. For each pair of subtype classifiers, classification concordance was assessed on both the full dataset and considering only patients classified with margins above a user-defined cutoff.

The consensusOV classifier was implemented using a random Forest classifier trained on concordantly subtyped tumors across multiple datasets. The random Forest method has previously been used for building a multi-class consensus classifier to resolve inconsistencies among published colorectal cancer subtyping schemes (19). In order to avoid normalizing expression values across datasets, binary gene pair vectors were used as feature space, as recently applied for breast cancer subtyping (20, 21). To address differences in gene expression scales due to different experimental protocols, consensusOV first standardizes genes in each dataset to the same mean and variance, and computes binary gene pairs from standardized expression values. Because the feature size of this classifier increases quadratically with respect to the size of the original gene set, we used the smallest gene set of the original subtype classifiers (the gene set of Verhaak and colleagues; ref. 5), which contains 100 gene symbols. The consensusOV classifier outputs the subtype classification and a real-valued margin score to discriminate between patients that are of well-defined or indeterminate subtype. Similarly to previously published subtype classifiers, a higher margin score indicates higher confidence of classification.

Performance of the consensus classifier for identifying concordantly classified subtypes was assessed using leave-one-dataset-out cross-validation (22). Concordant subtypes were identified to train the random Forest classifier using 14 of the 15 datasets, and subtype predictions were tested in the remaining left-out dataset. This process was repeated for all 15 datasets. While predicting the samples in any given dataset, the training set was subsetted to contain only the concordant subtypes in other datasets.

Subtype calls from the consensus classifier were analyzed for correlation with histopathology and tumor purity in the TCGA dataset. In order to best represent the most confident subtype calls, a default cutoff was used to include only the 25% of patients with the largest classification margins. Available histopathology variables included lymphocyte, monocyte, and neutrophil infiltration. Tumor purity was assessed using the ABSOLUTE algorithm (23), which estimates purity and ploidy from copy number and SNP allele frequency from SNP genotyping arrays (Synapse dataset syn3242754). Significance of associations was tested by one-way ANOVA for patient age, purity, and immune infiltration.

All results are reproducible using R/Bioconductor (24) and knitr (25) with LaTeX output at overleaf.com/read/srvqbpxpqbyz. Output of this code is provided as Supplemental File S1. Subtyping algorithms are provided by the open source consensusOV R package available from Bioconductor (http://bioconductor.org/packages/consensusOV).

We performed a meta-analysis of three published subtyping algorithms for HGSOC (5, 6, 9) and developed a new consensus classifier to identify unambiguously classifiable tumors (Table 1). Each of these algorithms identified four distinct HGSOC subtypes with specific clinical and tumor pathology characteristics (Fig. 1). We assessed the algorithms on a compendium of 15 datasets including over 1,700 patients with HGSOC (Table 2) with respect to concordance, robustness, and association to patient outcome. By modifying individual algorithms to discard tumors of intermediate subtype, we found that concordance between algorithms is greatly improved.

We reimplemented three published HGSOC subtype classifiers (5, 6, 9) (Table 1) and applied these methods to new datasets. We ensured correct implementation of classifiers by reproducing results from the original articles (Supplementary Information). When applied to independent datasets, pairwise concordance of the three methods was statistically significant (P < 10⁻⁵, χ² test; Fig. 2A) with the highest agreement observed for Helland and Konecny subtyping schemes (70.9%), followed by Verhaak and Helland (67.4%) and Verhaak and Konecny (58.9%). Cramer V coefficients (26) indicated a strong association between subtypes as identified by the different algorithms (>0.5).

The individual subtyping algorithms calculate numeric scores for each subtype and assign each tumor to the subtype with the highest score. A tumor with a large difference or “margin” between the highest and second highest scores can be considered distinctly classifiable, whereas a tumor with two nearly equal scores could be considered of intermediate subtype. We examined the effect of modifying the individual algorithms to prevent assignment of indeterminate cases at various thresholds. For each pair of subtype classifiers, we examined the classification concordance with increasing thresholds on the margins.

For all pairs of subtype classifier, classification concordance increased as additional marginal cases are removed, approaching over 90% concordance once the majority of tumors are left unclassified (Fig. 2B). Three-way concordance followed the same trend with lower overall concordance: a minimum of 23% for the proliferative subtype and maximum of 45% for the immunoreactive subtype when all tumors are classified. Restricting the concordance analysis to the top 50% of tumors by margin value resulted in an increased overlap between 35% (proliferative) and 65% (immunoreactive). At a strict threshold of where only 10% of tumors are classified, 88% of tumors overall are concordantly classified by all three published subtyping algorithms (Fig. 2C). This large gain in concordance results from large reductions in both singleton calls, tumors assigned to one subtype by one algorithm, but not by the other two algorithms, and in 2-to-1 calls, tumors assigned to one subtype by two algorithms, but not by the third (Fig. 2D). This indicates that tumors distinctly classifiable by a single algorithm are more likely to be concordantly classified by the other algorithms, and conversely, tumors that appear ambiguous to one algorithm are less likely to be classified in the same way by the other algorithms.

All proposed subtyping algorithms classified patients into groups that significantly differed in overall survival (Fig. 3A, P < 10⁻⁵ for each subtyping algorithm, log-rank test). Comparing low-risk with high-risk subtypes for each algorithm, the HRs increase from approximately 1.5 as marginal cases are removed (Fig. 3B), suggesting that marginal cases may contribute to the intermediate survival profiles between subtypes.

Robust molecular subtyping should be replicable in multiple datasets. We performed de novo clustering in 15 independent ovarian datasets using the authors’ original gene lists and clustering methods. We compared these de novo clusters with the labels from our implementation of the published classifiers to assess robustness using the PS statistic (12). For PS estimation, we included validation datasets with at least 100 HGSOC tumors. Overall, we observed low robustness for all classifiers, with PS values under 0.6 for the three algorithms across datasets (Supplementary Fig. S2), none meeting the 0.8 threshold typically indicating robust classes (12, 17).

To assess whether low confidence predictions are driving the PS estimation, we recomputed the robustness of each algorithm set to classify varying fractions of the tumors with the highest margins. We used the largest dataset available, the TCGA dataset, as the validation set, and varied margin cutoffs of the Tothill and Konecny classifiers to require them to classify between 25% and 100% of the cases. From 10 random clustering runs, we report the median PS for the dataset. Clustering was performed on the full TCGA dataset and tumors of low margin values were removed subsequent to clustering and after the classifier was fully defined, in order to avoid optimistically biasing the apparent strength of clusters. We observed that the robustness of each algorithm is substantially improved by preventing them to classify ambiguous cases. The Tothill algorithm achieved almost perfect robustness (PS = 0.96) when allowed to leave 75% of cases unclassified (Fig. 4).

To maximize concordance across classifiers, we developed consensusOV, a consensus subtyping scheme facilitating classification of tumors of well-defined subtypes (Fig. 5). This classifier uses binary gene pairs (20, 21) to support application across gene expression platforms. The consensusOV classifier exhibits overall pairwise concordance of 67% to 78% with each of the other three algorithms, when classifying all tumors, and 94% concordance with tumors that are concordantly classified by the other three algorithms (Fig. 5A). The margins of consensusOV are higher for concordantly classified cases than for nonconcordantly classified cases, and this difference in margins is greater than for any of the other three classifiers (Fig. 6A). Accordingly, consensusOV was also most effective in identifying concordantly classified cases, although it was similar to the Konecny classifier in this respect (AUC = 0.76; Fig. 6B). As expected, differences in survival of subsets identified by consensusOV are similar to those identified by previous classifiers. The highest risk subtypes are proliferative [HR, 1.44; 95% confidence interval (CI), 1.07−1.94] and mesenchymal (HR, 1.97; 95% CI, 1.46−2.67) when removing 75% of indeterminate low-margin tumors, with similar HRs for the concordant cases (Fig. 5B).

The existence of four distinct and concordant molecular subtypes of HGSOC has been reported in several studies of large patient cohorts (4–6, 9), but also called into question by another effort (2) that could not identify subtypes, and by an independent validation effort that reported only two or three reproducible subtypes (27). Meanwhile, significant effort is being expended to translate these subtypes to clinical practice, for example, to predict response to the angiogenesis inhibitor bevacizumab in the ICON7 trial (28, 29). Our study pursues three major objectives: (i) reproduction of published subtype classification algorithms as an open-source resource; (ii) evaluation of the robustness and prognostic value of each proposed subtyping scheme in independent data; and (iii) consolidation of proposed subtyping schemes into a consensus algorithm.

We find that although the proposed four-subtype classifications demonstrate significant concordance and association with patient survival, none is robust to retraining in new datasets. By modifying any of these algorithms to prevent classification of tumors of ambiguous subtype, robustness and concordance of subtyping algorithms improve dramatically. We propose a “consensus” classifier that can identify the most unambiguously classifiable tumors, although a continuous trade-off exists between classifying more tumors versus having greater confidence in those classified.

Ambiguity in tumor classification might arise from a heterogeneous admixture of different subtypes, or from a more homogeneous composition of indeterminate subtype. This distinction has implications for the therapeutic value of the proposed subtypes. Lohr and colleagues estimated that 90% of tumors in the TCGA HGSOC dataset are polyclonal (30), and clonal spread of HGSOC has been directly inferred from single-nucleus sequencing (31). However, it remains unclear whether multiple clones in a tumor are consistently classifiable to the same subtype. If a tumor consists of multiple clones of different subtypes, then a subtype-specific therapy will likely lead to relapse as other clones survive and continue to grow. If this situation is common, even unambiguously classifiable tumors might be contaminated by small amounts of another subtype that could lead to relapse after subtype-specific therapy. This question could not be resolved by the current datasets, but may eventually be addressed by single-cell RNA sequencing (32), which is expected to further improve precision HGSOC molecular subtyping.

Several findings stand out in the validation of published subtyping algorithms. First, although previous studies reported inconsistent findings on whether subtypes differ by patient survival, our analysis in independent data showed clear survival differences. The 5-year survival rate for patients with different subtypes ranged from as low as 20% to as high as 50%. Second, published algorithms do not meet previously defined standards of robustness in terms of PS, a measure of consistency between subtype classifiers trained in independent datasets. Finally, the concordance of three algorithms, established independently by different research groups from different patient cohorts, is only moderate but can be greatly improved by modifying the original algorithms to allow them to leave ambiguous tumors unclassified. In their original forms, all-way concordance of the four defined classes occurs in 23% to 45% of tumors. As the individual algorithms are modified so they are allowed to leave ambiguous cases unclassified, the minority of remaining tumors can be classified with over 90% concordance between the three algorithms. This is a novel finding of interest, because an alternate possibility was that classifiers trained on different datasets would suffer low concordance no matter how they treated uncertain tumors. This finding suggests a subset of tumors of “pure” subtype; unfortunately, such unambiguous cases account for as few as 25% of HGSOC tumors. This places important limitations on the potential for clinical application of HGSOC subtypes. The proposed alternative, consensusOV, identifies the consensus of published HGSOC subtype classifiers. By training on multiple datasets, using binary (pairwise greater-than or less-than) relationships between pairs of genes, and using a relatively small gene set, it is designed to identify robustly classifiable HGSOC tumors across gene expression platforms and datasets.

Moving forward, general agreement on how molecular subgroups of ovarian cancer are defined would facilitate the use of expression data in clinical management. (33). The current subgroups, although prognostically important, are not yet clinically meaningful. Much like other prognostic factors such as age, ascites, and histology, they do not alter clinical management. However, a better understanding of the biology underlying the subgroups will provide a more rational targeted treatment of those patients (perhaps first in trial) such as seen in HRD tumors with PARP inhibitors. The use of algorithms that can classify the tumor of an individual patient, while allowing some tumors to remain unclassified, along with assessment of subtype robustness in independent datasets by PS, would move the field closer to this goal.

No potential conflicts of interest were disclosed.

Conception and design: G.M. Chen, L. Kannan, M. Birrer, B. Haibe-Kains, L. Waldron

Development of methodology: G.M. Chen, L. Kannan, G. Parmigiani, M. Birrer, B. Haibe-Kains, L. Waldron

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): D.M.A. Gendoo, M. Birrer, B. Haibe-Kains

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): G.M. Chen, L. Kannan, Z. Safikhani, G. Parmigiani, M. Birrer, B. Haibe-Kains, L. Waldron

Writing, review, and/or revision of the manuscript: G.M. Chen, L. Kannan, L. Geistlinger, M. Birrer, B. Haibe-Kains, L. Waldron

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): L. Kannan, L. Geistlinger, V. Kofia, Z. Safikhani, B. Haibe-Kains, L. Waldron

Study supervision: B. Haibe-Kains, L. Waldron

Other (developed the MetaGxOvarian package): D.M.A. Gendoo

The authors thank Brad Nelson for his feedback regarding the prognostic value of molecular subtypes in HGSOC and Andrew Cherniak for providing ABSOLUTE purity and ploidy estimates for tumors from The Cancer Genome Atlas. G.M. Chen was supported by the Canadian Institutes of Health Research and the Terry Fox Research Institutes. D.M.A. Gendoo was supported by the Ontario Institute for Cancer Research through funding provided by the Government of Ontario. Z. Safikhani was supported by The Cancer Research Society (Canada). G. Parmigiani was supported by The National Cancer Institute 5P30CA006516-53. B. Haibe-Kains was supported by the Gattuso-Slaight Personalized Cancer Medicine Fund at Princess Margaret Cancer Centre, the Canadian Institutes of Health Research, and the Terry Fox Research Institute. L Waldron was supported by grants from the NCI at the NIH (1R03CA191447-01A1 and U24CA180996). This work was part of the immunoTherapy Network supported by the Terry Fox Research Institute (Translational Research Program Grant #1060).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.