Diagnostic Distinction of Malignant Melanoma and Benign Nevi by a Gene Expression Signature and Correlation to Clinical Outcomes

Melanoma is now the seventh most common cancer among adults, and approximately 2 million melanocytic lesions are biopsied annually in the United States for clinical suspicion of melanoma (1). Although the majority of these lesions are benign, melanoma will account for an estimated 10,130 deaths in 2016 (2). However, many melanomas are curable if detected early, with a 10-year survival rate of 95% for stage IA melanoma compared to only 10% to 15% for stage IV (3). This marked difference in biologic potential and clinical outcome underscores the value of early and accurate diagnosis.

Many melanocytic lesions can be accurately classified as benign or malignant by a skilled histopathologist using conventional light microscopy; however, numerous studies have demonstrated that pathologists arrive at different diagnoses in 15% to 38% of cases when evaluating the same specimens (1, 4–6). Studies have also shown that the consensus diagnosis of multiple experienced dermatopathologists may not align with ultimate clinical outcomes (7, 8).

To improve diagnostic accuracy, ancillary techniques have been sought to provide adjunctive diagnostic information for pathologists confronting histopathologically ambiguous melanocytic lesions. These methods include array comparative genomic hybridization (aCGH) and fluorescence in-situ hybridization (FISH), both of which rely on the detection of chromosomal copy number aberrations within neoplastic melanocytes (9–11). Recently, a gene expression signature has been developed to evaluate 23 genes that are differentially expressed in malignant melanoma and benign melanocytic nevi using quantitative reverse transcription polymerase chain reaction (qRT-PCR) (12, 13).

Previous validations of the gene signature have demonstrated that this test is capable of differentiating melanoma from benign nevi with greater than 90% accuracy compared with consensus diagnosis of multiple expert dermatopathologists (12, 13). These studies are in line with standard practice to validate new diagnostic methods against the current diagnostic standard or clinical outcomes (14). Collecting long-term clinical follow-up for melanocytic lesions can be a challenge, given the duration of follow-up required for benign cases and the high rate of excision of suspected melanoma. In addition, the requirement for metastatic disease to prove malignancy may limit the types of lesions available for analysis. For example, some lesions that fulfill morphologic criteria for melanoma may lack metastatic capability (15). For this reason, aCGH, FISH, and the 23-gene signature have all been previously validated against the current diagnostic standard of histopathology. Although the known limitations of histopathology may impact the reported diagnostic accuracy in such studies, the extent of this impact is unknown. In addition, variability in the diagnostic accuracy of histopathology across different melanocytic subtypes suggests that the magnitude of this effect is likely dependent on the composition and size of the validation cohort.

To date, evaluation of adjunctive melanoma diagnostic methods against clinical outcomes has been limited to relatively small samples sizes (9, 16–24), and previously published validations of the 23-gene signature have been based on histopathologic diagnosis. However, validation against a large, clinical cohort is necessary to accurately assess the performance of a diagnostic test for clinical use. Therefore, the current study aimed to evaluate the diagnostic accuracy of the 23-gene signature utilizing a large cohort of melanocytic lesions with clinical outcomes. This was done by determining the sensitivity of the gene signature in primary cutaneous melanomas with proven distant metastases and the specificity in melanocytic nevi with long-term event-free follow-up.

The study was conducted with Institutional Review Board oversight and was approved with a waiver for individual patient informed consent (Quorum Review IRB). Archived melanocytic neoplasms not previously assessed by the gene expression signature that met inclusion criteria were independently identified and submitted for testing by the Cleveland Clinic (Cleveland, OH), Nottingham University (Nottingham, United Kingdom), and Stanford University (Stanford, CA). Additional samples were acquired from Avaden Biosciences. Cases were selected based upon pre-specified clinical outcome measures to identify malignant and benign lesions.

Malignant lesions were stage I, II, or III primary cutaneous melanomas that produced distant metastases subsequent to diagnosis of the primary lesion, where distant metastases were defined as melanoma present at anatomic sites not including the proximal sentinel lymph node(s), given the propensity for borderline Spitz lesions to spread exclusively to sentinel lymph nodes. Melanomas known to be metastatic at the time of initial diagnosis were excluded. No minimum length of follow-up was required for cases with documented distant metastases. Benign lesions were required to be diagnosed as benign nevi and have clinical follow-up after the initial biopsy that demonstrated no evidence of local recurrence or in-transit, regional, or distant metastases. A disease-free follow-up time of at least 5 years was recommended for benign sample submission/acquisition from participating institutions. Samples were excluded from patients with a previous history of melanoma, a diagnosis of synchronous or metachronous melanoma or other cancer during the follow-up period, or who received immune therapy or chemotherapy prior to diagnosis of melanoma. Re-excision specimens, metastatic melanomas, and lesions that did not contain at least 10% tumor volume were also excluded in accordance with the technical specifications of the assay (13).

De-identified clinical data for each sample were obtained from the submitting institution and included patient age, gender, anatomic site of the primary lesion, Breslow depth, ulceration status, sentinel lymph node biopsy status, anatomic location of metastases, time to metastasis, and length of follow-up. Eleven melanomas were received without ulceration status indicated. The presence or absence of ulceration was determined by review of hematoxylin and eosin (H&E) stained sections of the lesions by a dermatopathologist (L.E. Clarke) at the testing institution. Histopathologic subtypes (e.g., superficial spreading melanoma, nodular melanoma, etc.) were determined for all cases by a panel of three dermatopathologists (J.S. Ko, S.D. Billings, and L.E. Clarke) who were blinded to the diagnosis of the submitting pathologist(s) and the gene expression score.

Gene expression analysis (Myriad Genetic Laboratories, Inc.) was carried out on archival formalin-fixed paraffin-embedded (FFPE) tissue sections from each lesion according to previously described methods (12, 13, 25). Briefly, a pathologist identified representative areas of the lesion using a single H&E-stained section. Lesions with <10% tumor volume were excluded from testing based on previous evidence that these lesions can produce false-negative results due to dilution by nonlesional cells (13). The representative area of the lesion identified for testing was macrodissected from unstained tissue sections and pooled into a single tube for RNA extraction. The qRT-PCR assay measured the differential expression of 23 genes (14 tumor marker genes and 9 housekeeper genes, Table 1).

The gene signature included three categories of tumor marker genes that are differentially expressed in melanoma: PRAME (cell differentiation), S100A9 and 4 related genes (cell signaling response to tissue damage), and 8 immune group genes (tumor immune response signaling). The S100A9 and immune group categories consisted of genes with highly correlated expression and similar biological functions. The differential expression of all genes in each category was consolidated into average component scores. A weighted algorithm was applied to normalize the tumor marker genes to the average differential expression of the housekeeper genes and produce a single numeric score plotted on a scale ranging from −16.7 to +11.1 (12). Scores from −16.7 to −2.1 were reported as “likely benign,” scores from −2.0 to −0.1 were reported as “indeterminate,” and scores from 0.0 to +11.1 were reported as “likely malignant.”

The correlation between the gene expression test result and the clinical outcome (benign or malignant) of the neoplasm was assessed for all eligible cases. Sensitivity (proportion of correctly identified positive/malignant cases) and specificity (proportion of correctly identified negative/benign cases) were calculated after excluding lesions that produced indeterminate results (scores between −2.0 and 0.0).

A total of 293 samples (135 benign, 158 malignant) were submitted by the participating institutions. Of these, 53 did not satisfy the pre-designated inclusion criteria (Fig. 1). An additional 58 samples failed to produce a valid gene expression score (Fig. 1). This failure rate may be due to insufficient mRNA quality, as mRNA extracted from archival samples is more prone to fragmentation compared with recently prepared FFPE lesions (12). In support of this, lesions within the current cohort that were tested within 5 years of the primary biopsy had a much lower failure rate (2/22, 9.1%) compared with lesions tested more than 5 years after biopsy (56/218, 25.7%).

Table 2 summarizes the clinical and histopathologic characteristics of the final cohort of 182 cases, which included 83 benign nevi and 99 malignant melanomas. The average age for patients with malignant and benign lesions was 63 and 42 years, respectively. Sixty-seven percent of melanomas were from males, whereas 33% were from females. Median follow-up time for benign lesions was 74.9 months, with 70% (58/83) of cases having at least 5 years (60 months) of clinical follow-up. Median time to metastasis for the malignant samples was 18 months.

The final cohort included a diverse range of histopathologic variants (Table 3). Overall, 12 distinct melanoma subtypes were tested, including nodular melanoma, superficial spreading melanoma, acral melanoma, and nevoid melanoma. The cohort also contained 18 distinct benign subtypes, including compound nevi, intradermal nevi, compound dysplastic nevi, and Spitz nevi. Median Breslow depth for the melanoma samples was 2.8 mm. The Breslow depth was 0 to 1.0 mm for 16 of the melanomas (including one melanoma in situ), 1.01 to 2.0 mm for 20, and >2.0 mm for 62. Ulceration was present in 39% of melanomas (Table 2).

Of the 99 melanoma cases included in the final cohort, 91 produced positive (“likely malignant”) scores, 6 produced negative (“likely benign”) scores, and 2 produced indeterminate scores (Table 3, Supplementary Fig. S1). The score accurately classified as malignant all thin (Breslow thickness ≤ 1.0 mm) melanomas (n = 16; Supplementary Figs. S2 and S3), all “nevoid” melanomas (n = 7; Supplementary Fig. S4), all but one melanoma arising within pre-existing nevi (n = 6; Supplementary Fig. S5), and all lentigo maligna melanomas (n = 2). Sentinel lymph node biopsy was performed in 14 outcome-based melanoma cases, six of which were positive for melanoma. The primary melanomas from all 14 cases produced positive gene expression scores (Table 4).

Of the six metastasis-proven melanomas that produced scores within the benign range of the scale (false-negative results), five had a Breslow thickness >2.0 mm (Table 5). Three of the melanomas that produced false-negative results were desmoplastic melanomas (“pure type”; Supplementary Fig. S6), two were superficial spreading melanoma (one with a substantial vertical growth phase, the other a melanoma arising within a nevus in which much of the targeted tissue appears to have been nevus), and one was a nodular melanoma (“spindle-cell type”). Two melanomas produced scores within the indeterminate range. This included a superficial spreading melanoma as well as a lesion classified as melanoma, not otherwise specified, in which isolated clusters of malignant melanocytes were surrounded by dense aggregates of lymphocytes.

Of the 83 benign cases, 75 produced negative (“likely benign”) scores, 3 produced positive (“likely malignant”) scores, and 5 produced indeterminate scores (Table 3; Supplementary Fig. S1). Of the three lesions with event-free follow-up that produced malignant scores (apparent false positives), two were compound dysplastic nevi with severe atypia (344 and 84 months of event-free follow-up, respectively), and one was a compound nevus with congenital and dysplastic features (180 months of event-free follow-up). One of these lesions was re-excised with a margin of normal skin. Another was not re-excised because the original biopsy site could not be definitively identified due to extensive solar damage and prior treatment with liquid nitrogen for nonmelanoma skin cancer (Supplementary Fig. S7). Treatment for the third lesion was unknown. Five lesions without adverse events detected during the follow-up period produced scores within the indeterminate zone of the scale. Four were dysplastic nevi and one was a compound nevus with “special site” features (Table 5).

The sensitivity and specificity of the signature was determined for all eligible samples that produced “likely benign” or “likely malignant” test results (n = 175). The resulting sensitivity and specificity were 93.9% and 96.2%, respectively. When benign samples with less than 5 years of clinical follow-up are excluded, the specificity was 94.6%.

Histopathology is an accurate diagnostic method for many melanocytic lesions and is currently considered the “gold” standard for the diagnosis of melanoma. In line with standard practice, adjunctive molecular tests to aid in the diagnosis of melanoma have largely been developed and evaluated using histopathology as a reference standard (9–12). Previous validation studies of the 23-gene expression signature used here have reported greater than 90% diagnostic accuracy by comparison with consensus diagnosis of multiple expert dermatopathologists (12, 13). However, a separate study of adjunctive diagnostic methods reported a sensitivity of only 55% and a specificity of 88% when the gene signature was compared with consensus histopathologic diagnosis (26). This variability in performance is consistent with the performance of FISH relative to histopathologic diagnosis, with reported sensitivities ranging from 35% to 87% (10, 18, 24, 26). Important differences in study design, such as the tumor subtypes analyzed and the methods for histopathologic review, likely drive the variability in reported sensitivities of ancillary diagnostic tests. In particular, histopathologic review can vary between studies based on the number of reviewing dermatopathologists, requirement for consensus, and whether reviewers were blinded to other diagnoses made by histopathology or other adjunctive tests. This highlights that although histopathology may be a reasonable reference standard in a carefully designed study, a more robust reference is needed to confirm diagnostic accuracy.

Clinical outcome data offer an objective standard for determining the accuracy of diagnostic methods; however, the challenge of compiling a sufficient number of cases to support robust clinical validation presents significant challenges. This includes the limited proportion of melanoma patients who develop distant metastases, the large number of patients who are lost to follow-up before a benign diagnosis can be definitively confirmed, and the difficulties encountered in acquiring corresponding clinical outcome data and primary biopsy tissue from multiple separate institutions. For these reasons, most reports validating molecular diagnostic tests against clinical outcomes for melanocytic lesions have been small subset analyses of larger studies that use histopathology as the reference standard (9, 16–24).

The current study evaluated the accuracy of a 23-gene expression signature in distinguishing clinical outcome proven melanomas from benign nevi. The cohort included 99 melanomas that metastasized following initial diagnosis and 83 nevi with long-term disease-free follow-up. Excluding indeterminate cases, the signature differentiated these lesions with 93.9% sensitivity and 96.2% specificity.

Many of the melanomas in this cohort were relatively large tumors (63% with a Breslow depth of greater than 2.0 mm). However, a substantial number of early-stage (“thin”) melanomas were also included, several of which would have been considered relatively low risk for metastasis by histopathologic staging (i.e., Breslow depth < 1.0 mm, no dermal mitoses, no ulceration; Supplementary Figs. S2 and S3). More than 70% of melanomas in the United States are now <1.0 mm in thickness at the time of diagnosis and account for up to 27% of fatal melanomas (27). The gene expression signature accurately classified all tumors <1.0 mm in thickness included here, suggesting it may prove useful in detecting the subset of thin melanomas with metastatic capacity. Conversely, five of the six false-negative results were produced by “thick” melanomas (Breslow depth greater than 2.0 mm) that may be less likely to pose diagnostic challenges. Although this may be expected given the prevalence of melanomas >2.0 mm thick within the study cohort, it is also possible that molecular differences between thick and thin melanomas may influence the accuracy of the gene expression signature (28–30).

The data presented here show high sensitivity and specificity across the most common clinico-histopathologic melanoma and nevus subtypes. One potential exception appears to be a reduced sensitivity in desmoplastic melanomas (Supplementary Fig. S6). The study cohort contained three desmoplastic melanomas, all of which produced a false-negative result. Other ancillary diagnostic techniques have also shown diminished sensitivity in desmoplastic melanomas relative to other melanoma types, suggesting that desmoplastic melanomas may possess unique molecular characteristics that make them less amenable to detection by existing molecular methods (31–33).

A primary aim of this study was to determine the accuracy of the gene expression signature in detecting lesions with unequivocal proof of malignancy. Therefore, the inclusion criteria of the study defined melanomas as lesions that produced distant metastases subsequent to initial biopsy. Some lesions that satisfy histopathologic criteria for the diagnosis of melanoma but lack metastatic capability would therefore have been excluded. It is also possible that some of the apparent false positives were in fact genuine melanomas that were “cured” by the initial biopsy or excision. Indeed, the histopathologic features of several of the “false-positive” cases were sufficiently concerning that all three reviewing dermatopathologists were unable to exclude melanoma and would have recommended treatment as melanoma (Supplementary Fig. S7). This suggests the benign outcome in these cases might have been due to early surgical intervention. In such cases, the gene expression signature can provide additive information to support excision and close clinical surveillance as is typically recommended for melanocytic neoplasms with “severe” atypia, for example.

Overall, multiple lines of evidence suggest that the gene expression signature differentiates benign and malignant melanocytic lesions with a high degree of accuracy. The data presented here show that the gene signature had high diagnostic accuracy relative to long-term clinical outcomes. In combination with previous validations that used consensus histopathologic diagnosis as the reference standard, this supports the use of the gene signature as an adjunctive diagnostic test to enable the early and accurate diagnosis of melanoma. Additional studies on specific melanoma and nevus subtypes are ongoing and will provide additional insight regarding the performance characteristics of the test.

J. Kim is a consultant/advisory board member for Myriad. C. Rock is employed at Myriad Genetic Laboratories, Inc. as medical science liaison. D.D. Flake II is employed at Myriad Genetic Laboratories, Inc. as a biostatistician, and has ownership interest (including patents) in Myriad Genetics. M.B. Warf is a scientist (II) at Myriad Genetic Laboratories, Inc. C. Miller is a clinical data manager at Myriad Genetics, Inc. K. Rushton is the senior director (Clinical Research) at Myriad Genetic Laboratories, Inc. A.R. Hartman was employed at Myriad Genetic Laboratories, Inc. at the time of this study as a Senior Director (Clinical Development). L.E. Clarke is the vice president (Medical Affairs) at Myriad Genetic Laboratories, Inc. No potential conflicts of interest were disclosed by the other authors.

Conception and design: J.S. Ko, C. Rock, D.D. Flake II, M.B. Warf, A.R. Hartman, R.J. Wenstrup, L.E. Clarke

Development of methodology: C. Rock, M.B. Warf, K. Rushton, A.R. Hartman, L.E. Clarke

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): J.S. Ko, B. Matharoo-Ball, S.D. Billings, B.J. Thomson, J.Y. Tang, K.Y. Sarin, E. Cai, J. Kim, C. Rock, H.Z. Kimbrell, C. Miller, K. Rushton, R.J. Wenstrup, L.E. Clarke

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): J.S. Ko, J.Y. Tang, J. Kim, D.D. Flake II, M.B. Warf

Writing, review, and/or revision of the manuscript: J.S. Ko, B.J. Thomson, J.Y. Tang, J. Kim, C. Rock, H.Z. Kimbrell, D.D. Flake II, M.B. Warf, C. Miller, K. Rushton, A.R. Hartman, R.J. Wenstrup, L.E. Clarke

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): B. Matharoo-Ball, S.D. Billings, J.Y. Tang, K.Y. Sarin, H.Z. Kimbrell, J. Nelson, T. Davis, C. Miller

Study supervision: K. Rushton, R.J. Wenstrup, L.E. Clarke

We acknowledge the efforts of the clinicians and patients who have participated in this study. We also acknowledge Krystal Brown for her assistance with manuscript preparation.

This work was supported by Myriad Genetic Laboratories, Inc.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.