Purpose:

Programmed cell death receptor-1 (PD-1) inhibitors are frontline therapy in advanced melanoma. Severe immune-related adverse effects (irAEs) often require immunosuppressive treatment with glucocorticoids (GCCs), but GCC use and its correlation with patient survival outcomes during anti–PD-1 monotherapy remains unclear.

Experimental Design:

In this multicenter retrospective analysis, patients treated with anti–PD-1 monotherapy between 2009 and 2019 and detailed GCC use, data were identified from five independent cohorts, with median follow-up time of 206 weeks. IrAEs were tracked from the initiation of anti–PD-1 until disease progression, initiation of a new therapy, or last follow-up. Correlations between irAEs, GCC use, and survival outcomes were analyzed.

Results:

Of the entire cohort of 947 patients, 509 (54%) developed irAEs. In the MGH cohort [irAE(+) n = 90], early-onset irAE (within 8 weeks of anti–PD-1 initiation) with high-dose GCC use (≥60-mg prednisone equivalent once a day) was independently associated with poorer post-irAE PFS/OS (progression-free survival/overall survival) [post-irAE PFS: HR, 5.37; 95% confidence interval (CI), 2.10–13.70; P < 0.001; post-irAE OS: HR, 5.95; 95% CI, 2.20–16.09; P < 0.001] compared with irAEs without early high-dose GCC use. These findings were validated in the combined validation cohort [irAE(+) n = 419, post-irAE PFS: HR, 1.69; 95% CI, 1.04–2.76; P = 0.04; post-irAE OS: HR, 1.97; 95% CI, 1.15–3.39; P = 0.01]. Similar findings were also observed in the 26-week landmark analysis for post–irAE-PFS but not for post–irAE-OS. A sensitivity analysis using accumulated GCC exposure as the measurement achieved similar results.

Conclusions:

Early high-dose GCC use was associated with poorer PFS and OS after irAE onset. Judicious use of GCC early during anti–PD-1 monotherapy should be considered. Further prospective randomized control clinical trials designed to explore alternative irAE management options are warranted.

Translational Relevance

Anti–PD-1 monotherapy as the standard-of-care treatment for patients with advanced melanoma has brought in substantial survival improvement at the cost of irAEs. Instinctively, the use of immunosuppressive agents (systemic glucocorticoids, GCC) to treat severe irAEs might impair the efficacy of anti–PD-1 monotherapy. Here, we report the results of a large sample cohort study [in total n = 947; irAE(+) n = 509] demonstrating that early onset severe irAEs (within 8 weeks after anti–PD-1 monotherapy initiation) that led to early use of high-dose GCC (≥60-mg prednisone equivalent once a day) are associated with more rapid disease progression and poorer OS in both exploratory and combined validation cohorts. These findings trigger caution, given the widespread use of high-dose GCCs for the treatment of irAEs.

The addition of programmed cell death receptor-1 (PD-1) inhibitory antibodies to the arsenal of therapies for the treatment of advanced melanoma has greatly improved the outcome of these patients, with unique but generally manageable toxicities (1–3). The mechanism of action of anti–PD-1 therapy gives rise to a potential immune response against self-antigens that can lead to the emergence of immune-related adverse effects (irAEs; refs. 4, 5). On the basis of this intrinsically shared mechanism behind anti–PD-1 efficacy and irAEs, it has been reported that irAEs were associated with higher objective response rate (ORR) in advanced melanoma and longer relapse-free survival (RFS) when given in the adjuvant setting (6, 7). A long-standing concern has been that immunosuppressive agents (generally systemic glucocorticoids, GCCs) to treat moderate or severe irAEs, might impair the efficacy of anti–PD-1. This concern has been supported by accumulating data in similar situations, specifically high-dose GCCs for ipilimumab-associated hypophysitis (8), baseline GCC use before anti–PD-1/PD-L1 blockade (9) and shorter RFS with extended use of GCCs in the adjuvant setting (7). Still, the impact of high-dose GCCs for the management of irAEs after anti–PD-1 monotherapy initiation (5) remains unclear. To address these issues, we performed a retrospective analysis on 947 patients with advanced unresectable stage III or stage IV melanoma (Supplementary Table)treated with anti–PD-1 monotherapy at five independent cancer centers and analyzed the correlation between survival, irAE and the impact of GCC exposure. We defined irAE as any AEs deemed immunologic in nature by treating clinical team.

Patients

Patients with advanced melanoma treated with anti–PD-1 antibody monotherapy (no prior anti–PD-1 exposure, initiated between May 2009 and Aug 2019) and with detailed GCC use data available were identified at Massachusetts General Hospital Cancer Center (MGH), Vanderbilt University Medical Center (VUMC), Memorial Sloan Kettering Cancer Center (MSKCC), Melanoma Institute Australia (MIA), and Dana-Farber Cancer Institute (DFCI). Last follow-up was in January 2021. All patients treated both within and outside clinical trial settings were included. Medical notes of each patient were reviewed and data independently QCed. The following clinical data were collected: baseline demographics, melanoma pertinent information [subtype, mutational status, AJCC stage, baseline lactate dehydrogenase (LDH)], previous treatment(s), survival, irAEs, and detailed use of GCC(s) (when given at the peak dose of above 30-mg prednisone equivalent per day), including peak dose (expressed as mg prednisone equivalent), and detailed tapering schedule. Effectiveness of anti–PD-1 monotherapy was determined by local radiologists or interpretation of radiology reports/physical exam by treating physicians. All participant centers used 12 weeks as the standard imaging schedule. Our primary goal was annotation of GCC(s) treatment and its correlation with anti–PD-1 monotherapy effectiveness (survival outcomes). This study has been conducted in compliance with Declaration of Helsinki, and informed written consent was obtained from each patient or each patient's guardian when local IRB considered necessary.

Assessments of irAE and effectiveness of anti–PD-1 monotherapy

irAEs were defined as any AE deemed immunologic in nature by treating clinicians. They were assessed until disease progression (PD), initiation of a new therapy, or last follow-up. They were graded on the basis of clinical descriptions from medical notes, clinical trial data, and/or by the National Cancer Institute Common Terminology Criteria for Adverse Events version 4.0 when objectively based and to the best of the clinical judgement. All datasets were independently evaluated by a second, independent clinician to assure accuracy. All irAEs were included for the cohorts of MGH, VUMC, MSKCC and MIA for irAE subtype and GCC exposure specific analyses. The initial irAE, first GCC-associated irAE, and first grade 3/4 irAE were included in the cohort of DFCI for the purpose of GCC exposure-specific analyses. Because of feasibility and scientific rationale (10), the detailed usage of GCC was collected only for GCC above 30-mg prednisone equivalent per day. IrAE of interest was defined as the first irAE that led to the use of GCC above 30-mg prednisone equivalent once a day or the initial irAE of a patient if no GCC above 30-mg prednisone equivalent once-a-day dose was given. For progression-free survival (PFS) and overall survival (OS), the start date of anti–PD-1 monotherapy was used as the index date. Post-irAE PFS and post-irAE OS were defined as the period from onset date of irAE of interest to disease progression/death or last follow-up time and were chosen as survival endpoints for the association analysis between high-dose GCC use and survival to eliminate the immortal time bias. High-dose GCC was defined as equal to or above 60-mg prednisone equivalent once a day.

To allow for the duration and magnitude of the tapering schedule to be taken into consideration, a sensitivity test assessing the correlation between accumulated GCC exposure and survival outcomes was performed. R package “maxstat” was used to determine the threshold for dichotomization that maximizes the post-irAE PFS difference using log-rank as the computed statistic and Hothorn & Lausen as the approximation of P value (11). The cutoff value for the accumulated GCC exposure dichotomization was calculated in the MGH cohort as 306-mg prednisone equivalent (as an example, 60 mg daily for 5 days would be 300 mg). Thus, patients with accumulated GCC exposure of above 300-mg prednisone equivalent was defined as high-accumulated GCC exposure.

Statistical analysis

The MGH cohort was used as the exploratory cohort. VUMC, MSKCC, MIA, and DFCI were combined as validation for correlation analysis of initial irAE and survival, as well as GCC exposure and correlation with post-irAE survival. Because of the discrepancy of data collection for irAEs, only VUMC, MSKCC, and MIA (DFCI excluded) was used as the validation cohort for the irAE subtype specific analyses (Fig. 1). Categorical variables were summarized by frequency and percentage whereas the continuous variables were summarized by median and range.

The relationships between irAE incidence and PFS/OS were analyzed using univariate and multivariate extended Cox models with onset of irAE as a time-dependent covariate to adjust for the immortal time bias. Analyses of specific irAE subtypes with respect to PFS/OS were conducted between the patients who experienced the specific subtype of irAE and patients who did not, regardless of the presence of other irAE subtypes.

The distributions of post-irAE PFS and post-irAE OS were summarized using Kaplan–Meier analysis and compared by the log-rank test. Multivariable Cox proportional hazards regression models were used to adjust for potential imbalanced prognostic factors among different groups, including previous treatment(s), baseline LDH, M stage, melanoma subtypes, and hospital/institute (for the combined validation cohort).

For landmark analyses, two preset landmarks, namely week 8 (2 months, approximation of median high-dose GCC–associated irAE onset time in the MGH exploratory cohort) and week 26 (6 months, approximation of 90th percentile of high-dose GCC–associated irAE onset time in the MGH exploratory cohort) were used (Fig. 1). Patients who already experienced the progression and survival events by landmark time points were excluded from the post-irAE PFS and post-irAE OS landmark analyses, respectively. High-dose GCC–associated irAEs (defined as those led to the use of GCC with peak dose ≥60-mg prednisone equivalent once a day; approximation of guideline-recommended dosage) occurring within 8/26 weeks after anti–PD-1 monotherapy initiation were treated as positive; otherwise, those occurring after this time frame, or irAEs that did not lead to high-dose GCC use, were treated as negative. In the sensitivity analysis, high-accumulated GCC exposure–associated irAEs (defined as those led to the accumulated exposure of GCC > 300-mg prednisone equivalent) occurring within 8/26 weeks after anti–PD-1 monotherapy initiation were treated as positive; otherwise, those occurring after this time frame, or irAEs that did not lead to high-accumulated GCC exposure, were treated as negative.

Statistical tests were two-sided, and P < 0.05 was defined as statistically significant. All analyses were performed using R version 3.6.0. (R packages survival, survminer, maxstat, and ggplot2).

In total, 947 patients with advanced melanoma treated with anti–PD-1 monotherapy were identified from five independent centers in the US and Australia (MGH, n = 169; VUMC, n = 246; MSKCC, n = 311; MIA, n = 114; DFCI, n = 107). The median follow-up time was 206 weeks (IQR, 146–274). Baseline characteristics and general survival data of the entire cohort are shown in Supplementary Table S1 (online only).

IrAEs of any grade occurred in 509/947 (54%) of the combined MGH, VUMC, MSKCC, MIA, and DFCI cohorts. Median irAE onset time was 9 weeks after anti–PD-1 monotherapy initiation. Detailed irAE characteristics are listed in Supplementary Tables S2–S4 (online only). Occurrence of any irAE (regardless of type or severity) was not significantly associated with PFS in both exploratory and validation cohorts. Occurrence of any irAE was not associated with OS in the exploratory cohort (HR, 0.80; 95% CI, 0.48–1.33; P = 0.38), but was associated with better OS in the validation cohort (HR, 0.63; 95% CI, 0.51–0.78; P < 0.001; Supplementary Table S5, online only). We then explored the correlation between irAEs and survival outcomes based on the system affected after adjustment for immortal time bias in the cohorts with detailed data of all irAEs available (MGH as exploratory and VUMC/MSKCC/MIA combined as validation). In multivariate analysis, no irAE subtypes were associated with either PFS or OS in the MGH (exploratory) cohort that was validated in the VUMC/MSKCC/MIA combined cohort (Supplementary Table S6, online only).

GCC use and correlation with PFS/OS

Overall, 164/509 (32%) patients who were irAE(+) received GCC with the peak dose >30-mg prednisone equivalent per day, among whom 59 (11%) had the peak dose ≥100 mg, 60 (12%) between 60 and 100 mg, and 45 (9%) had between 30 and 60 mg. When taking into account the duration of GCC use, 73 (15%) patients had the accumulated GCC exposure (calculated as the area under the GCC dose-time curve) above 1,000-mg prednisone equivalent, 63 (12%) had between 300 and 1,000 mg. The median time of tapering to a lower dose was 6 days. Details are listed in Table 1.

Table 1.

GCC usea in the irAE(+) subgroup in different cohorts (n = 509).

Number (%)
Entire cohortMGHCombined validation cohort
GCC use(n = 509)(n = 90)(n = 419)
GCC peak dose (mg once-a-day prednisone equivalent) 
 [300– > 1,000] 11 (2) 6 (7) 5 (1) 
 [100–300] 48 (9) 10 (11) 38 (9) 
 [60–100] 60 (12) 13 (14) 47 (11) 
 [30–60] 45 (9) 6 (7) 39 (9) 
 [0–30] 345 (68) 55 (61) 290 (69) 
Accumulated GCC exposure (mg prednisone equivalent)a 
 [3,000– >10,000] 18 (4) 6 (7) 12 (3) 
 [1,000–3,000] 55 (11) 10 (11) 45 (11) 
 [500–1,000] 42 (8) 9 (10) 33 (8) 
 [300–500] 21 (4) 4 (4) 17 (4) 
 [0–300] 28 (6) 6 (7) 22 (5) 
 0 345 (68) 55 (61) 290 (69) 
Mean tapering schedule (d)a,b (n = 164) (n = 35) (n = 129) 
 Median (d) 
 (7– > 14) 52 (32) 11 (31) 41 (32) 
 [5–7] 42 (26) 10 (29) 32 (25) 
 [3–5] 44 (27) 5 (14) 39 (30) 
 [1–3] 26 (16) 9 (26) 17 (13) 
Number (%)
Entire cohortMGHCombined validation cohort
GCC use(n = 509)(n = 90)(n = 419)
GCC peak dose (mg once-a-day prednisone equivalent) 
 [300– > 1,000] 11 (2) 6 (7) 5 (1) 
 [100–300] 48 (9) 10 (11) 38 (9) 
 [60–100] 60 (12) 13 (14) 47 (11) 
 [30–60] 45 (9) 6 (7) 39 (9) 
 [0–30] 345 (68) 55 (61) 290 (69) 
Accumulated GCC exposure (mg prednisone equivalent)a 
 [3,000– >10,000] 18 (4) 6 (7) 12 (3) 
 [1,000–3,000] 55 (11) 10 (11) 45 (11) 
 [500–1,000] 42 (8) 9 (10) 33 (8) 
 [300–500] 21 (4) 4 (4) 17 (4) 
 [0–300] 28 (6) 6 (7) 22 (5) 
 0 345 (68) 55 (61) 290 (69) 
Mean tapering schedule (d)a,b (n = 164) (n = 35) (n = 129) 
 Median (d) 
 (7– > 14) 52 (32) 11 (31) 41 (32) 
 [5–7] 42 (26) 10 (29) 32 (25) 
 [3–5] 44 (27) 5 (14) 39 (30) 
 [1–3] 26 (16) 9 (26) 17 (13) 

aGCC data collected at the peak dose of >30-mg prednisone equivalent once a day only.

bMean duration of GCC given at each single dose during tapering.

When using the peak dose of ≥60-mg prednisone equivalent once a day as the threshold of high-dose GCC (approximation of guideline-recommended dosage), there were 119 (23%) patients treated with high-dose GCC.

The most commonly seen irAE subtypes that led to the use of high-dose GCC involved gastrointestinal (n = 33, 28%), respiratory (n = 31, 26%), and hepatic (n = 20, 17%) systems (Table 2). In uni- and multivariate analyses adjusting for prognostic factors, including previous treatment(s), baseline LDH, M stage, melanoma subtypes, BRAF mutational status, and hospital/institute (for the combined validation cohort), high-dose GCC–associated irAE (treated as a time-dependent variable, adjusted for immortal time bias) was correlated with significantly poorer PFS in the MGH cohort (HR, 1.79; 95% CI, 1.05–3.06; P = 0.03), and with a trend in the multicenter validation cohort (HR, 1.29; 95% CI, 0.95–1.75; P = 0.11; Table 3). High-dose GCC–associated irAE was not associated with OS in either MGH exploratory or combined validation cohort (Supplementary Table S7, online only). A sensitivity test using accumulated GCC exposure as the measurement also demonstrated significant correlation between high accumulated GCC exposure and poorer PFS and the lack of correlation in OS (Supplementary Table S8).

Table 2.

Overview of irAEs that led to GCC with peak dose ≥60-mg prednisone equivalent once a day (n = 119).

Number (%)
Entire cohortMGH cohortCombined validation
IrAE leading to high-dose GCC use(n = 119)(n = 29)cohort (n = 90)
Median onset time (range, wk) 15 (0.1–269) 10 (1–96) 16 (0.1–269) 
Grade 
 1 5 (4) 2 (7) 3 (3) 
 2 42 (35) 8 (28) 34 (38) 
 3 58 (49) 14 (48) 44 (49) 
 4 13 (11) 4 (14) 9 (10) 
 NA 1 (1) 1 (3) 
Anti–PD-1 monotherapy 
 Discontinuation 77 (65) 19 (66) 58 (64) 
 No discontinuation 41 (34) 10 (34) 31 (34) 
 NA 1 (1) 1 (1) 
System involved 
 GI 33 (28)a 8 (28) 25 (28)a 
 Endocrine 4 (3) 3 (10) 1 (1) 
 Musculoskeletal 2 (2) 2 (2) 
 Skin 6 (5) 6 (7) 
 Respiratory 31 (26) 5 (17) 26 (29) 
 Liver 20 (17)a 5 (17) 15 (17)a 
 Others 24 (20) 8 (28) 16 (18) 
Number (%)
Entire cohortMGH cohortCombined validation
IrAE leading to high-dose GCC use(n = 119)(n = 29)cohort (n = 90)
Median onset time (range, wk) 15 (0.1–269) 10 (1–96) 16 (0.1–269) 
Grade 
 1 5 (4) 2 (7) 3 (3) 
 2 42 (35) 8 (28) 34 (38) 
 3 58 (49) 14 (48) 44 (49) 
 4 13 (11) 4 (14) 9 (10) 
 NA 1 (1) 1 (3) 
Anti–PD-1 monotherapy 
 Discontinuation 77 (65) 19 (66) 58 (64) 
 No discontinuation 41 (34) 10 (34) 31 (34) 
 NA 1 (1) 1 (1) 
System involved 
 GI 33 (28)a 8 (28) 25 (28)a 
 Endocrine 4 (3) 3 (10) 1 (1) 
 Musculoskeletal 2 (2) 2 (2) 
 Skin 6 (5) 6 (7) 
 Respiratory 31 (26) 5 (17) 26 (29) 
 Liver 20 (17)a 5 (17) 15 (17)a 
 Others 24 (20) 8 (28) 16 (18) 

Abbreviation: NA, not available.

aOne patient developed irAEs-affecting GI and liver simultaneously.

Table 3.

High-dose GCC (peak dose ≥60-mg prednisone equivalent once a day)–associated irAEs and their association with PFS in the MGH and multicenter validation cohorts.

MGH cohortValidation cohort
(n = 169)(n = 778)
Survival type Analysis type HR (95% CI) P HR (95% CI) P 
PFS Univariate 2.21 (1.32–3.70) 0.003 1.33 (0.98–1.79) 0.06 
 Multivariatea 1.79 (1.05–3.06) 0.03 1.29 (0.95–1.75) 0.11 
MGH cohortValidation cohort
(n = 169)(n = 778)
Survival type Analysis type HR (95% CI) P HR (95% CI) P 
PFS Univariate 2.21 (1.32–3.70) 0.003 1.33 (0.98–1.79) 0.06 
 Multivariatea 1.79 (1.05–3.06) 0.03 1.29 (0.95–1.75) 0.11 

aOther covariates included in the multivariate Cox proportional hazard regression model were melanoma subtype (cutaneous vs. noncutaneous), previous treatment (yes vs. no), BRAF mutation status (BRAF V600 mutant versus wild-type), M stage (M1a, M1b, M1c, M1d, M0), LDH elevation at baseline (yes vs. no), different melanoma institute within the validation cohort (VUMC, MSKCC, MIA, DFCI).

GCC was given either intravenously or orally. Intravenous GCC was generally given at a higher dose (Supplementary Table S9) and was always tapered down to oral GCC. Further analyses comparing the survival between patients with intravenous GCC use versus without yielded negative results after the adjustment of GCC peak dose and other above-mentioned prognostic factors (Supplementary Table S10, online only).

Timing of GCC use and its correlation with post-irAE PFS/OS in patients who are irAE(+)

Our previous data showed that major antitumor response was established within 6 months after anti–PD-1 monotherapy initiation (12), we thus hypothesized the timing of high-dose GCC use in patients who were irAE(+) may affect subsequent outcomes. Noticeably, there was a high degree of variability in terms of onset time of high-dose GCC–associated irAEs (Table 2). Thus, to directly assess the influence of GCC on survival outcomes and to eliminate the lead-in time bias of irAE occurrence, we used the endpoints of post-irAE PFS/OS and two preset landmarks (namely week 8 and 26).

Early onset (both within 8 and 26 weeks after anti–PD-1 monotherapy initiation) irAEs that led to the use of high-dose GCC were associated with poorer post-irAE PFS in the MGH cohort (with statistical significance for 8-week landmark analysis and marginal significance for 26-week) and this was validated in the combined cohort (Table 4). The median post-irAE PFS was 8 weeks (95% CI, 6 to not reached) versus 90 weeks (95% CI, 67 to not reached) in the MGH cohort (HR, 5.37; 95% CI, 2.10–13.70; P < 0.001, multivariate analysis), and 38 weeks (95% CI, 25–85) versus 114 weeks (95% CI, 87–182) in the combined validation cohort (HR, 1.69; 95% CI, 1.04–2.76; P = 0.04, multivariate analysis), for patients with and without irAEs leading to high-dose GCC use within 8 weeks after anti–PD-1 initiation, respectively. In the 26-week landmark analysis, corresponding median post-irAE PFS was 77 weeks (95% CI, 26 to not reached) versus not reached (95% CI, 90 to not reached) in the MGH cohort (HR, 2.49; 95% CI, 0.87–7.12; P = 0.09, multivariate analysis), and 67 weeks (95% CI, 38 to not reached) versus 326 weeks (95% CI, 173 to not reached) in the combined validation cohort (HR, 1.93; 95% CI, 1.19–3.13; P = 0.008, multivariate analysis), respectively.

Table 4.

Early high-dose GCC–associated irAEs and their correlations with post-irAE PFS and post-irAE OS.

MGH cohortValidation cohort
LandmarkSurvival typeAnalysis typeHR (95% CI)PHR (95% CI)P
8-week 
   (n = 83; positive, n = 11) (n = 404; positive, n = 26) 
 Post-irAE PFS Univariate 6.28 (2.67–14.75) <0.001 1.78 (1.11–2.85) 0.02 
  Multivariate 5.37 (2.10–13.70) <0.001 1.69 (1.04–2.76) 0.04 
   (n = 88; positive, n = 12) (n = 418; positive, n = 29) 
 Post-irAE OS Univariate 5.46 (2.45–12.16) <0.001 1.91 (1.14–3.21) 0.01 
  Multivariate 5.95 (2.20–16.09) <0.001 1.97 (1.15–3.39) 0.01 
26-week 
   (n = 52; positive, n = 12) (n = 321; positive, n = 37) 
 Post-irAE PFS Univariate 2.26 (0.90–5.68) 0.08 1.79 (1.13–2.82) 0.01 
  Multivariate 2.49 (0.87–7.12) 0.09 1.93 (1.19–3.13) 0.008 
   (n = 84; positive, n = 25) (n = 397; positive, n = 48) 
 Post-irAE OS Univariate 2.15 (0.99–4.65) 0.05 1.17 (0.73–1.88) 0.51 
  Multivariate 2.25 (0.96–5.31) 0.06 1.18 (0.71–1.98) 0.52 
MGH cohortValidation cohort
LandmarkSurvival typeAnalysis typeHR (95% CI)PHR (95% CI)P
8-week 
   (n = 83; positive, n = 11) (n = 404; positive, n = 26) 
 Post-irAE PFS Univariate 6.28 (2.67–14.75) <0.001 1.78 (1.11–2.85) 0.02 
  Multivariate 5.37 (2.10–13.70) <0.001 1.69 (1.04–2.76) 0.04 
   (n = 88; positive, n = 12) (n = 418; positive, n = 29) 
 Post-irAE OS Univariate 5.46 (2.45–12.16) <0.001 1.91 (1.14–3.21) 0.01 
  Multivariate 5.95 (2.20–16.09) <0.001 1.97 (1.15–3.39) 0.01 
26-week 
   (n = 52; positive, n = 12) (n = 321; positive, n = 37) 
 Post-irAE PFS Univariate 2.26 (0.90–5.68) 0.08 1.79 (1.13–2.82) 0.01 
  Multivariate 2.49 (0.87–7.12) 0.09 1.93 (1.19–3.13) 0.008 
   (n = 84; positive, n = 25) (n = 397; positive, n = 48) 
 Post-irAE OS Univariate 2.15 (0.99–4.65) 0.05 1.17 (0.73–1.88) 0.51 
  Multivariate 2.25 (0.96–5.31) 0.06 1.18 (0.71–1.98) 0.52 

Note: Other covariates included in the multivariate Cox proportional hazard regression model were melanoma subtype (cutaneous vs. noncutaneous), previous treatment (yes vs. no), BRAF mutation status (V600 mutant versus wild-type), M stage (M1a, M1b, M1c, M1d, M0), LDH elevation at baseline (yes vs. no), and melanoma institute within the validation cohort (VUMC, MSKCC, MIA, DFCI).

We further tested the correlation between irAEs that led to the use of high-dose GCC and post-irAE OS. Notably, in the 8-week landmark analysis, irAEs that led to high-dose GCC were associated with poorer post-irAE OS in both cohorts. For patients with and without high-dose GCC–associated irAEs within 8 weeks after anti–PD-1 monotherapy initiation, the median post-irAE OS was 48 weeks (95% CI, 39 to not reached) versus not reached (95% CI, 189 to not reached) in the MGH cohort (HR, 5.95; 95% CI, 2.20–16.09; P < 0.001, multivariate analysis). In the combined validation cohort, it was 126 weeks (95% CI, 70 to not reached) versus 289 weeks (95% CI, 240 to not reached; HR, 1.97; 95% CI, 1.15–3.39; P = 0.01, multivariate analysis; Fig. 2, Table 4; Supplementary Fig. S1, online only). In the 26-week landmark analysis, marginal significant negative correlation between high-dose GCC–associated irAEs and post-irAE OS was only observed in the MGH exploratory cohort but not in the combined validation cohort (Table 4, Supplementary Fig. S1, online only).

Figure 1.

Patient populations included in this retrospective study. Overview of IrAEs and correlation with survival. *Selected irAEs are defined as the initial irAE, first GCC-associated irAE, and first grade 3/4 irAE per individual patient. #High-dose GCC exposure was defined as peak dose ≥60-mg prednisone equivalent once a day. For 8/26-week landmark analysis, high-GCC exposure–associated irAEs occurring within 8/26 weeks after anti–PD-1 monotherapy initiation, respectively, were treated as positive; otherwise, those occurring after this time frame, or irAEs that did not lead to high-dose GCC use, negative.

Figure 1.

Patient populations included in this retrospective study. Overview of IrAEs and correlation with survival. *Selected irAEs are defined as the initial irAE, first GCC-associated irAE, and first grade 3/4 irAE per individual patient. #High-dose GCC exposure was defined as peak dose ≥60-mg prednisone equivalent once a day. For 8/26-week landmark analysis, high-GCC exposure–associated irAEs occurring within 8/26 weeks after anti–PD-1 monotherapy initiation, respectively, were treated as positive; otherwise, those occurring after this time frame, or irAEs that did not lead to high-dose GCC use, negative.

Close modal
Figure 2.

Week-8 landmark analysis for high GCC exposure–associated irAEs and its correlation with post-irAE PFS and post-irAE OS. [Positive if the patient was with high-dose GCC–associated irAEs (defined as those led to the use of GCC with peak dose ≥60-mg prednisone equivalent once a day) occurring within 8 weeks after anti–PD-1 monotherapy initiation; otherwise, negative.] A, Post-irAE PFS in the MGH exploration cohort. B, Post-irAE PFS in the combined validation cohort. C, Post-irAE OS in the MGH exploration cohort. D, Post-irAE OS in the combined validation cohort. In the MGH cohort, median post-irAE PFS was 8 weeks (95% CI, 6 to not reached) and 90 weeks (95% CI, 67 to not reached) for patients with and without high-dose GCC-associated irAEs within 8 weeks after anti–PD-1 monotherapy initiation, respectively. In the combined validation cohort, median post-irAE PFS was 38 weeks (95% CI, 25–85) and 114 weeks (95% CI, 87–182) for patients with and without high-dose GCC–associated irAEs within 8 weeks after anti–PD-1 monotherapy initiation, respectively. In the MGH cohort, median post-irAE OS was 48 weeks (95% CI, 39 to not reached) and not reached (95% CI, 189 to not reached) for patients with and without high-dose GCC–associated irAEs within 8 weeks after anti–PD-1 monotherapy initiation, respectively. In the combined validation cohort, median post-irAE OS was 126 weeks (95% CI, 70 to not reached) and 289 weeks (95% CI, 240 to not reached) for patients with and without high-dose GCC–associated irAEs within 8 weeks after anti–PD-1 monotherapy initiation, respectively.

Figure 2.

Week-8 landmark analysis for high GCC exposure–associated irAEs and its correlation with post-irAE PFS and post-irAE OS. [Positive if the patient was with high-dose GCC–associated irAEs (defined as those led to the use of GCC with peak dose ≥60-mg prednisone equivalent once a day) occurring within 8 weeks after anti–PD-1 monotherapy initiation; otherwise, negative.] A, Post-irAE PFS in the MGH exploration cohort. B, Post-irAE PFS in the combined validation cohort. C, Post-irAE OS in the MGH exploration cohort. D, Post-irAE OS in the combined validation cohort. In the MGH cohort, median post-irAE PFS was 8 weeks (95% CI, 6 to not reached) and 90 weeks (95% CI, 67 to not reached) for patients with and without high-dose GCC-associated irAEs within 8 weeks after anti–PD-1 monotherapy initiation, respectively. In the combined validation cohort, median post-irAE PFS was 38 weeks (95% CI, 25–85) and 114 weeks (95% CI, 87–182) for patients with and without high-dose GCC–associated irAEs within 8 weeks after anti–PD-1 monotherapy initiation, respectively. In the MGH cohort, median post-irAE OS was 48 weeks (95% CI, 39 to not reached) and not reached (95% CI, 189 to not reached) for patients with and without high-dose GCC–associated irAEs within 8 weeks after anti–PD-1 monotherapy initiation, respectively. In the combined validation cohort, median post-irAE OS was 126 weeks (95% CI, 70 to not reached) and 289 weeks (95% CI, 240 to not reached) for patients with and without high-dose GCC–associated irAEs within 8 weeks after anti–PD-1 monotherapy initiation, respectively.

Close modal

Sensitivity tests using accumulated GCC exposure as the measurement yielded similar results at both 8- and 26-week landmarks (Supplementary Table S11).

PD-1 blockade has improved survival outcomes for patients with advanced melanoma at the expense of irAEs. Early data (6, 7) suggest improved outcomes (ORR and RFS in the palliative and adjuvant settings, respectively) in patients with irAEs versus those who do not experience them, but there are no existing datasets that explore the impact of irAE management on long-term outcomes. This report presents the first large, multinational group of patients with both mature follow-up and detailed information on dose and duration of steroids. Importantly, we find that early onset irAEs (within 8 weeks) leading to the use of high-dose GCC (≥60-mg prednisone equivalent once a day) is correlated with poorer post-irAE PFS and post-irAE OS compared with later-onset irAEs with high-dose GCC use, or low-dose/no GCC use.

Although exploratory and observational, the demonstration of early high-dose GCC–associated attenuation of anti–PD-1 effectiveness may factor into the decision-making of treating clinicians when considering the risk/benefit of immunosuppressive treatments of irAEs, particularly in less severe and non–life-threatening toxicities during the early anti–PD-1 treatment window. Furthermore, this information should strengthen efforts to identify toxicity/mechanism-specific approaches to dampen auto-immunity. It also warrants further prospective randomized control trials (RCTs) testing alternative modalities of irAE management.

The major limitation of this study is that it is a retrospective analysis, making it susceptible to potential selection, measurement, and reporting biases. Although we used objective measurements for most cases, those biases cannot be entirely excluded. Over half of the irAEs that led to early use of high-dose GCC also led to the early discontinuation of anti–PD-1 monotherapy, which may contribute to the poorer survival. Given the contribution of data from multiple high-volume centers, the spectrum of irAEs differed among the five cohorts. Of note, we initially considered Peking University Cancer Hospital from China as well in the validation cohort, but elected to exclude due to conspicuous difference in terms of irAE occurrence (lower incidence of grade 3/4 irAEs), significantly less frequent use of GCC, and higher proportion of acral and mucosal melanomas, which are associated with poorer anti–PD-1 efficacy (13).

We report that the occurrence of irAE regardless of subtype or severity was not correlated with PFS but longer OS (in the large combined validation cohort) after adjustment for immortal time bias, consistent with two previous reports (6, 14). Notably, if we adopted naïve COX proportional hazard regression model without immortal time bias correction, we reached the false conclusion that irAE occurrence was positively correlated with both PFS and OS, a finding consistent with a previous analysis using the same model (15). This discrepancy originated from the fact that the naïve COX model does not take into consideration the intrinsic correlation between higher irAE incidence and longer anti–PD-1 monotherapy exposure, and thus is not appropriate to use in this setting. We did not observe any correlation between different types of irAEs and survival in this study, not even in skin irAEs. Of note, we did not differentiate vitiligo, which is known for its correlation with better survival in melanoma (16), from other skin irAEs, implicating that different types of irAEs, even when affecting the same system, may have distinct correlation patterns with survival. Further studies with higher granularity are warranted.

It has been reported that the use of high-dose GCCs was associated with reduced survival in a subset of ipilimumab [anti-cytotoxic T-lymphocyte antigen-4 (CTLA-4) antibody]-treated melanoma and anti–PD-1 monotherapy treated patients with non–small cell lung cancer (NSCLC; used before/at anti–PD-1 initiation, significant for palliative indications; refs. 8, 9, 17). Also reported is a negative impact of GCC use over 30 days on RFS in patients with stage III melanoma who received adjuvant pembrolizumab after surgery (7). In concert with these previous reports, we observe that high-dose GCC is associated with poorer PFS in both the exploratory and validation cohorts with statistical or marginal significance (Table 3), and this observation is confirmed by the sensitivity analysis using the accumulated GCC exposure as the measurement.

We also note that the use of GCC differed greatly between different individual patients, in terms of starting time, peak dose, tapering schedule, and accumulated exposure (Tables 1 and 2). Of note, compared with the combined large validation cohort, MGH demonstrated high early GCC use, higher peak doses, and higher total exposure, whereas the likelihood of early termination of anti–PD-1 were well balanced between MGH and the combined validation cohort.

We hypothesized that early high-dose GCC exposure, before the full establishment of antimelanoma immune responses triggered by anti–PD-1 antibody, together with early termination of anti–PD-1, may be more detrimental than if this occurred later. Therefore, we carried out two pre-set landmark analyses at weeks 8 and 26. Our data demonstrated that high-dose GCC–associated irAEs that occurred within 8 weeks after anti–PD-1 initiation were associated with both poorer post-irAE PFS and post-irAE OS, which was confirmed by multivariate analysis and validated by a large, multinational validation cohort, and was further confirmed by a sensitivity analysis using the accumulated GCC exposure as the measurement. This finding is supported by an earlier observation of the deleterious effect of baseline GCC (in this case >10 mg of prednisone or its equivalent) use in patients with NSCLC (9). We acknowledge that the number of patients with early onset high-dose GCC–associated irAEs is small, but considering the sample size of the reference group, we believe that our observation is not artifactual. Of note, compared with week 8, the hazard ratio of week 26 landmark analysis dropped but was still marginally or statistically significantly associated with poorer post-irAE PFS in uni- and multivariate analyses in the exploratory and validation cohorts, respectively; but not post-irAE OS. This suggests that the timing of irAE onset, subsequent GCC use, and anti–PD-1 termination matters. We notice that the magnitude of correlation varied across different cohorts and acknowledge that there might be patient selection and/or other factors contributing to this phenomenon aside from the fact that MGH is the center where GCC use was heaviest. Of note, this negative correlation between early high-dose GCC and post-irAE PFS/OS might at least partially be attributed to associated early termination of anti–PD-1 monotherapy observed in more than half of these patients. Because of the observational and exploratory nature of this study, we acknowledge that it is challenging to deconvolve whether high-dose GCC exposure, early anti–PD-1 discontinuation, or some unknown factors may contribute to our observations. To answer this question, future prospective RCTs with large sample size focusing on irAE management should be implemented to assess this question in a more controlled fashion. Because of the lack of other superior therapeutic modalities for the management of irAEs, currently GCC remains the important option for these patients. Experimental arms of the future RCTs should consider, including irAE management recommendations, including early intervention for mild irAE by delay of anti–PD-1 antibody with or without the use of lower dose GCC exposure. It may also be relevant to consider quicker tapering strategies to limit the total GCC exposure, and possible delayed use of GCC. Given that a subset of irAEs occur abruptly with high severity and may be life-threatening, high-dose GCC therapy continues to be necessary in select patients, which adds another layer of complexity on optimal trial design. Of greater relevance to the future of the field of immunotherapy, this work highlights the need to develop better biomarkers to diagnose emerging toxicities as well as need for RCTs testing alternative management options (both in terms of alternative GCC dosing schedules and also alternative drugs) for irAEs.

X. Bai reports other support from Bristol Myers Squibb outside the submitted work. A.B. Warner reports personal fees from LG Chem Life Sciences, Nanobiotix, Iovance Biotherapeutics, Novartis, Shanghai Jo'Ann Medical Technology, and BluePath Solutions outside the submitted work. L. Pallan reports personal fees from Bristol Myers Squibb outside the submitted work. J.V. Cohen reports personal fees from Bristol Myers Squibb and Sanofi-Genzyme outside the submitted work. R. Fadden reports other support from Apricity Health outside the submitted work. K. Rubin reports personal fees from Bristol Myers Squibb and Merck Sharpe & Dohme outside the submitted work. K.T. Flaherty reports personal fees from Loxo Oncology, Clovis Oncology, Strata Oncology, Vivid Biosciences, Checkmate Pharmaceuticals, Kinnate Biopharma, Scorpion Therapeutics, X4 Pharmaceuticals, PIC Therapeutics, Apricity, Oncoceutics, Fog Pharma, Tvardi, xCures, Monopteros, Vibliome, Sanofi, Amgen, Asana Biosciences, Adaptimmune, Aeglea Biosciences, Shattuck Labs, Tolero Pharmaceuticals, Neon Therapeutics, Eli Lilly, Novartis, Genentech, Bristol Myers Squibb, Merck Sharpe & Dohme, Takeda, Verastem, Boston Biomedical, Pierre-Fabre, and Debiopharm during the conduct of the study. O.E. Rahma reports personal fees from Imvax, Bayer, Gennentech, Sobi, GlaxoSmithKline, Boehringer Ingelheim, Puretech, and Maverick Therapeutics outside the submitted work and a patent for Methods of using pembrolizumab and trebananib pending. G.V. Long reports personal fees from Aduro Biotech Inc., Amgen Inc., Array Biopharma Inc., Boehringer Ingelheim International GmbH, Bristol Myers Squibb, Evaxion Biotech A/S, Hexell AG, Highlight Therapeutics S.L., Merck Sharpe & Dohme, Novartis Pharma AG, OncoSec, Pierre-Fabre, QBiotics Group Limited, Regeneron Pharmaceuticals Inc., SkylineDX B.V., and Specialized Therapeutics Australia Pty Limited outside the submitted work. A.M. Menzies reports personal fees from Bristol Myers Squibb, Merck Sharpe & Dohme, Novartis, Roche, Pierre-Fabre, and QBiotics outside the submitted work. J. Guo reports other support from Merck Sharpe & Dohme, other support from Roche, Bayer, Novartis, Simcere Pharmaceutical Group, Shanghai Junshi Biosciences, and Oriengene outside the submitted work. A.N. Shoushtari reports grants and personal fees from Bristol Myers Squibb and Immunocore and grants from Xcovery, Polaris, Novartis, Pfizer, and Checkmate Pharmaceuticals outside the submitted work. D.B. Johnson reports other support from Array Biopharma, Bristol Myers Squibb, Catalyst, Iovance, Jansen, grants from Incyte, and other support from Merck Sharpe & Dohme, Novartis, and Oncosec outside the submitted work. R.J. Sullivan reports grants and personal fees from Merck Sharpe & Dohme, grants from Amgen, personal fees from Asana Biosciences, Array Biopharma, Bristol Myers Squibb, Eisai, Iovance, AstraZeneca, Novartis, OncoSec, Pfizer, and Replimune outside the submitted work. G.M. Boland reports grants from Takeda Oncology, personal fees from Nektar Therapeutics, grants from Palleon Pharmaceuticals, Olink Proteomics, InterVenn Biosciences, and personal fees from Merck Sharpe & Dohme, Novartis, and NW Biotherapeutics outside the submitted work. No disclosures were reported by the other authors.

X. Bai: Conceptualization, resources, data curation, software, formal analysis, validation, investigation, visualization, methodology, writing–original draft, writing–review and editing. J. Hu: Software, formal analysis, validation, investigation, visualization, methodology, writing–original draft, writing–review and editing. A.B. Warner: Data curation, writing–review and editing. H.T. Quach: Data curation, writing–review and editing. C.G. Cann: Data curation, writing–review and editing. M.Z. Zhang: Data curation, writing–review and editing. L. Si: Data curation, writing–review and editing. B. Tang: Data curation, writing–review and editing. C. Cui: Data curation, writing–review and editing. X. Yang: Data curation, writing–review and editing. X. Wei: Data curation, writing–review and editing. L. Pallan: Data curation, writing–review and editing. C. Harvey: Data curation, writing–review and editing. M.P. Manos: Data curation, writing–review and editing. O. Ouyang: Data curation, writing–review and editing. M.S. Kim: Data curation, writing–review and editing. G. Kasumova: Data curation, writing–review and editing. J.V. Cohen: Data curation, writing–review and editing. D.P. Lawrence: Data curation, writing–review and editing. C. Freedman: Data curation, writing–review and editing. R.M. Fadden: Data curation, writing–review and editing. K.M. Rubin: Data curation, writing–review and editing. T. Sharova: Data curation, writing–review and editing. D.T. Frederick: Data curation, writing–review and editing. K.T. Flaherty: Conceptualization, resources, data curation, formal analysis, supervision, validation, investigation, writing–original draft, project administration, writing–review and editing. O.E. Rahma: Data curation, writing–review and editing. G.V. Long: Data curation, writing–review and editing. A.M. Menzies: Data curation, writing–review and editing. J. Guo: Data curation, writing–review and editing. A.N. Shoushtari: Data curation, writing–review and editing. D.B. Johnson: Data curation, writing–review and editing. R.J. Sullivan: Conceptualization, resources, data curation, formal analysis, supervision, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing. G.M. Boland: Conceptualization, resources, data curation, formal analysis, supervision, validation, investigation, visualization, methodology, writing–original draft, project administration, writing–review and editing.

The Editor-in-Chief of Clinical Cancer Research is an author on this article. In keeping with AACR editorial policy, a senior member of the Clinical Cancer Research editorial team managed the consideration process for this submission and independently rendered the final decision concerning acceptability.

The authors thank F. Steven Hodi for access to his IRB-approved database for this project.

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.

1.
Robert
C
,
Schachter
J
,
Long
GV
,
Arance
A
,
Grob
JJ
,
Mortier
L
, et al
Pembrolizumab versus Ipilimumab in advanced melanoma
.
N Engl J Med
2015
;
372
:
2521
32
.
2.
Robert
C
,
Long
GV
,
Brady
B
,
Dutriaux
C
,
Maio
M
,
Mortier
L
, et al
Nivolumab in previously untreated melanoma without BRAF mutation
.
N Engl J Med
2015
;
372
:
320
30
.
3.
Larkin
J
,
Chiarion-Sileni
V
,
Gonzalez
R
,
Grob
JJ
,
Cowey
CL
,
Lao
CD
, et al
Combined Nivolumab and Ipilimumab or monotherapy in untreated melanoma
.
N Engl J Med
2015
;
373
:
23
34
.
4.
Boutros
C
,
Tarhini
A
,
Routier
E
,
Lambotte
O
,
Ladurie
FL
,
Carbonnel
F
, et al
Safety profiles of anti–CTLA-4 and anti–PD-1 antibodies alone and in combination
.
Nat Rev Clin Oncol
2016
;
13
:
473
86
.
5.
Haanen
J
,
Carbonnel
F
,
Robert
C
,
Kerr
KM
,
Peters
S
,
Larkin
J
, et al
Management of toxicities from immunotherapy: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up
.
Ann Oncol
2017
;
28
:
iv119–iv42
.
6.
Weber
JS
,
Hodi
FS
,
Wolchok
JD
,
Topalian
SL
,
Schadendorf
D
,
Larkin
J
, et al
Safety profile of Nivolumab monotherapy: a pooled analysis of patients with advanced melanoma
.
J Clin Oncol
2017
;
35
:
785
92
.
7.
Eggermont
AMM
,
Kicinski
M
,
Blank
CU
,
Mandala
M
,
Long
GV
,
Atkinson
V
, et al
Association between immune-related adverse events and recurrence-free survival among patients with stage III melanoma randomized to receive pembrolizumab or placebo: a secondary analysis of a randomized clinical trial
.
JAMA Oncol
2020
;
6
:
519
27
.
8.
Faje
AT
,
Lawrence
D
,
Flaherty
K
,
Freedman
C
,
Fadden
R
,
Rubin
K
, et al
High-dose glucocorticoids for the treatment of ipilimumab-induced hypophysitis is associated with reduced survival in patients with melanoma
.
Cancer
2018
;
124
:
3706
14
.
9.
Arbour
KC
,
Mezquita
L
,
Long
N
,
Rizvi
H
,
Auclin
E
,
Ni
A
, et al
Impact of baseline steroids on efficacy of programmed cell death-1 and programmed death-ligand 1 blockade in patients with non–small cell lung cancer
.
J Clin Oncol
2018
;
36
:
2872
78
.
10.
Buttgereit
F
,
da Silva
JA
,
Boers
M
,
Burmester
GR
,
Cutolo
M
,
Jacobs
J
, et al
Standardised nomenclature for glucocorticoid dosages and glucocorticoid treatment regimens: current questions and tentative answers in rheumatology
.
Ann Rheum Dis
2002
;
61
:
718
22
.
11.
Hothorn
T
,
Lausen
B
. 
On the exact distribution of maximally selected rank statistics
.
Comput Stat Data Analysis
2003
;
43
:
17
.
12.
Bai
X
,
Kim
M
,
Kasumova
G
,
Si
L
,
Tang
B
,
Cui
C
, et al
Radiological dynamics and SITC-defined resistance types of advanced melanoma during anti–PD-1 monotherapy: an independent single-blind observational study on an international cohort
.
J Immunother Cancer
2021
;
9
:
e002092
.
13.
Shoushtari
AN
,
Munhoz
RR
,
Kuk
D
,
Ott
PA
,
Johnson
DB
,
Tsai
KK
, et al
The efficacy of anti–PD-1 agents in acral and mucosal melanoma
.
Cancer
2016
;
122
:
3354
62
.
14.
Robert
C
,
Hwu
W-J
,
Hamid
O
,
Ribas
A
,
Weber
JS
,
Daud
AI
, et al
Long-term safety of pembrolizumab monotherapy and relationship with clinical outcome: A landmark analysis in patients with advanced melanoma
.
Eur J Cancer
2021
;
144
:
182
91
.
15.
Rogado
J
,
Sánchez-Torres
JM
,
Romero-Laorden
N
,
Ballesteros
AI
,
Pacheco-Barcia
V
,
Ramos-Leví
A
, et al
Immune-related adverse events predict the therapeutic efficacy of anti–PD-1 antibodies in cancer patients
.
Eur J Cancer
2019
;
109
:
21
27
.
16.
Teulings
H-E
,
Limpens
J
,
Jansen
SN
,
Zwinderman
AH
,
Reitsma
JB
,
Spuls
PI
, et al
Vitiligo-like depigmentation in patients with stage III-IV melanoma receiving immunotherapy and its association with survival: a systematic review and meta-analysis
.
J Clin Oncol
2015
;
33
:
773
81
.
17.
Ricciuti
B
,
Dahlberg
SE
,
Adeni
A
,
Sholl
LM
,
Nishino
M
,
Awad
MM
. 
Immune checkpoint inhibitor outcomes for patients with non–small cell lung cancer receiving baseline corticosteroids for palliative versus nonpalliative indications
.
J Clin Oncol
2019
;
37
:
1927
34
.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs International 4.0 License.