Objective Responses in Patients with Malignant Melanoma or Renal Cell Cancer in Early Clinical Studies Do Not Predict Regulatory Approval

Contemporary drug development in oncology is extremely resource intensive. Only one in ten drugs that enter the clinic is eventually approved for use (1) and the dollar cost to bring a drug to market can approach U.S. $800 million (2). The human cost of late-phase development of inactive therapeutics is also considerable, and includes toxicity suffered by research subjects, unrealistic hope generated by seemingly promising new agents, and the wasted accrual of the limited pool of patients who are eligible for trials. Hundreds, or even thousands, of patients may be exposed to a new agent only to eventually determine that it does not provide clinical benefit, sometimes after completion of multiple phase III trials. Objective tumor response remains the most commonly used end point or surrogate of drug efficacy, and tumor response has been shown to correlate with survival (3–6). In lung cancer, for example, a response rate of 15% is usually considered sufficient to merit further investigation of an agent (7).

The decision whether to continue the development of a new therapeutic based on the results of early clinical trials (phase I or II) may be challenging, and is based on an evaluation of the therapeutic index. Whereas new therapeutics can be developed despite manageable toxicities, a critical consideration is the antitumor activity of the agent. Some clues can be gleaned from preclinical studies. Voskoglou-Nomikos et al. (8) have shown that preclinical data may predict for tumor responsiveness to cytotoxic agents in phase II studies. Measures of tumor control in in vitro human tumor cell line models, murine xenograft models, and murine tumor allograft models were compared with the phase II response rates in studies of non–small-cell lung cancer, colon, breast, and ovarian carcinoma. Comparisons were made between the preclinical control among both individual and overall grouped tumor types and the phase II responses among one of the four specific tumor types or among all tumor types combined. Several conclusions were reached: (a) murine allograft models were not predictive; (b) human xenograft models had good tumor-specific predictive value for non–small-cell lung cancer and ovarian cancer; and (c) in vitro cell line growth control was most predictive of both overall and disease-specific phase II responses, with the exception of colon cancer.

However, preclinical data largely guide the selection of drugs for clinical studies rather than later clinical development. Antitumor activity in phase I trials may help guide the selection of tumor types for phase II study. Estey et al. (9) found no difference between the median response rate of a drug in phase I trials and the later demonstration of phase II activity as defined by a minimum response rate. Sekine et al. (10) categorized response rates of various agents in phase I studies and correlated these response categories to the likelihood of a successful phase III trial. They found that each increment in response category had a small but significant increased odds ratio (OR = 1.14) of predicting for a successful phase III study. They concluded that overall phase I study response rates predicted for a successful phase III outcome, albeit marginally. An additional observation was that responses in renal cell carcinoma or melanoma were not predictive of successful phase III studies. Another study assessed anecdotally the relationship between responses seen in phase I studies and the eventual marketing of a drug, and concluded that drugs inactive in phase I trials did not achieve marketing approval (11).

One potential pitfall in basing drug development plans on objective responses is the propensity for some cancers to remit spontaneously. Whereas several adult solid cancers have been documented to occasionally regress in the absence of therapy, melanoma and renal cell carcinoma (renal cell carcinoma) compose the significant majority of such observations (12). Metastatic renal cell carcinoma may regress partially or completely in up to 6% to 7% of cases and durations may be prolonged (13, 14). Such responses to placebo have led to the suggestion that a minimum response rate of 18% be required of an agent in phase II studies before proceeding to phase III studies (14). Pathologic assessment of melanoma specimens suggests some degree of remission in 10% to 36% of cases (15).

Despite such spontaneous remissions, melanoma and renal cell carcinoma are among the most treatment resistant of cancers (16, 17). As a result of the lack of effective standard agents, these patients are often early candidates for experimental protocols.

Given the number of patients with renal cell carcinoma and melanoma who participate in phase I studies, and the known incidence of spontaneous remissions in these diseases, it is not surprising to note that isolated responses in melanoma or renal cell carcinoma are frequently reported, and have led to the initiation of larger trials in melanoma or renal cell carcinoma, or the development of agents that otherwise seem inactive. We were interested in assessing whether responses seen in early clinical trials, especially in melanoma or renal cell carcinoma, were predictive of later efficacy and conducted a retrospective review of our own trials and the literature.

Trial data acquisition. Published phase I studies and abstracts of cytotoxic drugs were identified for the period 1985 to 1999 by searching Medline and the National Cancer Institute of Canada Clinical Trials Group databases and by examining the bibliographies of review articles and identified studies. The search terms used were antineoplastic drugs with the limit of Clinical Trials, Phase I. Only publications of cytotoxic compounds tested in adults were considered. Phase Ib trials, phase I trials of combinations of agents, trials of drugs abandoned during phase I study because of unmanageable toxicity, and trials using tumors markers to measure response were excluded. The interval of 1985 to 1999 was chosen because it provided an adequate span to incorporate a good number of studies. Further, the average period of clinical investigation takes 10 years (18), the period allowed for a reasonable period for most phase II and III studies to be completed to allow for potential drug approval. Cytotoxic drugs were chosen to the exclusion of immune therapies, hormonal manipulation, or newer directed molecules as more mature data are available and the mechanisms of action are more homogeneous. For each phase I study identified, the following information was extracted: the total number of subjects, the total number of responses, the number of subjects with melanoma and renal cell carcinoma, and the responses observed among subjects with melanoma and renal cell carcinoma.

Because of the heterogeneity of these trials, data were not collected with respect to drug dose or schedule, gender, age, or prior treatments of subjects. The total response rate for a given drug was determined by dividing the total number of responses in all phase I trials for that drug (regardless of tumor type) by a denominator of all treated subjects. In melanoma and renal cell carcinoma, the denominators were the total number of subjects reportedly treated with the specific tumor type treated with a given drug.

All published phase II trials corresponding to each drug studied in the phase I trials were identified for the period 1985 to 2002 using the search terms of the generic and trade names of the drug with the limit of Clinical Trials, Phase II. Trials using tumor markers to measure response were excluded. Studies were divided among disease types including melanoma, renal cell carcinoma, non–small-cell lung cancer, breast cancer, ovarian cancer, colorectal cancer, and other solid tumors. The total number of complete or partial responses in each tumor type for a given drug was recorded. The denominator for response rate in this case was the total number of patients enrolled in all the phase II trials for a given drug and tumor type. As per the phase I studies, data were not collected with respect to drug dose or schedule, gender, age, or prior treatments of subjects.

The U.S. Federal Drug Administration web site, the European Medicines Evaluation Agency web site, and the Canadian Compendium of Pharmaceuticals and Specialties were searched to assess whether each drug had been approved in any jurisdiction, and if so, for what indication. Whereas phase III data are typically used when regulatory agencies assess a given agent, more than one phase III study may be required and additional toxicity data from earlier phase studies may be incorporated into an approval decision. It was thus felt that the end point of regulatory approval was best used to assess drug activity rather than a successful phase III trial. As it happened, all drugs approved in our database were approved in the United States, whereas approval in the other two jurisdictions was variable.

Statistical analysis. For each drug, phase I response rates among all subjects, subjects with renal cell carcinoma, and subjects with melanoma were classified into four categories according to the method of Sekine et al. (10): (a) response rate = 0%, (b) response rate >0% and ≤5%, (c) response rate >5% and ≤10%, and (d) response rate >10%. These rates are reflective of patients typically enrolled in the phase I setting (i.e., patients who have failed all other treatment options). Phase II response rates are typically higher. To usefully differentiate response rates in the phase II group, rates among all subjects and among each of the specified disease types were categorized as follows: (a) response rate = 0%, (b) response rate >0% and ≤10%, (c) response rate >10% and ≤20%, and (d) response rate >20%.

The relationship between the above response categories and the probability of the regulatory approval was assessed by the exact Cochran-Armitage linear trend test. An exact logistic regression model was used to obtain the OR of a drug being approved with each increase in response category and the associated 95% confidence interval. Comparisons were made between the overall response rate and regulatory approval for any indication, the overall response rate in the specific disease types noted and approval for an indication in that disease, and the overall response rate in specific disease types and approval for an indication in any disease. Correction of the α error was not made for multiple comparisons as tests were exploratory. All analyses were done using SAS software.

Predictive value of phase I studies. A total of 58 drugs (Table 1) were assessed in 100 phase I studies and 3,608 patients. Responses were seen in 161 subjects (Table 2). The median overall response rate among all drugs was 3.0% (range 0-18.8%), with 20 drugs demonstrating no objective response. Overall response rates among melanoma and renal cell carcinoma were both 6.3%. Eleven of the 58 agents were approved for at least one indication in at least one jurisdiction.

Response categories were assessed for correlation with marketing approvals for melanoma, renal cell carcinoma, and all tumor types combined (Table 3). Drugs demonstrating responses in melanoma or renal cell carcinoma were not more likely to be granted regulatory approval than drugs without responses. Among 40 agents studied in phase I trials treating subjects with melanoma, no drug was approved for melanoma and all seven drugs that were approved for another indication evoked no response in melanoma. Similarly, among 34 drugs tested on subjects with renal cell carcinoma, five of the six drugs that were approved evoked no response in renal cell carcinoma, and no drug was approved for use in renal cell carcinoma.

There was a significant association between higher overall response rate in any tumor type and the likelihood of a drug being approved in any indication (P = 0.03). The OR of a drug being approved with each increase in response category was 2.0 (95% confidence interval, 1.0-4.2). The association was still significant after removing patients with melanoma and renal cell carcinoma from the calculation of overall response rate (P = 0.02); the OR was 2.2 (95% confidence interval, 1.1-5.1).

Predictive value of phase II studies. Out of the 58 drugs tested in phase I studies, 46 drugs had reported results from at least one of the 499 phase II studies that met the inclusion criteria (Table 1). Among these 46 drugs, a total of 16,093 subjects were treated with a median overall response rate of 6.0% (range 0-28%), with eight drugs having a 0% response rate (Table 4). Nineteen drugs were studied in melanoma and 15 were studied in renal cell carcinoma, with 13 drugs being studied in both sites. In one instance for each of melanoma and renal cell carcinoma, a drug was studied in this tumor type but in no other disease category. In nine instances, a drug was studied in all disease categories.

Overall, there was a highly significant association between higher overall response rate category and the likelihood of a drug being approved in any indication (P < 0.0001; Table 5). The OR of a drug being approved with each increase in response category was 4.7 (95% confidence interval, 1.7-17.9).

When assessed by tumor type, higher overall response rate categories among breast, non–small-cell lung cancer, colorectal, and other solid cancers were associated with an increased probability of drug approval for an indication in that cancer type as well as for any indication. However, higher overall response rate categories in melanoma and renal cell carcinoma failed to predict for approval for an indication in those diseases or for any indication. Higher response rates for ovarian cancer were marginally associated with the higher probability of drug approval for that indication but significantly predicted for any approval (Table 5). Restricting the analysis to the subset of drugs assessed in phase II trials in melanoma (n = 19) and renal cell carcinoma (n = 15), a statistically significant association between higher response rate and approval was seen when separately analyzing breast (P = 0.02), ovarian (borderline, P = 0.06), and other (P = 0.009) carcinomas, and a positive trend was seen for lung (P = 0.13) and colorectal carcinomas (P = 0.11).

Given the frequent participation of patients with melanoma and renal cell carcinoma in phase I as well as phase II trials, and the well-characterized spontaneous remission rate in these tumors, we were interested in confirming the value of objective response in these tumor types in predicting subsequent regulatory drug approval.

As expected, the overall response rate in phase I trials (4.5%) was lower than that seen in phase II trials (13.5%). Interestingly, overall response rates were predictive of later marketing approval in any indication (OR = 2.0 for each increase in response category). A more pronounced effect was seen among phase II studies (OR = 4.7 for each increase in response category). For phase II trials, higher response rate categories in colorectal, non–small-cell lung cancer, breast, and other solid cancers also predicted for drug approval for an indication in that disease and for approval in any indication, confirming previous observations.

In contrast, higher response rate categories in melanoma or renal cell carcinoma, in either phase I or phase II studies, failed to predict for subsequent marketing approval for an indication in those diseases or for approval in any indication. We postulate that the responses reported were at least in part due to spontaneous remissions, limiting the predictive value of such responses for future success. It should be noted that the relatively small number of drugs studied in these two diseases could limit our ability to detect a positive relationship between response and approval. However, using that same drug subset of phase II studies, an association was still seen between response and approval for other tumor types.

One confounding factor in studies such as this is the effect of negative bias on the conclusions; drugs with no activity in the phase I setting may be abandoned early or may undergo very limited phase II testing. Such drugs would, of course, never be approved. In this case, our study might be revealing tendencies toward early termination of development. Our data suggest this is not the case; there was no difference in phase I median response rates for drugs which underwent phase II development compared with those drugs that did not (median 2.6% versus 4.6%, P = 0.2).

Our results confirm the preliminary findings of other investigators, but provide more recent and complete information. As compared with the work of Sekine et al. (10), our phase I trials were extracted from a later period and reporting is likely to be more detailed (1985-1999 versus 1976-1993). Second, by primarily examining phase II studies for tumor type specific predictive power, more accurate tumor type reporting and homogeneous populations could be obtained. In our study, the phase I and II data were consistent with respect to the lack of predictiveness of renal cell carcinoma and melanoma response rates and with respect to the general ability of phase I and II response rates to predict drug approval. Our study thus permits a broader comment on the predictive utility of response rates in early-phase studies.

In contrast to drug success as measured by a positive phase III study, we used the end point of drug approval as the measure of success. Both end points suffer from elements of subjectivity. In the study of Sekine et al. (10), for example, either response rate, progression-free survival, overall survival, or quality of life needed to be superior to define a positive study. Further, only one positive study needed to be found to categorize a drug as successful. By contrast, a regulatory agency may not be swayed by response rate alone in a setting of metastatic disease and may take into account conflicting study results in assessing effectiveness and approval.

Our study has limitations. It is conceivable that not all of the agents that we examined have had sufficient time to complete the approval process. To overcome this in part, our study sought to capture the earliest drug approval by assessing three Western jurisdictions: the United States, Europe, and Canada. All drugs approved in our database were approved in the United States, whereas approval in the other two jurisdictions was variable. Whereas we sought to use a broad period to incorporate mature data, developmental processes may have changed over time. Many agents in this study were relatively inactive and the predictive power of response rates may not carry over into cohorts of more response-inducing agents. Furthermore, studies of some less active agents may not have been published. The predictive ability of some tumor types may have been understated by the limited number of studies available, although significant results were found in several tumor types among phase II trials. For reasons of heterogeneity among trials, we did not record variables such as drug dose, schedule, or response definitions. However, we selected all published data and one would assume that reasonable methodology was used in determining dose and schedule. In addition, we limited our analysis to studies of solid tumors incorporating tumor measurements, as response definitions are generally consistent within this group.

Another potential problem is that the relationship between response rate and survival is unclear in melanoma and renal cell carcinoma. Whereas higher response rates in melanoma have been seen with combination chemotherapy and some immune therapies, these have not translated into improved survival over dacarbazine alone (19). Responses rates are variable in renal cell carcinoma and similarly do not obviously correlate with survival (20). In fact, approval for interleukin-2 in melanoma and renal cell carcinoma was based on prolonged duration remissions in very small numbers of patients with complete responses (21). Presently, it is unclear whether a relationship between response and survival will be seen with newer agents, such as those inhibiting angiogenesis.

In conclusion, we have confirmed the general predictive value of objective responses in early clinical trials for later marketing approval. Responses in melanoma and renal cell carcinoma were not predictive for regulatory approval and should be used cautiously as a guide for the future development of a drug. Phase II studies in melanoma and renal cell carcinoma might best be undertaken using randomized trials; although more resource intensive and having limited power, such trials may prevent larger unwarranted phase III trials. Given the limited efficacy of available approved drugs, such phase II trials could reasonably be conducted in the first line setting against a comparator of chosen standard treatment or in the second line setting against a placebo. With the lack of data to support an association between tumor response and survival, investigators should consider alternative end points such as time to disease progression. The applicability of these observations to newer targeted agents will require further elucidation.