The Inconvenience of Convenience Cohorts: Rhabdomyosarcoma and the PAX-FOXO1 Biomarker

Convenience cohorts represent a type of nonprobability sampling where study participants thought to represent the general population are selected on the basis of their availability, rather than at random (1). Because an unknown portion of the population is excluded and the degree of true population representation in convenience cohorts is not known, these methods are subject to bias and may be misleading (2). While they are essential to generate hypotheses, convenience cohorts are limited in their ability to definitively confirm the role of potential clinical biomarkers.

Rhabdomyosarcoma is the most common soft tissue sarcoma in children and adolescents (3). Rhabdomyosarcoma is usually divided into 2 broad histologic groups: embryonal rhabdomyosarcoma (ERMS), representing approximately 70% of cases and associated with a more favorable prognosis, and alveolar rhabdomyosarcoma (ARMS), representing 30% of cases, and associated with poorer prognosis (4). Clinical factors, including the primary site, completeness of resection before chemotherapy, tumor size, regional nodal involvement, and the presence of distant metastases, are also used to define risk groups, with the goal of better stratifying treatment regimens to promote optimal survival with minimal toxicity (5). “Low-risk” patients, with localized ERMS, have approximately 90% long-term failure-free survival (FFS), whereas “high-risk” patients, with metastatic rhabdomyosarcoma, have an expected FFS of less than 20% (6). “Intermediate-risk” patients represent a heterogeneous group of both patients with ERMS and ARMS with FFS ranging between 50% and 80% (4, 5, 7–9). To optimize the allocation of therapy intensity based upon the risk of recurrence, further improvement to the stratification of patients is necessary, particularly for those in the heterogeneous intermediate-risk group.

Among patients with ARMS, a translocation between the PAX3 or PAX7 gene and the FOXO1 gene is present in approximately 80% of cases (10–14). These chromosomal translocations generate novel proteins in which the DNA-binding portions of PAX are fused to the carboxyl terminus of FOXO1; the PAX-FOXO1 fusion protein acts as a potent transcriptional activator that influences the expression of genes ultimately controlling cell proliferation, apoptosis, differentiation, and motility. This “fusion-positive” status leads to expression of a potent transcriptional activator, which effects growth, apoptosis, differentiation, and motility (3). Several studies have suggested that fusion status is associated with outcome and should, therefore, be incorporated into preliminary risk stratification schemata (7–9, 15–19). Each of these studies was based on findings from “convenience samples,” however, and their results are inconsistent. This review aims first to highlight several clinical correlation studies using convenience cohorts to assess the PAX-FOXO1 translocation as a biomarker, and then to illustrate the challenges of using such methods to draw clinical conclusions.

Initial clinical reports suggest the PAX3-FOXO1 and PAX7-FOXO1 translocations are associated with distinct frequencies and clinical phenotypes. The PAX3-FOXO1 translocation is more common, present in 60% to 70% of ARMS cases, in contrast to the PAX7-FOXO1, present in 10% to 20% of cases (8, 10, 13, 18,20). Patients with the PAX3-FOXO1 translocation tended to be older, a finding traditionally associated with worse prognosis, and had more aggressively behaving tumors (11, 15, 21, 22). Small case series among patients with known presence of a PAX-FOXO1 translocation again suggested inferior clinical factors and outcomes [Kelly and colleagues (15); Anderson and colleagues (16)], leading to larger cohort studies to determine the association between PAX-FOXO1 translocation status and prognosis (Table 1).

The first relatively large analysis of the relationship between PAX-FOXO1 translocation status and survival used samples from intergroup rhabdomyosarcoma study (IRS)-IV (23–25), which enrolled patients from 1991–1997 (Sorensen and colleagues; ref. 7). Only 141 (11%) of all IRS-IV patients had centrally banked fresh frozen tissue suitable for molecular studies. An additional 27 ARMS cases were identified from local, institutional banks to create a combined cohort of 171 patients, including 78 with ARMS. Potential cases were reviewed by central pathology to confirm alveolar histology, and reverse transcriptase PCR (RT-PCR) was carried out by established methods at a single institution. PAX3- and PAX7-FOXO1 fusion transcripts were detected in 55% and 22% of patients with ARMS, respectively; 23% were fusion-negative. All other rhabdomyosarcoma specimens lacked detectable fusion transcripts. Fusion status was not associated with outcome differences in patients with localized ARMS; however, among those with metastatic disease, PAX3-FOXO1 was associated with inferior 4-year overall survival (OS, 8% vs. 75%, P = 0.0015).

To confirm these observations, the same investigators analyzed a separate, retrospective cohort from the IRS-III study (open from 1984–1991; ref. 26), including 78 archived formalin-fixed, paraffin-embedded (FFPE) specimens of ARMS tumors (Barr and colleagues; ref. 17). Satisfactory RT-PCR results were obtained in 59 cases (30% of total ARMS cases). The distribution of fusion types was similar to previous studies: PAX3-FOXO1, 59%; PAX7-FOXO1, 19%; and fusion-negative, 22%. Investigators were unable to detect differences in FFS among assayed ARMS cases using a classical alpha level of 0.05 for statistical significance (P = 0.17). However, being a member of the cohort (i.e., having FFPE tissue available) was associated with superior outcomes. Those without available fusion data appeared to have inferior outcomes. The HR for relapse among nonassayed cases was 2.1 [95% confidence interval (CI), 1.2–3.5; P = 0.0075] and the HR for death was 2.4 (95% CI, 1.3–4.1; P = 0.00027). Secondary analyses were unable to identify an explanation for this finding. The 2 groups did not differ significantly with respect to distribution of prognostic clinical variables, arguing that the convenience cohort was, indeed, similar to the larger patient population. Investigators suggested that superior outcomes among assayed patients was due to the fact that these cases were more likely to come from larger institutions (Mantel–Haenszel trend test, P = 0.067) They also postulated that factors such as unmeasured socioeconomic status, distance from treating centers, or insurance status may have affected outcomes. Finally, they noted the limitations of retrospective studies of molecular–clinical correlations and underscored the idea that results from convenience samples must be interpreted with caution.

The limitation inherent in a convenience cohort was also seen in a separate, retrospective analysis done within the German Cooperative Soft Tissue Sarcoma Study Group (CWS), reporting results from 4 consecutive trials open from 1984 to 2004 (Stegmaier and colleagues; ref. 18). To evaluate the prognostic value of PAX-FOXO1 fusion status, 121 ARMS specimens were selected for fusion status evaluation (27% of total patient population) based on availability of pretreatment frozen or FFPE tissue. RT-PCR was carried out by established methods at 2 institutions. Patients with PAX3-FOXO1–positive tumors tended to be older than those with PAX7-FOXO1 (63% vs. 17% were older than 10 years, respectively, P = 0.0001) and had higher rates of metastatic disease (50% vs. 24%, P = 0.017). There were no detected differences in 5-year event-free survival (EFS) between patients with localized disease, stratified by fusion status: PAX3-FOXO1, 38.9%; PAX7-FOXO1, 18.2%; fusion-negative, 11.7% (P = 0.235). Overall, the 5-year EFS for all localized patients and fusion data were 28.7%, compared with 65% seen on IRS-III and -IV (4, 26, 27). For patients with metastatic disease, fusion-positive status patients tended toward an inferior 5-year EFS compared with fusion-negative: PAX3-FOXO1, 9.3%; PAX7-FOXO1, 14.3%; fusion negative, 60% (P = 0.145).

In contrast to the IRS-III series, members of the CWS convenience cohort with localized disease had inferior EFS compared with those without tissue available for analysis (28.7% vs. 50.8%, P = 0.009), regardless of fusion status. The distribution of patients and tumor-related parameters was similar in both groups, except that analyzed patients had a greater proportion of unfavorable tumor sites such as extremities. Investigators suggested that tumor site may have contributed to the inferior outcomes seen in analyzed patients. Similarly, they highlighted the substantial differences in EFS among patients with localized disease who did and did not have tissue available and suggested these differences were due to small numbers of patients and limited representativeness of the convenience sample. They again noted the limitations of convenience samples for correlative studies. EFS for patients with metastatic disease was similar regardless of whether or not cases were analyzed.

A recent series studied the impact of rhabdomyosarcoma histology and fusion status using a convenience cohort of 101 patients with available frozen tissue plus an additional 109 with only clinical data available (8). RT-PCR was carried out by established methods; all samples that were negative for PAX-FOXO1 fusions underwent further, confirmatory pathologic review. ERMS cases were not assessed for fusion status. Patients were treated over several years and with various regimens of multiagent chemotherapy plus or minus surgery and/or radiation therapy. ARMS fusion-positive patients (either PAX3- or PAX7-FOXO1) had inferior 5-year EFS compared with ARMS fusion-negative and patients with ERMS (20% vs. 60% and 55%, respectively, P < 0.001). The relative risk of death for fusion-positive patients was 2.5 after adjustment for stage and histology (95% CI, 1.2–5.1). ARMS fusion-positive patients were more likely to have unfavorable sites of disease (79% vs. 53% and 57%, respectively, P = 0.002) and metastatic disease (43% vs. 8% and 12%, respectively, P < 0.001). The number of cases for which tissue was not available to analyze was not reported, so it is not possible to determine the extent of patient selection in this series. There were no comparisons of outcome by whether or not tissue was available for analysis. The authors concluded that fusion status, rather than histology, be used to risk-stratify patients with rhabdomyosarcoma.

A retrospective evaluation restricted to older (adolescent and adult) patients with rhabdomyosarcoma treated with various regimens at a single institution between 1957 and 2001 evaluated the role of PAX-FOXO1 translocations (9). One hundred and five of 251 consecutively treated patients with either ARMS or ERMS had available FFPE tissue samples and 52 (21% of all potential cases) yielded interpretable FISH results. Among the ARMS specimens, 67% had a detectable fusion gene, and these patients were more likely to have metastatic disease (39%) than those with fusion-negative alveolar or embryonal disease (both 22%, P = 0.0081). No associations were detected between fusion transcript type and survival; however, variable treatment regimens, stages of disease, and abilities to detect metastatic sites over this 40-year time period may have precluded appropriate survival comparisons.

This review highlights the limitations of using convenience cohorts in clinical correlation studies attempting to define a biomarker for disease risk. Such studies characterize selected subsets of patients, often treated with different regimens, spanning decades of time. Among patients with rhabdomyosarcoma, this is particularly important as risk criteria and histologic classifications have changed over time (5). Nonetheless, some investigators have concluded that fusion status is associated with prognosis (21); for example, studies commonly suggest that fusion-negative patients have better outcomes than fusion-positive, and PAX7 seems to be associated with less risk than PAX3. Other investigators have suggested such conclusions are premature (27) and can only be verified with more robust, prospective studies.

The inconsistent data about the prognostic significance of fusion status and type may be explained, in part, by the limitations of convenience sampling methodologies. Perhaps the strongest examples stem from the 2 studies that noted differences in survival based on membership in the convenience cohort alone [Barr and colleagues; (17); Stegmaier and colleagues; (18)]. Not only did these studies reveal conflicting results, but they reported findings that made little clinical sense. Why should the presence of archived tumor material predict a patient's prognosis? Clearly, the samples epitomize some unrecognized or unstudied selection bias. They are not necessarily representative of the whole population.

Other limitations include the fact that the relatively small number of patients in these studies makes it difficult to detect differences between groups or to fully adjust for potential covariance. For example, while patients with PAX3-FOXO1 tend to be older, it is unclear whether age and translocation status are independent or interacting factors of adverse outcomes. Similarly, the variable association with primary disease site and stage preclude consistent conclusions. Interpreting differences in survival status is limited by the fact that each study assessed different populations. Some included both ARMS and ERMS, or both metastatic and nonmetastatic disease in the same cohort. Other prognostic indicators, such as tumor site or baseline stage were not always included in multivariate models.

“Nonprobability sampling” like that used in convenience cohorts involves nonrandom selection of samples. This does not necessarily mean that convenience samples fail to represent the general population; rather, it implies that convenience samples cannot rely on probability theory and are, therefore, subject to bias. Unfortunately, there are no clear methods to detect or control for such biases and results from such studies must be interpreted carefully. The benefit of convenience cohorts, however, is that they provide efficient and important exploratory information that may be used in larger prospective studies with more uniform assessment of the prognostic factor.

The investigators in aforementioned studies appropriately attempted to characterize if and how their samples might differ from the larger population of patients with ARMS. Likewise, they qualified the limitations of their findings and suggested that future research include baseline tumor tissue for all enrolled patients. Indeed, the current and future Children's Oncology Group rhabdomyosarcoma clinical trials require tumor specimens from all patients and will ultimately be able to describe the true relationship between fusion status and clinical outcomes among patients with rhabdomyosarcoma.

This review underscores the fact that convenience samples are critical for hypothesis generation, but less compelling for confirmatory assessments. Rather, prospective studies which include timely molecular assessments of all patients may better elucidate true risk categorization. Ultimately, such studies will enable investigators and clinicians alike to risk stratify their patients more accurately.

No potential conflicts of interest were disclosed.

Conception and design: A.R. Rosenberg, D.S. Hawkins

Development of methodology: A.R. Rosenberg, D.S. Hawkins

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): A.R. Rosenberg, D.S. Hawkins

Writing, review, and/or revision of the manuscript: A.R. Rosenberg, S.X. Skapek, D.S. Hawkins

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): A.R. Rosenberg

The authors thank Jim Anderson, PhD, for his thoughtful assistance with the development and review of the manuscript.

This work was supported by the Ruth L. Kirschstein National Research Service Award T32CA009351.