Publication and Reporting Conduct for Pharmacodynamic Analyses of Tumor Tissue in Early-Phase Oncology Trials

See commentary by Iannone, p. 6398

A growing number of early-phase cancer trials involve collection of tumor tissue for pharmacodynamic (PD) analysis or biomarker development (1). Biopsies in early-phase trials are burdensome, and often have no value in terms of the clinical management of subjects (2–5). Although research biopsies are generally considered safe (6), the procedures are associated with pain (7–11), bleeding (12), infection, or other complications (13, 14). In at least one instance, a patient died as the result of a research biopsy (15). According to the norms of research ethics (16–18), such burdens should be justified by benefit through advancement of knowledge. This benefit is often called “knowledge value” (3–5).

Some commentators question whether the scientific value of biopsies in early-phase cancer trials redeems their burdens (19–22). The practical value of correlative PD studies is confounded by small sample sizes, heterogeneous tumor material, sample degradation, and uncertain use of assay methods (23). Disputes over the value of research biopsy are difficult to mediate owing to the subjective nature of knowledge value. We set out to inform this discussion by measuring 2 basic, objective components of knowledge value: publication and perceived reporting quality. Our premise is that studies can only produce knowledge value insofar as they are reported in ways that enable a broader scientific community to use their findings for planning new investigations. This study focuses on publication practices and reporting conduct (we do not address the extent to which PD actually informs drug development).

Our study was divided into 2 parts. In the first, we identified early-phase cancer trials with nondiagnostic tissue biopsies and determined whether investigations resulting from these biopsies were published. In the second, we surveyed corresponding authors of trials using biopsies to assess their perceptions of publication and reporting quality.

To assess the publication rate of PD data in early-phase cancer trials, we used a design previously described elsewhere (24, 25). This approach uses abstracts from major meetings as a baseline of research activity (because they are publicly available but provide incomplete reporting), and then tracks the proportion that result in subsequent journal publications. We selected 2005 as a cutoff date for all searches to allow sufficient time for trials to reach publication. Search strategies were tailored to the different database interfaces to produce the greatest volume of relevant results.

We began by creating a sample of studies by identifying abstracts published online by the American Society of Clinical Oncology (ASCO, 1995–2005 inclusive; ref. 26), using the following search terms: “biopsy,” “phase I,” and “pharmacodynamics.” To focus on the types of biopsy procedures where knowledge value is most critical from an ethical standpoint, we included only trials that described use of invasive and nondiagnostic tissue procurement and were early-phase (phase I or II). We excluded articles where biopsies were clearly based in standard-of-care (e.g., tissue collection from surgically excised tumors).

We supplemented this with a search of AACR abstracts in 2004 to 2005 (articles predating 2004 were not available through AACR online database). First, we searched the abstract database for entries that used any combination of the key words “biops*” and “pharmacodynamic” in the title or abstract, as well as any of the keywords “phase,” and “1, 2, I, or II” in the title. Second, we searched using the keywords “biops*” and “pharmacodynamic” as well as “aspirate” and “pharmacodynamic” in abstract or title. Our exclusion criteria were similar to above, except we also excluded studies that did not involve PD analysis (e.g., studies aimed at developing predictive biomarkers). The search of ASCO and AACR databases was piloted by G.A. Freeman and reviewed by J. Kimmelman.

Eligible meeting abstracts were matched to final publications in PubMed using subject–patient cancer types, investigational agent, comparison of author lists, dose range, and schedule. Minor variances in dosing schedules were tolerated. The matching of meeting abstract to final publication was piloted by G.A. Freeman and reviewed by J. Kimmelman. Publication status of abstracts involving invasive research procedures were classed into 4 categories: no publication of the early-phase trial in which the PD procedures were embedded, trial publication without PD data, trial publication with incomplete PD data, or trial publication with complete PD data. Nonpublication of PD data was defined as a matched final publication that did not report any of the PD studies listed in the meeting abstract. Incomplete publication was defined as a matched final publication that included some, but not all PD markers listed in the meeting abstracts. When non or incomplete publication of PD data occurred, we contacted corresponding authors electronically on published early-phase trials to determine whether the PD data were reported elsewhere, and if not, reasons for nonpublication. Authors were queried 5 times before nonresponses were considered a refusal. We did not contact authors on meeting abstracts that resulted in no final publication of the trial itself as our study focused on nonreporting of PD data in particular; these trials were not included in our analysis.

We identified corresponding authors on a sample of articles using nondiagnostic biopsy for PD study in early-phase cancer trials to assess perceptions of reporting conduct, quality, and reasons for nonpublication. To create our sample, we searched PubMed for oncology trials published from 2000 to 2010 using the following algorithms: (i) Search: PD and biopsy, limits: humans, clinical trial, cancer, publication date from 2000 to 2010; (ii) Search: correlative and biopsy and (cancer or oncology), limits: clinical trial, clinical trial, phase I, clinical trial, phase II, clinical trial, phase III, publication date from 2000 to 2010; and (iii) search: serial and biopsy and (cancer or oncology) and (tumo* or tumou* or marrow) not correlative, limits: clinical trial, clinical trial, phase I, clinical trial, phase II, clinical trial, phase III, publication date from 2000 to 2010. Articles were excluded where the nondiagnostic nature of tissue collection was ambiguous, where biopsy was not carried out, or where the trial did not involve patients with cancer. We included all unique corresponding authors in our sample; we also included investigators identified through our study of nonpublication, as described earlier.

Investigators were presented electronically with a questionnaire containing structured questions and one qualitative question (Supplementary Materials). As mentioned above, 5 queries without reply were scored as nonresponse. Initial analysis of the qualitative question was carried out by G.A. Freeman to identify the common themes. Final analysis and thematic coding was carried out independently J. Kimmelman and G.A. Freeman. J. Kimmelman carried out coding blinded to the identity of the respondents. There was close agreement (92.6% concurrence) in the independent coding carried out by G.A. Freeman and J. Kimmelman.

This study met requirements of Tricouncil Policy for the ethical conduct of research involving humans and received expedited ethics approval from the McGill REB (16).

Ninety abstracts met our eligibility criteria. Basic characteristics of abstracts are shown in Table 1. Figure 1 illustrates loss of PD data in the progression of cancer trials from abstract to full publication.

We sent surveys to 33 authors who had published articles with missing or incomplete PD data. Five authors could not be contacted. Fourteen of the remaining 28 authors responded (50%). Of responding authors, 9 (64.3%) indicated that PD had been published elsewhere. Of these, 7 provided a citation. In 4 instances, we were able to confirm full publication on analysis of publications offered by authors (57.1%); frequency estimates in Fig. 1 were adjusted accordingly.

We measured the time between meeting abstract presentation and final publication. Average time to publication was 2.9 years for ASCO abstracts and 1.9 years for AACR abstracts. Complete reporting of PD data was not associated with a longer time to publication (2.7 years vs. 2.9 years for ASCO, 1.9 years vs. 1.9 years for AACR). We encountered 1 instance of full publication before meeting abstract presentation.

We sent questionnaires to 92 corresponding authors. Thirty-three (35.9%) were corresponding authors on the 37 final publications we identified as having missing or incomplete PD data. Fifty-nine (64.1%) were corresponding authors in our sample of 77 cancer trials identified in our PubMed search of articles involving research biopsy for PD study. We received 53 responses. Ten authors were not reachable and 7 respondents actively declined participation; the response rate for our survey was 64.6%. Demographics of respondents are depicted in Table 2. As indicated, respondents did not seem grossly different from our overall sample in terms of location, credentials, and productivity (defined as the number of PubMed articles authored 2005–2010, limited to publications related to cancer). Primary affiliation of corresponding author did vary slightly between the contacted and responding groups, with university affiliated authors representing a smaller percentage of the responding group than the contacted group (53% vs. 61%).

We asked the investigators identified through the PubMed search whether they had ever taken part in a study where they had collected biopsies for PD, but had not published all analyses. 46.7% of respondents answered in the affirmative.

Authors were asked how often they thought trials were published without including all PD results (Fig. 2A). All respondents claimed that incomplete PD publication sometimes occurred. The highest proportion of respondents (40%) indicated that incomplete publication occurred more often than not. Authors were asked to rate the quality of reporting of PD data in early-phase cancer trials (Fig. 2B). The largest proportion of respondents (56.7%) considered the quality of reporting to be “fair.”

We asked authors to identify the main reasons PD data go unpublished. Recurrent themes, and their frequency of identification in open-ended questioning are depicted in Fig. 3. As indicated, the largest points of data loss were from poor patient accrual (44.1% of respondents), poor quality or incomplete assay (41.2% of respondents), and strategic considerations in publication (identified by 58.8% of respondents). These categories are nonexclusive. “Strategic considerations in publication” included studies where results were dismissed as not interesting, not informative, contradictory, or difficult to interpret, and “scientific disagreement posttrial either within the research group or with research sponsor” prevented the publication of PD results. Drug company resistance, although not frequently cited, was mentioned in some responses: “In [2] of my personal studies, the PD data was [sic] negative…in both cases the data was published with a lot of resistance from the drug companies.” Other investigators described problematic use of posthoc analysis: “The primary [PD] endpoint is not clearly defined and eventually, some significant result is found and reported (“fishing expedition”).”

Correlative biopsy data has the potential to ascertain target-specific, mechanistic activity of study interventions (2, 27). This can provide clues as to how to troubleshoot agents that fail to recapitulate disease activities shown in animals (28) it can inform dosing decisions for phase II studies, eligibility criteria for trials using enrichment designs, or it can provide corroborating evidence that an agent should be advanced into later stages of development. However, the invasive nature of the procedures, their lack of diagnostic use for subjects, and concerns over the validity and predictive value of assays bring a measure of controversy concerning their ethical justification (19–23, 29, 30). This study was not directed toward adjudicating this dispute. Instead, we sought to identify factors that intercept the accrual of knowledge value in the hopes of informing the planning, implementation, and ethical review of such studies.

The first factor frustrating accrual of knowledge value is nonpublication. Publication is the main process through which findings of individual research teams are disseminated to a wider community. Any interruption of this process frustrates the integration of isolated findings into a broader corpus of knowledge. In our study, self-reported publication practices were used as the primary indicator of knowledge accrual interruption. The second indicator was perceived reporting quality among practitioners. We reasoned that scientists who actually conduct PD and biomarker studies are in a privileged position to judge practices and quality in publication and reporting.

We found that 41.1% of all studies involving invasive nondiagnostic tissue procurement in trials were either unreported or incompletely reported in full publication. This number increases to 52.8% when we exclude from our analysis those studies carried out in the context of trials that themselves were never published. Our results are comparable to figures for nonpublication in early-phase studies (24, 25). 22.2% of the phase I and II trials in our sample resulted in no full publication.

The loss of information through nonpublication or poor reporting raises several concerns. Nonpublication frustrates the redemption of burdens asked of volunteers. It also incompletely honors the spirit of altruism with which subjects consent (16, 25, 31). Finally, incomplete or poor reporting can lead to inappropriate decision-making in treatment, policy, and research design. This puts future patients and scarce resources at risk (16, 25, 32–35).

A majority of respondents stated that nonpublication occurred “more often than not.” Barriers to publication reflected in survey responses were similar to those reported in other studies of nonpublication (24, 25, 36–47). For instance, a 2005 study identified industry influence as a cause of nonpublication in 4% of trials (24). In our study, industry pressure was identified as a barrier to publication by 5.7% of respondents. One barrier to publication that has been described elsewhere, but not in our study, was author relocation (24, 25)

Our study is subject to several limitations. First, our sample size was small, owing to difficulty building a larger sample of abstracts. Furthermore, the search terms used for the ASCO abstract database were restrictive; this may have limited our study capture. Our search strategy was selected to strike a workable balance between sensitivity and specificity. Second, our questionnaire response was somewhat low, though similar to that obtained in a study of phase I trial nonpublication (24). Nevertheless, demographics of respondents and our overall sample were similar. Third, our questionnaire, in aiming to facilitate response among busy practitioners, was tightly structured and left little room for nuanced responses. Fourth, publication status and perceptions of reporting quality represent crude measures of knowledge value. Indeed, although our study points to ways the risk-benefit balance might be improved, we make no claims concerning the overall ethical justifiability of nondiagnostic biopsies in cancer trials, or whether current practices fall below a threshold of acceptability.

These limitations aside, our study nevertheless identifies barriers to full attainment of knowledge value for invasive, nondiagnostic biopsies. These findings can inform the design and review of PD studies. For example, when developing proposals to conduct PD using biopsied tissue, investigators should present (and IRBs should seek) realistic recruitment plans, evidence of assay validation (48), and a well-developed plan for sample handling. This would provide greater assurance that favorable risk-benefit balances decided prospectively will be maintained in execution. As well, journal editors can contribute to a favorable risk-benefit balance by eliciting better study reporting from authors. Specific reporting quality guidelines, like the CONSORT statement for randomized trials (32) and REMARK for tumor marker prognostic studies (36) should be developed for PD.

On the basis of the findings of this study, we recommend opening up a greater dialogue with the community of early-phase research oncologists regarding publication practices and quality of reporting with the aim of developing publication and reporting guidelines for PD studies.

No potential conflicts of interest were disclosed.

Conception and design: J. Kimmelman

Development of methodology: J. Kimmelman

Acquisition of data (provided animals, acquired and managed patients, provided facilities, etc.): J. Kimmelman

Analysis and interpretation of data (e.g., statistical analysis, biostatistics, computational analysis): G. A. Freeman, J. Kimmelman

Writing, review, and/or revision of the manuscript: G. A. Freeman, J. Kimmelman

Administrative, technical, or material support (i.e., reporting or organizing data, constructing databases): G. A. Freeman, J. Kimmelman

Study supervision: J. Kimmelman

The authors thank Janet Dancey and Jose Monzon for their comments on an earlier draft.

This work was funded by the Canadian Institutes of Health Research (EOG 102823).

The costs of publication of this article were defrayed in part by the payment of page charges. This article must therefore be hereby marked advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate this fact.