Skip to Main Content

Advertisement

Skip Nav Destination

Improving Reproducibility

Improving Reproducibility

To improve the reproducibility of published research articles, authors are urged to provide additional important details about their reported studies. These details include identifiers for the reagents and tools used in the study, step-by-step protocols, software code, and experimental design details such as sex, blinding, randomization and power analysis. At this point in time, addition of this information is strongly encouraged, but not required. The AACR provides access to tools and services that aid authors in enhancing the reproducibility of their work ahead of publication; it also encourages use of certain free resources to increase the likelihood of study replicability. An introduction to these services and resources is provided below.

Extended Methods Announcement: Effective December 2021

As part of the ongoing efforts of the AACR journals to improve the reproducibility of the research they publish, the "Materials and Methods" or "Methods" section in original research articles will no longer be included when calculating the length of a submitted article.

The Reproducibility Project: Cancer Biology has clearly highlighted the need for published articles to contain more detail of exactly how a study was conducted. Article length requirements should not constrain authors in this regard, nor should important methods information be relegated to the supplementary materials where citations to prior work fail to be properly indexed. Moreover, the AACR journals will begin to actively screen for manuscripts that appear to have overly short descriptions of the methods, and journal staff will ask authors to expand them as deemed necessary.

Authors are also strongly encouraged to publish the step-by-step protocol(s) used in their study at protocols.io and then include the unique digital object identifier(s) (DOI) in the methods section of their research article as described in detail below.

SciScore

This automated methods-review tool screens submitted manuscript text for adherence to experimental rigor and reporting requirements.

To assist authors in improving the reproducibility of submitted manuscripts, the AACR journals have integrated our manuscript submission system, SmartSubmit, with SciScore.

SciScore scans the methods section of an author’s submitted manuscript and uses machine learning methods to check for multiple indicators of experimental rigor and proper reporting of the resources used. The tool then provides the author with a report containing easy-to-use tables on rigor adherence and key resources such as antibodies, experimental models, recombinant DNA, and software. The report clearly communicates what the tool did and did not find, and displays an overall numerical score from 0 to 10, based on the findings.

Importantly, SciScore also checks whether the experiments described in the manuscript used any known misidentified or contaminated cell lines. If any are detected, the SciScore report prominently highlights this.

Authors are strongly encouraged to use the SciScore report to ensure that key information related to reproducibility is included in their manuscript whether they are revising it for submission to an AACR journal or another publisher.

The SciScore report is made available to the journal editors and reviewers but isn’t currently a formal part of their evaluation. Prior to acceptance, authors are asked to use the report’s list of key resources to add RRIDs to their manuscript for all listed resources (see RRID section below). The SciScore report will also be used by journal staff when performing standard manuscript quality checks before acceptance. Authors will be asked to address any outstanding reporting issues.

For more information, including how you can use your ORCID to run the report for free yourself prior to submission, please see the frequently asked questions page on the SciScore site and this sample report.

Learn more about SciScore in this AACR Journals Webinar
SciScore: An Effective Tool that Verifies Rigor Criteria and Research Resources

Computational Methods and Associated Data

This free cloud-based computing environment provides reviewers and readers with seamless browser-based access to the code and data behind manuscripts so that they can reproduce the analyses used by the authors.

Researchers are increasingly relying on algorithms implemented in software to process and analyze their data. Without the computer code and associated test or real data, it is difficult for others to reproduce or reuse the authors’ analysis methods. To help solve this problem, the AACR journals have integrated their submission system, SmartSubmit, with Code Ocean, a free cloud-based service that allows authors to provide the data, code, and computing environments used in their analyses in the form of a Code Ocean “compute capsule.” This capsule will be linked to their manuscript so that editors and reviewers may easily reproduce and examine the exact data analyses performed by the authors. Upon publication, the capsule may be linked to the published manuscript so that the compute capsule is accessible inline in the published article through a widget (see examples below). Readers may then easily reproduce and build on the authors’ work.

During manuscript submission, authors are asked if they wish to provide code associated with the work described in their submission. If they already have a git repository of the pertinent code, it can be imported with the click of a button, cleaned up, validated, and provided to editors, reviewers, and readers. We strongly encourage all authors who used code for any of their data analysis to provide the software code, with either test or actual raw data, in a Code Ocean compute capsule.

We therefore encourage authors to provide their software code, with either test or actual raw data, in a Code Ocean compute capsule so that others to easily run, edit, and reuse the code in a cloud-based computing environment from their web browser.

All authors with a Code Ocean compute capsule associated with a manuscript should include a statement in their manuscript describing the capsule and a link in the form: https://codeocean.com/capsule/XXXXXXX/ (where X is the unique identifier of the capsule).

Learn more about CodeOcean in this recent AACR Journals Webinar:
Code Ocean: A Tool to Support the Delivery of Reproducible Analysis with Dr. Benjamin Haibe-Kains

Examples of published articles from AACR journals with code and data are provided below.
 



Research Resource Identifiers (RRIDs)

Author inclusion of unique searchable IDs for the antibodies, experimental models, recombinant DNA, and software used their manuscript, provides critical unambiguous methodological information for readers.

To uniquely identify the biological reagents and software tools reported in the Materials and Methods of published research articles, the AACR asks its authors to include unique searchable Research Resource Identifiers (RRIDs), as assigned by the Resource Identification Initiative. The process of identifying resources in need of RRIDs and assigning the correct RRID is greatly facilitated by the SciScore report described above.

In addition to uniquely identifying each reagent and tool, these RRIDs can be used to link readers to external resources and enable search engines to return all articles in which a resource was used. We believe these are important steps toward ensuring reproducible methods and providing critical data to help researchers identify suitable reagents and tools, and we are now asking authors to include RRIDs in their manuscripts.

How to find an RRID

When submitting a manuscript to the AACR journals, the corresponding author will receive a SciScore report containing a Key Resources Table listing all the reagents and other resources that SciScore detected in the methods section of the manuscript. Beside each resource is a suggested RRID and link to the resource description. Click the link to verify that this is the exact resource used in the described research.

To obtain an RRID for a resource mentioned somewhere other than the methods section of a submitted manuscript, please visit the Resource Identification Portal and enter appropriate search term(s) for the reagent you used. Tips for searching are provided below.

  • Antibodies: searching for the catalog number usually narrows the search to only a few relevant results.
  • Cell Lines: searching for the catalog number of an established cell line is usually best, searching for common cell lines such as HeLa cells is expected to produce several pages of results. The cell line information page will indicate whether the cell line is one of over 480 misidentified cell lines cataloged in the Register of Misidentified Cell Lines. Please also see our cell line authentication policy.
  • Organisms: you may include PubMed IDs (PMIDs) in your search or filter your search results by PMID, species, phenotype, and other criteria.
  • Software Tools: usually the name of the tool (MATLAB or ImageJ) or the institution where it is housed will return relevant results.

How to include RRIDs

Once you have located an RRID, please insert "RRID:" plus the identifier in the appropriate location in the Materials and Methods section of the manuscript. For example:

  • Antibodies: "Sections were stained with a rabbit polyclonal antibody against ERK1 (Abgent Cat# AP7251E, RRID: AB_2140114)."
  • Cell Lines: "Hela cells (CLS Cat# 300384/p699_HeLa_S3, RRID: CVCL_0058) were obtained from...”
  • Organisms: "Fgf9Eks/Fgf9+ mice (RRID: MGI_3840442) were fed..."
  • Software Tools: "...terminals were mapped with a computer-assisted mapping program (Neurolucida, v 10; MicroBrightField RRID: nif-0000-10294)."

How to request an RRID

If you cannot find an RRID for a resource that you used, please submit the reagent as detailed below so that it can be given proper identifiers for future use. The Resource Identification Portal includes mouse, zebrafish, worm, fruit fly, and rat model organisms as well as many commercial antibodies and some lab-sourced ones. When you cannot find an RRID for one of these five model organisms, please see these instructions for submitting a new organism to the relevant model organism database. Antibodies can be added via the Antibody Registry; please note that a login is required on that site.

Step-by-Step Protocols

A free platform allows authors to publish clear step-by-step methods protocols that can be directly linked to from their research articles.

It is generally recognized that the traditional Methods section in published research articles has insufficient details for readers to reproduce the results. To help address this, we encourage authors to publish the step-by-step protocol(s) used in their study at protocols.io and then include the unique digital object identifier(s) (DOI) in the Materials and Methods of their research article.

How to publish your Protocol at protocols.io

Protocols.io provides a free platform for researchers to publish their protocols and control who can view them. An available app allows each protocol to be conveniently run in a mobile device at the bench.

  • Create a free account at protocols.io.
  • Enter the step-by-step details and comments for your protocol.
  • Select "Get DOI" to issue your protocol with a unique DOI (digital object identifier)
  • Optional: Select "Publish" to make your protocol public. If you don't make your protocol(s) public immediately, reviewers will still be able to access it through a DOI link in your submitted manuscript (see below).

How to link to your Protocol from your manuscript

By linking to protocols from within a submitted manuscript, both reviewers and readers will be able to directly access and read the full protocol at protocols.io.

  • Find and copy the DOI link generated above.
  • Include this link in the Material and Methods section of your manuscript in the form: http://dx.doi.org/10.17504/protocols.io.xxxxxxx (where xxxxxxx is the unique DOI).
  • If you did not select "Publish" as described above, only people with access to this link (e.g., editors and reviewers) will be able to view your protocol.
  • If your article is published, any referenced protocols will automatically be made public without any intervention on your part.

American Association for Cancer Research
615 Chestnut Street  |  17th Floor  |  Philadelphia, PA 19106-4404
Phone: (215) 440-9300  |  Fax: (215) 440-9354  |  E-mail:  pubs@aacr.org

Close Modal

or Create an Account

Close Modal
Close Modal