How I learned to stop worrying and love preprints

Researchers in genetics and neuroscience are starting to embrace a new form of scientific communication: the preprint. Preprints are manuscripts that are posted online prior to being published in a journal.

Unlike traditional manuscripts, a preprint submission does not undergo peer review and goes live within days rather than months. Preprints are generally open access, which means new ideas, data and methods quickly become available to the scientific community and the wider world.

Preprints have been commonplace in the physical sciences and mathematics for almost 30 years. However, bioRxiv, the most popular preprint server for biology, launched only in late 2013. bioRxiv is gaining in popularity: The number of preprints has been increasing exponentially each year.

Still, many biologists are reluctant to submit preprints, in part out of concern that doing so will allow others to “scoop” their work and undermine their chances of publication in a prestigious journal. I would like to rebut that concern, among others, and to share our research group’s first experience submitting a preprint manuscript.

What-ifs:

In an informal poll of autism researchers during dinner at an autism meeting in April, I confirmed that the No. 1 fear about submitting to a preprint server is this: What if someone uses our findings to scoop us?

For those of us who entered science in this historic era of risk-gene discovery, this fear is practically tattooed into our psyche. We have heard stories about colleagues presenting a new risk gene at a conference and getting beat to the punch in publication.

The effect of this fear on the field is palpable in the halls of the annual meeting of the American Society of Human Genetics, where it is rare to see a talk on a genetic discovery that is not already published.

The next great concern is this: How will a preprint publication alter our chances of getting a paper published in a major journal?

Big-name journals value novelty, and their editorial positions on preprints vary. Some publishers collaborate with preprint servers, whereas others ban publishing preprint manuscripts. If researchers submit manuscripts to multiple publishers, they increase their risk of running into a problem.

Researchers now have to stay informed about journal policies, which are evolving and often complex. For example, an editorial from Nature Biotechnology in its December 2017 issue highlighted the editors’ general support of preprints, but noted that primary research findings cannot be based on new methods exclusively published as preprints.

We too had some of these fears, but, as we learned, our fears turned out to be overblown and without much basis. 

Signs of progress:

The first significant bioRxiv manuscript caught my attention in November 2015. Science magazine ran a story, as did Nature, on the paper by the Exome Aggregation Consortium (ExAC). (Exomes are the protein-coding portions of the genome.)

The study described a collection of exomes from more than 60,000 individuals. I had seen a talk on these data, and my colleagues and I were eager for the paper to be published. Such a significant work appearing in bioRxiv put the preprint model of publishing on our radar.

Some funding agencies now recommend that their grantees submit preprints. In May 2016, one of our primary funders, the Simons Foundation Autism Research Initiative (SFARI), announced that its grantees are strongly encouraged to publish on preprint servers. (SFARI is Spectrum’s parent organization.) At our next lab meeting, we discussed the preprint model of publishing and its implications at length.

At scientific meetings, I approached editors from Cell Press, Nature Publishing Group and Science to ask about their policies and the potential impact of a preprint on manuscript submissions to their journals. I got the sense that they were not keen on the idea but were taking a wait-and-see approach. All assured me that, if their policies allowed it, the existence of a preprint would not hinder a manuscript review. Seeing the ExAC manuscript eventually appear in Nature provided some reassurance.

Another sign of progress came from the Wellcome Trust and the Bill & Melinda Gates Foundation, which in early 2017 began allowing researchers to include preprints in grant submissions and progress reports. When the National Institutes of Health (NIH) did the same in May 2017, it was a game changer. The NIH biosketch, which highlights an investigator’s credentials and productivity, is a widely used short-form curriculum vitae.

Preprints allow a researcher to cite new work or progress on a grant application months earlier than peer-reviewed publications. This can make all the difference for securing grant funding — especially for early-career scientists who do not have long track records.

Bombs away:

In 2016, we put the finishing touches on a long manuscript describing results from a multi-year project. The project focused on ‘mosaic’ mutations in a large autism cohort. (Mosaic mutations are those that occur after conception and in only some cells.)

Within a week of presenting the main results at the American Society of Human Genetics Meeting, we submitted a manuscript to bioRxiv. There was no required format and the manuscript length was unlimited. By the end of the day, our paper was live and had been tweeted about.

By comparison, the same paper took 10 months to appear in a traditional journal. (The journal received it in early November, accepted it late the following July and published it online at the end of August.)

According to bioRxiv metrics, our abstract was viewed more than 1,500 times in its first two months. Our paper is also getting traffic on the wider web, according to Altmetric, which tracks mentions of scholarly work on multiple websites. By the end of 2017, it had an Altmetric attention score of 41, which put it in the top fifth percentile of all scores.

These numbers show that our paper had immediate impact, which was the main rationale for submitting it as a preprint.

Initial bioRxiv submissions are permanent, which means you can’t fix mistakes without submitting revisions. (Double-check your manuscript before hitting send.)

When we submitted the paper to a journal, we were able to add analyses suggested by the peer reviewers. This made our final manuscript more robust. I must admit, however, that because the main results had already been posted, I was less motivated to work through the difficult revisions. This was the most consequential negative impact of having a preprint out in the world.

However, our study did not get scooped, and our paper was published in our first-choice journal without issue. In addition, a 2016 review cited our preprint long before the final manuscript was published.

Many people believe that preprints serve as a priority claim, although it isn’t clear how widespread that view is. Importantly, several of us on the team benefited from citing the preprint in progress reports and grant applications.

To all researchers out there, I encourage you to stop worrying and love the preprint. Submit your manuscripts, but also read preprints and make comments. The biggest thing missing from the current biologic preprint ecosystem is the same type of critical evaluation authors receive during peer review.

Our group’s journal clubs have shifted almost exclusively from peer-reviewed articles to preprints. Recognizing that others are doing the same, Daniela Saderi, a graduate student at my university, has co-founded PREreview, an online portal for posting and sharing citable preprint reviews from journal clubs around the world. This is a great potential training mechanism for students and should help drive the type of direct feedback currently missing from bioRxiv preprints.

Who knows? With the feedback you receive on a preprint, you might head off a major concern from the infamous ‘Reviewer #3’ in advance.

Brian O’Roak is assistant professor of molecular and medical genetics at Oregon Health and Science University in Portland.