GigaScience – making open peer review more open: Q&A with Publons co-founder, Andrew Preston

July 3, 2014

At GigaScience, one of our major goals is to improve transparency and reproducibility of research and one of the ways we do this is through open peer review. After the unusual “meta peer review” of our Assemblathon2 paper (see more in biome), we thought our peer review couldn’t get more open, but a small New Zealand-based start-up, Publons, who also happens to be the world’s largest open peer-review platform, approached and told us about their exciting, innovative approach that gives Peer Reviewers due credit for the work that they do. GigaScience peer reviewers can now get further recognised for their efforts through our partnership with Publons, which was recently announced via the BMC Blog. Following some of our author and reviewer Q&A’s, here, we present a Q&A with Publons co-founder, Andrew Preston, who gives us a personal insight into the rationale for doing what they do.

Why is open peer review so important to you?

Well it’s more that science — expanding the sphere of human knowledge — is important to us. It happens that peer review is one of the pillars of scientific research and is in most obvious need of improvement.

Peer review is incredibly important as it’s the best tool we have for evaluating the quality (trustworthiness) and significance (worth reading?) of research. However, while researchers are rewarded handsomely for publishing papers, the reviewers we all rely on get no credit.

There’s room in the world for both open and closed review (and also pre- and post-publication review). All types of review are important. That said, we believe that reviewers add an incredibly valuable corpus of context and knowledge to the publications they review that, among other benefits, is crucial to assessing the reproducibility of the research.

One of our goals is to encourage open review so that we can codify and index it, and make it a formal part of the sphere of human knowledge.

What was the reasoning behind you and Dan setting up Publons?

I got my start in physics. During my PhD and postdoc I published and reviewed papers and learned how frustrating and inefficient the process is. I think the critical point leading to us founding Publons was learning that of the 150 days (on average) it takes to publish a paper, 120 of them are spent in peer review.

We figured that we could speed up science by focusing on that problem.

Why should reviewers get credit?

The first answer is that reviewers should get credit because it’s only fair.  As one of our top reviewers Eugene d’Eon put it:

“I love the idea that you can receive recognition for your reviews, especially when you point out a major flaw or much simpler approach during the review process and the authors fix it or use your idea. You should get credit for that.”

There’s far more to it than that though. We (publishers, authors, and society as a whole) all benefit if the speed and quality of peer review improves. Our fundamental thesis is that we can improve both the speed and quality of peer review by making it a rewarding activity, as opposed to a distraction or a chore. Giving credit to reviewers results in a better peer review process for everyone.

How does Publons work for reviewers?

It’s pretty simple. We help you to build a profile of all the reviews you’ve done and turn it into something you can reference or export and add to your resume.

We work to verify these reviews with the journals you reviewed for, allowing you to prove conclusively that you did a review, even if it was a blind review of a paper that has not yet been published.

We also directly interface with journals (like GigaScience) to collect, verify, curate and index your peer review for you.

Of course you are always in control of how much information is displayed about your reviews on Publons.

We try to provide some light-hearted forms of credit too. For example, we just announced a reviewer rewards program where our top reviewers get prizes from cutting edge companies like Altmetric and ScienceExchange

What are your hopes for Publons in the future?

Our vision is to get as much of the world’s reviews on to Publons as is possible. The more reviews we have, the easier it is for our reviewers to have complete and verified profiles of the peer review work they’ve done. We also believe that critical mass would legitimize review as a research output that can contribute to funding and hiring decisions, and really become a great benefit to reviewers.

At the same time we’d also like to encourage an open and ongoing discussion around published research. We feel that this is the key to ensuring that research is of a high quality, but also reproducible. So we really want to start encouraging reviewers to contribute post-publication reviews along with their pre-publication work.

GigaScience was very interested to hear that the reports from the controversial Science #ArsenicLife paper are available to view on Publons. Can you tell us a bit more about how you managed that, and what scope there is for throwing more light on other similar controversial studies, such as the very topical Nature acid-bath stem-cell papers?

Great question!

All of the pre-publication reviews for the Science Arsenic paper are available at publons.com/p/2751/. The reviews were actually released into the public domain as the result of a FOIA request by USA Today (this was possible since the work was done on behalf of NASA).

All reviews on Publons have a source license and attribution so we can host reviews like this pretty easily (not just CC-SA). That’s all part of our goal to support any and all peer review (including blind or open, and published or unpublished reviews).

Our basic belief is that peer review, like this, provides valuable context to any published paper. It doesn’t matter if the result itself is eventually discovered to be incorrect, what’s important is the process and the thinking that’s going on behind the scenes — that’s scientific research in action.

Our goal is to encourage that process in a way that benefits the reviewer and the journal. It should be a positive process, and that’s the culture we’re trying to build. We believe this will lead to a more reproducible literature in the long run.