Monday, June 03, 2013

Guest post from Jake Scott: Building trust: a sine qua non for successful acceptance of preprints in the biological sciences

Today I am happy to have a guest post from my friend and colleague Jake Scott.  The topic of the day is preprints in biology and medicine.

Hi - I'm Jake Scott.  I met Jonathan last year when he and I spoke at TEDMED 2012. Both Jonathan and I have posted recently about the need for, and (slowly) growing movement in the biological sciences to post #preprints of manuscripts in openly accessible fora to circumvent some problems associated with standard academic publishing.  Most worrisome are the issues surrounding #openaccess and the length of time it takes to get information from one's brain to the literature - drastically slowing down the pace of science.

This has worked GREAT in the physics community, where this trend really began quite some time ago when the high energy physicists started the arXiv.  Now, the precedent is set, and no one in physics bats an eye about sticking their paper on the arXiv, and cite other works presented there as standard publications.

The climate in biology, sadly, is much different. Whether this is because of a more competitive climate for funding, or just a field diluted by more talented scientists, I don't know.  But there is a pervasive attitude of fear and mistrust around the idea of preprints.

Before you read on (and become biased by my opinions) take a few second (really, probably 1.5 minutes) and take this quick survey:

When I preach to my biological colleagues about the virtue of pre-print servers, I most often, I hear:

Why should I post my papers on a pre-print server where anyone can see it before it is published!?  They could scoop me!

I honestly don't understand this argument, but I hear it all the time.  By nature of pre-print servers, like the arXiv, the idea is yours! Time and date stamped. And, better yet, it is completely #openaccess, free of charge, and helps move science along at a better pace.  Only a very few journals have problems with posting of pre-prints before they get their (greedy) hands on the results of all your hard work, but most are totally OK with it.

The arXiv isn't really interested in shopping its (free) service out to the biological sciences, not because they don't think it would be of value, but because it just doesn't have the infrastructure to support it.  This is a problem that is being with newly created repositories like Nature Precedings, PeerJ and soon, the bioRxiv.  So, the only thing holding us up is, IMHO, trust.

How can we rectify this?

I think the way forward is to create something that we are all missing now, except when we are at our home conference, among friends or if we got into a time machine and went back 100 years - community.

Science is such a juggernaut now that putting your work onto a pre-print server where anyone in the world can see your as of yet unvetted work can be daunting.  Worse, the idea of commenting on it is a tough sell when the world is a witness.  I think we need to (re)create micro-communities of our specialist peers where these initial discussions can be held.  Two examples of this are Haldane's Sieve and more recently created, an initiative I'm involved with, Warburg's Lens.  These two sites are micro-communities where population and evolutionary biologists, and mathematical oncologists (respectively) congregate to discuss pre-prints culled from any repository but necessarily of interest to the micro-community.

This does two things: it allows a common place for easy browsing in topics of interest to a specialist (like reading your favorite journal), and increases the chances that the readers and commenters are your (at large) peers.

So, those are my two cents. #Openaccess for all is coming, and preprints are a part of the wave.  The sooner we all adopt an open science attitude, the sooner we'll come to the conclusions and make the discoveries that make doing science AWESOME.  There is no better job than science, and sharing and communication are central to it

So START SHARING your science.  Commit to this - when you are ready to submit your next paper, put that version on a pre-print server as you start the submission process. Then tweet about it, G+ about it, blog about it, do whatever, but let your peers know!

Anyone else interested in starting a micro-community discussion forum, or to just discuss this issue further, please contact me.

If you are against it - please leave some comments about why, I'd love to try to convince you otherwise!  If you are a biologist (or know one) who DOES post pre-prints, weigh in and share your good experiences!

About me: I am a radiation oncologist and I approach the understanding of cancer like my original training in physics taught me - from the ground up, using the descriptive language of mathematics.  Using established mathematics in new ways, guided by the principles of evolution, I hope to better understand (and maybe treat!) cancer.  I am a proud member of the Integrated Mathematical Oncology group at the Moffitt Cancer Center and the Centre for Mathematical Biology at Oxford University.  You can follow me on twitter @CancerConnector or read my blog Connecting the Dots.


  1. I do not understand the advantages of publishing pre-prints in the biological sciences as it can frequently take years to revise and publish a significant finding in a major journal. There is no precedent for receiving credit for a pre-print; why should we expect that to change any time soon in a competitive funding atmosphere?

  2. The advantage is that you can disseminate your findings that much earlier - and the pace of science can quicken. You are right that there is, as of yet, no precedent for receiving credit for preprints, but we hope to change that...

    In a perfect world (which we aren't in, but are working towards!) the 'credit' is dissemination itself. Further, there is more and more a movement towards article level metrics being the coin of the realm, and these are a part of the new biological repositories.

  3. Matt Damon writes: "I do not understand the advantages of publishing pre-prints in the biological sciences as it can frequently take years to revise and publish a significant finding in a major journal."

    That is precisely why posting a pre-print is so important. Unless of course you think it doesn't matter whether people actually read and use your work.

    1. There is a distinct difference between the natural sciences and mathematics in presenting evidence and establishing an idea as valid. Mathematical proofs are largely self-evident, whereas the natural sciences rely upon the peer review process for meeting a reasonable burden of proof in defining a key concept. It is not sufficient to presuppose an idea through a non-peer reviewed publication and still receive creative credit for it. Do not consider this distinction as trivial because Nobel Prizes have been won and lost on it, from PCR to elucidation of transcriptional machinery.

      OA journals exist for individuals who would like to expedite the peer-reviewed publishing process (and they have succeeded admirably). Conferences fulfill their role as a minimalistic pre-print access point. I find it improbable that biology will adopt the policies of math or physics, given current funding exigencies and the need to protect funding sources.

    2. Putting aside whether math is "self-evident" (?), preprints are also the norm in economics and physics. You might be interested in Paul Krugman's discussion of how the economics community stopped caring about journal publications decades ago [here]. An excerpt:

      "[J]ournal publication? Well, tenure committees needed that, but it was so slow relative to the pace of ongoing work that it no longer acted as an information conduit. I presented my paper on target zones at a 1988 conference; by the time it was formally published, in 1991, I had to add a section on the subsequent literature, because there were around 150 derivative papers already out there."

    3. But obviously economics is just a branch of applied math, really, rather than a actual science with messy experimental techniques. Despite being a microbiologist by training, I did a postdoc in a Computer Science department with a strong theoretical focus and things were much like how Krugman describes economics -- everybody read conference proceedings and working papers rather than journals. But that's because the truth of a mathematical proposition is in the pudding -- either the proof is valid or it is not. These days, there are even computer programs that can validate proofs by themselves.

      Experimental science isn't so simple; unless you have performed similar experiments yourself it is hard to see if the methods described really are reliable, and yet the audience for an experimental paper is much wider than just these people; bioinformaticans use experimental results as raw data, for example. So I don't see the need for formal peer review going away in biology -- or chemistry (another solidly journal-based science).

    4. So I don't see the need for formal peer review going away...

      All the communities we're discussing (economics, physics, math) do have "formal" peer-review; it's just done after a paper is posted to a preprint server and made public.

    5. Any peer review that happens after something is public is by definition merely informal. Obviously sometimes informal critiques are very useful, such as Rosie Redfield's critiques of the arsenic life paper on her blog (although she followed that up with a peer reviewed study of her own), but even at their best, they can't do what formal peer review can, which is prevent bad science from ever reaching the public (not that it is 100% free of either false positives or negatives in that regard)

    6. "Any peer review that happens after something is public is by definition merely informal."

      Not always. See for example F1000 Research, which publishes submissions immediately and then solicits formal peer-reviews.

      "They can't do what formal peer review can, which is prevent bad science from ever reaching the public."

      That is a comforting delusion. Pre-publication peer-review doesn't come anywhere close to achieving this. It's not there are a few false positives here and there, it's that whole structure is a house of cards. See for example Classical peer review: an empty gun.

  4. This comment has been removed by the author.

  5. I would be very interested in seeing data showing that pre-prints actually speed up the pace of research in biology.

    I would be happy to be shown I'm wrong, but it seems to me that after an initial burst where manuscripts currently languishing on desks go up on the pre-print servers, the publication cycle would remain largely the same. The difference being that instead of eagerly awaiting the next set of papers from a lab, we are instead eagerly awaiting the next set of pre-prints. The publication cycle hasn't sped up, it has only shifted.

    Except now as a field we are combing through manuscripts that haven't benefited from peer-review.

    1. You assume that every paper is independent of every other paper and all science acts in parallel. But some aspects of science are serial. Suppose there were five studies needed to make a major breakthrough. Suppose further that they had to happen in a specific order with each one needing the publication of the previous one to proceed. If it took two years to carry out the work for each study and one year to publish the results from each then the final discovery would take fifteen years. If the time to publish was cut to three months then the total time for all five studies in series would be 11 years and three months. Sure this is a bit artificial as science is both parallel and serial but changing the time line to publish and share results accelerates science much more than just shifting all of science a bit in one direction.

  6. We ARE the peer review. And, the pace is sped up because once your idea is out there, MANY people can think about it and move forward their thinking and therefore their work. You are right, in the first iteration it is a shift, and it would not affect the pace if only one person read it, but since MANY can move their thinking forward, it does speed the pace. You are shifting an exponential curve, so the pace does quicken.

    1. I'd buy the "the scientific public is the peer review" argument more if it wasn't the case that hardly anyone comments on papers on PLOS ONE, for example.

  7. I fully agree with the idea of pre-prints, mainly because I find it very tiring to have to wait for the long publication process to end and to have to grovel in front of other people's opinion who are not necessarily the people I would want to convey the message to in the first place. When data are finally published, the mind has long moved on to another level of research. The only issue that holds me back is the uncertainty about how journals will respond to pre-publication. Is there an easy way to find out whithout having to read through journal policies which involves a time-consuming effort?