tag:blogger.com,1999:blog-10781944.post8805758323591351574..comments2024-03-28T00:36:36.460-07:00Comments on The Tree of Life: Guest post from Jake Scott: Building trust: a sine qua non for successful acceptance of preprints in the biological sciencesJonathan Eisenhttp://www.blogger.com/profile/07953790938128734305noreply@blogger.comBlogger16125tag:blogger.com,1999:blog-10781944.post-781208930249115542013-06-07T13:39:19.956-07:002013-06-07T13:39:19.956-07:00"Any peer review that happens after something..."Any peer review that happens after something is public is by definition merely informal."<br /><br />Not always. See for example F1000 Research, which publishes submissions immediately and <i>then</i> solicits formal peer-reviews.<br /><br />"They can't do what formal peer review can, which is prevent bad science from ever reaching the public."<br /><br />That is a comforting delusion. Pre-publication peer-review doesn't come anywhere close to achieving this. It's not there are a few false positives here and there, it's that whole structure is a house of cards. See for example <a href="http://breast-cancer-research.com/content/12/S4/S13" rel="nofollow"> Classical peer review: an empty gun</a>.<br />Mike Taylorhttps://www.blogger.com/profile/06039663158335543317noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-38974526172575150272013-06-07T13:16:51.296-07:002013-06-07T13:16:51.296-07:00Any peer review that happens after something is pu...Any peer review that happens after something is public is by definition merely informal. Obviously sometimes informal critiques are very useful, such as Rosie Redfield's critiques of the arsenic life paper on her blog (although she followed that up with a peer reviewed study of her own), but even at their best, they can't do what formal peer review can, which is prevent bad science from ever reaching the public (not that it is 100% free of either false positives or negatives in that regard)Jonathan Badgerhttps://www.blogger.com/profile/04921990886076027719noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-10270586593112352012013-06-06T20:18:11.190-07:002013-06-06T20:18:11.190-07:00So I don't see the need for formal peer review...<i>So I don't see the need for formal peer review going away...</i><br /><br />All the communities we're discussing (economics, physics, math) do have "formal" peer-review; it's just done after a paper is posted to a preprint server and made public.Joe Pickrellhttps://www.blogger.com/profile/03756271491303196763noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-5989011975772989212013-06-05T20:35:22.843-07:002013-06-05T20:35:22.843-07:00You assume that every paper is independent of ever...You assume that every paper is independent of every other paper and all science acts in parallel. But some aspects of science are serial. Suppose there were five studies needed to make a major breakthrough. Suppose further that they had to happen in a specific order with each one needing the publication of the previous one to proceed. If it took two years to carry out the work for each study and one year to publish the results from each then the final discovery would take fifteen years. If the time to publish was cut to three months then the total time for all five studies in series would be 11 years and three months. Sure this is a bit artificial as science is both parallel and serial but changing the time line to publish and share results accelerates science much more than just shifting all of science a bit in one direction.Jonathan Eisenhttps://www.blogger.com/profile/07953790938128734305noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-28714718685702819112013-06-05T13:37:23.110-07:002013-06-05T13:37:23.110-07:00But obviously economics is just a branch of applie...But obviously economics is just a branch of applied math, really, rather than a actual science with messy experimental techniques. Despite being a microbiologist by training, I did a postdoc in a Computer Science department with a strong theoretical focus and things were much like how Krugman describes economics -- everybody read conference proceedings and working papers rather than journals. But that's because the truth of a mathematical proposition is in the pudding -- either the proof is valid or it is not. These days, there are even computer programs that can validate proofs by themselves.<br /><br />Experimental science isn't so simple; unless you have performed similar experiments yourself it is hard to see if the methods described really are reliable, and yet the audience for an experimental paper is much wider than just these people; bioinformaticans use experimental results as raw data, for example. So I don't see the need for formal peer review going away in biology -- or chemistry (another solidly journal-based science).Jonathan Badgerhttps://www.blogger.com/profile/04921990886076027719noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-26967224525514478122013-06-05T11:38:08.202-07:002013-06-05T11:38:08.202-07:00I'd buy the "the scientific public is the...I'd buy the "the scientific public is the peer review" argument more if it wasn't the case that hardly anyone comments on papers on PLOS ONE, for example. Jonathan Badgerhttps://www.blogger.com/profile/04921990886076027719noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-38789280581547884972013-06-05T07:02:13.422-07:002013-06-05T07:02:13.422-07:00See this "list of academic journals by prepr...See this <a href="http://en.wikipedia.org/wiki/List_of_academic_journals_by_preprint_policy" rel="nofollow"> "list of academic journals by preprint policy" </a> Jonathan Eisenhttps://www.blogger.com/profile/07953790938128734305noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-17956181977691076802013-06-05T03:36:39.884-07:002013-06-05T03:36:39.884-07:00I fully agree with the idea of pre-prints, mainly ...I fully agree with the idea of pre-prints, mainly because I find it very tiring to have to wait for the long publication process to end and to have to grovel in front of other people's opinion who are not necessarily the people I would want to convey the message to in the first place. When data are finally published, the mind has long moved on to another level of research. The only issue that holds me back is the uncertainty about how journals will respond to pre-publication. Is there an easy way to find out whithout having to read through journal policies which involves a time-consuming effort? Anonymoushttps://www.blogger.com/profile/02209027084404977468noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-91001937749979908802013-06-04T10:43:17.855-07:002013-06-04T10:43:17.855-07:00We ARE the peer review. And, the pace is sped up ...We ARE the peer review. And, the pace is sped up because once your idea is out there, MANY people can think about it and move forward their thinking and therefore their work. You are right, in the first iteration it is a shift, and it would not affect the pace if only one person read it, but since MANY can move their thinking forward, it does speed the pace. You are shifting an exponential curve, so the pace does quicken.Jacob Scotthttps://www.blogger.com/profile/12788368243256158841noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-39322743200829355072013-06-04T10:05:22.418-07:002013-06-04T10:05:22.418-07:00I would be very interested in seeing data showing ...I would be very interested in seeing data showing that pre-prints actually speed up the pace of research in biology. <br /><br />I would be happy to be shown I'm wrong, but it seems to me that after an initial burst where manuscripts currently languishing on desks go up on the pre-print servers, the publication cycle would remain largely the same. The difference being that instead of eagerly awaiting the next set of papers from a lab, we are instead eagerly awaiting the next set of pre-prints. The publication cycle hasn't sped up, it has only shifted.<br /><br />Except now as a field we are combing through manuscripts that haven't benefited from peer-review.<br />Ryan Kirkbridehttps://www.blogger.com/profile/11743085276457326205noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-55433920284081589862013-06-03T19:13:33.371-07:002013-06-03T19:13:33.371-07:00Putting aside whether math is "self-evident&q...Putting aside whether math is "self-evident" (?), preprints are also the norm in economics and physics. You might be interested in Paul Krugman's discussion of how the economics community stopped caring about journal publications decades ago [<a href="http://krugman.blogs.nytimes.com/2012/01/17/open-science-and-the-econoblogosphere/" rel="nofollow">here</a>]. An excerpt:<br /><br />"[J]ournal publication? Well, tenure committees needed that, but it was so slow relative to the pace of ongoing work that it no longer acted as an information conduit. I presented my paper on target zones at a 1988 conference; by the time it was formally published, in 1991, I had to add a section on the subsequent literature, because there were around 150 derivative papers already out there."Joe Pickrellhttps://www.blogger.com/profile/03756271491303196763noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-16116571464693824332013-06-03T17:59:04.832-07:002013-06-03T17:59:04.832-07:00This comment has been removed by the author.Ryan Kirkbridehttps://www.blogger.com/profile/11743085276457326205noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-90855566004645812752013-06-03T16:16:57.034-07:002013-06-03T16:16:57.034-07:00There is a distinct difference between the natural...There is a distinct difference between the natural sciences and mathematics in presenting evidence and establishing an idea as valid. Mathematical proofs are largely self-evident, whereas the natural sciences rely upon the peer review process for meeting a reasonable burden of proof in defining a key concept. It is not sufficient to presuppose an idea through a non-peer reviewed publication and still receive creative credit for it. Do not consider this distinction as trivial because Nobel Prizes have been won and lost on it, from PCR to elucidation of transcriptional machinery.<br /><br />OA journals exist for individuals who would like to expedite the peer-reviewed publishing process (and they have succeeded admirably). Conferences fulfill their role as a minimalistic pre-print access point. I find it improbable that biology will adopt the policies of math or physics, given current funding exigencies and the need to protect funding sources.Matt Damonhttps://www.blogger.com/profile/06714642774706231676noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-18085699174506412522013-06-03T12:23:33.293-07:002013-06-03T12:23:33.293-07:00Matt Damon writes: "I do not understand the a...Matt Damon writes: "I do not understand the advantages of publishing pre-prints in the biological sciences as it can frequently take years to revise and publish a significant finding in a major journal."<br /><br />That is <i>precisely</i> why posting a pre-print is so important. Unless of course you think it doesn't matter whether people actually read and use your work.<br />Mike Taylorhttps://www.blogger.com/profile/06039663158335543317noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-67748274076444239412013-06-03T09:53:09.018-07:002013-06-03T09:53:09.018-07:00The advantage is that you can disseminate your fin...The advantage is that you can disseminate your findings that much earlier - and the pace of science can quicken. You are right that there is, as of yet, no precedent for receiving credit for preprints, but we hope to change that...<br /><br />In a perfect world (which we aren't in, but are working towards!) the 'credit' is dissemination itself. Further, there is more and more a movement towards article level metrics being the coin of the realm, and these are a part of the new biological repositories.Jacob Scotthttps://www.blogger.com/profile/12788368243256158841noreply@blogger.comtag:blogger.com,1999:blog-10781944.post-48925821790669240572013-06-03T09:36:05.520-07:002013-06-03T09:36:05.520-07:00I do not understand the advantages of publishing p...I do not understand the advantages of publishing pre-prints in the biological sciences as it can frequently take years to revise and publish a significant finding in a major journal. There is no precedent for receiving credit for a pre-print; why should we expect that to change any time soon in a competitive funding atmosphere?Matt Damonhttps://www.blogger.com/profile/06714642774706231676noreply@blogger.com