Painful, kind of hilarious, typesetting error in a #PLOSOne paper from my lab - should I try to get it fixed?

Saw this tweet:

@phylogenomics Funny/interesting typesetting trainwreck in equation 4 of PLOS PhyEco paper. English letters were replaced by Greek!
— Michael Hall (@mikehall_dal) March 30, 2015

So I went and found the paper.

Wu D, Jospin G, Eisen JA (2013) Systematic Identification of Gene Families for Use as “Markers” for Phylogenetic and Phylogeny-Driven Ecological Studies of Bacteria and Archaea and Their Major Subgroups. PLoS ONE 8(10): e77033. doi:10.1371/journal.pone.0077033

And discovered what he was pointing to

Then I looked at the Pubmed Central version of the paper and it was the same.  So I wnet and found the arXiv version of the paper and it looked correct.

Story Behind the Paper: Comparative Analysis of Functional Metagenomic Annotation and the Mappability of Short Reads (by Rogan Carr and Elhanan Borenstein)

Here is another post in my "Story Behind the Paper" series where I ask authors of open access papers to tell the story behind their paper.  This one comes from Rogan Carr and Elhanan Borenstein.  Note - this was crossposted at microBEnet.  If anyone out there has an open access paper for which you want to tell the story -- let me know.

---

We’d like to first thank Jon for the opportunity to discuss our work in this forum. We recently published a study investigating direct functional annotation of short metagenomic reads that stemmed from protocol development for our lab. Jon invited us to write a blog post on the subject, and we thought it would be a great venue to discuss some practical applications of our work and to share with the research community the motivation for our study and how it came about.

Our lab, the Borenstein Lab at the University of Washington, is broadly interested in metabolic modeling of the human microbiome (see, for example our Metagenomic Systems Biology approach) and in the development of novel computational methods for analyzing functional metagenomic data (see, for example, Metagenomic Deconvolution). In this capacity, we often perform large-scale analysis of publicly available metagenomic datasets as well as collaborate with experimental labs to analyze new metagenomic datasets, and accordingly we have developed extensive expertise in performing functional, community-level annotation of metagenomic samples. We focused primarily on protocols that derive functional profiles directly from short sequencing reads (e.g., by mapping the short reads to a collection of annotated genes), as such protocols provide gene abundance profiles that are relatively unbiased by species abundance in the sample or by the availability of closely-related reference genomes. Such functional annotation protocols are extremely common in the literature and are essential when approaching metagenomics from a gene-centric point of view, where the goal is to describe the community as a whole.

However, when we began to design our in-house annotation pipeline, we pored over the literature and realized that each research group and each metagenomic study applied a slightly different approach to functional annotation. When we implemented and evaluated these methods in the lab, we also discovered that the functional profiles obtained by the various methods often differ significantly. Discussing these findings with colleagues, some further expressed doubt that that such short sequencing reads even contained enough information to map back unambiguously to the correct function. Perhaps the whole approach was wrong!

We therefore set out to develop a set of ‘best practices’ for our lab for metagenomic sequence annotation and to prove (or disprove) quantitatively that such direct functional annotation of short reads provides a valid functional representation of the sample. We specifically decided to pursue a large-scale study, performed as rigorously as possible, taking into account both the phylogeny of the microbes in the sample and the phylogenetic coverage of the database, as well as several technical aspects of sequencing like base-calling error and read length. We have found this evaluation approach and the results we obtained quite useful for designing our lab protocols, and thought it would be helpful to share them with the wider metagenomics and microbiome research community. The result is our recent paper in PLoS One, Comparative Analysis of Functional Metagenomic Annotation and the Mappability of Short Reads.

The performance of BLAST-based annotation of short reads across the bacterial and archaeal tree of life using the 'top gene' protocol. See the manuscript for full details. Figure and text adapted from: Carr R, Borenstein E (2014) Comparative Analysis of Functional Metagenomic Annotation and the Mappability of Short Reads. PLoS ONE 9(8): e105776 

Guest post by Jay Kaufman: A Bad Taste That Keeps Not Getting Any Better....

Guest post by Jay Kaufman.  Jay and I have been having some email discussions about a paper in PLOS One.  I offered to let him write a guest post to my blog about his concerns.

-------------------------------

Jonathan Eisen already posted on this blog about a PLoS ONE paper by Mason et.. published on 23 October 2013.  And he posted related comments on the PLoS ONE website.  I also commented at this site, in reference to his comments and the authors' response. The purpose of my comments here are just to review those concerns and comment additionally on the PLos ONE response and what this means for the journal's publication model and the progress of science.

The paper by Mason and colleagues analyzes data on 48 people in each of 4 self-identified ethnic groups (African American, Caucasian, Chinese, and Latino). These study subjects are apparently volunteers, and the paper only states that they are non-smokers over 18 years old who are free of a list of diagnosed diseases and who have not recently had their teeth cleaned. Based on the text of the published paper, there is no consideration of their age, diet, social class, or even gender. The authors culture bacterial species from the study subjects and process the data through an algorithm that maximizes the prediction of racial group membership based on these measured data.

The prediction is moderately successful, but this could result from any number of unsurprising reasons. For example, if alcohol consumption affects some particular bacterial species, and whites drink more than Asians in central Ohio, then whatever species is diminished by alcohol exposure would help predict that a sample was from a white rather than from an Asian volunteer. And likewise for any of a million possible lifestyle, social class and demographic differences.  In fact, this is a general problem with data mining exercises that Lazer et al describe in the current issue of Science.

The authors provide no information about how balanced this sample is with respect to any of these variables. Maybe the 48 Hispanics are younger than the 48 whites on average, or have more tooth decay or eat more refined sugar or any of a million other possibilities. The fact that these countless potentially imbalanced factors get represented in the oral bio-environment hardly seems surprising, and the fact that these behaviors and exposures might be differential by race is an observation that is completely trivial from a sociological perspective.

My concern here, however, the authors assert in the published text that these differences do not arise from any of these myriad environmental factors, but from some innate genetic characteristics of the groups. In the Discussion section on page 3 they state that "ethnicity exerts a selection pressure on the oral microbiome, and...this selection pressure is genetic rather than environmental, since the two ethnicities that shared a common food, nutritional and lifestyle heritage (Caucasians and African Americans) demonstrated significant microbial divergence." Here is a remarkable statement, that Caucasians and African Americans experience no differential dietary or lifestyle factors. It is directly contradicted by thousands of published papers in sociology, epidemiology and anthropology that document these differences for reasons of culture, geographic origin, social class and discrimination.

Jonathan Eisen's post directly confronted the authors on this point, and they responded with the following explanation:

"Subjects were selected based on extensive questionnaire surveys and clinical examinations to ensure homogeneity. These questionnaires evaluated educational level, socio-economic status, diet and nutritional history, systemic health status, oral hygiene habits and dental visits, among other things."

This is surely an important statement about the research design, but the problem is that it appears nowhere in the peer-reviewed text of the published paper. What exactly do the authors mean when they insist that the study subjects were perfectly balanced on factors such as socio-economic status and nutritional history? These complex social and lifestyle variables are notoriously difficult to define and measure. While the authors describe the laboratory techniques in baroque detail, they do not even mention in the published paper that they measured these factors, let alone how these variables were defined and considered in the analysis. This represents a profound limitation for the reader in assessing the validity of these measures and adjustments, and therefore the adequacy of the claimed "homogeneity". The complete omission of these crucial aspects of the analysis in the paper prevents the reader from investing much confidence in the boldly stated claim that observed differences are "genetic rather than environmental" in origin.

I expressed these concerns in my own post at the journal website on 14 November 2013, but the authors did not respond.  Therefore, at Jonathan's suggestion, I addressed this concern to the PLoS ONE editors in an e-mail on 22 November 2013. I got passed along from one editor to another, and finally I got a very nice response from Elizabeth Silva on 4 December 2013. She wrote:

I wanted to let you know that I am discussing this article and your concerns with both the Academic Editor and the authors, as well as with my colleagues. We take such concerns very seriously and will ensure that appropriate measures are taken to correct any errors or discrepancies. 

Then I waited.  After 2 months I had heard nothing, so I wrote to Dr. Silva again asking for any word on progress, but received no reply.  So I waited another month.

On 4 March it had been 3 months since the note from PLoS ONE promising appropriate measures to correct any errors or discrepancies, so I wrote again, this time a bit more insistently.  This did message did finally generate a quick and reassuring response from Dr. Silva:

I really am very sorry for the extended delay in replying to you, and for my neglect in providing you with an update. Following your correspondence I contacted the authors to ask them for additional information relating to their statement that they corrected for confounding factors, and details of these methods. They promptly replied with a table of details of the baseline variables that they corrected for, and that they described in the comment on their article (see attached), as well as an additional correction to one of their figure legends. I then contacted the Academic Editor, Dr. Indranil Biswas with the full details of your concerns, as well as the table sent by the authors and the correction they requested for their figure legend. We asked Dr. Biswas to revisit the manuscript, in light of this new information, and he has informed us that he feels the conclusions of the manuscript are sound. We will now work with the authors to draft and issue a formal correction to the published article to update the methods to include the table, and to amend the figure legend in question.

The table that Dr. Silva forwarded displayed a list of variables and a p-value for some kind of test between the values in the 4 race groups. The test is not specified (t-test? chi-square test?) but presumably it is for any difference in means or proportions between the 4 groups.  Most of the p-values are large, indicating little evidence for any difference between the groups in income, age, education, or frequency of tooth-brushing, etc.  Based on this table, the populations differed only in their diets, which were characterized as "Asian Diet", "Hispanic Diet" and "American Diet".  Not unsurprisingly, the Asians were significantly more likely to report an "Asian Diet" and the Hispanics were significantly more likely to report a "Hispanic Diet".  The Blacks and Whites had similar reported consumption of the "American Diet", which presumably was the basis for the authors' assertion that these groups have identical social environments.

To date, there has been no correction made to the Mason et al paper at the PLoS ONE website.  Therefore it is perhaps somewhat premature to speculate on how the authors will address the concern voiced by Jonathan Eisen's posted comment and blog post that balance across a handful of measured covariates does not in any way imply balance across all relevant factors except for genetics. Indeed, it has long been argued in the epidemiology literature that one cannot make indirect inferences about genes by measuring and adjusting for a few environmental exposures and attributing all remaining differences to genes.  The argument that Blacks and Whites in Ohio experience identical environments is clearly false, even if a handful of measured covariates are not significantly different in their small convenience sample, the exact origin of which is still obscure.

There are many observations that can be made from this episode. I offer just a few:

1. These authors are assiduous in describing their lab techniques, but regarding the study design and analysis they are quite cavalier.  Presumably the reviewers were not population scientists, and so they failed to point out these embarrassing flaws. This raises the question of whether a multidisciplinary journal such as PLoS ONE has the relevant expertise to screen out scientifically invalid papers. The fact that Dr. Silva suggested that the authors' table of covariates and p-values solves the issue demonstrates a wide gap of understanding.  Specialty journals that handle a narrow disciplinary range are not faced with this kind of crisis of competence.
2. PLoS ONE is such a large operation with so many papers, that quality control seems to suffer. These beleaguered editors are responsible for an enormous publishing volume. Has quantity overwhelmed quality to the extent that gross errors of logic slip through? Months later, the Mason paper has been accessed thousands of times and generated a great deal of media attention, and yet no correction or erratum has appeared, despite the fact that the authors freely admit that the methods in their published paper are not accurate. 
3. The publishing model gives PLoS ONE a big incentive (almost $2000) to accept a paper, but once it is published, little incentive to correct or withdraw it. 

Sadly, this is not an isolated example.  This week, PLoS ONE published a paper by Wikoff et al which makes a similar logical gaffe about observed racial difference proving a genetic difference.  We could post a comment online, but it seems that nobody (neither the authors nor the editors) has much time to spend monitoring such comments, nor much incentive to care about them.  The authors have their publication, the journal has its $2000, and another tiny piece of horrific misinformation has been released into the world.  The basic philosophy of PLoS ONE is to reduce the gate-keeper role of scientific publication. I am starting to become convinced that a little gate-keeping is not such a bad idea.

Bad Ome-like word of the week: symbiome

Well I got pointed to this paper: Transgenerational Transmission of the Glossina pallidipes Hytrosavirus Depends on the Presence of a Functional Symbiome

And as many might guess - the word "symbiome" did not sit well with me.  Alas, they don't define it in the paper.  So I can't really quibble with their definition.  But I did find some other stuff out there that, well, at least helps see how other people are using the word:

• Symbiomism - Wikipedia, the free encyclopedia
• iPhylo: Visualising the symbiome: hosts, parasites, and the Tree of Life
• Observing the Coral Symbiome Using Laser Scanning Confocal
• "Symbiomes: Systems Metagenomics of Host Microbe interactions ...

I can't really tell from most of these if "symbiome" can be a useful term or not sometimes.  Certainly the iPhylo example above has potential.  But in general, the word seems awkward at best.  Now - as far as I can tell, nobody is using it in the context of "genomics" so this does not fit in with my "badomics" obsession.  But it still does not make me feel warm and fuzzy so I am going to give it a pseudo-award - the Bad Ome-like word award.

If you are into microbial diversity and use R this may be worth checking out: phyloseq

Just got pointed (by the lead author) to this new paper: PLOS ONE: phyloseq: An R Package for Reproducible Interactive Analysis and Graphics of Microbiome Census Data

Abstract:

Background
The analysis of microbial communities through DNA sequencing brings many challenges: the integration of different types of data with methods from ecology, genetics, phylogenetics, multivariate statistics, visualization and testing. With the increased breadth of experimental designs now being pursued, project-specific statistical analyses are often needed, and these analyses are often difficult (or impossible) for peer researchers to independently reproduce. The vast majority of the requisite tools for performing these analyses reproducibly are already implemented in R and its extensions (packages), but with limited support for high throughput microbiome census data.
Results
Here we describe a software project, phyloseq, dedicated to the object-oriented representation and analysis of microbiome census data in R. It supports importing data from a variety of common formats, as well as many analysis techniques. These include calibration, filtering, subsetting, agglomeration, multi-table comparisons, diversity analysis, parallelized Fast UniFrac, ordination methods, and production of publication-quality graphics; all in a manner that is easy to document, share, and modify. We show how to apply functions from other R packages to phyloseq-represented data, illustrating the availability of a large number of open source analysis techniques. We discuss the use of phyloseq with tools for reproducible research, a practice common in other fields but still rare in the analysis of highly parallel microbiome census data. We have made available all of the materials necessary to completely reproduce the analysis and figures included in this article, an example of best practices for reproducible research.
Conclusions
The phyloseq project for R is a new open-source software package, freely available on the web from both GitHub and Bioconductor.

Seems similar in some ways to the WATERs Kepler Workflow that we released a few years ago.   Anyway - if you use R and are into microbial diversity studies this may be worth checking out.  As a bonus - it has a strong emphasis on reproducibility - which is a good thing.

PLoSOne paper: Parallel polymorphisms for pepper population phylogenetics, from #UCDavis, not #Pepperspray

Interesting new paper from colleagues at UC Davis: PLOS ONE: Characterization of Capsicum annuum Genetic Diversity and Population Structure Based on Parallel Polymorphism Discovery with a 30K Unigene Pepper GeneChip.

Press release is here: http://www.news.ucdavis.edu/search/news_detail.lasso?id=10497

Good to see something on peppers from UC Davis not about spraying.

A title and figures say it all: An In-Depth Analysis of a Piece of Shit (note - a bit gross)

I had heard about this paper a few weeks ago and was just reminded about it by a friend. To understand this paper all you really need are the title and Figure 1 and 2 which are below. Enjoy.

An In-Depth Analysis of a Piece of Shit: Distribution of Schistosoma mansoni and Hookworm Eggs in Human Stool.

Important & neglected aspect of lab studies of animals : effect of habitat change on microbiome

By Aaron Logan via Wikipedia 

Very very interesting paper came out recent from some colleagues at UC Davis: PLOS ONE: Routine Habitat Change: A Source of Unrecognized Transient Alteration of Intestinal Microbiota in Laboratory Mice

Abstract: The mammalian intestine harbors a vast, complex and dynamic microbial population, which has profound effects on host nutrition, intestinal function and immune response, as well as influence on physiology outside of the alimentary tract. Imbalance in the composition of the dense colonizing bacterial population can increase susceptibility to various acute and chronic diseases. Valuable insights on the association of the microbiota with disease critically depend on investigation of mouse models. Like in humans, the microbial community in the mouse intestine is relatively stable and resilient, yet can be influenced by environmental factors. An often-overlooked variable in research is basic animal husbandry, which can potentially alter mouse physiology and experimental outcomes. This study examined the effects of common husbandry practices, including food and bedding alterations, as well as facility and cage changes, on the gut microbiota over a short time course of five days using three culture-independent techniques, quantitative PCR, terminal restriction fragment length polymorphism (TRFLP) and next generation sequencing (NGS). This study detected a substantial transient alteration in microbiota after the common practice of a short cross-campus facility transfer, but found no comparable alterations in microbiota within 5 days of switches in common laboratory food or bedding, or following an isolated cage change in mice acclimated to their housing facility. Our results highlight the importance of an acclimation period following even simple transfer of mice between campus facilities, and highlights that occult changes in microbiota should be considered when imposing husbandry variables on laboratory animals.

I personally think that we as a community are going to have to come to grips with the fact that the microbial communities in / on research organisms (of all kinds) may have a profound effect on experimental results.  This may explain many of the differences seen in experiments between facilities or over time within a facility.  In general, I think either controlling the microbes more carefully in lab experiments (e.g., using defined flora) or at least monitoring them is going to be very important to best interpret studies of plants and animals in the lab (or for that matter - in the field too).  Anyway -this paper is a tiny window into one of the ways that controlling for microbiomes may be important in lab studies.


Citation: Ma BW, Bokulich NA, Castillo PA, Kananurak A, Underwood MA, et al. (2012) Routine Habitat Change: A Source of Unrecognized Transient Alteration of Intestinal Microbiota in Laboratory Mice. PLoS ONE 7(10): e47416. doi:10.1371/journal.pone.0047416

PCR amplification and pyrosequencing of rpoB as complement to rRNA

Figure 1. Number of OTUs as
 a function of fractional sequence difference
 (OTU cut-off) for the 16S rRNA marker
 gene (A) and the rpoB marker gene (B).

Interesting new paper in PLoS One: PLoS ONE: A Comparison of rpoB and 16S rRNA as Markers in Pyrosequencing Studies of Bacterial Diversity

In the paper they test and use PCR amplification and pyrosequencing of the rpoB gene for studies of the diversity of bacteria. Due to the lower level of conservation of rpoB than rRNA genes at the DNA level they focused on proteobacteria. It seems that with a little perseverance once can get PCR for protein coding genes to work reasonably well for even reasonably broad taxonomic groups (not totally new here but I am not aware of too many papers doing this with pyrosequencing). Anyway, the paper is worth a look.


Citation:
 Vos M, Quince C, Pijl AS, de Hollander M, Kowalchuk GA (2012) A Comparison of rpoB and 16S rRNA as Markers in Pyrosequencing Studies of Bacterial Diversity. PLoS ONE 7(2): e30600. doi:10.1371/journal.pone.0030600 Vos, M., Quince, C., Pijl, A., de Hollander, M., & Kowalchuk, G. (2012). A Comparison of rpoB and 16S rRNA as Markers in Pyrosequencing Studies of Bacterial Diversity PLoS ONE, 7 (2) DOI: 10.1371/journal.pone.0030600

To be discussed in journal club here today: eukaryotic metatranscriptomics

Going to be discussing this in a journal club today

Will post some comments / notes later / as I have them but any comments from others welcome too ...

PLoS ONE: Metatranscriptomics Reveals the Diversity of Genes Expressed by Eukaryotes in Forest Soils

Note - am copying the whole text here to mark it up a bit since it it awkward to try and mark it up at the PLoS One site

PLoS picture of the day: Simon Chan from #UCDavis sports #PLoSOne shirt when presenting to @BillGates

Good to see here that Simon Chan, from UC Davis, knows what is the best outfit to wear to present his work to Bill Gates.  

Thanks to Simon for sending me the photo and to the Bill and Melinda Gates Foundation for approving it'sits use here.

Calling all authors, bloggers, reporters - please help with aggregating discussions of scientific papers

There has been a lot of hand wringing over whether we should and if we should how should we link discussions about scientific papers with the papers themselves.  For example, if someone writes a news story about a paper in BMC Genomics - should the online version of the paper show links to the news story?  I think so, as so many others.  If someone writes a blog post discussing the paper, should there be a tracked link on the journal site?  Again, I think so.  I think this is even more important in social media discussions of papers, which I find fascinating much of the time.   Very few people go to journal sites and post comments and open up discussions of papers.  But lots of people post comments to twitter, facebook, and other social media sites.  So why not bring those posts into the fold?

Now there are lots of efforts out there to collect comments about papers.  Faculty of 1000.  The Third Reviewer.  Research blogging. And much more. For other discussions of the issue see:

• Science in the Open » Blog Archive » The future of research ...
• Implementing the “Publication as Aggregation” - Cameron Neylon
• Aggregating, Tagging and Integrating Biodiversity Research
• How to Publish a Scientific Comment in 1 2 3 Easy Steps
• Soft peer review: Social software and distributed scientific evaluation
• and much much more

I am not really going to get into a discussion of all the ideas out there in this area.  Some are good.  Some are bad.  Some are probably both.  I personally think aggregation is going to be a very useful tool in post publication sharing and discussion and searching.  But that is not per se what I am here to talk about.  What I am here to talk about is what anyone can do right now to help with this in a very simple way.

My simple suggestion:
If you see ANY online discussion of a paper - a news story - a blog - even some smaller thread somewhere.  Find the journal article online and use the comments function is the journal has one to post  a comment saying "There was a news story discussing this paper in the New York Times.  See ...."  Or something like that.  And presto, people who go see the paper online will also have potential to find the link you post.

I have been doing this for a while.  It is relatively easy for PLoS Papers.  For example for my paper on "Stalking the Fourth Domain in Metagenomic Data" I posted all sorts of links using the PLoS One comment function.  I posted links to my blog.  I posted links to positive news stories.  I posted links to critiques.  Anything I could find.

And this worked out pretty well.

ISI - late to index #PLoS One but now marketing that they do so

Well, just a mini post here. In case you did not know - PLoS One is now being indexed by ISI (see their announcement: PLoS ONE and see the PLOS blog post here
and see Erik Svensson's blog post for an interesting take) and will get an impact factor and be in their Citation Index and all such things. Now mind you, I think "Impact Factor" is a silly thing overall in that we should evaluate papers not journals per se.

So why am I writing this - because I find it pretty funny that despite being slow to recognize PLoS One ISI is now promoting the fact that they are indexing PLoS One on their home page. See the screen capture above.

Please - bash my latest paper - for the benefit of humanity

My lab has a new paper that just came out on the sequencing and analysis of the genome of a pretty cool (or hot actually) bacterium, Thermomicrobium roseum, which was isolated from a Toadstool Spring, an alkaline siliceous hotspring in Yellowstone National Park. This paper is from a grant we had when I was at TIGR as part of the "Assembling the Tree of Life" program at NSF. Our grant was focused on generating genome sequences from phyla of bacteria for which no genomes were available.

At the time this species was a representative of a phylum that had no genomes. After we started sequencing, the phylum was dissolved, but never mind that for now. We report what I think are some very interesting things in the paper. Among them:

• We report the first example of a plasmid that encodes all the genes needed for chemotaxis including all the genes for making a flagellum. Given that they are on a plasmid this suggests that motility could be easily transfered between species.
• We report experimental work and genome analysis that helps understand the novel membrane and cell wall structure in this species.
• This is the first thermophile known to oxidize carbon monoxide
  

But I am not writing per se about the things I like about our paper. I am instead asking people out there to find things wrong with our paper. Why am I doing this? Because this paper is part of a broader experiment in publishing in that it is in PLoS One. And one of the main benefits of PLoS One is the features that allows commenting on publications. I personally believe such features are part of the future of scientific publication. But it is currently unclear just how effectively such commenting features are used (note Euan Addie is doing a survey about comments on PLoS One papers here).

So I am offering up my paper as a case study. If you comment and ask questions or make critiques, I will try to respond. And if you think something in our paper is wrong or weird, please say so. If you think something in our paper is supported by other work we do not cite, please say this too. If you have anything useful to say, please make comments.

How do you do this?

• Go to the paper at the PLoS One Web Site.
• In the upper right click on "Login" if you have an account or "Create account" if you do not.
  
• Return to the paper once you are logged in
• Find some part of the text you want to comment on
  
• Highlight that text and click over on the right "Add a note" or "Make a comment"
  
• Fire away.
  

I see PLoS in everything

Seen recently at the California Railroad Museum. I guess the people from Nature were right - PLoS One is leaving its mark everywhere.

Only Nature could turn the success of PLoS One into a model of failure

Now, mind, you I like Nature as a publishing unit. They publish some very fine journals. Now, most of them are not Open Access, so I choose not to publish there if I can avoid it. But I still like them. And many of the editors and reporters there are excellent - smart, creative, insightful and such. But Nature the publisher can also be completely inane when it comes to writing about Open Access and PLoS. In a new article by Declan Butler, Nature takes another crack at the PLoS "publishing model"

The problem with PLoS now is ... wait for this ... the success of PLoS One. PLoS One it turns out is publishing a lot of papers (including one by me, today). And bringing in a decent amount of money to PLoS apparently (note for full disclosure - I am involved in PLoS Biology as "Academic Editor in Chief" and PLoS Computational Biology as an Academic Editor ... although I should note I am not involved in financial discussions at PLoS in any way).

So why is the success of PLoS One a problem? Well, because it allows Nature to do the old good cop bad cop routine and to write, again, about the "failings" of the PLoS publication model. Now, mind you, the article does not quote a single source for what the PLoS publication model is. But they do say it has failed. From what I can tell here is the logic of the failure argument:

1. Nature believes PLoS' model for success revolved solely around PLoS Biology and PLoS Medicine and some of the other other PLoS journals being self sustaining after a few years.
2. Analysis of some financial information suggests that PLoS Biology and Medicine currently are not breaking even
3. PLoS One is apparently wildly successful and thus is brining in some money to PLoS.
4. PLoS One publishes a lot of papers (they discuss this a bit and imply that this is a bad thing because some of the papers must be bad. Note - they do not back this up with any evidence. Silly for me to ask a science journal to use evidence)
5. Therefore, the entire PLoS Publication model is a failure.

The problems with this logic are, well, large. Here are some:

1. Does Nature really think that there ever was a single "model" for how PLoS should be evaluated?
2. If so, where is the documentation of what this model actually was?
3. Even if there was a PLoS model and even if it turns out to be not exactly what PLoS is doing now, what is the big deal? If you were a stockholder of any company and they told you "we are never going to change our business model no matter what happens in the world around us" I would recommend you not buy their stock. It is simply farcical to expect any entity to stick to a single simple model forever.
4. Does not Nature supplement some of their bigger journals with their higher volume other journals?
5. Most companies these days use high profile entities such as PLoS Biology and PLoS Medicine to attract attention to other portions of their company in order to help bring in money. Is this somehow not allowed by PLoS? Doesn't Nature do the same thing?
6. If you look at the figure Nature shows of PLoS $$$, it shows income rising in 2007 and expenses going down. How did that get turned into a bad thing?

So - I still do like Nature publishing because much of the time it has high quality stuff. It even has high quality stuff commenting/criticizing the Open Access movement and pointing out some of the challenges with it. But this article by Butler is not an impressive piece of work. I really wanted to give him an award but could not think of what to give.

See also (thanks to Bora for pointing out a bunch of these links)

• FrontalBlogotomy
• Drug Monkey: Nature offers a completely objective and unbiased review of PLoS.
• Gnxp
• Greg Laden
• Grrrlscientist
• Peter Suber
• Alex Holcombe/Ceptional Nature targets financial weakness of PLoS journals
  
• Plausible Accuracy
• Bora at Blog Around the Clock
• Bill Hooker
• Health Science and Libraries Blog
• Questionable Authority
• WordMunger
• Neurotic Physiology When Journals Pounce
• Class of the business models from A Man with a PhD
  
• Bjorn Brembs
• Hank at Scientific Blogging
• Timo Hannay at Nature
• Nature Re-Attacks Open Access and PLoS
  
• Nature vs. PLoS by John Dupuis
  
• Commentary: Open access equals bulk publishing? by Lars Jensen
  
• The ‘threatening’ success of PLoS; now heard aloud!
• Science Literature and Changing Times
• The future of scientific publishing
• PLoS, Nature and the community backlash
• PLoS In Nature : The Big Picture
• New salvos in the publishing wars
• Science Commons’ John Wilbanks on the Declan Butler article
  
• PLoS and the future of publishing - as framed by Nature
  
• OA and the broader social good
  
• The economics of open access, part two
  
• What’s the deal with PLoS One?
  

My first PLoS One paper .... yay: automated phylogenetic tree based rRNA analysis

Well, I have truly entered the modern world. My first PLoS One paper has just come out. It is entitled "An Automated Phylogenetic Tree-Based Small Subunit rRNA Taxonomy and Alignment Pipeline (STAP)" and well, it describes automated software for analyzing rRNA sequences that are generated as part of microbial diversity studies. The main goal behind this was to keep up with the massive amounts of rRNA sequences we and others could generate in the lab and to develop a tool that would remove the need for "manual" work in analyzing rRNAs.

The work was done primarily by Dongying Wu, a Project Scientist in my lab with assistance from a Amber Hartman, who is a PhD student in my lab. Naomi Ward, who was on the faculty at TIGR and is now at Wyoming, and I helped guide the development and testing of the software.

We first developed this pipeline/software in conjunction with analyzing the rRNA sequences that were part of the Sargasso Sea metagenome and results from the word was in the Venter et al. Sargasso paper. We then used the pipeline and continued to refine it as part of a variety of studies including a paper by Kevin Penn et al on coral associated microbes. Kevin was working as a technician for me and Naomi and is now a PhD student at Scripps Institute of Oceanography. We also had some input from various scientists we were working with on rRNA analyses, especially Jen Hughes Martiny

We made a series of further refinements and worked with people like Saul Kravitz from the Venter Institute and the CAMERA metagenomics database to make sure that the software could be run outside of my lab. And then we finally got around to writing up a paper .... and now it is out.

You can download the software here. The basics of the software are summarized below: (see flow chart too).

• Stage 1: Domain Analysis


• Take a rRNA sequence
• blast it against a database of representative rRNAs from all lines of life
• use the blast results to help choose sequences to use to make a multiple sequence alignment
• infer a phylogenetic tree from the alignment
• assign the sequence to a domain of life (bacteria, archaea, eukaryotes)


• Stage 2: First pass alignment and tree within domain
• take the same rRNA sequence
• blast against a database of rRNAs from within the domain of interest
• use the blast results to help choose sequences for a multiple alignment
• infer a phylogenetic tree from the alignment
• assign the sequence to a taxonomic group


• Stage 3: Second pass alignment and tree within domain
• extract sequences from members of the putative taxonomic group (as well as some others to balance the diversity)
• make a multiple sequence alignment
• infer a phylogenetic tree

From the above path, we end up with an alignment, which is useful for things such as counting number of species in a sample as well as a tree which is useful for determining what types of organisms are in the sample.

I note - the key is that it is completely automated and can be run on a single machine or a cluster and produces comparable results to manual methods. In the long run we plan to connect this to other software and other labs develop to build a metagenomics and microbial diversity workflow that will help in the processing of massive amounts of sequence data for microbial diversity studies.

I should note this work was supported primarily by a National Science Foundation grant to me and Naomi Ward as part of their “Assembling the Tree of Life” Program (Grant No. 0228651). Some final work on the project was funded by the Gordon and Betty Moore Foundation through grant #1660 to Jonathan Eisen and the CAMERA grant to UCSD.

Wu, D., Hartman, A., Ward, N., & Eisen, J. (2008). An Automated Phylogenetic Tree-Based Small Subunit rRNA Taxonomy and Alignment Pipeline (STAP) PLoS ONE, 3 (7) DOI: 10.1371/journal.pone.0002566

Oakley Keeping his Eye on Evolution

I saw an interesting talk today and thought I would post on it. I was not expecting to be able to go to the talk as I was supposed to be at a workshop in San Francisco but had to bail on it because my kids have been sick. But I did go in to Davis for a brief spell to go to a seminar by Todd Oakley who gave a talk here today.

It was a generally insightful phylogenomic tour of the evolution of opsins and eyes in animals. He also mentioned a paper he had recently in PLoS One on Animal Opsin evolution. This is from the work of his graduate student David Plachetzki and it does a nice job of doing "phylogenomic" analysis in the way I think of phylogenomics -- that is -- a integration of evolutionary and genomic analyses. (NOTE - I think it is kind of lame that people use the term phylogenomics, which I coined by the way, to refer to "using genomes to infer evolutionary trees). It is a paper worth checking out if you are interested in the origin of novelty.

I am putting links here to some of their figures and embedding them in this blog because, well, I can since PLoS One uses a broad Creative Commons license. For example - see Figure 6 from their paper below (the citation is Plachetzki DC, Degnan BM, Oakley TH (2007) The Origins of Novel Protein Interactions during Animal Opsin Evolution. PLoS ONE 2(10): e1054. doi:10.1371/journal.pone.000105).

Figure 6. Ancestral state reconstruction of G protein-binding interactions for each metazoan opsin-mediated phototransduction cascade obtained by simulated mutational mapping [64] (see methods).For each class of opsin, the P value of the reconstructed ancestral G α interactions is represented in pie graphs. Ancestral G protein interactions in phototransduction cascades mediated by ciliary, rhabdomeric and G_o opsins can be significantly resolved (P>0.95) but the ancestral states of the rhabdomeric+G_o, and cnidops clades are equivocal. ML state reconstructions shown for each node as colored branches. Red, G_i/t; Blue, G_q; Green, G_o; Black, no G protein interaction (as is the case for RGR/Retinochrome opsins); Grey, equivocal reconstruction from ML. Reconstructed ancestral amino acid motifs of the 4th cytoplasmic loop region of opsin are shown along branches in logos. Maximum vertical height scales to P = 1.0. We obtained clear reconstructed states for most of the residues in a conserved tripeptide motif (residues 310, 311 and 312, horizontal bar) for the ciliary, rhabdomeric and G_o /RGR nodes. For the most part, the remainder of the residues in can be unequivocally reconstructed to the level of Dayhoff classes. B = HRK, X = LIVM, J = GATSP, Z = DENQ..

























See also Figure 2

"Figure 2. Unrooted metazoan-wide phylogeny of opsins, new cnidarian genes in bold, branches proportional to substitutions per site. Circles at nodes indicate Bayesian posterior probabilities (White = 1.0, Red>0.90, Blue>0.80, Green>0.70, Yellow>0.60, Black>0.50). cil = ciliary, rh = rhabdomeric."

Open Access dinosaurs and way to go Paul Sereno

Well, I met Paul Sereno, the dinosaur hunter, for the first time at SciFoo camp (for more about that see here). I confess I was skeptical when he said he was committed to Open Access. But now he has really proven his OA chops. He has a new paper in PLoS One on some friggin cool dinosaur fossils.

The paper is "Structural Extremes in a Cretaceous Dinosaur" by Paul C. Sereno^1*, Jeffrey A. Wilson², Lawrence M. Witmer³, John A. Whitlock², Abdoulaye Maga⁴, Oumarou Ide⁴, Timothy A. Rowe⁵

Check it out at PLoS One.

GME 2007 - Getting feedback on PLoS ONe

While at the GME meeting - I have now been approached by 3 people saying they really liked the discussion I initiated on PLoS One for a paper on metagenomics a few weeks ago. They were not saying they liked my comments but that there was an active discussion about some important topics. I think the function on PLoS One has great potential to engage the broad scientific community in discussions that might have previously been limited to journal clubs. So it is nice to see (1) that people are reading stuff on PLoS One and (2) that they seem to like the commenting function.