The microbiome in the news: risk of overselling but not always bad coverage

Well, in the space of about five or so years we have gone from everyone ignoring the "cloud" of microbes that live in and on various plants and animals (the so called microbiomes of these species) to everyone now basically implying that the microbiomes do EVERYTHING.  Over the last few years I started to get stressed about this and started giving out "Overselling the microbiome" awards here.  Some previous posts on this topic include:

• Overselling the microbiome award: Scientists look to mummies for obesity cure
• Probiotics are the new viagra & the risks of overselling of probiotics
• Probiotic use spreading, lots of money being made, known benefits still murky
• Overselling the microbiome award #2: The Marshall Protocol
• Overselling the microbiome award: Stephen Barrie on pre and probiotics at the Huffington Post

That was five or so posts over a few years.  But i certainly have seen more cases of overhyping and it does seem to be getting worse. One of the key aspects of overhyping is the continuous danger of correlation vs. causation.  Microbial communities can be very very complex ecosystems.  What many people/researchers do is the following:

1. identify a few groups of hosts (e.g., healthy vs. disease) 
2. collect samples and characterize the microbial communities in the samples
3. carry out some clustering/correlation analysis to look for features of the microbial community that are correlated with the host classes (e.g., healthy vs disease)

And given the massive number of variables in the microbial communities once can almost always find some feature that is highly correlated - or even perfectly correlated - to the host classes.  The problem with this is that you expect many such correlations by chance.  So how do you know when you have found one that is not spurious?  That is, not a false positive?  In reality, you can't know this without follow up studies.  In addition, and more important - suppose you found a consistent correlation between some microbial feature and the host phenotype/disease state/diet/etc?  What would that mean?  Well, one thing one CANNOT conclude is that the differences in the microbes CAUSED the host phenotypic differences.  All you know is that there is a correlation.  Perhaps the host phenotypic differences themselves drove changes in the microbes and were caused by something else.  Or perhaps some other issue (e.g., inflammation) caused both a change in the microbes and a change in the host.

So - please - if you are doing a microbiome study be careful about making conclusions based on correlations.  And if you are reading about microbiome studies - be careful about believing claims made by the authors/reporters.

So - I worry about these things OK?  And my gut (pun intended) says there is a lot of this going on.  So I decided to check out recent news on the topic of the human microbiome.  And of course I went to Google News and searched for "microbiome".  And I decided to look in more detail at a few of these story lines including

• Microbe connection to colorectal cancer
• Gut bacteria and metabolic syndrome
• A story in Food Consumer on diet and aging

Story 1: Gut microbes and colorectal cancer

Some studies of course do an OK job of trying to test whether observations are correlations or have some causative connection.  One seemingly well done case from the scientific publication point of view involves a recent paper on gut microbes and colon cancer: Gut Microbes Implicated in the Development of Colorectal Cancer.  Alas it is not an open access paper so what most people out there have to go on is the press coverage of the work.  The paper itself is quite interesting and the authors do a pretty good job of discussing how they went about testing the roles of specific microbes and even specific genes in the etiology of disease in a mouse model.  

The press coverage has not been so clear alas.  Some examples of the press coverage are below:

• Health News - Genes carried by E.Coli bacteria linked to colon cancer
• E. coli strain linked to cancer in mice
• Clear Links Found Between Inflammation, Bacterial Communities and Cancer
• Gut Bug Links Inflammation to Colorectal Cancer
• Inflammation, Bacteria and Colorectal Cancer

In many of the stories the key distinction between correlation vs. causation is nowhere to be found.  Of course, I don't always expect the press to cover such distinctions, but the more this is discussed in blogs, press stories the better off we all will be.  

Story 2: Gut microbes and pre-diabetes risk

Consider also recent stories on the microbiome and "pre-diabetes" risk.  

In the press release from the authors, a clear distinction is made between cause and effect: "We can’t infer cause and effect, but it’s an important step forward that we’re starting to identify bacteria that are correlated with clinical parameters ...". And in the paper such distinctions are also pretty clear: Analysis of the Gut Microbiota in the Old Order Amish and Its Relation to the Metabolic Syndrome.  Note - this paper was in PLoS One so is freely and openly available to all to read. Some quotes include "although the cross-sectional nature of this study makes it difficult to infer cause and effect with these data alone"

And much of the press coverage recapitulates these notes of caution - probably because they copied a lot of material from the PR.  See for example

• Amish gut study shows specific bacteria correlated to metabolic syndrome
• http://zeenews.india.com/...
• http://www.voanews.com/co...
• http://topnews.net.nz/con...
• http://www.healthcanal.co...

So - in these two cases the papers and PRs do an OK job of discussing correlation vs. causation and some of the press coverage does too.  Not perfect.  But not so bad. (I note - I originally flagged these cases as possible recipients of "Overselling the microbiome" awards but upon further examination discovered that the authors/PR people did an OK job ..

Story 3: Food consumer article on aging

Alas - if we look at how some others are making use of the microbiome studies that are coming out we see many more problems.  For example consider this site: One of the Best Foods You Can Eat to Defy Aging from some group called FoodConsumer.Org.  They discuss the human microbiome project and other studies of the microbiome, list many of the things the microbiome has been shown to do in humans and then, well, go overboard by telling how to ensure a healthy gut

• "A healthy diet is the ideal way to maintain a healthy gut, and regularly consuming traditionally fermented or cultured foods is the easiest way to ensure optimal gut flora."
• "Just make sure to steer clear of pasteurized versions, as pasteurization will destroy many of the naturally occurring probiotics. For example, most of the "probiotic" yogurts you find in every grocery store these days are NOT recommended. Since they're pasteurized, they will be associated with all of the problems of pasteurized milk products instead."

No evidence I know of supports these latter claims.  It is for this reason that I have previously given the person behind this site an  "Overselling the microbiome award".  But alas this is not the only place with some bad science on microbiomes.  Stay tuned - I still feel like I will be giving out many other awards in the near future ...

UPDATE 1: 7 AM 8/20

Ewen Callaway from Nature News asks on Twitter

@phylogenomics Jobin paper shows causal link between colon cancer and E coli in mouse model of colitis, no? How is our coverage bad?
— Ewen Callaway (@ewencallaway) August 20, 2012

I responded

@ewencallaway 1) for that story I said "The press coverage has not been so clear alas." - did not say it was "bad"
— Jonathan Eisen (@phylogenomics) August 20, 2012

@ewencallaway 2) but discussion of human side did not make clear that it was only correlation for human's w/ colibactin vs. cancer
— Jonathan Eisen (@phylogenomics) August 20, 2012

Some more details on Ewen's article - which I did find to be good, just unclear on the human side of things.  Here is some of the discussion of human colon cancer.  I have flagged sections that I wish had made more clear than in humans there is no evidence that the colibactin producing bacteria cause cancer.

Many humans also harbour bacteria that produce colibactin. The researchers found them in the stools of 20% of 24 healthy people, 40% of 35 people with inflammatory bowel disease and 66% of 21 people with colorectal cancer. But how the colibactin-producing bacteria lead to cancer isn’t clear, Jobin says. 

He hypothesizes that gut inflammation causes colibactin-producing strains to bloom while simultaneously weakening epithelial cells that line the gut, making them more susceptible to DNA damage. If this happens for long enough, a cell will turn cancerous, Jobin suggests.

Working out these steps in the human gut could help to prevent cancer, he adds. Doctors could use DNA sequencing to survey their patients’ guts for microbes producing genes that cause cancer, and then eliminate them with antibiotics. Similarly, probiotics could displace cancer promoting bacteria.

Pollard says that people already do this. Some fruits and vegetables seem to stave off cancer, whereas red meat and other foods are associated with higher cancer risks. Perhaps, Pollard says, foods prevent and promote cancer by shaping the microbiome. 

In this ending section on humans it is not made clear that the new study does not in any way show that colibactin producing bacteria cause cancer in humans.  Furthermore, it would have been good to add some serious caveats to the discussion of probiotics and displacement of cancer promoting bacteria.  Overall, a decent news story but it went a bit overboard on the "bacteria cause cancer in humans" angle without making clear that this was not shown.

UPDATE 2: Example of not so good coverage of a microbiome correlation issue

Here is an example of a recent news coverage that really does a bad job of dealing with the issue of cause vs. effect.  This relates to a recent study from the Murdoch Children's Hospital about bacteria and eczema.  Examples of news stories on the topic include:

• "Study shows bacteria could prevent eczema" (an interview on PM radio)
• Early exposure to bacteria could prevent eczema (ABC online story)
• Infant gut bacteria linked to eczema (in the Australian)

These appear to relate to a paper in Pediatric Allergy and Immunology: Reduced gut microbial diversity in early life is associated with later development of eczema but not atopy in high-risk infants.

The abstract reads

Background:  Alterations in intestinal microflora have been linked to the development of allergic disease. Recent studies suggest that healthy infant immune development may depend on the establishment of a diverse gut microbiota rather than the presence or absence of specific microbial strains. 

Objectives:  We investigated the relationship between diversity of gut microbiota in the early postnatal period and subsequent development of eczema and atopy in the first year of life. 

Methods:  Fecal samples were collected 1 wk after birth from 98 infants at high risk of allergic disease, who were followed prospectively to age 12 months. Fecal microbial diversity was assessed by terminal restriction fragment length polymorphism (T-RFLP) using restriction enzymes Sau96I and AluI, with a greater number of peaks representing greater diversity of bacterial communities. 

Results:  Microbial diversity at day 7 was significantly lower in infants with eczema at age 12 months as compared to infants without eczema (AluI mean number of peaks 13.1 vs. 15.5, p = 0.003, 95% CI for difference in means −3.9, −0.8; Sau96I 14.7 vs. 17.2, p = 0.03, 95% CI −4.9, −0.3). No differences were observed for atopic compared to non-atopic infants, or infants with two allergic parents compared to those with one or no allergic parent. 

Conclusions:  A more diverse intestinal microbiota in the first week of life is associated with a reduced risk of subsequent eczema in infants at increased risk of allergic disease. Interventions that enhance microbial diversity in early life may provide an effective means for the prevention of eczema in high-risk infants.

The key part really is in the conclusion.  What they showed was a correlation - a higher level of microbial diversity (in fecal samples) was correlated with reduced risk of eczema.  No causal connection was shown.  Alas the press coverage and the quotes/words of the authors in the press stories do not reflect any level of caution in the presentation.

For example in the Study shows bacteria could prevent eczema story from PM radio examples of troubling sections (with some comments by me in underlined words include):

• "Research by the Murdoch Childrens Research Institute shows that infants with low bacteria levels are more susceptible to eczema and asthma" (no susceptibility differences were detected .. all that was shown was that kids with eczema had different bacteria).
• "STEPHANIE SMAIL: Associate Professor Tang says the study shows introducing good bacteria into a child's diet could prevent eczema from developing. But she also says exposing children to common germs would help alleviate the problem" (well, excessive cleanliness is probably a bad thing in many cases ... but I know of no evidence that exposure to germs helps protect from eczema - and certainly this was not in this new study)

Or consider Early exposure to bacteria could prevent eczema.

• "This suggests that altering the mix and amount of bacteria in our guts in early life could be an effective approach to the prevention of eczema, especially for those with an increased risk of developing allergic disease." (no evidence has been presented that the microbes even cause the eczema - so it is way to early to speculate that changing the microbes could prevent anything).

The study done on eczema is quite interesting and potentially suggestive ... but the jump from "we observe differences in microbes" to "changing the microbes can probably prevent eczema" is a bit too much of a jump for me.