Please make it stop - overselling the microbiome award for rugby, exercise, microbiome stories

Update added 11/2/14 - for all my posts on Overselling the Microbiome go here. 

Well, I think today's lesson is, many people, including many scientists and science reporters, just do not get that there is a difference between correlation and causation.  I know - this is like beating a dead horse since many write about this issue.  But it just needs to be called out every time until it stops.  And today's fun comes from stories and the original research articles about how exercise supposedly alters the gut microbiome.

I was pointed to this just a few minutes ago on Twitter:

Seems like more appropriate title would b "rugby players have different diets" http://t.co/nzNsnHCYtA @phylogenomics

— Bernat Olle (@bernatolle) June 10, 2014

In this Tweet Bernat Olle points to a "news" story in Medpage Today: Exercise Boosts Gut Microbiome Diversity by Kristina Fiore.   Well, so of course I started digging around.  And, not surprisingly, the study that this is based on shows absolutely no causal connection between exercise and the gut microbiome.  The study is in the journal "Gut": Exercise and associated dietary extremes impact on gut microbial diversity.  And here is what they did:

• They selected subjects - 40 "elite" rugby players.
• They identified healthy male "controls" with similar age and size and from similar place. 
• Then they collected faecal and blood samples from participants and did surveys about their nutrition and clinical data.
• Among many measurements, they did 16S sequencing from the fecal samples
• Then they did some bioinformatics and found differences between the rugby players and the controls in many features including microbiomes.

And amazingly, from this they report, in their abstract

The results provide evidence for a beneficial impact of exercise on gut microbiota diversity but also indicate that the relationship is complex and is related to accompanying dietary extremes.

The key part of this to me is 

The results provide evidence for a beneficial impact of exercise on gut microbiota diversity

For which they have no support.  They do not in any way show that exercise has ANY affect on the microbiota.  They show it is correlated to the microbiota.

And sadly there is a commentary on the article in the same issue of Gut that makes the same mistake.  Georgina Hold in The gut microbiota, dietary extremes and exercise writes:

The article is the first report that exercise increases gut microbiota richness/diversity and highlights that exercise is another important factor in the complex relationship among the host, host immunity and the microbiota.

No.  They did not show exercise increased gut microbiota diversity.  How can the difference between correlation and causation be missed in these articles?  Are these not even reviewed?  Sure - this is consistent with exercise affecting microbiomes but it is also consistent with rugby players having different diets and other behaviors.  There is a big difference between showing cause and effect and showing correlation.  For not distinguishing between correlation and causation regarding the rugby player microbiomes I am giving all involved here an "Overselling the Microbiome Award".

Here is a microbiome theory I will leave you with.  I hypothesize that these papers, and all the other ones that oversell the microbiome, themselves cause major changes in the microbiome of many people.  Evidence for this?  Well, none yet.  But I have a correlation.  The correlation is, after reading these papers,  I feel sick to my stomach.  That must be proof right?

---

UPDATE 6/11/14

Author of the Medscape Medpage today article Kristina Fiore says she will update the article to more accurately reflect the science. See some of the thread below

.@davidkroll @KristinaMFiore @medpagetoday they showed correlation between exercise & microbiome, but no causal relationship tested or shown
— Jonathan Eisen (@phylogenomics) June 11, 2014

.@Sandlot1313 @davidkroll @KristinaMFiore @medpagetoday exactly - whole effect could be diet, other behaviors, social contacts, travel, etc.
— Jonathan Eisen (@phylogenomics) June 11, 2014

we'll do that. @phylogenomics @Sandlot1313 @davidkroll @medpagetoday
— Kristina Fiore (@KristinaMFiore) June 11, 2014

---

UPDATE 2: 6/11/14.

The press release from Gut associated with this paper contains many inaccurate statements.

Examples include:

• Title: Exercise boosts diversity of gut bacteria
• Text: Exercise boosts the diversity of the bacteria found in the gut, indicates the first study of its kind published online in the journal Gut.

Somewhat surprised that such mistakes would come from the journal itself.

---

UPDATE 3: 6/12/14.

Kritina Fiore has fixed the Medpage article.  Nice.

---

UPDATE 4: 6/12/14.

Science Magazine gets the causation vs. correlation issue wrong in their little news piece about this.  Yuck.

So @sciencemagazine also does not understand causation vs. correlation - see http://t.co/FZoIsrqbzj vs. http://t.co/5UqR6EG9rb #Yuck
— Jonathan Eisen (@phylogenomics) June 12, 2014

---

UPDATE 5: 6/12.

Alexandra Sifferlin has a good article about this at Time

I spoke to scientists abt the line between microbiome fervor and being critical of new science http://t.co/OVptCV9Poe cc @phylogenomics
— Alexandra Sifferlin (@acsifferlin) June 12, 2014

Nailed it: @acsifferlin on "Exercise Helps Your Gut Bacteria: Amazing, Overblown, or Plain Incorrect?" http://t.co/ESWjHfE4f2 in @TIMEHealth
— Jonathan Eisen (@phylogenomics) June 12, 2014

---

UPDATE 6.

More accurate coverage by Claire O'Connell in the Irish Times Generally a good article here: Rugby players show good guts

---

UPDATE 7.

Popular Science messes it up too

---

UPDATE 8.

Keeping track of some of the Tweets about this on Storfy.

---

UPDATE 9 6/13/14.

NPR News Falls for the Hype

@nprhealth Seriously? No. You need to actually read the paper. Correlation ≠ Causation — just ask @phylogenomics https://t.co/8NbLR3r5Fy
— Jonathan Jacobs (@bioinformer) June 13, 2014

---

UPDATE 10: 6/13/14.

Just found another inaccurate claim in the original paper

---

UPDATE 11: 6/13/14

Oh FFS. Now I have found some articles reporting not only that exercise affects gut microbial diversity but that this is why exercise reduces obesity. See Exercise lowers obesity risk by stimulating diverse gut bacteria in the NVO News, for example.

 

Quote from the story:

A latest research suggests that exercise actually lowers obesity risk by stimulating diverse gut bacteria

---

UPDATE 12: 6/13/14

Fox News did better with the science (at least in their headline) than many other News Agencies (and much better than NPR).  They report "Exercise may lead to healthier gut bacteria".

Just that word "May" makes me happy.  I know.  Low bar.  But I will take what I can get.

UPDATE 13: 6/13/14

Genome Web also is reporting on the story and on the "overselling" that was done.

UPDATE 13: 6/18/14

And now the New York Times joins the fray: Exercise and the ‘Good’ Bugs in Our Gut where Gretchen Reynolds writes:

The findings suggest that, in addition to its other health benefits, frequent exercise may influence our weight and overall health by altering the kinds of organisms that live inside of us.

No - the findings do not suggest that.  The findings are consistent with that theory but they are consistent with many many many other theories.  FFS this is maddening.  And the article ends with a quote from one of the authors:

But even in advance of those findings, he said, it seems likely that any amount of exercise should make your gut more welcoming to the bacteria that you want residing there.

I note - I found out about this article via Twitter

@eswillwalker In this case, it was not just the journalists, it seems.... http://t.co/ddgBN3XsXu
— Britt Koskella (@bkoskella) June 18, 2014