Blind trust in unblinded observation in Ecology, Evolution and Behavior (Guest Post by Melissa Kardish)

This is a guest post from Melissa Kardish - a PhD student at UC Davis - writing about a recent paper from work she did at her prior position.  The citation for the paper she is writing about is below:

Kardish MR, Mueller UG, Amador-Vargas S, Dietrich EI, Ma R, Barrett B and Fang C-C (2015) Blind trust in unblinded observation in Ecology, Evolution, and Behavior. Front. Ecol. Evol. 3:51. doi: 10.3389/fevo.2015.00051

Here is her post.

---

Blind trust in unblinded observation in Ecology, Evolution and Behavior

We recently published our study in Frontiers in Ecology and Evolution where we found that a remarkable number of studies that could be affected by observer bias didn’t indicate whether or not they blinded their research. In fact only 13.3% of studies reported this:

We tried to make this a very transparent study. In addition to journal level data in the main article, we include in our supplemental material a table with the score for every article we read for this study (a summary of these scores per journal can be found in Figure S2 included here). If anything, our results under-represent the amount of studies that could have been scored blind (the real underreporting/underuse of blind observation is probably less than the 13.3% we report). For instance, we did not assess that there was potential for bias in the scoring of microsatellite markers (scored as unlikely to have observer bias). However, we did identify one study which was based on data from microsatellites which did blindly score their markers and report this scoring in their methods (and was therefore scored as “blind” in our study).  We also considered a study blind in its entirety for the purposes of our scoring if only one aspect is reported even if other experiments could also have been influenced by observer bias (Check out our supplemental methods for more ways we conservatively scored in our study).

We recognize that not all EEB studies can be blinded due to a variety of logistical or hypothesis driven reasons; however, we encourage such studies to accurately report this rationale and consider and attempt to minimize observer bias when designing experiments.

Thus far we have had a great response from the surveyed journals. Many of them have notified their editors about the lack of blind observation that we found reported in their journal. One journal has even notified us of plans already in place to address this issue at their next editorial board meeting.

We’re excited to have this work out there and hope this will inspire people to blind their studies and accurately report the science they are doing. We’re also excited to have the study published in an open-access format where we hope the encouragement for blind observation can reach all levels of science. Finally, as reporting of science in our fields improves in the coming years, we hope this study can serve as a template to address other potential concerns in experimental design and reporting.

For our article check out:  http://journal.frontiersin.org/article/10.3389/fevo.2015.00051/full

For our supplemental text and methods:  http://journal.frontiersin.org/file/downloadfile/10238/octet-stream/Data%20Sheet%201.PDF/711/2/142945

For our data table with each of the articles we surveyed: http://journal.frontiersin.org/file/downloadfile/10235/octet-stream/Table%201.XLSX/711/1/142945