This is a guest post from Melissa Kardish - a PhD student at UC Davis - writing about a recent paper from work she did at her prior position. The citation for the paper she is writing about is below:
Kardish MR, Mueller UG, Amador-Vargas S, Dietrich EI, Ma R, Barrett B and Fang C-C (2015) Blind trust in unblinded observation in Ecology, Evolution, and Behavior. Front. Ecol. Evol. 3:51. doi: 10.3389/fevo.2015.00051
Here is her post.
Blind trust in unblinded observation in Ecology, Evolution and Behavior
We recently published our study in Frontiers
in Ecology and Evolution where we found that a remarkable number of studies
that could be affected by observer bias didn’t indicate whether or not they
blinded their research. In fact only 13.3% of studies reported this:
We tried to make this a very transparent study. In addition to
journal level data in the main article, we include in our supplemental material
a table with the score for every article we read for this study (a summary of
these scores per journal can be found in Figure S2 included here). If anything,
our results under-represent the amount of studies that could have been scored
blind (the real underreporting/underuse of blind observation is probably less
than the 13.3% we report). For instance, we did not assess that there was
potential for bias in the scoring of microsatellite markers (scored as unlikely
to have observer bias). However, we did identify one study which was based on
data from microsatellites which did blindly score their markers and report this
scoring in their methods (and was therefore scored as “blind” in our
study). We also considered a study
blind in its entirety for the purposes of our scoring if only one aspect is
reported even if other experiments could also have been influenced by observer
bias (Check out our supplemental methods for more ways we
conservatively scored in our study).
We recognize that not all EEB studies can be blinded due to a
variety of logistical or hypothesis driven reasons; however, we encourage such
studies to accurately report this rationale and consider and attempt to
minimize observer bias when designing experiments.
Thus far we have had a great response from the surveyed
journals. Many of them have notified their editors about the lack of blind
observation that we found reported in their journal. One journal has even
notified us of plans already in place to address this issue at their next editorial
board meeting.
We’re excited to have this work out there and hope this will
inspire people to blind their studies and accurately report the science they
are doing. We’re also excited to have the study published in an open-access
format where we hope the encouragement for blind observation can reach all
levels of science. Finally, as reporting of science in our fields improves in
the coming years, we hope this study can serve as a template to address other
potential concerns in experimental design and reporting.
For our article check out:
http://journal.frontiersin.org/article/10.3389/fevo.2015.00051/full
For our supplemental text and methods:
http://journal.frontiersin.org/file/downloadfile/10238/octet-stream/Data%20Sheet%201.PDF/711/2/142945
For our data table with each of the articles we surveyed:
http://journal.frontiersin.org/file/downloadfile/10235/octet-stream/Table%201.XLSX/711/1/142945
