Case Study - what to do when there are disagreements about whether a paper has problems

In light of the ongoing PLOSOne #Creationgate controversy (e.g. see this write up here), I thought I would share a story I have been working on about a case where there are disagreements about whether a paper has problems or not.

---

So I got this email the other day. It was from an author of a paper who I know who was upset about a paper that was published a while back for which I was the editor. This person wrote to me and the authors of the new paper somewhat angrily critiquing them for some aspects of their paper that related to this person's work. The authors responded to the critique and, well, did not agree with the points of the letter writer. The letter writer wrote again to all of us and again somewhat angrily critiqued the authors.

Then the letter writer wrote just to me asking what I thought should be done - detailing further what they viewed as mistakes of the paper.  The letter writer was quite clear, clearly upset, and had some good points. And the letter writer wanted advice about what should be done here.  I thought about this for some time and wrote and rewrote an email to the letter writer.  The challenge with this case was that this really seemed to be more of a disagreement than a case where an Editor could say "This is right and this is wrong."  So this is what I wrote

Letter Writer

I appreciate your comments and your intent here.  I think the best course of action is for you to publish more public, formal or informal, comments about the paper.  I am not sure I would support any type of attempt to require the authors to officially revise their paper.  The paper was reviewed by multiple reviewers and the published version is the final outcome of the review process.  I believe the process was fair, rigorous and thorough.  That does not mean of course that it was perfect (note - I am not making any statement here about whether I think your comments and concerns are valid or not).  But as far as I can tell, even if your claims and comments were 100% valid and correct, I probably would still not recommend undertaking any action that would require modifications of the paper by the authors.  I therefore would recommend that you pursue other options including 

(1) submitting comments about this on the journal web site 
(2) submitting comments at Pubmed Commons or other such sites 
(3) submitting a formal response via the journal 

If you wish to pursue further the possibility of requesting a modification of the paper by the authors, I am happy to forward this on to the higher ups at the journal and they can tell you how to do this. But as I note, I am not sure this would be the right thing to do here.

Jonathan Eisen

I don't know if I did the right thing here but I just felt that this was a case where people could just disagree about what was correct.  Any suggestions for how to handle such cases or other examples would be welcome.