Another Issue with Peer Review

May 3rd, 2012

I’ve written in the past about peer review (and links therein), particularly my annoyances with how some referees don’t seem to be constructive about it, and in fact can be condescending assholes.  I made my suggestions about improving it while keeping it anonymous.  Another option is to remove anonymity, which I think does have drawbacks, but that’s the suggestion in light of “primate behavior” ala Matt Ridley’s review of David Maestripieri’s “Games Primates Play.”

Dr. Maestripieri’s most intriguing chapter is entitled “Cooperate in the Spotlight, Compete in the Dark.” He describes how people, like monkeys, can be angels of generosity when all eyes are on them, but devils of spite in private. Famously, the citizens of New York City turned to crime when the lights went out in the blackout of July 13, 1977-not because they were evil but because the cost-benefit calculus was altered by the darkness.

Dr. Maestripieri then offers a fascinating analysis of the conundrum of peer review in science. Peer review is asymmetric: The author’s name is known, but the reviewers remain anonymous. This is to prevent reciprocal cooperation (or “pal review”): I’ll be nice about your paper if you’re nice about mine.

In this it partly works, though academics often drop private hints to each other to show that they have done review favors. But peer review is plagued by the opposite problem-spiteful criticism to prevent competitors from getting funded or published. Like criminals in a blackout, anonymous reviewers, in the book’s words, “loot the intellectual property of the authors whose work they review” (by delaying publication while pinching the ideas for their own projects) and “damage or destroy the reviewed authors’ property” (by denying their competitors grants and publications).

Studies show that peer reviewers are motivated by tribal as well as individual rivalry. Says Dr. Maestripieri: “I am a Monkey-Man, and when I submit a grant application for peer review, I am terrified that it might fall into the hands of the Rat-People. They want to exterminate all of us…(because our animals are cooler than theirs).”

His answer (and it applies to far more fields than science) is total transparency with the help of the Internet. The more light you shine, the less crime primates commit. Once everybody can see who’s reviewing whose papers and grant applications, then not only will spite decline, but so will nepotism and reciprocity. Anonymity alters the cost-benefit balance in favor of competition; transparency alters it in favor of cooperation.

I’m a little concerned, however, that junior referees will hesistate to point out problems on papers authored by more senior scientists.  I mean, I’m in favor of cooperation and constructive reviews, but we might get reviewers trying to suck up to more senior people, or junior referees declining to review papers at all.  That’s not healthy either.  I think I like my suggestions better, although I’m torn if the question is simply whether anonymity is better than transparency.  In any event, I’d like referees evaulated and those evaluations used by editors whether the reviewers are anonymous or not.

Share/Bookmark

Tags: , , , , ,

You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.