Sitting at the conference banquet and feeling bored? Here’s an easy way to spice things up: finish that glass of wine, mention that you’ve recently submitted a paper for review, and wryly comment on the inanity of the process. With a little bit of luck, presto! One Thanksgiving-with-the-family-style debate on the pros and cons of peer review.
In all seriousness, few topics in academia inspire the same level of passionate debate as peer review does. Its defenders point to decades of peer-reviewed work as an argument that it serves a fundamental-if-flawed check against poor research. Its detractors point to arbitrary reviews, increasing rates of retraction, long delays between submission and publication, and unfair distributions of review requests to argue that the system is broken and needs replacement. Love it or hate it, though, all of us academic researchers have to deal with it.
At its core is a simple premise: new ideas require some degree of vetting, if only to reduce the amount of pseudoscience that enters the greater body of human knowledge. Obviously, experts should conduct this vetting process if at all possible. Who better qualified, then, to evaluate a researcher’s new idea than that researcher’s peers? Presumably, they fit the bill as experts, and if they’re true peers, they most likely have a good understanding of the field in which the new idea lies. Even better, it’s reciprocal: if I review your new idea, you can review my new idea, and everyone wins.
As with all human enterprises, of course, reality strays from the ideal. Peer review can fail; poorly-conducted studies can enter the system and become deeply rooted, weakening the foundations and, when uncovered, poisoning public trust of science. No one argues that this never happens — the debate instead centers over just how terrible the situation is.
In my time in graduate school, I’ve found myself falling into the “it’s the worst option except for all the rest” camp. I agree with the critiques that peer review is often arbitrary, slows innovation, eschews novelty in favor of dogma, etc. But I fail to see how any other approach — paid, dedicated staff reviewers, for example — addresses the fundamental problem that the most qualified, knowledgeable individuals to evaluate one’s research are, almost always, one’s peers.
Perhaps more promising are the ideas for tweaking/tuning/innovating on peer review. A few of my personal favorites (no, I’m not a fan of the idea of outright paying reviewers) include:
Add post-publication reviews to the process. In my field (computer engineering), most conference papers see between 3 to 7 reviews before publication, but after publication, any reviews are just researchers taking personal notes. Offering a formal means of providing post-publication feedback to authors won’t eliminate the need for retractions, but it ameliorate the phenomenon of peer reviewed-work immediately becoming accepted dogma.
Create a global publishing currency. One of the more intriguing ideas in my opinion, some scholars propose the creation of a “currency” to encourage peer review. In the proposed scheme, researchers would earn 1 credit for each review, while submitting a paper for review would cost 3 credits. It would, in effect, “privatize” the peer review system. The devil is in the details, of course: how do papers get recycled? What about young, infrequent, or returning researchers? And what happens if some journals/publishers refuse to use the system?
Increase the use of pre-prints. The explosion of pre-print servers like arxiv in fields like physics have led some researchers to propose that all work (at least) start on these servers. Only after significant review by the greater community, then, can it be accepted for final publication. In effect, this suggestion calls for throwing open the peer review process at the potential risk of significantly lengthening the publication process.
In the end, the simple answer is (as always) that there are no simple answers. The critiques of peer review will only continue to grow, and rightly so in my opinion. We in academia need to stop talking about innovating and start actually doing it, whether that means increasing the number of reviewers or co-opting the cryptocurrency movement. Now is the time to do what we do best: experiment, before public trust in peer review is lost forever.