Peer Review In Need of Reform – Reproducing Results Entails Too Much Time And Effort, Says Philip Campbell of Nature

The Swiss online NZZ here  has a report on the peer-review process and the problems plaguing it. Peer-review has become a leading topic in science in Europe and all over the world. The NZZ starts off:

Irritated by multiple scandals over the last years, many scientists are up in arms over the peer-review of scientific articles and are hoping for improvement.

Many say there’s a need for reform.

Cartoon Source: http://medinfo.netbib.de/archives/2009/01/23/2975

Scientists say manipulation is as easy as pie.As a whole, the field of science is becoming overwhelmed by the flood of papers seeking publication. The NZZ writes that according to a Finnish study, already in 2006  1.35 million peer-reviewed papers were published, and the trend is accelerating.

And it is unclear whether the current large-scale peer-review process yields the correct, important results every time. Also publishers and peer-reviewers can make mistakes, as hanky panky like copying text and manipulating charts is especially easy in today’s computer age.

The NZZ does not name any particluar science field here, or anywhere else in the report. But for those familiar with climate science and the CRU e-mails, it sounds all too familiar.

The topic of peer-review has gained much importance over the years, as it’s the lifeblood of scientific careers. That’s one reason why the recent European Science Forum in Turin in early July was so jammed pack.

The NZZ asks: Is the current system the best we have?

Based on a 2009 study by Adrian Mulligan of Elsevier publishing, the NZZ reports that one third of the scientists replied with yes, one third with no, and the other third were undecided. The survey sampled 4000 scientists.

With those results one could reasonably assume that half are not really convinced by the peer-review process. That tells me it needs to be reformed. There are many problems with it. The NZZ mentions some of them.

For example, some scientists say that papers do not even get reviewed by experienced scientists, but are often passed down to younger, less experienced colleagues who don’t really know how to do it.

The NZZ writes about being able to reproduce results:

Increasingly, peer reviewers are no longer able to reproduce the results of studies on their own – because the time and effort simply would be too much. Philip Campbell of Nature brought up that point in Turin.

Campbell says errors will always get by peer-reviewers, and sometimes even outright fraud. But that’s rarely the case, as studies such as one from Daniele Fanelli of the University of Edinburgh 2009 in the journal PLoS ONE shows. NZZ writes:

According to the results, an average of 2% of scientists admitted to having made at least one falsification. And up to 34% admit to having committed a dubious act.

Plagiarism is another problem. Fortunately, plagiarism-recognition software such as CrossCheck help to detect plagiarism. The software is based on publication data-banks.

Diagram manipulation is detected using the algorithms in programs for graphic processing. Other offences committed include ghostwriting, including co-authors who did not take part in the study and salami publication.

In my view, the NZZ report places too much of the blame on the sheer volume of papers that need to be reviwed and on human nature, and completely ignores the political aspects that have corrupted the process.

But in the end, its conclusion is correct: Peer-review needs to be reformed, as for now there is no other alternative system available.

 

 

4 responses to “Peer Review In Need of Reform – Reproducing Results Entails Too Much Time And Effort, Says Philip Campbell of Nature”

  1. DirkH

    I suggest public blog review as done at WUWT.

  2. JP

    It’s simple. Publish everything, let the “marketplace” decide.
    I.e., publications that are useful will “tend” (nothing is guaranteed)
    to become more and more read, and be used as the basis
    for future research.

    This simple solution, of course, means that granting agencies,
    universities, official journals, major publishers, etc., will have
    to change their ways. And that is not going to happen.

  3. Erik

    Another vote for public blog review as done at WUWT.