Doesn’t peer review offer some guarantee of quality?, suggests Harford. “Peer review is of minimal value” is the response to this, “…checkability is what really guarantees quality”. Senn goes on to suggest that scientists sign an undertaking to provide raw original data to anyone who requests it.

This post was prompted by a discussion at the Bad Science forum, where the idea of applying industrial-style quality assurance to journal articles was raised. This would mean that there would be some sort of checklist that a reviewer would have to go through, and this would be checked to make sure it had been done. It would not be much use to do this informally; there would need to be some formal way of doing it.

Traditional methods of peer review are coming under strain as the volume of manuscripts and the number of forums for manuscript submission rise. These pressures can result in poorer quality reviews, extended publication times, and higher costs to the organisations that fund research. In this paper we describe a method for reducing reviewing burden, expediting feedback and shortening publication times. Furthermore, by its nature, the method produces leading (as opposed to lagging/trailing) publication metrics for authors and the manuscripts they write, and we show how these metrics can be used by search engines to provide more useful orderings of search results. Finally, we briefly discuss the potential to apply the underlying mechanism of the method to application domains beyond research publishing, such as the web as a whole.

if we really want to develop "some sensible measure of scholarly impact on the social Web" then we have to step back and consider three questions:\n\n * what do we want to measure?\n * what can we measure?\n * how can bring these two things close enough together to create something useful?\nThese two are clearly related, at least in the sense that someone's level of engagement in a community (their connectedness if you like) clearly increases the exposure of their work but is also indicative of the credibility they have within that community. Having said that, my gut feeling is that credibility, at least for the purposes of scholarly communication, can only really be measured by some kind of a peer-review (i.e. human) process.

There is an economic solution to the problem that bears consideration: Charge for submission. This would induce self-selection; authors would be loathe to submit unless they thought the paper had a fair chance of acceptance. Consider a conference or journal with a 25% acceptance rate that charged, say, $50 per submission. (The right amount may be different; I use this figure just as an example.) Authors who tended to write and submit average quality papers would be confronted with a cost of some $200 (in expectation) per published paper. If they wanted to reduce that cost, the expedient method would be to submit fewer papers and papers with higher average quality. The most plausible approach is to refrain from submitting the lowest quality papers, but other methods of improving quality would work as well. This has several positive effects: reduced reviewing load, higher average submission quality, less “salami-slicing“, and revenue generation to boot. To avoid disenfranchisement of scholars with more limited means, fee waivers should be supplied in exigent circumstances, as they are for page, figure, and other publication charges by many journals. The application process for the fee waiver would be separated from the editorial process to prevent mercenary considerations from affecting editorial decisions. Submission fees have a further benefit over publication fees in eliminating any economic incentive for lowering quality standards as a means for increasing revenue, as discussed in detail by McCabe and Snyder.

Science Comment is dedicated to the advancement of sciences. Feel free to add your comment on publications, be it inspiring thoughts or critics, to propulse the scientific discussion and to enhance future results. Please use this site to help clarify certain aspects of science. Let's make a difference!

With a shift from proprietary to non-proprietary systems of peer review, initial experience has now been garnered from SCOAP [4] and the Springer experiments [5] at UKB, MPG, Göttingen University and, lately, California University [6]). This conversion can be speeded up if disciplinary communities, universities, and research funders actively enter the market of the peer review organisers by calling for tenders and inviting publishers to submit proposals for a non-proprietary design of the peer review process. Given the current situation – with the American legislature and the European Commission having clearly taken a stand in favour of Open Access – one can expect that such tenders will certainly produce interesting proposals. The time is ripe!