[Dean's World] Aziz P: peer review: necessary, but not sufficient

[notify at powerblogs.com]

Posted by Aziz P:
peer review: necessary, but not sufficient
http://www.deanesmay.com/posts/1179325806.shtml


   The Real Climate folks had a good post a while back about [1]the
   limits of peer review.

   They note that peer review forces science to be slow and incremental,
   and that new "bombshell" ideas are usually less reliable and robust
   than ideas that take their time, wind through the system, and
   gradually attract support. After all, as they rightly note, "It is
   extremely unlikely that any new study will immediately overthrow all
   the past knowledge", and then comment,

     Scientists would find the apparent contradiction interesting and
     worthy of further investigation, and would devote further study to
     isolating the source of the contradiction. They would not suddenly
     throw out all previous results. Yet, one often gets the impression
     that scientific progress consists of a series of revolutions where
     scientists discard all their past thinking each time a new result
     gets published. This is often because only a small handful of
     high-profile studies in a given field are known by the wider public
     and media, and thus unrealistic weight is attached to those
     studies. New results are often over-emphasised (sometimes by the
     authors, sometimes by lobby groups) to make them sound important
     enough to have news value. Thus "bombshells" usually end up being
     duds. ... even when it initially breaks down, the process of
     peer-review does usually work in the end. But sometimes it can take
     a while. Observers would thus be well advised to be extremely
     skeptical of any claims in the media or elsewhere of some new
     "bombshell" or "revolution" that has not yet been fully vetted by
     the scientific community.

References

   1. http://www.realclimate.org/index.php/archives/2005/01/peer-review-a-necessary-but-not-sufficient-condition/