{"id":855,"date":"2009-08-14T08:52:42","date_gmt":"2009-08-14T13:52:42","guid":{"rendered":"http:\/\/www.realclimate.org\/?p=855"},"modified":"2009-08-25T00:10:15","modified_gmt":"2009-08-25T05:10:15","slug":"resolving-technical-issues-in-science","status":"publish","type":"post","link":"https:\/\/www.realclimate.org\/index.php\/archives\/2009\/08\/resolving-technical-issues-in-science\/","title":{"rendered":"Resolving technical issues in science"},"content":{"rendered":"
\n

One of the strengths of science is its capacity to resolve controversies by generally accepted procedures and standards. Many scientific questions (especially more technical ones) are not matters of opinion but have a correct answer. <\/p>\n

Scientists document their procedures and findings in the peer-reviewed literature in such a way that they can be double-checked and challenged by others. The proper way to challenge results is, of course, also through the peer-reviewed literature, so that the challenge follows the same standards of documentation as did the original finding.
\n<\/p>\n

Such a challenge can either be in form of a new, independent paper, or in the form of a comment to a published paper. The latter is the appropriate avenue if the challenge is not based on new data (and is thus a piece of research in its own right), but is a criticism of the methods used in a paper.<\/p>\n

Such technical comments are routinely published in journals, and RealClimate authors have of course also been involved in writing or receiving such comments. One prominent example was a comment in Science<\/i><\/a> showing that a challenge by Von Storch et al. (2004) to the \u201chockey stick\u201d climate reconstruction of Mann et al. (1998) \u201cwas based on incorrect implementation of the reconstruction procedure\u201d. We discussed the implications on Realclimate<\/a> after the comment appeared. Another recent example was a comment by Schmith et al. <\/a> on a Science<\/i> paper on sea level rise by Stefan<\/a>, noting that he failed to account for the effect of smoothing on the autocorrelation in the data he used. In his response<\/a>, Stefan acknowledged this mistake but showed that it does not affect his main conclusions.<\/p>\n

That the original authors are allowed to respond to a comment in the same journal issue, and the comment\u2019s authors get to consider this response before deciding to go ahead with their comment, are key hallmarks of a fair procedure, in addition to a neutral journal editor and independent reviewers overseeing the process. Even if the authors of comment and reply continue to disagree to some extent, this comment process in most cases resolves the issue to the satisfaction of the scientific community. It lays out the facts in a fair and transparent way and gives outsiders a good basis for judging whom is right. In this way it advances science.<\/p>\n

There is however a different way of criticizing scientific papers that is prevalent in blogs like ClimateAudit. This involves challenging, ‘by all means necessary’, any paper whose conclusions are not liked. This can be based on simple typos, basic misunderstandings of the issues and ‘guilt by association’ though there is sometimes the occasional interesting point. Since these claims are rarely assessed to see if there is any actual impact on the main result, the outcome is a series of misleading critiques, regardless of whether any of these criticisms are in fact even valid or salient, that give the impression that every one of these papers is worthless and that all their authors incompetent at best and dishonest at worst. It is the equivalent of claiming to have found spelling errors in a newspaper article. Fun for a while, but basically irrelevant for understanding any issue or judging the worth of the journalist.<\/p>\n

While commentary — even quite negative commentary — of papers on blogs is entirely reasonable (after all, we do it here occasionally), claims that a particular paper has been ‘discredited’ or ‘falsified’ that have not withstood (at minimum) the process of peer-review should be viewed with extreme skepticism. So should accusations of dishonesty or misconduct that have not already been conclusively and unequivocally substantiated.<\/p>\n

This brings us to the recent claim by Hu McCulloch that a post on ClimateAudit.org<\/a>, detailing an error in Steig et al\u2019s paper in Nature<\/i> on Antarctic temperature change, was not given due credit by Steig et al. when they published a Corrigendum<\/a> earlier this month. In this case, McCulloch\u2019s comment on the paper were perfectly valid, but he chose to avoid the context of normal scientific exchange — instead posting his comments on ClimateAudit.org — and then playing a game of ‘gotcha’ by claiming plagiarism when he wasn\u2019t cited.<\/p>\n

McCulloch accuses Steig et al. of appropriating his ‘finding’ that Steig et al. did not account for autocorrelation when calculating the significance of trends. While the published version of the paper didn’t include such a correction, it is obvious that the authors were aware of the need to do so, since in the text of the paper it is stated that this correction was made. The corrected calculations were done using well-known methods, the details of which are available in myriad statistics textbooks and journal articles. There can therefore be no claim on Dr. McCulloch’s part of any originality either for the idea of making such a correction, nor for the methods for doing so, all of which were discussed in the original paper. Had Dr. McCulloch been the first person to make Steig et al. aware of the error in the paper, or had he written directly to Nature<\/i> at any time prior to the submission of the Corrigendum, it would have been appropriate to acknowledge him and the authors would have been happy to do so. Lest there be any confusion about this, we note that, as discussed in the Corrigendum<\/a>, the error has no impact on the main conclusions in the paper.<\/p>\n

There is nothing wrong with constructive criticism, and pointing out errors — even fairly minor ones — is important and useful. The difference, though, between people who want to find out something about the real world and people who just want to score political points, is what is made of those errors. That is the test of constructive scientific dialog. Specious accusations of fraud, plagiarism and the like don\u2019t pass such a test; instead they simply poison the atmosphere to everyone’s loss. <\/p>\n\n<\/div> ","protected":false},"excerpt":{"rendered":"

One of the strengths of science is its capacity to resolve controversies by generally accepted procedures and standards. Many scientific questions (especially more technical ones) are not matters of opinion but have a correct answer. Scientists document their procedures and findings in the peer-reviewed literature in such a way that they can be double-checked and […]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[12,1,35,24,34],"tags":[],"class_list":{"0":"post-855","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-arctic-and-antarctic","7":"category-climate-science","8":"category-communicating-climate","9":"category-reporting-on-climate","10":"category-skeptics","11":"entry"},"aioseo_notices":[],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/855","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/comments?post=855"}],"version-history":[{"count":76,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/855\/revisions"}],"predecessor-version":[{"id":1044,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/855\/revisions\/1044"}],"wp:attachment":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/media?parent=855"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/categories?post=855"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/tags?post=855"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}