{"id":8744,"date":"2011-09-06T09:02:33","date_gmt":"2011-09-06T14:02:33","guid":{"rendered":"http:\/\/www.realclimate.org\/?p=8744"},"modified":"2011-09-06T11:50:22","modified_gmt":"2011-09-06T16:50:22","slug":"resignations-retractions-and-the-process-of-science","status":"publish","type":"post","link":"https:\/\/www.realclimate.org\/index.php\/archives\/2011\/09\/resignations-retractions-and-the-process-of-science\/","title":{"rendered":"Resignations, retractions and the process of science"},"content":{"rendered":"
\n

Much is being written<\/a> about the very public resignation<\/a> of Wolfgang Wagner from the editorship of Remote Sensing over the publication of Spencer and Braswell (2011) – and rightly so. It is a very rare situation that an editor resigns over the failure of peer review, and to my knowledge it has only happened once before in anything related to climate science – the mass resignation of 6 editors at Climate Research in 2003 in the wake of the Soon and Baliunas debacle<\/a>. Some of the commentary this weekend has been reasonable, but many people are obviously puzzled by this turn of events and unsupported rumours are flying around.<\/p>\n

<\/p>\n

The primary question of course is why an editor would resign over a published paper. Wagner (who I have never met or communicated with) explains it well in his letter:<\/p>\n

\nAfter having become aware of the situation, and studying the various pro and contra arguments, I agree with the critics of the paper. Therefore, I would like to take the responsibility for this editorial decision and, as a result, step down as Editor-in-Chief of the journal Remote Sensing.
\n…
\nWith this step I would also like to personally protest against how the authors and like-minded climate sceptics have much exaggerated the paper\u2019s conclusions in public statements. [UAH press release<\/a>. Forbes article<\/a> etc.]
\n<\/a><\/p><\/blockquote>\n

He clearly feels as though he, and his fledgling journal, were played in order to get a politicised message to the media. A more seasoned editor might well have acted differently at the various stages and so he resigned to take responsibility for the consequences of not doing a better job, and, presumably, to try and staunch the impression that Remote Sensing<\/em> is a journal where you can get anything published.<\/p>\n

This was nonetheless a very unusual step. Many bad papers are published (some of which are egregiously worse than the one in question here) and yet very few editors resign over the way the process was handled. (In fact, I think this is unique – the resignations at Climate Research in 2003 were not of the editors involved in dealing with Soon and Baliunas, but the other members of the board protesting at the inability and\/or unwillingness of the publisher to deal with the resulting mess). <\/p>\n

But what makes a paper ‘bad’ though? It is certainly not<\/strong> a paper that simply comes to a conclusion that is controversial or that goes against the mainstream, and it isn’t that the paper’s conclusions are unethical or immoral. Instead, a ‘bad’ paper is one that fails to acknowledge or deal with prior work, or that makes substantive errors in the analysis, or that draws conclusions that do not logically follow from the results, or that fails to deal fairly with alternative explanations (or all of the above). Of course, papers can be mistaken or come to invalid conclusions for many innocent reasons and that doesn’t necessarily make them ‘bad’ in this sense. <\/p>\n

So where does S&B11 fall on this spectrum?<\/p>\n

The signs of sloppy work and (at best) cursory reviewing are clear on even a brief look at the paper. Figure 2b has the axes mislabeled with incorrect units. No error bars are given on the correlations in figure 3 (and they are substantial – see figure 2 in the new Dessler paper<\/a>). The model-data comparisons are not like-with-like (10 years of data from the real world compared to 100 years in the model – which also makes a big difference). And the ‘bottom-line’ implication by S&B that their reported discrepancy correlates with climate sensitivity is not even supported by their own figure 3. Their failure to acknowledge previous work on the role of ENSO in creating the TOA radiative changes they are examining (such as Trenberth et al, 2010<\/a> or Chung et al, 2010<\/a>), likely led them to ignore the fact that it is the simulation of ENSO variability, not climate sensitivity, that determines how well the models match the S&B analysis (as clearly demonstrated in Trenberth and Fasullo’s guest post<\/a> here last month). With better peer review, Spencer could perhaps have discovered these things for himself, and a better and more useful paper might have resulted. By trying to do an end run around his critics, Spencer ended up running into a wall.<\/p>\n

Of course, Spencer does not see this in the same light at all. His comments both before<\/a> the publication of the paper and subsequent to the editor’s resignation indicate that he thinks that he is being persecuted by (unnamed) ‘IPCC Gatekeepers’ who are conspiring to suppress his results – he even insists that this was “one damn fine and convincing paper<\/a>“. As well as straining credulity to the maximum, I find this both unfortunate and curious. It is unfortunate because this attitude makes it almost impossible for him to take on board constructive criticism, and given that none of us are perfect, there are many times when doing so is essential. It is also curious because there is no evidence of any grand conspiracy, just people disagreeing with and criticising his conclusions (which as a scientist, you really just have to get used to!). It was S&B’s desire to avoid dealing with that, that likely led them to a non-standard journal, whose editor very likely followed the authors suggestions for (friendly) reviewers, whose resulting reviews didn’t do very much (if anything) to strengthen the paper. <\/p>\n

Reactions to this turn of events have been decidedly mixed (though falling along existing lines for the most part). A few people<\/a> have (I think correctly) noted that the paper itself was of ‘minor consequence’ and does not explicitly claim anything much other than correlation analysis over a short time period isn’t going to constrain climate sensitivity, and that at first glance, there was a mismatch between models and observations in a particular calculation. The first claim is actually uncontroversial (despite what Spencer would have you believe), and the second turns out to be less interesting than it first seems (see Trenberth and Fasullo’s RC post<\/a>). However, the media and blogospheric interest in the paper had very little to do with the actual paper, rather it was provoked by the over-exaggerated press release<\/a> from UAH and the truly absurd piece<\/a> in Forbes by the Heartland Institute’s James Taylor. <\/p>\n

Roger Pielke Sr. has accused Wagner of ‘politicizing’ the situation by resigning, but this is completely backwards. The politicisation of the situation came almost entirely from Spencer and Taylor, and Wagner’s resignation is a recognition that he should have done a better job to prevent that. Statements from Ross McKitrick<\/a> that Wagner is a “grovelling, terrified coward” for his action are completely beyond the pale (as well as being untrue, possibly libelous, and were stated with no evidence whatsoever). <\/p>\n

The question has also arisen why the paper itself has not been retracted (and indeed will not be<\/a>). However, that would be a really big step. I can only think of two climate science related papers that have been retracted in recent years – one was for plagiarism (among other problems: Said et al, 2008<\/a>) and the other was because of a numerical calculation error that fatally undermined the reported results<\/a>. There are of course many, many more papers that are wrong, mistaken and\/or ‘bad’ (in the sense defined above) and yet very few retractions occur. I think (rightly) that people feel that the best way to deal with these papers is within the literature itself, and in this case it is happening this week in GRL (Dessler, 2011<\/a>), and in Remote Sensing in a few months. That’s the way it should be, and neither resignations nor retractions are likely to become more dominant – despite the amount of popcorn being passed around.<\/p>\n\n<\/div> ","protected":false},"excerpt":{"rendered":"

Much is being written about the very public resignation of Wolfgang Wagner from the editorship of Remote Sensing over the publication of Spencer and Braswell (2011) – and rightly so. It is a very rare situation that an editor resigns over the failure of peer review, and to my knowledge it has only happened once […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[1,24],"tags":[],"class_list":{"0":"post-8744","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-climate-science","7":"category-reporting-on-climate","8":"entry"},"aioseo_notices":[],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/8744","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/comments?post=8744"}],"version-history":[{"count":23,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/8744\/revisions"}],"predecessor-version":[{"id":8769,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/8744\/revisions\/8769"}],"wp:attachment":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/media?parent=8744"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/categories?post=8744"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/tags?post=8744"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}