{"id":2452,"date":"2009-12-17T13:38:45","date_gmt":"2009-12-17T18:38:45","guid":{"rendered":"http:\/\/www.realclimate.org\/?p=2452"},"modified":"2010-07-27T10:27:16","modified_gmt":"2010-07-27T15:27:16","slug":"please-show-us-your-code","status":"publish","type":"post","link":"https:\/\/www.realclimate.org\/index.php\/archives\/2009\/12\/please-show-us-your-code\/","title":{"rendered":"Please, show us your code"},"content":{"rendered":"
\n

The 1991 Science<\/em> paper by Friis-Christensen & Lassen, work by Henrik Svensmark (Physical Review Letters<\/em>), and calculations done by Scafetta & West (in the journals Geophysical Research Letters<\/em>, Journal of Geophysical Research<\/em>, and Physics Today<\/em>) have inspired the idea that the recent warming is due to changes in the sun, rather than greenhouse gases.<\/p>\n

We have discussed these papers before here on RealClimate (here<\/a>, here<\/a>, and here<\/a>), and I think it’s fair to say that these studies have been fairly influential one way or the other. But has anybody ever seen the details of the methods used, or the data? I believe that a full disclosure of their codes and data would really boost the confidence in their work, if they were sound. So if they believe so strongly that their work is solid, why not more transparency?<\/p>\n

<\/p>\n

There is a recent story<\/a> in the British paper The Independent<\/em>, where Friis-Christensen and Svensmark responded to the criticism forwarded by Peter Laut (here<\/a>). All this would perhaps be unnecessary if they had disclosed their codes and data.<\/p>\n

Gavin and I published a paper<\/a> in Journal of Geophysical Research<\/em>, where we tested the general approach used by Scafetta & West, and tried to repeat their analysis. We were up-front about our lack of success in a 100% replication<\/a> of their work, but we argue that the any pronounced effect – as claimed by Scafetta & West – should be detectable even if the set-up is not 100% identical.<\/p>\n

However, Scafetta does not accept our analysis and has criticized me for lacking knowledge about wavelet analysis – he tells me to read the text books. So I asked him to post his code openly on the Internet so that others could repeat our test with their code. That should settle our controversy.<\/p>\n

After repeated requests, he told me that he doesn’t really understand why I’m not able to write my own program to reproduce the calculations (actually, I did in the paper together with Gavin, but Scafetta wouldn’t accept our analysis), and keeps insulting me by telling me to take a course on wavelet analysis. Furthermore, he stated that there “are several other and even more serious problems” in our work. I figure then that the easiest way to get to the bottom of this issue it to repeat our tests with his code.<\/p>\n

A replication<\/a> in general doesn’t require full disclosure of source code because the description in the paper should be sufficient, though in this case it clearly wasn’t. So to both save having us do it again and perhaps miss some other little detail – in addition to using an algorithm that Scafetta is happy with – it’s worth getting the code with which to validate our efforts. <\/p>\n

It should be a common courtesy to provide methods requested by other scientists in order to speedily get to the essence of the issue, and not to waste time with the minutiae of which year is picked to end the analysis. <\/p>\n

The reason why Gavin and I were not able to repeat Scafetta’s analysis in exact details is that his papers didn’t disclose all the necessary details. The first point he raised was that we used periodic instead of reflection boundaries<\/a>. The fact that the paper referred to the expression ‘1\/2 A sin (2 pi t)’ to describe the temperatures or solar forcing would normally suggest that they used periodic rather than reflection boundaries. There was no information in the paper about reflection boundary. But this is no big deal, as we have subsequently repeated the analysis with reflection boundary, and that doesn’t alter our conclusions.<\/p>\n

After further communication, we found out that Scafetta re-sampled the data in such a way that the center of the wavelet band pass filter was located exactly on the 11 and 22 year solar cycles, which were the frequencies of interest. He also informed me that a reasonable choice of the year when the reflection boudary was made should be the year 2002-3 when the sun experienced a maximum for both the 11 and 22 year cycles. This information was not provided in the papers.<\/p>\n

I’m no psychic<\/a>, so I couldn’t have guessed that all this was needed to reproduce his result. But since Scafetta has lost faith in my ability to repeat his work, I think it’s even a greater reason to disclose his code so that others can have a go.<\/p>\n

For the record, we did not just use wavelets to filter the data – we obtained the same conclusion with an ordinary band-pass filter.<\/em><\/p>\n\n<\/div> ","protected":false},"excerpt":{"rendered":"

The 1991 Science paper by Friis-Christensen & Lassen, work by Henrik Svensmark (Physical Review Letters), and calculations done by Scafetta & West (in the journals Geophysical Research Letters, Journal of Geophysical Research, and Physics Today) have inspired the idea that the recent warming is due to changes in the sun, rather than greenhouse gases. We […]<\/p>\n","protected":false},"author":11,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[1,26,34],"tags":[],"class_list":{"0":"post-2452","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-climate-science","7":"category-rc-forum","8":"category-skeptics","9":"entry"},"aioseo_notices":[],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/2452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/comments?post=2452"}],"version-history":[{"count":45,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/2452\/revisions"}],"predecessor-version":[{"id":4660,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/2452\/revisions\/4660"}],"wp:attachment":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/media?parent=2452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/categories?post=2452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/tags?post=2452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}