Please, show us your code

The 1991 Science paper by Friis-Christensen & Lassen, work by Henrik Svensmark (Physical Review Letters), and calculations done by Scafetta & West (in the journals Geophysical Research Letters, Journal of Geophysical Research, and Physics Today) have inspired the idea that the recent warming is due to changes in the sun, rather than greenhouse gases.

We have discussed these papers before here on RealClimate (here, here, and here), and I think it’s fair to say that these studies have been fairly influential one way or the other. But has anybody ever seen the details of the methods used, or the data? I believe that a full disclosure of their codes and data would really boost the confidence in their work, if they were sound. So if they believe so strongly that their work is solid, why not more transparency?

There is a recent story in the British paper The Independent, where Friis-Christensen and Svensmark responded to the criticism forwarded by Peter Laut (here). All this would perhaps be unnecessary if they had disclosed their codes and data.

Gavin and I published a paper in Journal of Geophysical Research, where we tested the general approach used by Scafetta & West, and tried to repeat their analysis. We were up-front about our lack of success in a 100% replication of their work, but we argue that the any pronounced effect – as claimed by Scafetta & West – should be detectable even if the set-up is not 100% identical.

However, Scafetta does not accept our analysis and has criticized me for lacking knowledge about wavelet analysis – he tells me to read the text books. So I asked him to post his code openly on the Internet so that others could repeat our test with their code. That should settle our controversy.

After repeated requests, he told me that he doesn’t really understand why I’m not able to write my own program to reproduce the calculations (actually, I did in the paper together with Gavin, but Scafetta wouldn’t accept our analysis), and keeps insulting me by telling me to take a course on wavelet analysis. Furthermore, he stated that there “are several other and even more serious problems” in our work. I figure then that the easiest way to get to the bottom of this issue it to repeat our tests with his code.

A replication in general doesn’t require full disclosure of source code because the description in the paper should be sufficient, though in this case it clearly wasn’t. So to both save having us do it again and perhaps miss some other little detail – in addition to using an algorithm that Scafetta is happy with – it’s worth getting the code with which to validate our efforts.

It should be a common courtesy to provide methods requested by other scientists in order to speedily get to the essence of the issue, and not to waste time with the minutiae of which year is picked to end the analysis.

Page 1 of 2 | Next page