Please, show us your code

After further communication, we found out that Scafetta re-sampled the data in such a way that the center of the wavelet band pass filter was located exactly on the 11 and 22 year solar cycles, which were the frequencies of interest. He also informed me that a reasonable choice of the year when the reflection boudary was made should be the year 2002-3 when the sun experienced a maximum for both the 11 and 22 year cycles. This information was not provided in the papers.

I’m no psychic, so I couldn’t have guessed that all this was needed to reproduce his result. But since Scafetta has lost faith in my ability to repeat his work, I think it’s even a greater reason to disclose his code so that others can have a go.

For the record, we did not just use wavelets to filter the data – we obtained the same conclusion with an ordinary band-pass filter.

Page 2 of 2 | Previous page

233 comments on this post.
  1. Lynn Vincentnathan:

    I think we have to rethink the sun-factor.

    First of all, where would we be without the sun. Some call it global warming disaster, I call it life.

    Second, according to Hansen (see pp. 5-6 of ) we’ve been in a solar minimum for some years, while the world has warmed anyway. We’ll really be in hot water (maybe literally) once its irradiance starts increasing again.

    Not to mention other factors like the sun (and us) melting permafrost and hydrates, releasing gigatons of GHGs….

  2. Matthew:

    But has anybody ever seen the details of the methods used, or the data? I believe that a full disclosure of their codes and data would really boost the confidence in their work, if they were sound. So if they believe so strongly that their work is solid, why not more transparency?

    Since 1991, more and more journals have requested complete disclosure of data and code as a condition of publication. Just as you suggest. I think that has been the right course, though journals have not always enforced their own policies. Have you formally requested the data and code from those authors and been turned down? If so, have you filed FOIA requests? FWIW, I support your suggestion.

    (FWIW? OK, not worth so much. Just one statistician’s opinion.)

    [Response: My personal opinion is that FOIA requests are often used just a form of harrassement and I don’t think this case rises to the level of a legal case of great import. We’ll manage ok without the code, and in the event that the criticism is we still haven’t got it exactly right, people will be able to draw their own conclusions. (and yes, the code was requested (and promised ‘in a few days’ in August). – gavin]

  3. Matthew:

    26, comment by rasmus: Internal e-mails, however, are not very relevant – they only fuel charades and political spin because we will never know their true context and will never be able to understand their true meaning For instance, I often tell jokes and use sarcasm or irony when writing to close colleagues. Taken out of context, one would get a completely wrong impression.

    “You should never write anything in email that you would not want the whole world to read.” I received that advice about 20 years ago, and I pass it along whenever I can. Possibly, where you work, the IT managers have made you sign a piece of paper whereon you pledged never to use the computer system in any unprofessional manner whatsoever, or any manner that might bring dishonor to the institution. If so, this might be a good time to review that promise. If the CRU team had adhered to this ancient advice, or their computer use agreement, there would not be a “climategate”. Lots of the emails contain straightforward technical discussions of the best way to handle problems, and those are not a problem. But “to hide the decline” and a recommendation to delete emails that the law requires be kept — that was not good.

  4. DougT:

    He should release the code and the data.

  5. Ed:

    There is value – to the authors – in sharing code. I am a software engineer who reviewed partial folders of the CRU code release and I found obvious errors. Dr. John Graham-Cumming, in the UK also found potentially significant errors and expressed a view that the software quality did not meet necessary standards for an issue as serious as climate research. Many others have also found similar problems to us.

    Here at RealClimate you have written in the past that you had minimal budget for Software Quality Assurance (I believe it was .25 FTE) and you rely on others replicating the work. In the software world we apply a body of software engineering principles and practices to design, develop, verify and test that the software correctly implements the solution. Money is not infinite (unless you are a banker who makes money out of thin air) and there should be a desire to achieve desired quality levels in a cost efficient manner.

    From reading the comments here and on other blogs, a tremendous effort has been expended to attempt to replicate others’ work – sometimes with success and sometimes not. And often when something suspicious is found, a period of accusations and defense ensues. When all of it could be resolved quickly by looking at the source code.

    The arguments for not releasing source code have become increasingly hollow. If the CRU document dump had not occurred, neither they – nor we – would have been aware of their software coding errors. Now that the dump has occurred, we realize that much of the code did not meet acceptable standards and is questionably the basis upon which to base major decisions.

    [Response: The hacked documents were not ‘the CRU code’ in any real sense. They are random bits and pieces of research code that had been emailed to people at various times. I wouldn’t read too much into any bugs you found there given that you have no idea how old the code was or whether it was subsequently fixed (or abandoned). – gavin]

  6. Alex Harvey:

    You can add my voice to those of climate change skeptics agreeing that Scafetta et al. should release their code.

    Can I ask, does this mean that RealClimate scientists also now agree to release code? Even to Steve McIntyre? If so, this is a very happy development.

  7. AJ:

    Use the Monte Carlo for radiative transfer instead. Results will be unimpeachable. If you don’t, I will.

  8. macumazan:

    All men of goodwill must support you in your endeavour to check the code and data used.

  9. Matthew:

    After repeated requests, he told me that he doesn’t really understand why I’m not able to write my own program to reproduce the calculations (actually, I did in the paper together with Gavin, but Scafetta wouldn’t accept our analysis), and keeps insulting me by telling me to take a course on wavelet analysis. Furthermore, he stated that there “are several other and even more serious problems” in our work. I figure then that the easiest way to get to the bottom of this issue it to repeat our tests with his code.

    I am sorry. I did not notice the “More >>” button. You are correct: they ought to have honored your request for full disclosure.

  10. dhogaza:

    Can I ask, does this mean that RealClimate scientists also now agree to release code?

    What makes you think they haven’t already.

    Gavin Schmidt is a “Real Climate scientist”. After you’ve digested this source code, we can feed you links to more code.

  11. t_p_hamilton:

    Mike W asks for clarification, it looks like he is ready for
    skip through quickly to the multi-layer model. Basically, the radiation coming INTO a particular layer comes from above (fewer CO2) and below (more CO2). The net result is heating of the lower layers, and cooling of the upper layers (stratospheric cooling).

  12. Mesa:

    I think there is a reasonable standard of information and data that can be supplied in general cases to allow reproduction within reason of the essential results by skilled and informed practitioners. However, that standard of disclosure necessarily becomes much higher when the results are canonical in the field (ie on the cover of the IPCC report). Even higher standards of data, procedures, and algorithms should also apply to things like basic sets of temperature data that the entire field is based upon, and that are paid for on an ongiong basis by taxpayers.

  13. Doug Bostrom:

    Dennis Hamilton says: 17 December 2009 at 6:18 PM

    “Seems to me that the solar piece of this puzzle needs some serious attention…”

    Extensively investigated and here’s a fabulous place to review the results:

  14. skeptic:

    A breathrough moment! Both sides working together to establish an atmosphere of transparency and replicability of scientific work. I sincerely hope it works out.

    Next thing you know there will be meaningful debate without name-calling. Must be the Christmas Season.

    Of course he should turn it over!

    And more importantly: all future work should adhere to the same standard.

    When I went to school, it was the standard. But that was long ago.

  15. Mike W:

    Chris, caught your response to my thought experiment (#37)at #50.

    The conversion and balance of vibrational and translational energy is not a trivial matter, and the definition of terms really is important. Here’s a small extract from the Wiki on the speed of sound, odd place to find what I was looking for, but still entirely relevant.”since vibration modes in a polyatomic gas gives the gas additional ways to store heat which do not affect temperature, and thus do not affect molecular velocity and sound velocity.” – of course here they are using temperature in the sense of its meaning for an ideal gas, still a useful meaning.

    Even in the kinetic theory of gasses the pressure is explained as the exchange of momentum with the gas molecules and the wall of the container. That’s translational energy at work not vibrational.

    You can put more “heat” in the gas with IR but it’s unlikely to effect the translational energy, unless there is some sort of asymmetry whereby a vibrationally exited CO2 molecule is more likely to lose energy than gain energy during a number of collisions.

    IR “heating” of gasses is IR heating of gasses, there’s not much else to compare it to. Microwave radiation is a lower energy than IR and effects rotational energy in molecules, rotational levels are strongly affected by molecular collisions, whereas vibrational levels are weakly affected. Furthermore, the glass of water is a liquid, not a gas, absorbs radiation very differently.

  16. Steve Fish:

    Comment by Alex Harvey — 17 December 2009 @ 8:44 PM:

    You say– “..does this mean that RealClimate scientists also now agree to release code?”

    I say– What evidence do you have that they have not already released their code and data?


  17. Rod B:

    Mike W (37), from what I understand I think you have the basic processes down pretty well, though there are a couple of areas where it almost falls off the track. [And, disagreeing with Chris Crawford, I do think it is worth worrying over the details which many miss yet which is key in the physics. (Chris C. is none-the-less correct in his basic answer.).] Only the areas where I would demur:

    1) The absorption or emission of IR radiation to/from the vibration energy levels does not change the temperature of the gas. The temperature changes only when this energy is transferred to some other molecule’s translation energy (this molecule can be and most likely is a molecule that does not absorb IR, like O2 or N2.)

    2) The CO2 molecule absorbs, emits, or transfers all or none of the energy in a vibration (or rotation) mode — no partials.

    3) the transfer from vibration to translation is hardly restricted: a molecule’s translation energy is not quantized (for all practical purposes) and can take all and more of what the vibration wants to give it. Collisions are frequent (multi-billions/second). Once transferred via collision, the CO2 molecule has nothing to, and won’t, emit anything (other than with a totally new and different photonic sequence).

    4) the transfer from translation to vibration is less frequent but not terribly constrained, and is a completely independent process from the vib-to-trans transfer.

    5) Whether a CO2 molecule emits a photon or (to some extent) willing accepts energy from a collision is a determination of quantum mechanics. As a rule, at these temperatures it doesn’t “like” energy in a vibration mode, especially if colder. It will emit a photon if within a short period (whose length is problematic ala QM) it hasn’t spit it out through a collision. This is more likely to happen in the upper atmosphere where collisions are less frequent.

    6) an emitted photon will look exactly like the absorbed photon (ignoring doppler, etc per your post).

    7) the average overall translation energy (ala 1/2mv^2) might or might not stay the same. [It can be lost through other than the transfers described above.] Will if all is in equilibrium; might not if not — ergo the start of global warming.

    Hope this helps.

  18. MikeS:

    Hmmmmmm…. the whole finger-pointing toward solar forcing at a time when the real question is about who’s doing what to the data in the hallowed inner-circle of the IPCC… well, it just smacks of trying to change the subject and say “Johnny did it, too, Mom”. Considering the adjustment trend to the raw data that is coming out, I’d expect a little more attention to the subject at hand rather than trying to start a diversion.

  19. Eli Rabett:

    Mike W. in 37 describes a CO2 molecule absorbing a photon at 4.3 microns and then says that after collisions, etc. it will “then emit an IR photon almost identical to the one it absorbed (Kirchoff’s Law?)”

    Actually no and the reason is interesting. CO2 has four ways of vibrating. Absorbing the 4.3 micron photon excites what is called the a symmetric stretch. Hopefully this works, but that looks like

    O –>

    There is another mode, called the symmetric stretch, which looks like this

    C The C atom does not move

    These are both relatively high energy, The symmetric stretch has about 7-8 times more energy than the average thermal energy at room temperature, 295 K in science speak, but it cannot radiate. The asymmetric stretch, which can radiate (at 4.3 microns) has about 12 times the average thermal energy

    On the other hand there are two lower energy bending vibrations that look like
    (this IS gonna be tough)

    ^ ^
    | |
    | |
    O C O

    Nice animated pictures @

    The other bend in in the plane perpendicular to the screen. The energy involved in the bending vibrations is ~3.5 times thermal. What is going to happen is that the CO2 molecule that absorbs the photon is very rapidly going to lose that energy to collisions. This occurs much faster than direct radiation.

    Once that happens of course, some of the unexcited (ground state) CO2 can GAIN vibrational energy by collisions! Under most atmospheric conditions (pressure and temperature), the fraction of CO2 in the bending mode at any time according to a Boltzmann distribution will be g exp(-Evib/kT) where g is the number of vibrations at the same frequency (called the degeneracy) Evib the vibrational energy, k the Boltzmann constant and T the temperature. kT is the thermal energy, so the probability of finding a CO2 molecule vibrationally excited in one of the bending modes is 2 exp(-3.5) and in the asymmetric stretch exp(-12), or roughly 6% and 0.0006% respectively. So most of the emission from your balloon has to come from the excited bending vibration which at 17 microns.

    To get any significant emission at 4.3 microns we are talking about Venus hot and beyond, but even then more would come out at 17 microns

    Hope that helped.

    OTOH a

  20. Norman:

    John Lennon “Gimmie some Truth”

    “I’m sick to death of seeing things
    From tight-lipped, condescending, mama’s little chauvinists
    All I want is the truth
    Just gimme some truth now

    I’ve had enough of watching scenes
    Of schizophrenic, ego-centric, paranoiac, prima-donnas
    All I want is the truth now
    Just gimme some truth”

  21. Eli Rabett:

    Let me post something more relevant. In this case Rasmus and Gavin have made good faith efforts to replicate Scarfetta’s calculation. The codes have to be compared to figure out where they disagree.

    The situation that this best resembles is that of the UAH and RSS codes, which disagreed, and it was only on comparison that a series of small errors were dug out (on both sides).

  22. steven mosher:

    Nicola Scafetta played this same stupid game over at CA. I think we asked him for the code as well and he ran away. Kudo’s for keeping after him. Written descriptions of algorithms rarely given enough information to implement them and this little game he plays of trying to have you read his mind isnt funny. Its a waste of your valuable time.

    [Response: Much as it pains me to disagree with you, most published descriptions do allow for replication without posting of code. This one didn’t. It is a waste of our time though. – gavin]

  23. Greg Leisner:

    My son is at AGU this week (his field is Space Science). He said before going that he was going to go to some of the Climate Science panel discussions to watch the fights.

    And I thought that he was joking. But this thread “feels” like he read the pulse of the Climate Community correctly.

  24. Peter Backes:

    RE: 9

    LOL – Turnabout is fair play! Way to go RC.

    So now the question is whether or not the CRU hackers will go after Friis-Christensen and Svensmark code. I won’t hold my breath…

  25. steven mosher:

    Let me just direct people to some resources and give rasmus kudos for giving his code to Lucia.

  26. Spaceman Spiff:

    I’ll add a point to post #50, which is responding to post #37 re. the absorption of IR radiation and the temperature of the gas.

    When IR radiation is absorbed by molecules, the result is not always radiative de-excitation. A collision between the excited molecule and another molecule, before the former can decay radiatively, then increases the kinetic energy of the colliding molecule — collisional de-excitation. In fact in local thermodynamic equilibrium the collision rates exceed the radiative decay rates. I don’t model energy transfer in earth’s atmosphere for a living, but I would be surprised if the above were not important over most or even all of the troposphere. I hope that helps.

  27. Elliot:

    Free data and code for all. Perhaps both sides offend on this count it seems. At least he hasn’t been caught destroying it yet.

    Note you dont actually post the responses from Scafetta so we only have your word he responded in such ways.

  28. Peter:



    I agree about the need for general need for transparency in the data and methods.

    If they release the data and it shows that they took actual measurements of solar activity pumped it through a formula that itself induced a trend, mixed in disparate sets of proxies (i dunno – tanning behaviour of Germans 1940-1970, cataracts in corpses buried 1850-1910, whatever) then discarded the data that wasn’t convenient to show what ever they wanted to show then I would join in the pillory of sarcasm.

  29. Steve L:

    I find this post and thread a bit surprising, not because of the request for code (which I think is reasonable), but because there’s been little promotion of the primary literature as the best forum in which to have this scientific debate. Surely rather than criticize Rasmus, Scafetta should have criticized the approach in a peer-reviewed response. The editors/reviewers of the paper thought Rasmus and Gavin’s approach had some merit, so a clear explication in the literature would have been valuable. No?

  30. Neil Pelkey:

    R “script kiddies” can use the following to have their own look at the raw station data.
    You will need wget however.

    wget is used rather than R to get data due to firewall issues.

  31. Chris Crawford:

    Dennis Hamilton in #41 calls the changes in CO2 concentration “subtle” while at the same time asserting that the sun is showing “large changes” in radiative output. Both statements are incorrect. CO2 concentrations have increased by about 30%, while secular changes in solar radiative output are less than 1%

  32. Pierre-Andre Morin:

    Just go back and publich in the same journal.
    Nothing like a study that proves that the original results can not be repeated.

    Isn’t that science, you can repeat the results?
    You could also publish all versions of code.
    This you put the burden on them to prove you wrong.

  33. Mike W:

    Thankyou guys, always exiting to be able to draw back the curtain on the world like this!

    Rod, Eli (was hoping you’d contribute, I read some great stuff of yours written in 2007 in response to a similar question) Spaceman; great answers! Will need some time to digest them and do some more reading, may have some more qualifying questions.

    Figured there would have to be an asymmetry for it to work, the message I’m taking away is that it is “easier” for vibration to “become” translational energy than it is for translational energy to become vibrational energy. I’m just reading a bit about equipartition and the ‘freezing out’ of rotational and vibrational modes….I think I see how it fits into the picture here.

    Of course, now I want to know how much of the upwelling IR is thermalised? and “wasted” as translational energy, and how much is re-radiated.

    Eli et al, how is the orignal momentum of the absorbed photon conserved? It annoys me a little that it’s rarely mentioned and often just waved away.

    Cheers, Mike W.

  34. steven mosher:

    re 77. Eliot. In august steve mc asked for his code. I did. Lucia did. and Jeffid did.
    Scafetta played a little insulting “i know more than you” game that was entirely consistent with the descriptions that Rasmus gives in his account.

  35. steven mosher:

    re 72. Gavin. We can agree to disagree on the replication thing. People have actually tested whether or not people could replicate from a written description, even their own work!. However, I overstated the case by saying “rarely”.

  36. Allen:

    Moral equivalence: “they don’t share their methods or data either, therefore our recalcitrance is not a problem.”


    This article is my first contact with, thinking to take in your views unfiltered and judge for myself. Chosen essentially at random I happen on a moral equivalence argument attempting to justify the lack of transparency that is seemingly inherent to the practice of climate “science.” To my mind you have only affirmed your critics.

  37. Barton Paul Levenson:


    Okay, here’s the truth. John shouldn’t have committed adultery, divorced his wife to marry his girlfriend, taken hard drugs, abandoned his childhood beliefs, or fired Pete Best. And Yoko really wasn’t that great a musician.

  38. Eukaryt:

    Good timing with this article.

    And again the media fail in researching the topic enough.

  39. Steve:

    I think the problem with most of these rogue scientists is that when
    they started, there was still enough doubt for their views to be given
    credence. Now they have been largely discredited and feel they have
    nothing left to loose, so they keep fighting their corner.

    Some are forced to take money from oil companies and book sales instead of
    research grants because the scientific community has left them behind.


  40. Completely Fed Up:

    “At least he hasn’t been caught destroying it yet.”

    Neither has anyone in CRU.

    Unless you think the CRU have a crack paramilitary team of Ninjas with magnets and a box of matches to go to foreign libraries to burn all the books with data they don’t want?

  41. Completely Fed Up:

    “And I thought that he was joking. But this thread “feels” like he read the pulse of the Climate Community correctly.”

    My daddy taught me never to start a fight, but always finish it.

    (For the folksy.)

  42. Lawrence Coleman:

    Does anyone know to what extent a further 2C warming will cause glacial retreat and to which altitudes? As far as I can see ‘any’ additional warming from the current will cause catastrophic problems to the world’s fresh water supplies. The 2C figure presumably represents the global mean temp rise indicating that the arctic/arctic regions will record a higher figure..say 2.5C and that the equatorial regions a relatively lower one.say 1.5C. Following on that..I would assume that the arctic/antarctic glaciers will retreat faster and further and at higher altitude than the himalayan glaciers? Using good ol’ common sense even with the temp rise at present the world will experience critical water shortages in say 30-50 years time. If you envisage a further 1-2C rise even with agressive global emissions targets in play there will be masse population migrations to where is is still reliable water supplies. So any further rise of temp from the current will lock in a severe water crises in the medium I right?

  43. Completely Fed Up:

    “When I went to school, it was the standard. But that was long ago.”

    Rose-tinted glasses are not a reliable way to recall the truth of the past.

    Mind you, when I went to school, source code wasn’t copyrightable.

    But try and tell Oracle you have to have their source code now…

    PS NOTE: The raw data was never usually included in scientifit reports and the data that was was small enough to fit on a couple of pages in table format (with plenty of blank space to make it readable).

    Try that with a dataset of 4 trillion floating point numbers…

    Also note one interlocutor was 100% behind the idea of the raw data including the test spec and calibration results of the thermometers.

    Go and find ANY paper that produced or kept the calibration of their instruments.

    Just one.

    In the entire available history.

  44. greenJamie:

    Anyone with any support for Ian Plimer needs their head examining

  45. Completely Fed Up:

    Mesa: “and that are paid for on an ongiong basis by taxpayers.”

    But you don’t PAY Russian taxes. So why should YOU get free access to THEIR paid for data?

    Pay them for the data and they’ll give it to you. But you’ll have to agree to an NDA just like you do when you enter into Microsoft’s Shared Source initiative.

    Why? Because you could give it away to others. You paid only a small cost for it, not the entire cost. And, just like P2P sharing isn’t solved by the P2Per paying the price of the tracks because they distributed the stuff for free to others, you can’t solve the loss to the taxpayers when you share for free the data the Russian taxpayers paid for by saying “I paid for it! It’s mine!”.

  46. Completely Fed Up:

    “Here is an independent interview on the CRU data. He states he is not a sceptic and points out problems in the code. I would appreciate your take on this. Thanks”

    But posts on a blogsite called “hot air”???

    But first blush: he doesn’t know the physics, so how can he tell if it is right?

    PS pay your government more taxes for science work and they’ll be able to pay programmers (who cost more than scientists of the same stature) to refactor the code. This means more taxes, but you want GOOD QUALITY code, you gotta pay for it.

  47. Scott A. Mandia:

    #20 Rasumus:

    $urely $. Fred $inger (who $hare$ the Nobel for hi$ IPCC review work) doe$ not believe “it’$ the $un” de$pite what he $ay$ and write$ about global warming.

    BTW, wouldn’t a better title for this thread be “Touche'”?

  48. Completely Fed Up:

    Oh, he’s looking at the leaked code, pointing at the COMMENTED OUT “fudge factor” line and probably going: See, this is where we see they fudged the data.

    He’s pointing to GOTOs.

    Device drivers in Linux use GOTOs. Not using GOTOs is to make code clearer. But sometimes a GOTO is necessary. “Rules are for the guildance of wise men and obedience of by fools”. Or as I prefer “Rules are there to make you think before you break them”.

    And Case statements, etc boil down to JMP instructions which are GOTOs anyway.

    He points to “I have poor programming skills”. Well this means the programmer isn’t falling into Dunning-Kruger territory. He’s a good enough programmer to know that. And this is not proof that the code is wrong.

    ALL software has bugs. You pay for Windows, yet there are bugs. “My poor programming has struck again!”. And even Dr Cumming will have made mistakes.

    Does this prove he should be sacked and made to work tables instead?

  49. MB:

    Yes, all data and programs should be available for inspection to go along with any paper that wishes to become part of the body of knowledge. If results have not been independently verified and are not independently verifiable, then they are not part of science – by definition.

    I think you would struggle to find a single arguement why not to be completely open and transparent. If you have one, I would love to hear it.

    [Response: You are over-stating the issues. Openness + Transparency is of course necessary. However, replication in general doesn’t require turnkey code for every random paper. If someone says they used linear regression, you can do that yourself. If the say the used an EOF decomposition of the sea level pressure field, then you can do it yourself. I discussed these issues in a piece earlier this year. The code only becomes an issue here because, a) we tried to replicate on our own and didn’t match 100%, and b) the criticism of our paper is based strongly on that single issue. We don’t think it is particularly relevant (since the sensitivities to co-linearity and noise will still be there and were when we used different approaches), but rather than have a pointless argument about the exact method, we’d rather have a discussion on more interesting issues. – gavin]

  50. Ray Ladbury:

    Mike W.,
    You are ignoring the fact that the mode being excited by CO2 is vibrational and so represents kinetic energy (internal, to be sure). Thus, collisional de-excitation by a non-ghg molecule is quite probable. This HAS TO happen given equipartition, no?