A Mistake with Repercussions

Today, Science published an important comment pointing out that there were serious errors in a climate research article that it published in October 2004. The article concerned (Von Storch et al. 2004) was no ordinary paper: it has gone through a most unusual career. Not only did it make many newspaper headlines [New Research Questions Uniqueness of Recent Warming, Past Climate Change Questioned etc.] when it first appeared, it also was raised in the US Senate as a reason for the US not to join the global climate protection efforts. It furthermore formed a part of the basis for the highly controversial enquiry by a Congressional committee into the work of scientists, which elicited sharp protests last year by the AAAS, the National Academy, the EGU and other organisations. It now turns out that the main results of the paper were simply wrong.

Von Storch et al. claimed to have tested the climate reconstruction method of Mann et al. (1998) in model simulations, and found it performed very poorly. Now, Eugene Wahl, David Ritson and Caspar Amman show that the main reason for the alleged poor performance is that Von Storch et al. implemented the method incorrectly. What Von Storch et al. did, without mentioning it in their paper, was to remove the trend before calibrating the method against observational data – a step that severely degrades the performance of Climate Field Reconstruction (CFR) methods such as the Mann et al. method (unfortunately this erroneous procedure has already been propagated in a paper by Burger and Cubasch (GRL, 2005) where the authors refer to a personal communication with Von Storch to justify the use of the procedure). Another more recent analysis has shown that CFR methods perform well when used correctly. (See our addendum for a less technical description of what this is all about).

How big a difference does this all make? The calibration error in the temperature minimum around 1820, where one of the largest errors occurs, is shown as 0.6ºC in the standard case of 75% variance in the Von Storch et al analysis. This error reduces to 0.3ºC even in the seriously drift-affected ECHO-G run when the erroneous detrending step is left out. In the more realistic HadCM3 simulation, this error is just above 0.1ºC. The error margins (2 sigma) provided by Mann et al. and pictured in the IPCC report are ±0.17ºC (Fig. 2.21, the curves are reproduced in our addendum). It is therefore clear that the model test of Von Storch et al, had it been implemented correctly, would have shown a small but undramatic underestimation of variance and would have barely ruffled a feather.

Error made, error corrected, and all is well? Unfortunately not. A number of questions remain, which need to be resolved before the climate science community can put this affair to rest.

The first is: why did it take so long to correct this error, and why did the authors of the original paper not correct it themselves? The error is reasonably easy to spot, even for non-specialists (see addendum). And it was in fact spotted very soon after publication. In January 2005, a comment was submitted to Science which correctly pointed out that Von Storch et al. had calibrated with detrended data and had therefore not tested the Mann et al. method. As such comments are routinely passed to the original authors for a response, Von Storch et al. must have become aware of their mistake at this point at the latest. However, the comment was rejected by Science in May 2005.

Page 1 of 3 | Next page