RealClimate logo

Technical Note: Sorry for any recent performance issues. We are working on it.

The most common fallacy in discussing extreme weather events + Update

Filed under: — stefan @ 25 March 2014

Does global warming make extreme weather events worse? Here is the #1 flawed reasoning you will have seen about this question: it is the classic confusion between absence of evidence and evidence for absence of an effect of global warming on extreme weather events. Sounds complicated? It isn’t. I’ll first explain it in simple terms and then give some real-life examples.

The two most fundamental properties of extreme events are that they are rare (by definition) and highly random. These two aspects (together with limitations in the data we have) make it very hard to demonstrate any significant changes. And they make it very easy to find all sorts of statistics that do not show an effect of global warming – even if it exists and is quite large.

Would you have been fooled by this?

Imagine you’re in a sleazy, smoky pub and a stranger offers you a game of dice, for serious money. You’ve been warned and have reason to suspect they’re using a loaded dice here that rolls a six twice as often as normal. But the stranger says: “Look here, I’ll show you: this is a perfectly normal dice!” And he rolls it a dozen times. There are two sixes in those twelve trials – as you’d expect on average in a normal dice. Are you convinced all is normal?

You shouldn’t be, because this experiment is simply inconclusive. It shows no evidence for the dice being loaded, but neither does it provide real evidence against your prior suspicion that the dice is loaded. There is a good chance for this outcome even if the dice is massively loaded (i.e. with 1 in 3 chance to roll a six). On average you’d expect 4 sixes then, but 2 is not uncommon either. With normal dice, the chance to get exactly two sixes in this experiment is 30%, with the loaded dice it is 13%[i]. From twelve tries you simply don’t have enough data to tell.


In 2005, leading hurricane expert Kerry Emanuel (MIT) published an analysis showing that the power of Atlantic hurricanes has strongly increased over the past decades, in step with temperature. His paper in the journal Nature happened to come out on the 4th of August – just weeks before hurricane Katrina struck. Critics were quick to point out that the power of hurricanes that made landfall in the US had not increased. While at first sight that might appear to be the more relevant statistic, it actually is a case like rolling the dice only twelve times: as Emanuel’s calculations showed, the number of landfalling storms is simply far too small to get a meaningful result, as those data represent “less than a tenth of a percent of the data for global hurricanes over their whole lifetimes”. Emanuel wrote at the time (and later confirmed in a study): “While we can already detect trends in data for global hurricane activity considering the whole life of each storm, we estimate that it would take at least another 50 years to detect any long-term trend in U.S. landfalling hurricane statistics, so powerful is the role of chance in these numbers.” Like with the dice this is not because the effect is small, but because it is masked by a lot of ‘noise’ in the data, spoiling the signal-to-noise ratio.

Heat records

The number of record-breaking hot months (e.g. ‘hottest July in New York’) around the world is now five times as big as it would be in an unchanging climate. This has been shown by simply counting the heat records in 150,000 series of monthly temperature data from around the globe, starting in the year 1880. Five times. For each such record that occurs just by chance, four have been added thanks to global warming.

You may be surprised (like I was at first) that the change is so big after less than 1 °C global warming – but if you do the maths, you find it is exactly as expected. In 2011, in the Proceedings of the National Academy we described a statistical method for calculating the expected number of monthly heat records given the observed gradual changes in climate. It turns out to be five times the number expected in a stationary climate.

Given that this change is so large, that it is just what is expected and that it can be confirmed by simple counting, you’d expect this to be uncontroversial. Not so. Our paper was attacked with astounding vitriol by Roger Pielke Jr., with repeated false allegations about our method (more on this here).


European summer temperatures for 1500–2010. Vertical lines show the temperature deviations from average of individual summers, the five coldest and the five warmest are highlighted. The grey histogram shows the distribution for the 1500–2002 period with a Gaussian fit shown in black. That 2010, 2003, 2002, 2006 and 2007 are the warmest summers on record is clearly not just random but a systematic result of a warming climate. But some invariably will rush to the media to proclaim that the 2010 heat wave was a natural phenomenon not linked to global warming. (Graph from Barriopedro et al., Science 2011.)


Heat records can teach us another subtle point. Say in your part of the world the number of new heat records has been constant during the past fifty years. So, has global warming not acted to increase their number? Wrong! In a stationary climate, the number of new heat records declines over time. (After 50 years of data, the chance that this year is the hottest is 1/50. After 100 years, this is reduced to 1/100.)  So if the number has not changed, two opposing effects must have kept it constant: the natural decline, and some warming. In fact, the frequency of daily heat records has declined in most places during the past decades. But due to global warming, they have declined much less than the number of cold records, so that we now observe many more hot records than cold records. This shows how some aspects of extreme events can be increased by global warming at the same time as decreasing over time. A curve with no trend does not demonstrate that something is unaffected by global warming.


Drought is another area where it is very easy to over-interpret statistics with no significant change, as in this recent New York Times opinion piece on the serious drought in California. The argument here goes that man-made climate change has not played “any appreciable role in the current California drought”, because there is no trend in average precipitation. But that again is a rather weak argument, because drought is far more complex than just being driven by average precipitation. It has a lot to do with water stored in soils, which gets lost faster in a warmer climate due to higher evaporation rates. California has just had its warmest winter on record. And the Palmer Drought Severity Index, a standard measure for drought, does show a significant trend towards more serious drought conditions in California.

The cost of extreme weather events

If an increase in extreme weather events due to global warming is hard to prove by statistics amongst all the noise, how much harder is it to demonstrate an increase in damage cost due to global warming? Very much harder! A number of confounding socio-economic factors clouds this issue which are very hard to quantify and disentangle. Some factors act to increase the damage, like larger property values in harm’s way. Some act to decrease it, like more solid buildings (whether from better building codes or simply as a result of increased wealth) and better early warnings. Thus it is not surprising that the literature on this subject overall gives inconclusive results. Some studies find significant damage trends after adjusting for GDP, some don’t, tempting some pundits to play cite-what-I-like. The fact that the increase in damage cost is about as large as the increase in GDP (as recently argued at FiveThirtyEight) is certainly no strong evidence against an effect of global warming on damage cost. Like the stranger’s dozen rolls of dice in the pub, one simply cannot tell from these data.

The emphasis on questionable dollar-cost estimates distracts from the real issue of global warming’s impact on us. The European heat wave of 2003 may not have destroyed any buildings – but it is well documented that it caused about 70,000 fatalities. This is the type of event for which the probability has increased by a factor of five due to global warming – and is likely to rise to a factor twelve over the next thirty years. Poor countries, whose inhabitants hardly contribute to global greenhouse gas emissions, are struggling to recover from “natural” disasters, like Pakistan from the 2010 floods or the Philippines and Vietnam from tropical storm Haiyan last year. The families who lost their belongings and loved ones in such events hardly register in the global dollar-cost tally.

It’s physics, stupid!

While statistical studies on extremes are plagued by signal-to-noise issues and only give unequivocal results in a few cases with good data (like for temperature extremes), we have another, more useful source of information: physics. For example, basic physics means that rising temperatures will drive sea levels up, as is in fact observed. Higher sea level to start from will clearly make a storm surge (like that of the storms Sandy and Haiyan) run up higher. By adding 1+1 we therefore know that sea-level rise is increasing the damage from storm surges – probably decades before this can be statistically proven with observational data.

There are many more physical linkages like this – reviewed in our recent paper A decade of weather extremes. A warmer atmosphere can hold more moisture, for example, which raises the risk of extreme rainfall events and the flooding they cause. Warmer sea surface temperatures drive up evaporation rates and enhance the moisture supply to tropical storms. And the latent heat of water vapor is a prime source of energy for the atmosphere. Jerry Meehl from NCAR therefore compares the effect of adding greenhouse gases to putting the weather on steroids.

Yesterday the World Meteorological Organisation published its Annual Statement on the Climate, finding that “2013 once again demonstrated the dramatic impact of droughts, heat waves, floods and tropical cyclones on people and property in all parts of the planet” and that “many of the extreme events of 2013 were consistent with what we would expect as a result of human-induced climate change.”

With good physical reasons to expect the dice are loaded, we should not fool ourselves with reassuring-looking but uninformative statistics. Some statistics show significant changes – but many are simply too noisy to show anything. It would be foolish to just play on until the loading of the dice finally becomes evident even in highly noisy statistics. By then we will have paid a high price for our complacency.


Postscript (29 March):

The Huffington Post has the story of the letters that Roger Pielke sent to two leading climate scientists, perceived by them as threatening, after they criticised his article: FiveThirtyEight Apologizes On Behalf Of Controversial Climate Science Writer. According to the Huffington Post, Pielke wrote to Kevin Trenberth and his bosses:

Once again, I am formally asking you for a public correction and apology. If that is not forthcoming I will be pursuing this further. More generally, in the future how about we agree to disagree over scientific topics like gentlemen?

Pielke using the word “gentlemen” struck me as particularly ironic.

How gentlemanly is it that on his blog he falsely accused us of cherry-picking the last 100 years of data rather than using the full available 130 years in our PNAS paper Increase of extreme events in a warming world, even though we clearly say in the paper that our conclusion is based on the full data series?

How gentlemanly is it that he falsely claims “Rahmstorf confirms my critique (see the thread), namely, they used 1910-2009 trends as the basis for calculating 1880-2009 exceedence probabilities,” when I have done nothing of the sort?

How gentlemanly is it that to this day, in a second update to his original article, he claims on his website: “The RC11 methodology does not make any use of data prior to 1910 insofar as the results are concerned (despite suggestions to the contrary in the paper).” This is a very serious allegation for a scientist, namely that we mislead or deceive in our paper (some colleagues have interpreted this as an allegation of scientific fraud). This allegation is completely unsubstantiated by Pielke, and of course it is wrong.

We did not respond with a threatening letter – not our style. Rather, we published a simple statistics tutorial together with our data and computer code, hoping that in this way Pielke could understand and replicate our results. But until this day we have not received any apology for his false allegations.

Our paper showed that the climatic warming observed in Moscow particularly since 1980 greatly increased the chances of breaking the previous July temperature record (set in 1938) there. We concluded:

For July temperature in Moscow, we estimate that the local warming trend has increased the number of records expected in the past decade fivefold, which implies an approximate 80% probability that the 2010 July heat record would not have occurred without climate warming.

Pielke apparently did not understand why the temperatures before 1910 hardly affect this conclusion (in fact increasing the probability from 78% to 80%), and that the linear trend from 1880 or 1910 is not a useful predictor for this probability of breaking a record. This is why we decomposed the temperature data into a slow, non-linear trend line (shown here) and a stochastic component – a standard procedure that even makes it onto the cover picture of a data analysis textbook, as well as being described in a climate time series analysis textbook. (Pielke ridicules this method as “unconventional”.)

He gentlemanly writes about our paper:

That some climate scientists are playing games in their research, perhaps to get media attention in the larger battle over climate politics, is no longer a surprise. But when they use such games to try to discredit serious research, then the climate science community has a much, much deeper problem.

His praise of “serious research” by the way refers to a paper that claimed “a primarily natural cause for the Russian heat wave” and “that it is very unlikely that warming attributable to increasing greenhouse gas concentrations contributed substantially to the magnitude of this heat wave.” (See also the graph above.)


Update (1 April):

Top hurricane expert Kerry Emanuel has now published a very good response to Pielke at FiveThirtyEight, making a number of the same points as I do above. He uses a better analogy than my dice example though, writing:

Suppose observations showed conclusively that the bear population in a particular forest had recently doubled. What would we think of someone who, knowing this, would nevertheless take no extra precautions in walking in the woods unless and until he saw a significant upward trend in the rate at which his neighbors were being mauled by bears?

The doubling of the bear population refers to the increase in hurricane power in the Atlantic which he showed in his Nature article of 2005 – an updated graph of his data is shown below, from our Nature Climate Change paper A decade of weather extremes.



Related posts:

Extremely hot

On record-breaking extremes

The Moscow warming hole


[i] For the math-minded: if a dice has a probability of 1/n to roll a six (a normal dice has n=6) and you roll it k times, the probability p to find m sixes is p = k!/[(k-m)!m!] × (n-1)(k-m)/nk.

59 Responses to “The most common fallacy in discussing extreme weather events + Update”

  1. 51
    Simon C says:

    Perhaps a slightly more likely “Yes Minister” scenario (probabilities, anyone?):

    Minister: The latest IPCC report finds global warming to be in line with that predicted, and largely caused by humans…we’d better do something.
    Sir Humphrey: Oh dear Minister, I’m sure you must be mistaken. Addressing greenhouse warming would entail missing our spending targets.
    Minister: But look here, Sir Humphrey, we can’t just dismiss the massive costs caused by these extreme climate events, can we?
    Sir Humphrey: Well, if we insist they are natural, no-one can blame us for them, can they? Or failing that we could at least say that we can’t be absolutely certain to what extent humans are responsible.
    Minister. I suppose you’re right. But the scientists might say “we told you so!”
    Sir Humphrey: Yes, Minister, but we could always point out that some scientists disagreed with them …
    Minister: But those scientists were always a minority, and will probably turn out to be entirely wrong!
    Sir Humphrey: Yes, Minister, but I’m sure people will understand that we couldn’t possibly have known that at the time..

  2. 52
    Walter says:

    Hi, I saw this Pielke issue show up elsewhere. Anyone who takes this:

    “Once again, I am formally asking you for a public correction and apology. If that is not forthcoming I will be pursuing this further.”

    … as being “threatening” is both a chicken little claiming the sky is falling and the boy who cried wolf.

    More people than just Pielke could grow up a little, or at least seek some intelligent advice before jumping on the Paranoia Bus to nowhere. More than that, even IF that did mean legal action, then so what? people have rights and if they feel they have been infringed then expressing the outrage in a form that suggests you feel that deeply about that ‘offense’ you’re considering legal action is actually a RATIONAL comment to make to another to impress upon them the “importance” of the matter to yourself.

    BUT Pielke never said a word about legal action anyway, which logically follows that the one’s labeling his comments as some kind of “threat” is false hyper-sensitive paranoid and self-defeating. It’s an embellished beat up iow.

    and of course the absence of evidence is evidence of absence. I thought would be obvious, however luckily I discovered years ago that what is supposedly bleeding obvious, well simply isn’t to many on all kinds of levels. and that causes confusions on all sides and therefore a direct connection to very poor communication outcomes.

    Many wished and assumed the content of the AR4 would have been already “obvious” to all – fact is it wasn’t, still isn’t and the AR5 will make no difference because the communication system to the public remains unchanged. Pielke et al are not responsible for this and is a distraction form what is actually important and can be changed for the better. chow

  3. 53
    Gaudenz Mischol says:

    My oh my, just an example of someone who is so much overconfident what he knows.
    Our world isn’t simple physics. Sometimes it would be wise to be a little more humble..

  4. 54
    Hank Roberts says:

    “Walter” is probably alluding to

    (He doesn’t cite his source, that’s my guess about the basis for his story)

  5. 55
    Matthew R Marler says:

    Comment by Steve Metzler — 26 Mar 2014 @ 4:54 PM

    Thank you for the link to the quantile regression. We need more of that and generalized extreme value distribution modeling/estimation.

  6. 56
    Tom Scharf says:

    A couple comments.

    1. If absence of evidence is not evidence of absence, then what pray tell is? Monkeys? I get that we need to wait to see if a trend emerges, but this statement seems to not be a fallacy at all.

    2. If hurricane data is already sparse, erratic, and noisy, then it is reasonable to assume numerically that splitting this data set into smaller pieces (different basins, different strengths, etc.) only weakens the SNR further (smaller numbers). One would not expect to find useful information in the subsets if one is not finding correlations in the large set due to insufficient SNR. Spurious correlations are more likely with less data.

    3. It is reasonable to assume that the longer it takes for a trend to emerge, the smaller the effect is. Dice with 8 sixes will show an emerging trend change a lot faster than dice with only 4 sixes. One can be randomly unlucky and a trend could take longer to emerge then statistically expected, but this should not be assumed.

  7. 57
    Walter says:

    Hank, is that yet another complaint? I am unsure. You could google it I suppose, but why not simply reflect on the content of the article and the URL given in the article itself?

    Being :: in the 29th update section:

    People can always ask when confusion reigns. Assumptions not required. Rhetorically, how obvious do things need to be before it is no longer required (by at least one reader) to hold people’s hands and explain every single minutia?

    TYPO correction in caps

    ” and of course the absence of evidence is NOT evidence of absence ”

    it’s a fallacy and always will be. It depends on what evidence is declared absent, and why, and what is the query.

    Philosophically and mathematically it goes “the absence of evidence = the absence of evidence” and that’s it.

    So Tom S etc, i suggest to research the subject further, and think about the last time you lost your car or house keys. Maybe the monkeys took the car keys? :)

  8. 58
    Walter says:

    On “extreme weather events” and logic.

    ALL weather events, extreme or not are currently being driven by the global climate which is a dynamic and synergistic and integrated single system. Nothing exists in a vacuum when it comes to climate and weather.

    The climate is being forced by climate change drivers.

    The climate change drivers are being affected by major long term land use changes and CO2e emissions forcing the global temperature up, which in turn affects ALL – EVERY – SINGLE aspect of the dynamic Global climate system.

    Therefore Climate change IS already being reflected in all weather in all locations on this earth, right now, and has been for decades already.

    Statistics are irrelevant. The degree of X event % of CC influence is irrelevant no matter how many times a scientist gets asked that question. Attributions of xyz or *&#% how much X extreme weather event is a reflection of climate change is IRRELEVANT …. and self-defeating and unimportant.

    Stick to the known science … it already proves beyond reasonable doubt that there is a disaster in the making. That’s the main game, I recommend the science sticks to that… and just keep repeating that like this:

    “…. it is the climate that gives us the weather .. whatever the weather it IS a reflection of that. Fact is the global climate is changing and has already changed significantly and this will be seen in all weather including the (hurricane, storm, drought, flood, snow storm) we had yesterday.”

    Anything more than that is overkill and where precious resources and climate scientists time could better be directed to other climate and energy questions needing far more urgent answers that what % climate change affected a particular extreme weather event. imho.

    It may be an intriguing science question, but given the reality of the world and the lack of action to heed 25 years of warnings so far, AI would have thought there were more important and effective things to spend one’s time on, right now.


    What’s the difference between a traffic signal and an IPCC report?

    A traffic signal only shows one light at a time .. stop, go, or changing. Not all 3 at once.

  9. 59
    Doug Bostrom says:

    Roger Jr.: …I will be pursuing this further.

    Not by legal means but rather– if history is any guide– via another explosion of recklessly wrong blog posts. Given Roger’s lamentable track record of self-inflicted injuries to his own reputation a change in tactics is certainly indicated. Behaving like a gentleman is an excellent notion, a laudable goal. There’s always room for improvement, often a lot.

Switch to our mobile site