Data presentation: A trend lesson

The real trick, however, is to show all the short-term variations. Hourly and daily values would be an over-kill, but showing monthly values works. Climate change involves time scales of many years, and hence if emphasis is given to much shorter time scales, the trends will drown in noisy variations. This can be seen if we show annual mean anomalies (as shown below for exactly the same data), rather than the monthly anomalies (again, done with the same R-script)

A linear trend fit to the annual mean anomalies the last 17 years suggest similar warming rates as reported by Grant Foster and Stefan Rahmstorf. These trends are derived from exactly the same data as those used in the original figure, that was used to argue that the global warming had stopped – by two professors and a statistician, the very same who performed curve-fitting and removed data not fitting their conclusion.

Page 2 of 2 | Previous page


  1. G. Foster, and S. Rahmstorf, "Global temperature evolution 1979–2010", Environ. Res. Lett., vol. 6, pp. 044022, 2011.

119 comments on this post.
  1. John P. Reisman (OSS Foundation):

    It would be even more fun if you reverse the x or y axis. People don’t look at the numbers on the side anyway :)

  2. Michael A. Lewis, PhD:

    The real trick is deciding on the end points.

    Since global average surface temperature exhibits a long-term sinusoidal trend, one can display either a positive or negative trend with careful start and end point choices.

    Your mileage may vary depending on your destination.

  3. Utahn:

    “by two professors and a statistician…” So which one is Inferno?

  4. Salamano:

    This is another great example why archiving/providing as much of the data (and any manipulations to it) as possible is useful for science.

  5. Ron Manley:

    Since 30-year trends are the new ‘meme’ I’ve been looking at running 30-year trends. For example, the rate of increase from 1850 to 1880, from 1851 to 1881, and so on.Using three long-term global data sets it shows an underlying increase but clear fluctuation in the rate. There were maxima in the 30 years up to 1882, 1941 and 2004, and minima in the years 1907 and 1967.

  6. P.E. McDaniel:

    When I was a student studying chemistry I would frequently see a text for a class I regret not taking. Class was an upper division statistics class, the text was “How to Lie with Statistics”.

    Whenever I see graphs or data provided by politicians, advertising agencies or climate deniers, I think of that book.

    I think look for the visual ‘flaw’ in the graph …

  7. Chris Reynolds:

    #5 Ron Manley,

    Fascinating plot. The model ensemble should average out internal variability of the climate/weather system and leave only the forced response. This is seen in Johanessen et al 2009, “Arctic climate change: observed and modelled temperature and sea-ice variability.” They find that the recent warming appears in all model runs, whereas the 1940s event does not appear in model runs, although similar warming events appear at different times. See fig 1 and text.

  8. Hank Roberts:

    > Since global average surface temperature
    Since when?

    > exhibits

    > a long-term

    > sinusoidal

    > trend
    Why do you think any of the above is true?

    You refer on your blog to “possible future climate change over which we have no control” — why do you believe we’re not controlling climate now?

  9. Kasuha:

    There’s something I don’t understand. When I run linear regression over UAH data between 1994 and 2012 as per both the original graph and your reconstruction, I get the same trend value regardless whether I run it on monthly or annual averages. Of course it’s a positive trend exactly as in your fourth figure. The original graph however does not seem to talk about trends over that period but about “net change relative to average 1979-1988″ – which is supposed to be zero particularly for UAH. As I assume you have access to the whole paper rather than just the graph, could you please explain what net change is meant there?

  10. Hank Roberts:

    Oh, you’re a WTF guy. The ‘Hayduke’ nickname had me hoping for science, but, no.

  11. Michael A. Lewis, PhD:

    What does “WTF guy” mean and what does it have to do with Hayduke? What does Hayduke have to do with science, for that matter?

    Is this a blog where everyone trashes everyone else based on labels? What about content?

  12. Michael A. Lewis, PhD:

    With regard to “possible future climate change over which we have no control”…

    I hope this is not a site promoting the idea that only human activity influences climate change. This would be an unfortunate position, since it is quite evident that climate did indeed change before humans came on the scene, let alone before humans had any means of influencing climate.

    There are natural factors influencing climate over which we have no control, neh?

  13. Hank Roberts:

    > I hope this is not a site promoting the idea that only
    > human activity influences climate change.

    Not remotely. I’d recommend using the ‘Open Thread’ for the month if you want to talk about yourself and what you believe people here think, so we don’t distract from the topic here.

    I’ve put a response to you there.

  14. Ray Ladbury:

    Michael Lewis,
    Yes, of course there are natural factors that influence climate. They have very little to do with the rapid warming we are seeing at present, which, if we are not to deny the evidence, is overwhelmingly due to anthropogenic greenhouse gasses.

  15. dhogaza:

    Michael A. Lewis, PhD:

    I hope this is not a site promoting the idea that only human activity influences climate change.

    Despite being burdened with a PhD I think it’s reasonable for us to expect you to be capable of exploring the site a bit before making silly statements. Do you see the “start here” link at the top left corner of the home page? Hint: it’s meant to be taken literally.

    OK, so reading your site, you’re a PhD in Anthropology, and a survivalist with an interest in the eco-warrior fantasies of Edward Abbey (who himself was wise enough to make no effort, for the most part, to live them).

    Exactly the credentials that will convince people that we’re seeing a “long-term sinusoidal trend” (odd use of “trend”, BTW, you’re supposed to conclude that there’s *no* trend, maybe you should try copy-pasting denialist talking points directly rather than paraphrasing them?) rather than warming.

    I mean it’s clear that you’re right and all those professionals working in the field of climate science are wrong …

    Oh, this:

    What does “WTF guy” mean

    He’s suggesting you’re a denizen of Watts Up With That, a motely crew of folks who, when it comes to knowledge of science, for the most part fall far short of the blog’s host’s high-school degree.

  16. Doug H:

    No, this is a site where all the forcings on climate are investigated, human and natural. The fact that human factors are forcing current warming trends is one outcome of those investigations.

  17. Thomas:

    Eyeballing it (I don’t want to figure out how to get the raw numbers), I am convinced I’d get the same result as Kasuha, a significant linear trend if I did a least squares fit to a linear. But of course for the writer interested in disinformation, he can hand draw a flat line, and fool the uninitiated.

  18. John P. Reisman (OSS Foundation):

    #12 Michael A. Lewis, PhD

    What is your PhD in? Naivete and Ignorance?

    In other words your post displays a tremendous amount of ignorance. You jump straight to a conclusion and obviously have not looked at the tremendous wealth and depth of material discussed on this site.

    It’s funny, people think that just because someone has a PhD behind their name that those individuals will automatically make sense. You have displayed a good example of how that simply is not true.

    Start Here:

  19. David B. Benson:

    Michael A. Lewis, PhD — I strongly recommend that you begin with “The Discovery of Global Warming” by Spencer Weart, first link in the science section of the sidebar. Then you might care to listen/watch David Archer’s videotaped lectures, available online from his website. If you then want a solid background in climatology, study Ray Pierrehumbert’s “Principles of Planetary Climate”

  20. John P. Reisman (OSS Foundation):

    #12 Michael A. Lewis, PhD

    And while we’re on the subject:

    “…let alone before humans had any means of influencing climate.”

    Aren’t you ignoring the unknown unknowns here. Isn’t it possible that some humans may have figured out how to influence pre-human climate by teleporting thought waves through space/time anomalies?

    I mean if we are going to fantasize, hey, why not go big!

  21. Owl905:

    There’s probably no stopping the stats trench warfare anytime soon. The pro-pollutionist put number-crunched honey in the trap, and whistled a short tune. Now everything from climate science to the world’s largest pollution problem is on sabbatical.

    Suggestion (and it won’t force you to fold up your spreadsheet):

    Brag about the size of your graph!

    Brag about your graph respecting all the time series; all the weather-noise; and all the chaos! Brag about how your graph satisfies all the evidene!

    Draw attention to their tiny little graphs! To their short attention span! To their need to compensate for an inability to satisfy the evidence!

  22. GSW:


    Is there a link to the original work/paper? Foster & Rahmstorf is referenced but not Solheim et al. Which journal is this from?

    A couple of comments; I’m guessing, without seeing the paper, that the 17yrs is from Santer et al (2011?). Also, I’m struggling to see the “trick” in plotting monthly values rather than year anomalies for such a short time period. The calculated trend should be the same.

    [Response:Yes. The point there I believe is that doing so can give the impression of there being more truly relevant data to the question at hand than there actually are--you really only need the yearly averages to get the longer term trend.--Jim]

  23. Robert Butler:

    A quick visit to Wiki and Amazon shows “How to Lie with Statistics” by Darrell Huff is well known, in print, and available.

  24. Richard Simons:

    Owl905: I am far from clear what point you are trying to make. Are you trying to criticize the author (Rasmus) without saying anything that could be used against you, while being a smart Aleck?

  25. Andrew Park:

    The subject of time series trends using different degrees of discrimination raises an important statistical point. What is the importance of autocorrelation in these series and how should it be accounted for, if at all.

    I am no time series expert, but my understanding is that autocorrelation articifically infaltes type I error rates by diminishing the effective degrees of freedom. Often such data sets are detrended to see the autoregressive structure of the data more clearly. But of course, we want to see any trends that exist in climate data, right? So is there a way to remove the autocorrelation – or at least to allow for it in such a way that you can analyze the trend with appropriate degrees of freedom, in the absence of autocorrelation?

    [Response:Yes there are a number of ways of doing so. Arguably the simplest is to just to compute the (lower) effective sample size based on the actual size and the estimated autocorrelation, and use that value to determine your probability (p). The formula for which escapes me at the moment however.--Jim]

  26. Michael A. Lewis, PhD:

    I’m just geting a feel for the discourse here. I see nothing different from the “denialist” lists.

    Thanks for the demonstration.

    [Response:You can learn a great deal of climate science from the discourse here if you set your mind to doing so. It's up to you really.--Jim]

  27. John E Pearson:

    2 Michael A. Lewis, Ph.D wrote: “sinusoidal trend”.

    Very interesting

  28. John E Pearson:

    5 Ron wrote there were maxima and minima:

    I’ve personally examined ALL the temperature data for the lifetime of the planet and I’ve made not one but two great discoveries. Great discovery 1: every pair of temperature maxima are separated by a temperature minimum. Great discovery 2: every pair of temperature minima are separated by a temperature maximum.
    There are as many increases as decreases. THE CONCLUSION IS INESCAPABLE!!! COOLING IS AS LIKELY AS WARMING!!!

  29. Radge Havers:

    MAL, PhD @ 26

    “I see nothing different from the “denialist” lists.”

    As my paleontology teacher used to say, there are two kinds of people in the world: lumpers and splitters. Looks like you’re a lumper. Of course you’d expect a scientist to be willing to invest time analyzing the classes in question before making that decision. And you’d especially expect an anthropologist to be nuanced in assessing the social context of a conflict like this.

    OTOH, are you by chance one of those magical Carlos Castaneda type anthropologists? One who thinks any naming is just an arbitrary cultural invention equal in value to all other namings — the only inferior system being that of the mad robot scientists who secretly rule the world?

  30. Jim Larsen:

    22 GSW said, ” Also, I’m struggling to see the “trick” in plotting monthly values rather than year anomalies for such a short time period. The calculated trend should be the same.”

    Me too. I google scholared “Solheim 2012″ and checked the first few pages of results. Nada of relevance to this post. (or I’m blind. That often happens)

    Who/when/what were the conclusions Rasmus was talking about? Trend line “they” said was ______? Why is it wrong? What caused the unstated difference in trend or significance? I’m sure for many folks this post works, but it’s frustrating for me. It assumes I know stuff I don’t. So, I’ll give my thoughts and hope a kind expert will correct my errors:

    17 data points has less significance than 204. Yearly values has been the standard forever. Going to months is a huge deviation, for which there must be a motive. Perhaps it’s to artificially increase the statistical significance.

    Also, adding in monthly values “pollutes” the data, hiding trends within short term variability. Obviously, this would randomly increase or decrease the trend. In this instance, it happened to decrease(?) the trend, and so the Monoskeptics used that happenstance to make a false point. (I’m real fuzzy as to how this works)

    Woo, a twofer! Both increases statistical significance and decreases the trend, and all done by going monthly instead of the standard yearly.

  31. John P. Reisman (OSS Foundation):

    #26 Michael A. Lewis, PhD

    Another ‘Pot meet Kettle’ moment… Well, thank you for the hypocrisy.

    Why is it that you think we/I don’t know how to use the internet? Why is it that you think we/I can’t see what you have written already? Why is it that you are inferring that we/I are somehow guilty of pointing out that you are hypocritical in your posting here?

    SInce your one of the writers on

    and have co-written an article with Anthony Watts stating:

    “Global warming hyperbole has been used to discredit free-thinking, independent scientific research, free expression, free thought and free action. The individuals and corporations funding this movement are laying the ground work for society controlled by corporate-government-military oligarchies to maintain the economic and political status quo.”

    using phrasing such as

    - “These activist dolts…”

    - “I love this retarded logic: …”

    - “Idiots.”

    - “…I hope some TV station sues the living crap out of these bozos…”

    Other articles you have written include phrases such as:

    - “The Trenberth article contains so many glaring errors and biased assumptions, it’s hard to know where to start.”


    - “Since we do not yet fully understand the natural geophysical processes that result in observed climate variations over geologic time periods, it is very difficult, if not impossible, for us to fully understand the contribution to global climate variation resulting from anthropogenic greenhouse gases.”


    “Their unceasing drum-beat for Anthropogenic Global Warming will ultimately discredit their otherwise worthwhile and necessary programs to reduce human pollution as a result of unrestricted human population and economic growth.”


    “it is clear from the tone of the article on the NCAR web site that this is ideologically driven publication, not scientific research.”

    You are basically arguing from your own lack of knowledge of the science, you feel justified coming into RC, playing innocent, and indirectly accusing folks (such as myself) of not being nice to you.

    You use tired old non-science arguments to say climate science doesn’t know everything therefore we do not have enough knowledge to make any decisions. That’s simply false.

  32. Ron Manley:

    John E Pearson #28
    From your comment its not clear whether or not you have looked at the link I gave. The significance was not that maxima were follwed by minima but the timing which suggests a multidecadal periodicity in the temperature trend. This is very obvious when you follow the link.

  33. Jim Larsen:

    27 John said, ” 2 Michael A. Lewis, Ph.D wrote: “sinusoidal trend”. Very interesting”

    English is a wonderful language that way. We can use words to convey thoughts beyond their original intent. He’s talking about the supposed ~60 year natural cycle over a benign recovery from the little ice age, as slightly influenced by mankind’s emissions.

    It all sounds grand, but unfortunately it just ain’t true.

    Michael, if you’re here to learn (they never are, sigh), then go to “Start here”. Skeptical Science has some great resources too. After spending a night digging, come back here to the Unforced Variations thread and give alternative hypotheses or ask questions. This site, used correctly, is an awesome resource.

    [Response:Unfortunately, he isn't, as his last boreholed comment demonstrates.--Jim]

  34. R. Gates:

    Excellent lesson in how to pick data cherries. Seems the skeptics of anthropogenic climate change really don’t need any lessons as they are quite the experts already, and the cherries they pick tend to be of the psychotropic variety, leading to all sort of interesting mental states.

    An of course while studies such as Foster & Rahmstorf and others have shown the underlying warming of the troposphere that is continuing when background noise is eliminated from the longer-term signal that growing greenhouse concentrations are causing, the important message of warming is not found in the troposphere at all, but in the cryosphere and oceans, where the last 10 years (rather than being somewhat flat like the tropopshere has) have been warming at accelerated rates. Given that the cryosphere and oceans are far better long-term indicators of changes in Earth’s energy balance than the much more “noisy” troposphere, for anyone to suggest that the warming of the Earth system has slowed or stopped over the past 10 years, means they are purposely ignoring the far bigger heat sinks of the cryrosphere and oceans, or they simply want to spout nonsense.

  35. Doug Rusta:

    #28 John E Pearson

    When all (or most) of the factors that contribute to climate are considered, it is obvious that cyclical patterns will result over long time scales. World climate conditions will change as the oceans and atmosphere seek new equilibrium conditions. Your capitalized conclusion is thus devoid of useful information content. Sure, there will be short term ups and downs in average global temperatures, but what matters is the longer trend toward a new “stable” condition. What will the world look like when this new equilibrium is achieved? That is the issue of concern, and it matters little which side wins the debate since our political system is so badly broken.

  36. Owl905:

    @Richard Simmons – no.

  37. Brian Dodge:

    @ Michael Lewis (not that he’s still listening)
    “Ice core records go back thousands of years, but are not helpful in the past 2,000 years.”
    Misinformation like this is what gets you jumped on. Who told you that? – see &

  38. Owl905:

    @Doug Rusta 35 – It is not obvious that cyclical patterns will evolve over long time scales. In fact, cyclical patterns grow and die subject to the physical and chemical evolution of the planet. The most recent big example is the anomaly drainage of the mid-North American ice-age sea (Lake Agassiz) into the Arctic. It may not be a co-incidence that the current long Holocene plateau is a function of that anomaly. Likewise, AGHG is not cyclical – it’s an event disruption that could very well overwhelm some basic natural cyclical patterns.

    JPearson’s remark was an excellent indictment of how to twist a statistical tautology into an agenda.

  39. John P. Reisman (OSS Foundation):

    Re. #33 Jim Bouldin’s note:

    I took a look at the Michael Lewis boreholed comment out of curiosity. How he is coming to his conclusions is baffling. He says the most recent warming period is the last 1.5 million years, but data indicates that we went form a more stable warmer state with smaller oscillations into deeper oscillations into colder states.

    This indicates a general cooling trend on average for the 1.5 million year period in question. How does he come to the idea that a cooling period is a warming period one can only wonder?

    I would like to know his source for this assessment. Michael, care to share? Or is your source for this data and your assessment of 1.5 million years of warming a secret?

    FYI: Michael, for the past million years the general warming and cooling cycle revolves around 100 kyr cycles

  40. Neil Fisher:

    The full graph look more like a step to me – would you be so kind as to re-plot the trends witha break-point analysis?

  41. John E Pearson:

    35: Doug: You’ve exemplified a big problem with Real Climate. I found the notion that someone thinks it’s profound that every two maxima are separated by a minimum to be pretty funny and on intellectual par with the vast majority of denialist arguments. I remember quite well one of my earlier attempts at humor here after Texas Rod claimed that a short lived downturn in the temperature record meant that x years of warming had been wiped out. I found Rod’s comment idiotic beyond belief so I wrote something to the effect that “alarmists call them seasons” or some such. I understand that humor is a dicey business for a professional physicist to engage in but still I can’t help myself. I was surprised, though, at the hostility on the part of otherwise intelligent people. I was immediately called an idiot by Hank Roberts and a bunch of other people some of whom made up some fairly silly stuff in order to attack what they perceived as denialism on my part. There is a well-heeled denial machine that in spite of the science operates with great effectiveness. You cannot defeat it with reason. You cannot defeat it in the schools. You can hope to defeat it with humor. Possibly you can defeat it with music. You cannot defeat it with humorlessness no matter how sincere you are.

  42. John P. Reisman (OSS Foundation):

    #28 John E Pearson

    There are as many increases as decreases. THE CONCLUSION IS INESCAPABLE!!! COOLING IS AS LIKELY AS WARMING!!!

    Of course, you are not accounting for reason in this statement.

    Attribution is critical to understanding. If you ignore the reasons why things happen and focus on the ups and downs, you can conclude much less to very little, if anything at all.

    It is the ‘reason’ that the temperature goes up and down at different times under different circumstances that counts toward understanding.

    As I have said in the past, two different people can dislike you at different times, for entirely different reasons, just as with two whom like you…

    Knowing some like you and some dislike you as data points provides no understanding as to why.

  43. flxible:

    John Pearson – FWIW, this non-physicist not only caught your humour but smiled at the targets expense . . . in both cases. ;)

  44. John P. Reisman (OSS Foundation):

    #28 John E Pearson

    hmmm… or were you just being humorous again?

  45. Hank Roberts:

    > John P. Reisman (OSS Foundation) says:… Michael Lewis

    Always possible he’s considering changing his mind, though. People do.
    I suggested going into that in the ‘Open Thread’ …

  46. John P. Reisman (OSS Foundation):

    #45 Hank Roberts

    I understand your concern. I do not believe I am being unduly harsh though. Reasonable people can learn and I hope he is one that can. But from what I’ve read of his writings on WUWT it seems clear he is not basing his assumptions on science, or science in context. In fact there are leanings indicated toward conspiracy.

    Conspiracy theorists are hard to turn around. I did a talk about a month ago and it was a particularly tough crowd. 61% thought global warming was a conspiracy. After one hour that number was dropped to 33%. If I had two hours I might have done even better. But I think it is much harder to turn a conspiracy theorist around via a blog. If Michael Lewis is actually basing his perspective merely on bad data and is not belief oriented then he has a better chance of learning.

    I for one am very curious how anyone can think Earth has been in a warming period for 1.5 million years? I have never heard that claim before and all the evidence I’ve seen indicates otherwise. I hope he can quote a source for that assumption though. I would like to see it.

  47. David B. Benson:

    When attempting humor whilst blogging, plase end with
    as there will always be some who don’t get it otherwise.

  48. Hank Roberts:

    Having trouble finding ‘The Bore Hole’?

    here’s how to find stuff.

    1) on this or any other web page: use “find” in your browser; it will find the words you type into the box; often (as here) they’re a link;
    2) on this and many other websites: use a Search box; here, it’s at the top right corner of each page;
    3) use a Site Search:
    4) on this website, look in the right sidebar (you’ll see the same words the “find” command hilights). That’s the last item under Categories. It’s a link; click it.
    5) ask.

    Note: people are people; some will be rude; some will be helpful; some will be both rude and helpful, as explained here: How To Ask Questions The Smart Way; and some will not say a word til they get a feel for whether you’re going to bother learning what’s available.

    Best of luck.

  49. Erica:

    So what you’re really saying then, is that a trend over the last 17 years is not enough to establish anything. 30+ is what is needed. Yes?

  50. Erica:

    John E Pearson, You speak of a “well-heeled denial machine”. Realistically though, what they spend is peanuts compared to what government spends on – adopting your own terminology style – what one might call an “alarmist machine”.

    [Response:One might call anything one might like, but you completely misunderstand the nature of funding for basic research and data gathering (a few $bn a year). To describe the funding for NPOESS or AURA or MODIS or SCIAMACHY or NCEP or MERRA as part of an 'alarmist machine' is simply a nonsense. - gavin]

  51. Hank Roberts:

    Categories (in the right sidebar) is a list of links, including The Bore Hole
    To look “Bore Hole” up: Search box, top right corner
    In the top navigation bar, left side: Start Here button.

  52. Jos Hagelaars:

    I think that the original plot on the “forskning” website is a product of Photoshop and the “AllinONE” graph you can find on the Climate4You website of Ole Humlum:

    The colours are exactly the same, the lay-out, the text in the graph, the axis text et cetera. The creator only removed the data before 1995, the y-axis values -0.5/-0.6 and the text “Climate4you graph” in the upper left corner.

  53. Hank Roberts:

    > called an idiot by Hank Roberts
    John, I’m glad to apologize for being fooled by a Poe; wouldn’t be the first time. I don’t always succeed at pretending to be patient. Someone using “Hank” has a few times posted nasty stuff that’s been mistaken for mine, so I’d like to know for sure.

  54. John E Pearson:

    41: I wrote: “You cannot defeat it with reason. ” but meant
    “You cannot defeat it with reason ALONE.”

  55. Ray Ladbury:

    The fact that denialists equate money spent on science with money spent on propaganda merely illustrates that the denialists haven’t the first idea what science is.

    I would welcome any amount of scientific investment by fossil fuel interests to research alternative models to the consensus model of Earth’s climate. I rather doubt they would get anywhere, but if they did, it would revolutionize the field.

    Here’s the problem: science requires a model. The model makes predictions and suggests avenues of inquiry. If those predictions are successful, they in turn suggest further predictions and avenues of research. If not, they suggest ways of modifying the model or if severe enough, development of a new model. That’s science, and the impressive successes of climate models show they are good models.

    That is not what the denialists are doing. They are executing a classic gish gallop to blind the public with bullshit–usually not even bothering to publish for a technical audience. That is classic anti-science.

    So, go with the scientists or go with the anti-scientists. That is the choice you have.

    The de

  56. MartinM:

    Erica, you don’t need a supercomputer to produce PR.

  57. Richard Simons:

    Erica says:

    So what you’re really saying then, is that a trend over the last 17 years is not enough to establish anything. 30+ is what is needed.

    Why 17? Why not 16 or 18 years? I suspect 17 is chosen because it starts with a particularly warm year. As the start year was chosen because it is an outlier, any statistical tests of significance you do are invalid as they all assume a random starting and ending points (random in the sense that they were not selected on the basis of their temperatures).

    There is no magic number of years that is needed to establish that a linear trend exists. It depends on the signal-to-noise ratio, so for global temperature in recent decades 20 years has been about enough, for CO2 concentration 4 years is more than enough while for hurricane frequency 50 years is probably too short. BTW: it can never be shown that no trend exists. The problem is that the question is poorly phrased.

  58. John P. Reisman (OSS Foundation):

    #52 Hank Roberts

    No worries. Michael expressed:

    that he wishes global warming were real or that some other event such as a comet of a Coronal Mass Ejection might wipe out the human race to prevent us form doing more damage apparently. Odd though since a comet or (truly large) CME would do just as much damage to the rest of nature on the planet. Hard to reconcile that logic, though I can understand the root frustration.

    BTW I gave up patience for lent ;)

  59. Arun:

    Interesting illustration that finding the signal in the noise is made easier by working with intermediate averages.

    The one thing which is arbitrary is the start of the year. I expect it will, but would you mind showing us that January-December annual mean anomaly shows the same trend as the April-March annual mean anomaly?

    Thanks in advance!

  60. Arun:

    The reason for my previous question is, suppose for simplicity, we have only two seasons winter and summer, and winters are getting warmer and summers are getting cooler (or vice versa) with anomalies in a series like this ( units of micro-Kelvin if you like). The annual anomaly is the average of two successive numbers in the sequence.

    1, -2, 4, -8, 16, -32, ….

    Depending on how I group to make the year, I get for the annual anomaly

    -0.5, -2, -8, …


    0.5, 1, 4, …..

    Am not saying that physically something like this is plausible, but it is a logical possibility that should be ruled out.

  61. GSW:

    @Richard, Erica,

    It really depends what the original paper was trying to show. I think Ben Santer had a paper out last year to counter “It hasn’t warmed for 10yrs argument therefor the models are wrong” argument.

    My recollection of it is- Even in individual Climate model runs there are periods ~10yrs with no warming, so it is unexceptional. Santer suggests a minimum of 17yrs would be required before it could be said there was a problem with models.

    My only reason for suggesting this is the coincidental recurrence of the 17yrs figure.

  62. GSW:

    [Apologies if this appears twice, got lost? the first time]
    @Richard, Erica,

    It really depends what the original paper was trying to show. I think Ben Santer had a paper out last year to counter “It hasn’t warmed for 10yrs argument therefore the models are wrong” argument.

    My recollection of it is- Even in individual Climate model runs there are periods ~10yrs with no warming, so it is unexceptional. Santer suggests a minimum of 17yrs would be required before it could be said there was a problem with models.

    My only reason for suggesting this is the coincidental recurrence of the 17yrs figure.

  63. John P. Reisman (OSS Foundation):

    #48 Erica

    Some focused model work has it down to 17 years to separate signal to noise. 30 years is good for most models. Think about it this way, if you can model out the noise of natural variation, you can see the human climate signal.

    There is now a clear and identifiable human climate signal above the noise with strong scientific confidence.

  64. Bob Loblaw:

    Going back in the thread a bit…

    Re: #9 “There’s something I don’t understand. When I run linear regression over UAH data between 1994 and 2012 as per both the original graph and your reconstruction, I get the same trend value regardless whether I run it on monthly or annual averages.

    Why is that a surprise? It’s a characteristic of linear systems. Perhaps it’s my age (I remember when I had do do linear regressions with a pencil and paper for the sums, and a slide rule to help with the squares and square roots), but a fundamental principle of a linear least squares regression is that the best fit line passes through the point represented by the mean X and mean Y values. A quick-and-dirty regression result can be had by calculating mean X and mean Y, and then using a ruler to draw the “best fit by eye” line through that point.

    [Response: Oddly enough linear regression is not actually linear, per se! It is based on minimizing the sum of squares of the departures of the data points from the linear model. Because of this, it is potentially sensitive to outliers (i.e. it is not "robust"). And in general, one could actually get a somewhat different trend using monthly data rather than the annual means of those data. This would be true, for instance, if the earliest or latest individual monthly values in the series are outliers--that will have more leverage on the trend in the monthly series than in the corresponding annual means. Try it out on some synthetic data and see for yourself. This is one of the weaknesses of linear regression--its not resistant to outliers. This is also the motivation for other alternatives such as robust trend estimation (which minimizes the distance rather than squared distance from the linear model; but it doesn't have the nice closed-form analytical solution that least squares has). -mike]

    A linear regression says “let’s allow our mean Y to depend on X in a linear fashion”, instead of using a single mean Y to represent the data (independent of X). If you have equally spaced points along the X axis, and decide to average them in groups of 12, you’re just pre-averaging the data before feeding it into the regression. Since it is a linear combination, where you do the averaging doesn’t matter when it comes to the resulting slope.

    Of course, the other things related to the regression (significance, standard error, etc.) will not be the same, so averaging the data before doing a regression is frowned upon (to say the least), but it won’t affect the trend.

  65. Ron Manley:

    Chris Reynolds #7
    I’ve had a look at the paper you reference. Although it looks mainly at Arctic sea ice the points it raises go to the heart of the wider climate debate. Climate models represent the late 20th century warming more accurately than any other period. If the warming in that period is caused wholly, or largely, by the forcing represented in those models then their projection are likely to be valid. (The paper you reference believes they are.) On the other hand, if much of that warming was due to pseudo oscillations not included in the models (AO, NAO, AMO, ENSO, PDO, etc)then their projections may be less valid.

  66. GSW:


    “if you can model out the noise of natural variation”

    Sorry John, I don’t know if you’ve just chosen your words badly, but you definitely cannot “model out” noise. It’s well, noise.

    [Response: One person's noise is another person's signal. - gavin]

    [Response:He's talking about its reduction relative to the signal, not an actual removal--Jim]

  67. John E. Pearson:

    52: Hank was several years ago. All was long since forgiven. Point is that people here are too touchy. I understand the touchiness but still…

    My first attempt at humor here was way too subtle. I was certain that yesterday’s attempt couldn’t possibly have been mistaken but that’s the way it is. Go figure.

    [Response:You need to get back to looking at all temperature data on the planet John. There's been a lot of it in the last 24 hours ;)--Jim]

  68. Hank Roberts:

    > people here are too touchy
    I resemble that remark, at times:

    I swear the ReCaptcha AI is watching me; it just asked me to type:

    focus jookes

  69. tamino:

    Re: response to #62

    [Response: Oddly enough linear regression is not actually linear, per se! It is based on minimizing the sum of squares of the departures of the data points from the linear model. Because of this, it is potentially sensitive to outliers (i.e. it is not "robust"). And in general, one could actually get a somewhat different trend using monthly data rather than the annual means of those data. This would be true, for instance, if the earliest or latest individual monthly values in the series are outliers--that will have more leverage on the trend in the monthly series than in the corresponding annual means. Try it out on some synthetic data and see for yourself. This is one of the weaknesses of linear regression--its not resistant to outliers. This is also the motivation for other alternatives such as robust trend estimation (which minimizes the distance rather than squared distance from the linear model; but it doesn't have the nice closed-form analytical solution that least squares has). -mike]

    Actually linear regression is indeed linear, because the result of a linear regression is a strictly linear function of the input data. Hence if the data x is the sum of two different sets of data x1 and x2, then linear regression on x is the sum of the linear regressions on x1 and x2. This is a remarkably useful property in many contexts, and is one of the advantages of linear regression which is rarely appreciated.

    The “basis” for linear regression is that if the noise (deviation from the model) follows the normal distribution, then linear regression is the maximum-likelihood solution for a straight-line fit. Extreme outliers can spoil the fit, not because of any inherent weakness of the method but because their presence indicates that the underlying assumption does not hold, i.e. the noise does not follow a normal distribution — not even approximately.

    There are lots of “robust” (i.e. resistant to outliers) regression methods (besides least-absolute-difference regression). They all have their own weaknesses (not just the lack of closed-form solution), including the fact that in some cases the solution may be ambiguous.

    Even when the noise isn’t normally distributed (e.g. in the presence of outliers), as long as the noise is unbiased, uncorrelated, and homoscedastic, when the number of available data gets large one can rely on linear regression because of the Gauss-Markov theorem, that linear regression gives the BLUE, i.e., “Best Linear Unbiased Estimator” of the regression line.

    If you work out the gory mathematical details you can see that the regression on annual means is *necessarily* very close to the regression on monthly means (assuming that the annual means are the averages of the monthly means). There can be tiny differences, but even with outliers at the extremes of the time range the results will still be close.

    Linear regression isn’t perfect — nothing is — but it’s probably the best single trend-line estimator for general use in more situations than any other. There’s a reason it’s the workhorse of trend-line estimation.

    Incidentally, I wouldn’t say that averaging the data before linear regression is frowned upon (although it’s less than optimal). In fact if your data are strongly autocorrelated, then averaging before regression can be a *good* idea because it reduces the autocorrelation of the data used for the regression. Linear regression on monthly temperature data, for instance, will give you a reliable trend, but the estimated *uncertainty* that most computer programs compute for the regression fit will be way off. If you do the same on annual means instead, the uncertainty most programs report will be much closer to reality.

    BUT: linear regression on *moving* averages is a very bad idea. Moving averages make the data strongly autocorrelated even if it wasn’t already, violating the assumptions of just about every regression method (even invalidating the Gauss-Markov theorem). Don’t do it!

  70. John P. Reisman (OSS Foundation):

    #64 GSW

    Far be it from me to always choose my words well :)

    I sometimes assume others see my context, which is sometimes foolish on my part.

    But I’ve never been afraid to play the part of the fool either. In my lifetime, i’ve probably been more wrong than right. If I have paid attention properly, it is possible I may have gained some relevant understanding and knowledge in the process.

    Thanks Gavin/Jim.

  71. John P. Reisman (OSS Foundation):

    #65 John E. Pearson

    And my apologies to you as it becomes more clear I misunderstood the joke. When I first read your post, I thought ‘that’s got to be a joke’… but then I thought well, maybe others might not think it’s a joke. And then I thought placing an analogy referring to it would help those that are following the thread.

    As I’m sure you do understand the sensitivity reasoning though… so many times, and so many things, said out of context by so many denialists… I certainly can be ‘touchy’ about such things at times.

  72. Steve Fish:

    John E. Pearson (~#65), I greatly appreciate your attempts to combat the humor denialists. Steve

  73. John P. Reisman (OSS Foundation):

    #64 GSW

    “if you can model out the noise of natural variation”

    Attempts at less bad (wow, that is truly bad grammar) wording:

    - if you can separate the human signal from the noise of natural variation
    - if the model can distinguish the human signal for natural variation
    - if natural variation signal can be identified in order to parse it from the human signal
    - if the natural variation noise can be reduced in order to more clearly see the human signal

    There’s an old saying in production: There are a million ways to do it (say it); a hundred ways to do it (say it) right; and three really bitchin ways. If you can at least get one of the hundred, then that’s not too bad.

  74. Hank Roberts:

    Lynn, you’ll save time and typing by using either the search box (upper right corner) or most any web browser’s Site Search function — for example:

    It’s been covered well previously.

    As to Chip K — “New Hope” sells “advocacy science”
    (one sided presentation, spin for paying clients to make their arguments)

  75. Kevin McKinney:

    #57–OT, OT, OT–

    “BTW I gave up patience for lent ;)”

    And now you can’t wait for Easter?

  76. John P. Reisman (OSS Foundation):

    #73 Kevin McKinney


    Alas, I fear all I have left to look forward to now is the next festival of Saturnalia and hope that Ba’al Hammon and Cronus would acquiesce and allow the crowning of a new lord of misrule that somehow the king could return disorder to the feast of fools that there may be some hope that our current insane reality can be restored to some semblance of the antithetical order of the now by achieving the antithesis of the faux facade we currently face in the circus of our social medium and its related maelstrom.

    Unfortunately, as far as I know, the last Saturnalia festival was five to six centuries ago seems to have died with the end of the Roman Empire… I might have to wait sometime I suppose.

  77. Bob Fischer:

    See the book, “How to Lie with Statistics”

  78. Chris Crawford:

    Darrell Huff’s classic 1954 book, “How to Lie With Statistics” might be of some value here.

  79. GSW:

    @John #71,#68,

    “Signal as noise, noise as signal?”

    Apologies John, I did know what you were getting at. Just being overly pedantic. ;)

    It’s certainly possible to filter out “weather” from the temperature record, but I’m uncomfortable ascribing the label “Human Signal” to what’s left – You could use “Climatic/Temperature trend” instead, and attribute an influence on that trend as being Anthropogenic in origin – which I think is what you mean.

    Possibly just being pedantic again, but view the words “Human Signal” as describing a “specific”/known quantity (poor choice of words) superimposed on top of the temperature record. I prefer to think of it as a mid to long term “bias” towards higher temperatures rather than a “signal”.

    But perhaps it just playing words.


  80. Kevin McKinney:

    #74–You might be surprised!

    Though apparently you’ll have to wait for the 2013 edition now.

    In another sense, we *always* seem to have a few ‘lords of misrule’ lounging about these days, if you know what I mean… some might even claim to be Lords…

  81. Steven Franzen:

    @65 John E. Pearson:

    Have you considered the use of emoticons? ;)

    On the internet, in the absence of body language and intonation, it’s often hard to interpret the more subtle forms of humour. It’s a remarkably thin line between those who are serious but appear humorous, and those who are satirising the former. Hence, a strategy can be to respond only to what people write and not what they might mean, given no explicit cues of intent. This in turn may appear touchy or overly serious, especially if responding in a similar tone as the original post. However, given your imitation, you know how obtuse some serious posts can be…
    I must add that your sprinkling of caps and exclamation points did not convince me in this case ;).

  82. Hank Roberts:

    Speaking of data presentations, here’s a good explanation of why these merit careful attention. It’s also an explanation of advocacy science at work:

    A brief excerpt with his quote of a

    “… former Congressional staffer ….:

    > I often (some would say usually) had absolutely no
    > idea what I was doing, substantatively, or procedurally
    > or politically. I was constantly in need of reliable
    > information, provided quickly, tailored to my specific
    > situation, dumbed down to my level.
    > And so, as you can imagine, my best experiences with
    > lobbyists were the ones who gave me useful information
    > when I needed it

    This is what happens when you underpay staffers–and for the scientists, they make the equivalent of a postdoc salary. If you want professional, competent, experienced help, then it’s going to cost us.

    And it does cost us, often greatly. This another reason why we can’t have nice things.”

    — end excerpt—-

  83. John P. Reisman (OSS Foundation):

    #79 GSW

    I understand your point and in a science forum it is easy show interest in hyper-accuracy from ones own perspective. But what I’m trying to do in my work is be hyper-meaningful to an audience that does not readily understand science. I always try to avoid words like anthropogenic as it does not quickly convey a point and increases dissonance.

    The human signal in temperature is that temperature that is there by human causation factors, such as induced by increased radiative forcing. To the natural signal I suppose the human signal is the noise though and this becomes a semantic point. I use the Meehl UCAR/NCAR chart in presentations to explain the difference:

  84. PKthinks:

    I must say this is a very good article for explaining the simple ways data cherrypicking and graphical presentation can misrepresent observations, would be very good reading for some journalists

    However I think it works both ways and good as it is there is a little self righteous hubris within

    How would you react to the following a sceptical blog say

    “I just came across an interesting way to eliminate the impression of slowing of the warming trend . A trick used to argue that the global warming is continuing at the same rate as the latter part of the 20th century, and the simple recipe is as follows:

    •Extend all of the measurements as far back into the 20th century as possible, at least 1979.

    •Plot annual averages of these 17 years to get fewer of data points and disguise true monthly averages.

    •A good idea is to show a streched plot with longer anomaly axis vertically to enhance temperatures.

    The third point used to excellent effect in the third graph (from 1979)by the way


  85. Hank Roberts:

    > PKthinks says: 26 Mar 2012 at 12:42 PM …
    > … How would you react to the following a sceptical blog say …

    I’d ask for a cite. Got one? Google can’t find the stuff you appear to be quoting; where did you get it?

  86. GSW:


    Not a problem John honestly. Its worth bearing in mind that climate response is not linear, so be cautious about ascribing distinct Human and Natural components to what is observed.


  87. Rafael Molina Navas, Madrid:

    Nothing as tricky – fraudulent actually – as:
    with the “argument”:
    “Except for the well known fact that temperature changes precede CO2 changes (!!),
    the supposed CO2-driven raise of temperatures works ok before temperature reaches
    max peak. No, the real problems for the CO2-rescue hypothesis appears when temperature
    drops again. During almost the entire temperature fall, CO2 only drops slightly.
    In fact, CO2 stays in the area of maximum CO2 warming effect. So we have temperatures
    falling all the way down even though CO2 concentrations in these concentrations where
    supposed to be a very strong upwards driver of temperature”.

  88. LazyTeenager:

    I have been taking some notice of the little tricks that are used to obscure temperature and other trends. One such that is currently in fashion is comparing gradients rather than actual values. This is effective since gradients visually exaggerate noise and therefore provide more cherry picking opportunities via range selection.

    On the other hand I don’t think they even need to do this. C.S.s are quite capable of looking directly at an upward trend graph and ignoring the first 99% of the range and focussing on the last 1% that makes them happy.

    Not sure how they do it though. Place an index card over the offending vision on their computer screen? Or do they just gouge out their left eyeball with their thumbs?

  89. Bojan:

    You question is certainly a legitimate one, albeit not completely symmetrical (longer time series is just beneficial for trend seeking). Symmetry notwithstanding, you should note from the discussion that objective analysis yields same result regardless of 2. and 3. step. So it’s not so much about ‘How to lie with statistics’, it’s more about ‘How to tweak the chart to fool human’s faculty to extract trends visually’.
    But there are more capable guys here to provide more authoritative answer.

  90. Brian Dodge:

    Often what is loosely called “noise”is not only not normally distributed(thank you tamino), but is in fact an interfering signal from a known source. For instance, the annual variations in CO2 rise aren’t strictly speaking noise, but a signal from vegetation in the northern hemisphere. Since they repeat fairly regularly, averaging for integral multiples of the annual cycle is very effective at quashing them, but an 18 month average fluctuates more than a 12 month average –

    True random noise would only be decreased by root n, where n is the number of samples averaged, but it would be better reduced by 18 averages than 12.

    Knowing the underlying physics is necessary for determining the appropriate model to use – Gaussian statistics, Fourier transform, linear or polynomial fits, and so on. When the underlying physics is isn’t yet known, testing which model fits best may give insight as to what the underlying processes may be, but randomly applying mathturbation until it gives an answer you like, then proudly proclaiming “It’s cycles” or “it’s a fifth order polynomial, and cooling is imminent” isn’t even wrong.

  91. Tom:

    Here’s another one. Calculate the previous 17 year trend line slope for each month and plot. The current slope is as low as it has been in the last 30 years. What happens over the next few years is going to be very interesting as to whether the growth snaps back or trends indicate a peak in global temps.

  92. Dan H.:

    Agreed. I have done the same calculations, and arrived at the same conclusion.

  93. Tony Weddle:

    This is an interesting paper that corrects temperature records for short term effects like ENSO, volcanoes and solar variability, which gives a marked upward underlying trend:

    Has this been covered here anywhere? It seems to provide a very good rebuttal for those who parrot the “global warming has stopped” line.

  94. Hank Roberts:

    For Tony Weddle: Yes.
    Here’s how to check:

    In the upper right corner of the web page, there’s a white rectangle; type the words there — for the paper you ask about, the authors’ names, or the journal cite can be pasted in there. Click the Search button next to it.

    That will find any previous mention of whatever you’re searching for.

  95. Hank Roberts:

    > the previous 17 year trend … is as low
    > as it has been in the last 30 years

    Please show your work. Got a web page somewhere?

  96. PKthinks:

    Yes I was aware of 2 and 3 being merely what I would call ‘spin’
    Hank , the issues are generic but well demonstrated in most 21st cent publications(being the converse of the sceptical trick you would expect this), and if you pushed me for a citation ‘classic’ I would look no further than F+R (and figure 8 especially)
    but it was a little tongue in cheek

  97. Kevin McKinney:

    #91, 92–

    And what will you gentleman conclude, should March be warmer than February was?

  98. John P. Reisman (OSS Foundation):

    #80 Kevin McKinney

    Amazing! Not only is Saturnalia back, it has a facebook page!!!

  99. Brian Dodge:

    “The current slope is as low as it has been in the last 30 years” Tom & DanH

    It’a pretty damn close to the trend since 1950.(0.013 vs 0.011 per year) I don’t see that these “trends indicate a peak in global temps.”

  100. Hank Roberts:

    “The current slope is as low as it has been in the last 30 years” Tom & DanH

    Seriously, guys, you both claim independently to have determined this is a fact. Please show your work. You’re not just porting someone else’s claims over here and saying you did the analysis yourself, so — show how you decided this was the case.

  101. Jim Larsen:

    90 Brian Dodge says, “Often what is loosely called “noise”is not only not normally distributed(thank you tamino), but is in fact an interfering signal from a known source. ”

    Quite true. I’d even lose the “known”. As science advances, more and more “noise” will become “forcings” and “feedbacks”.

    91 Tom said, “What happens over the next few years is going to be very interesting as to whether the growth snaps back or trends indicate a peak in global temps.”

    If you ignore what Brian said along with the rest of the science, then such a conclusion could appeal, but if you reduce some of that “noise” by attributing it to ENSO, solar forcing, and volcanoes, you’d reach a completely different conclusion. Read about Foster and Rahmstorf 2011 (and related stuff) here:
    and here

    92 Dan H said, “Agreed. I have done the same calculations, and arrived at the same conclusion.”

    Dan, I’m quite certain that you know about F&R2011, and fairly confident you can understand the basic concepts behind it and are able to read the main graph. Thus, your comment puzzles me. It’s almost as if you’re being deliberately tunnel-visioned.

    So, please explain your comment in light of F&R2011. Is it that their data is “inconvenient” and so should be discarded since this is a political issue for you, or do you have data or analysis which refutes them? (I’m assuming nobody could come to your conclusion while believing F&R2011 is worth more than fire starting material.)

    Since you’ve done calculations, I’m sure everyone here would love to see them. Please provide a link.

  102. Dan H.:

    Tom’s comment concerns the CRU data. Calculating the 17-year trend yields a current value of 0.06C/decade, the lowest since the 17-year period 12/63-11/80. The period 2/78-1/05 is the next lowest interval, registering 0.1C/decade.

    When using the GISS dataset, the previous two intervals are similar to the CRU data, but the most recent is higher, resulting in the ’78-’05 interval being lower than the present.

    Using the RSS data, which Santer used in determining his 17-yr minimum time needed for a human global warming signal, the most recent 17-yr trend is the lowest in the entire data series.

  103. Hank Roberts:

    Dan H
    claims to cite the specific result discussed here:

    Dan H
    falsely claims that cite supports his broad general statement
    asserting “a 17-yr minimum time needed for a global warming signal”

    Dan H
    his claim, in an animated GIF:

    Dan H, if you’re not getting paid for this, you’ve missed your calling.

  104. Dan H.:


    PLease explain the differences between these two statements:
    1. “Using the RSS data, which Santer used in determining his 17-yr minimum time needed for a human global warming signal”


    2. “Our results show that temperature records of at least 17 years in length are required for identifying human effects on global-mean tropospheric temperature.”

    BTW, Thanks for the compliment.

    Also, any comments on CRU showing the lowest 17-yr trend in almost three decades? or RSS showing the lowest 17-yr trend ever?

  105. Hank Roberts:

    > 92 Dan H said, “Agreed. I have done the same calculations,
    > and arrived at the same conclusion.”

    Dan says Tom’s comment concerns CRU data;
    Dan says “I have done the same calculations ….”
    Dan talks about GISS, then RSS, and the fog thickens.

    What calculations did you do that you are relying on for your statements?

  106. Phil Mattheis:

    A magical sequence to flesh out this thread. Dan H has helpfully stepped up, again, to demonstrate in real time how to misuse trend analysis to make one’s preferred point.

    He argues from personal authority (rhetoric tool used in lieu of logic or substance), without citation or discussion of how he chose his data start and end points. He takes the “17-year minimum” notion as a standard fixed-dimension template, ignoring the implied: “more years would be better” qualifier. He fits that template to graphed data looking for fortuitous matches, and calculates slopes for the ones he really likes, out of a nearly infinite series of potential 17-year intervals.

    He describes his results minimally, from authority as a couple numbers in the air, when a graph might be much more convincing. However, that graph might also look familiar to many, suggesting another version, The Escalator, which can be seen at Skeptical Science.
    Do you suppose Dan has an animated version? I also wonder if he gets a stipend from our moderators when he does a really good job as straight man…

  107. Susan Anderson:

    Since you mention in, here’s the Escalator for any neutral readers tending towards believing the insistent drumbeat of technical sounding language supporting erroneous conclusions:

    And while we’re doing simple and persuasive, this:

    captcha can be amazing: escapap

  108. Ric Merritt:

    This is all good clean fun (if your tastes run toward teaching pigs to sing, in spite of the doubtful success and the annoyance for the pigs), but could we step back for a bit?

    Everyone here knows, or should, that 10 years GT data is too short for climate conclusions, that 20 is barely enough, and that 30 is way better.

    Stipulate a moment for the sake of argument that the 2010-19 average turns out about the same as 2000-09. (No volcano issue.) Or even that the next decade follows suit, so we have 30 years of flattish temps. Somewhere in there, we would need, as we haven’t so far (!), significant changes in mainstream climate science. There would be lots of shouting and political fallout. Embarrassment for many.

    But would it be enough to convince anyone with sober judgment that (to pick a convenient benchmark opinion) Lindzen is about right? That AGW exists, a bit, but just isn’t a big deal? Not at all necessarily. Maybe by that time most of the field would be convinced that the 60-70-year quasi-kinda-periodic “cycle” was real, with some nice hypotheses about the underlying physics, and the coming decades would warm again. Maybe multi-decade variation is bigger than we thought, and sensitivity to CO2 doubling is on the low side, and we have longer to react than we fear today, but it wouldn’t put us out of the woods.

    To get to the shoulder-shrugging, no-big-deal stage, we would need serious cooling and/or reasonable confidence that the 21st and subsequent centuries will warm at a rate *well lower* than the 20th. Six tenths of a K per century is less alarming than 2 or 3 K, but it is still enough to pose a challenge at the level of world civilization.

    And, of course, to get to even the 2030 assumption, we had to posit 2 decades of temperature records drawn from Marc Morano’s dreams.

  109. SRJ:

    About 17 years trends, here is plotted all 17 year trends and 95% CI in
    a) GISTEMP Land-ocean index
    b) RSS

    Trends are plotted versus starting time. So the data point for 1995.0833 is the trend for the 17 year period from january 1995 to January 2012.
    Confidence intervals are corrected for autocorrelation in the same way as in Foster&Rahmstorf 2011.

    My figure b agrees with Dan’s observation that for RSS the recent trend is the lowest – . Regarding GISTEMP and CRU I am confused about what periods he refers to, he writes 2/78-1/05. That is 27 years, not 17.

    However, instead of arguing over how many years to calculate linear trends over, I would suggest another way to approach this.
    Gavin Simpson suggests at his blog fitting a local model to the entire time series. That model can then be used to evaluate over which periods the time series show significant changes. Then there is no need to discuss when to start the analysis since all data is used from the time series discussed. However, the approach described only deals with annual data. Follow the link to se more detalis and plots.

  110. Gavin Simpson:

    @SRJ (Comment 109) Thanks for the mention.

    The additive model approach I outline on my blog can easily be extended to monthly or daily data. The seasonal aspects of the data are modelled via a cyclic smoother on day-of-year or numeric month-of-year. The autocorrelation structure might need a bit more work to accommodate long and short term serial correlation in residuals. But the basic principles apply. I just didn’t get round to showing an example — the models take somewhat longer to fit as the number of data points increases. But perhaps I should?

  111. Phil Mattheis:

    If somebody was to redo The Escalator animation,
    - using a series of overlapping 17 year trends, it might be a good visual aid to show that specified time span is maybe a little better at prediction than the original sequence of shorter 6-8 year periods, but just as prone to cherry picking. It would be cool in animation, though, with the 17-yr lead point dancing around the long term trend line, dragging its streaming tail along behind in blurring definition of some weird confidence interval…not so much science, but fun to watch.

    Looks like several folks here (@91 and @92) already did the math for a selected bunch of flat and downward segments, so someone would just have to add a balancing upward collection. Could be a collaborative effort!

    (but would have to be some other of yall – recaptcha is “vacurry look”, which would describe my face after all that calculating…)

  112. Dan H.:


    Thanks for the graph. Yes, I made an addition mistake on the previous post. The appropriate time frame should be 78-05. I agree that we should not become fixed on a particular length of time for temperature trends. Maybe this will put an end to the 17-yr interval made popular by the Santer paper.

    I heartily endorse using the entire dataset, rather than picking and choosing over which time intervals we perform our analyses. This would be the only way to determine which inputs have resulted in which results.

  113. Brian Dodge:

    “Using the [cherry flavored &;>) RSS data, which Santer used in determining his 17-yr minimum time needed for a human global warming signal, the most recent 17-yr trend is the lowest in the entire data series." DanH

    "Here we show that trends in MSU channel 2 temperatures are weak because the instrument partly records stratospheric temperatures whose large cooling trend offsets the contributions of tropospheric warming."

    Using the UAH data, the slope for 1995-2012 is 0.013, almost identical to the HadCRUT trend of 0.012 since 1950.

    It might be worthwhile to compare how UAH and RSS differ in their handling of channel crosstalk, and what the trends of the various levels are; is the lower trend of RSS tropospheric temperatures reflective of greater stratospheric cooling?

    DanH - what you are doing is a sort of seat-of-the-pants Exploratory Data Analysis - which is OK, but is an incomplete way to understand science, especially something as complicated and comprehensive as climatology(there's a reason most papers have multiple authors, and still have to pass peer review). You might find it useful and interesting to study a more formal way of approaching this - see

    Note the following:

    "Classical techniques serve as the probabilistic foundation of science and engineering; the most important characteristic of classical techniques is that they are rigorous, formal, and "objective".
    EDA [Exploratory Data Analysis] techniques do not share in that rigor or formality…EDA techniques are subjective and depend on interpretation….”

    “In the real world, data analysts freely mix elements of all of the above three approaches[Classical, EDA, Bayesian] (and other approaches).”

    But the final result has to be rigorous, formal, and objective.

  114. t marvell:

    these charts can be made more meaningful by incorporating the known short term effects of el nino and volcanos.

  115. Dan H.:

    The UAH data has shown greater warming recently than either the RSS or CRU data. The discussion was concerning the recent changes in temperature trends. Yes, the most recent 17-year UAH trend (0.012) is higher than RSS (0.04), it is the lowest since 1/81 -12/97.

    It is quite likely that the lower RSS trend is due to greater stratospheric cooling. The UAH data is showing a decreased trend, similar to RSS in timing, but less in magnitude.

    The difficulty in using particular year trend lines is that they are one dimensional. Any changes which occur over smaller intervals become lost in larger trendlines. It would be best to examine the overall appearance of the temperature data, rather than focus on linear trends.

  116. Tony Weddle:

    Hank (#94), thanks for the tip on searching but I’ve never got anything from that site search for quite some time (just a blank results panel, not even a “zero results” message). However, as you confirmed that it had been covered before, I did a search on Google, adding the site parameter ( and found the reference. However, it was only covered in a brief news item, then referenced, even more briefly, in another article last month. Here is the news item

  117. Hank Roberts: (site search for “foster and rahmstorf” finds that one )

    scholar search for the DOI number finds

  118. Tony Weddle:

    Thanks, Hank. Still no results on my Firefox under Linux. However, your reply prompted me to try a little harder, so I installed the Chromium browser and, lo and behold, I got results.

    Thanks, again.

  119. Hank Roberts:

    Tony, yeah, searches are peculiar.

    Speaking of data presentation, it’s always possible your search provider is ‘personalizing’ your results; erase the cache and browser cookies, sign out of ‘oogle, try duckduckgo, or search as you did by starting with a fresh browser that hasn’t accumulated any cookies.

    The claim that the personalization will “improve” your browsing by giving you more of what you want to find really sucks when you’re looking for facts.

    Compare just for example this:

    to this:

    Yet another reason I trust a good librarian more than I trust myself.