RealClimate logo


The Montford Delusion

Filed under: — group @ 22 July 2010

Guest commentary by Tamino

Update: Another review of the book has been published by Alistair McIntosh in the Scottish Review of Books (scroll down about 25% through the page to find McIintosh’s review)

Update #2 (8/19/10): The Guardian has now weighed in as well.

If you don’t know much about climate science, or about the details of the controversy over the “hockey stick,” then A. W. Montford’s book The Hockey Stick Illusion: Climategate and the Corruption of Science might persuade you that not only the hockey stick, but all of modern climate science, is a fraud perpetrated by a massive conspiracy of climate scientists and politicians, in order to guarantee an unending supply of research funding and political power. That idea gets planted early, in the 6th paragraph of chapter 1.

The chief focus is the original hockey stick, a reconstruction of past temperature for the northern hemisphere covering the last 600 years by Mike Mann, Ray Bradley, and Malcolm Hughes (1998, Nature, 392, 779, doi:10.1038/33859, available here), hereafter called “MBH98″ (the reconstruction was later extended back to a thousand years by Mann et al, 1999, or “MBH99″ ). The reconstruction was based on proxy data, most of which are not direct temperature measurements but may be indicative of temperature. To piece together past temperature, MBH98 estimated the relationships between the proxies and observed temperatures in the 20th century, checked the validity of the relationships using observed temperatures in the latter half of the 19th century, then used the relationships to estimate temperatures as far back as 1400. The reconstruction all the way back to the year 1400 used 22 proxy data series, although some of the 22 were combinations of larger numbers of proxy series by a method known as “principal components analysis” (hereafter called “PCA”–see here). For later centuries, even more proxy series were used. The result was that temperatures had risen rapidly in the 20th century compared to the preceding 5 centuries. The sharp “blade” of 20th-century rise compared to the flat “handle” of the 15-19th centuries was reminiscent of a “hockey stick” — giving rise to the name describing temperature history.

But if you do know something about climate science and the politically motivated controversy around it, you might be able to see that reality is the opposite of the way Montford paints it. In fact Montford goes so far over the top that if you’re a knowledgeable and thoughtful reader, it eventually dawns on you that the real goal of those whose story Montford tells is not to understand past climate, it’s to destroy the hockey stick by any means necessary.

Montford’s hero is Steve McIntyre, portrayed as a tireless, selfless, unimpeachable seeker of truth whose only character flaw is that he’s just too polite. McIntyre, so the story goes, is looking for answers from only the purest motives but uncovers a web of deceit designed to affirm foregone conclusions whether they’re so or not — that humankind is creating dangerous climate change, the likes of which hasn’t been seen for at least a thousand or two years. McIntyre and his collaborator Ross McKitrick made it their mission to get rid of anything resembling a hockey stick in the MBH98 (and any other) reconstruction of past temperature.

Principal Components

For instance: one of the proxy series used as far back as the year 1400 was NOAMERPC1, the 1st “principal component” (PC1) used to represent patterns in a series of 70 tree-ring data sets from North America; this proxy series strongly resembles a hockey stick. McIntyre & McKitrick (hereafter called “MM”) claimed that the PCA used by MBH98 wasn’t valid because they had used a different “centering” convention than is customary. It’s customary to subtract the average value from each data series as the first step of computing PCA, but MBH98 had subtracted the average value during the 20th century. When MM applied PCA to the North American tree-ring series but centered the data in the usual way, then retained 2 PC series just as MBH98 had, lo and behold — the hockey-stick-shaped PC wasn’t among them! One hockey stick gone.

Or so they claimed. In fact the hockey-stick shaped PC was still there, but it was no longer the strongest PC (PC1), it was now only 4th-strongest (PC4). This raises the question, how many PCs should be included from such an analysis? MBH98 had originally included two PC series from this analysis because that’s the number indicated by a standard “selection rule” for PC analysis (read about it here).

MM used the standard centering convention, but applied no selection rule — they just imitated MBH98 by including 2 PC series, and since the hockey stick wasn’t one of those 2, that was good enough for them. But applying the standard selection rules to the PCA analysis of MM indicates that you should include five PC series, and the hockey-stick shaped PC is among them (at #4). Whether you use the MBH98 non-standard centering, or standard centering, the hockey-stick shaped PC must still be included in the analysis.

It was also pointed out (by Peter Huybers) that MM hadn’t applied “standard” PCA either. They used a standard centering but hadn’t normalized the data series. The 2 PC series that were #1 and #2 in the analysis of MBH98 became #2 and #1 with normalized PCA, and both should unquestionably be included by standard selection rules. Again, whether you use MBH non-standard centering, MM standard centering without normalization, or fully “standard” centering and normalization, the hockey-stick shaped PC must still be included in the analysis.

In reply, MM complained that the MBH98 PC1 (the hockey-stick shaped one) wasn’t PC1 in the completely standard analysis, that normalization wasn’t required for the analysis, and that “Preisendorfer’s rule N” (the selection rule used by MBH98) wasn’t the “industry standard” MBH claimed it to be. Montford even goes so far as to rattle off a list of potential selection rules referred to in the scientific literature, to give the impression that the MBH98 choice isn’t “automatic,” but the salient point which emerges from such a list is that MM never used any selection rules — at least, none that are published in the literature.

The truth is that whichever version of PCA you use, the hockey-stick shaped PC is one of the statistically significant patterns. There’s a reason for that: the hockey-stick shaped pattern is in the data, and it’s not just noise it’s signal. Montford’s book makes it obvious that MM actually do have a selection rule of their own devising: if it looks like a hockey stick, get rid of it.

The PCA dispute is a prime example of a recurring McIntyre/Montford theme: that the hockey stick depends critically on some element or factor, and when that’s taken away the whole structure collapses. The implication that the hockey stick depends on the centering convention used in the MBH98 PCA analysis makes a very persuasive “Aha — gotcha!” argument. Too bad it’s just not true.

Different, yes. Completely, no.

As another example, Montford makes the claim that if you eliminate just two of the proxies used for the MBH98 reconstruction since 1400, the Stahle and NOAMER PC1 series, “you got a completely different result — the Medieval Warm Period magically reappeared and suddenly the modern warming didn’t look quite so frightening.” That argument is sure to sell to those who haven’t done so. But I have. I computed my own reconstructions by multiple regression, first using all 22 proxy series in the original MBH98 analysis, then excluding the Stahle and NOAMER PC1 series. Here’s the result with all 22 proxies (the thick line is a 10-year moving average):

Here it is with just 20 proxies:

Finally, here are the 10-year moving average for both cases, and for the instrumental record:

Certainly the result is different — how could it not be, using different data? — but calling it “completely different” is just plain wrong. Yes, the pre-20th century is warmer with the 15th century a wee bit warmer still — but again, how could it not be when eliminating two hand-picked proxy series for the sole purpose of denying the unprecedented nature of modern warming? Yet even allowing this cherry-picking of proxies is still not enough to accomplish McIntyre’s purpose; preceding centuries still don’t come close to the late-20th century warming. In spite of Montford’s claims, it’s still a hockey stick.

Beyond Reason

Another of McIntyre’s targets was the Gaspe series, referred to in the MBH98 data as “treeline-11.” It just might be the most hockey-stick shaped proxy of all. This particular series doesn’t extend all the way back to the year 1400, it doesn’t start until 1404, so MBH98 had extended the series back four years by persistence — taking the earliest value and repeating it for the preceding four years. This is not at all an unusual practice, and — let’s face facts folks — extending 4 years out of a nearly 600-year record on one out of 22 proxies isn’t going to change things much. But McIntyre objected that the entire Gaspe series had to be eliminated because it didn’t extend all the way back to 1400. This argument is downright ludicrous — what it really tells us is that McIntyre & McKitrick are less interested in reconstructing past temperature than in killing anything that looks like a hockey stick.

McIntyre also objected that other series had been filled in by persistence, not on the early end but on the late end, to bring them up to the year 1980 (the last year of the MBH98 reconstruction). Again, this is not a reasonable argument. Mann responded by simply computing the reconstruction you get if you start at 1404 and end at 1972 so you don’t have to do any infilling at all. The result: a hockey stick.

Again, we have another example of Montford implying that some single element is both faulty and crucial. Without nonstandard PCA the hockey stick falls apart! Without the Stahle and NOAMER PC1 data series the hockey stick falls apart! Without the Gaspe series the hockey stick falls apart! Without bristlecone pine tree rings the hockey stick falls apart! It’s all very persuasive, especially to the conspiracy-minded, but the truth is that the hockey stick depends on none of these elements. You get a hockey stick with standard PCA, in fact you get a hockey stick using no PCA at all. Remove the NOAMER PC1 and Stahle series, you’re left with a hockey stick. Remove the Gaspe series, it’s still a hockey stick.

As a great deal of other research has shown, you can even reconstruct past temperature without bristlecone pine tree rings, or without any tree ring data at all, resulting in: a hockey stick. It also shows, consistently, that nobody is trying to “get rid of the medieval warm period” or “flatten out the little ice age” since those are features of all reconstructions of the last 1000 to 2000 years. What paleoclimate researchers are trying to do is make objective estimates of how warm and how cold those past centuries were. The consistent answer is, not as warm as the last century and not nearly as warm as right now.

The hockey stick is so thoroughly imprinted on the actual data that what’s truly impressive is how many things you have to get rid of to eliminate it. There’s a scientific term for results which are so strong and so resistant to changes in data and methods: robust.

Cynical Indeed

Montford doesn’t just criticize hockey-stick shaped proxies, he bends over backwards to level every criticism conceivable. For instance, one of the proxy series was estimated summer temperature in central England taken from an earlier study by Bradley and Jones (1993, the Holocene, 3, 367-376). It’s true that a better choice for central England would have been the central England temperature time series (CETR), which is an instrumental record covering the full year rather than just summertime. The CETR also shows a stronger hockey-stick shape than the central England series used by MBH98, in part because it includes earlier data (from the late 17th century) than the Bradley and Jones dataset. Yet Montford sees fit to criticize their choice, saying “Cynical observers might, however, have noticed that the late seventeenth century numbers for CETR were distinctly cold, so the effect of this truncation may well have been to flatten out the little ice age.”

In effect, even when MBH98 used data which weakens the difference between modern warmth and preceding centuries, they’re criticized for it. Cynical indeed.

Face-Palm

The willingness of Montford and McIntyre to level any criticism which might discredit the hockey stick just might reach is zenith in a criticism which Montford repeats, but is so nonsensical that one can hardly resist the proverbial “face-palm.” Montford more than once complains that hockey-stick shaped proxies dominate climate reconstructions — unfairly, he implies — because they correlate well to temperature.

Duh.

Guilty

Criticism of MBH98 isn’t restricted to claims of incorrect data and analysis, Montford and McIntyre also see deliberate deception everywhere they look. This is almost comically illustrated by Montford’s comments about an email from Malcolm Hughes to Mike Mann (emphasis added by Montford):

Mike — the only one of the new S.American chronologies I just sent you that already appears in the ITRDB sets you already have is [ARGE030]. You should remove this from the two ITRDB data sets, as the new version should be different (and better for our purposes).
Cheers,
Malcolm

Here’s what Montford has to say:

It was possible that there was an innocent explanation for the use of the expression “better for our purposes”, but McIntyre can hardly be blamed for wondering exactly what “purposes” the Hockey Stick authors were pursuing. A cynic might be concerned that the phrase actually had something to do with “getting rid of the Medieval Warm Period”. And if Hughes meant “more reliable”, why hadn’t he just said so?

This is nothing more than quote-mining, in order to interpret an entirely innocent turn of phrase in the most nefarious way possible. It says a great deal more about the motives and honesty of Montford and McIntyre, than about Mann, Bradley, and Hughes. The idea that MM’s so-called “correction” of MBH98 “restored the MWP” constitutes a particularly popular meme in contrarian circles, despite the fact that it is quite self-evidently nonsense: MBH98 only went back to AD 1400, while the MWP, by nearly all definitions found in the professional literature, ended at least a century earlier! Such internal contradictions in logic appear to be no impediment, however, to Montford and his ilk.

Conspiracies Everywhere

Montford also goes to great lengths to accuse a host of researchers, bloggers, and others of attempting to suppress the truth and issue personal attacks on McIntyre. The “enemies list” includes RealClimate itself, claimed to be a politically motivated mouthpiece for “Environmental Media Services,” described as a “pivotal organization in the green movement” run by David Fenton, called “one of the most influential PR people of the 20th century.” Also implicated are William Connolley for criticizing McIntyre on sci.environment and James Annan for criticizing McIntyre and McKitrick. In a telling episode of conspiracy theorizing, we are told that their “ideas had been picked up and propagated across the left-wing blogosphere.” Further conspirators, we are informed, include Brad DeLong and Tim Lambert. And of course one mustn’t omit the principal voice of RealClimate, Gavin Schmidt.

Perhaps I should feel personally honored to be included on Montford’s list of co-conspirators, because yours truly is also mentioned. According to Montford’s typical sloppy research I have styled myself as “Mann’s Bulldog.” I’ve never done so, although I find such an appellation flattering; I just hope Jim Hansen doesn’t feel slighted by the mistaken reference.

The conspiracy doesn’t end with the hockey team, climate researchers, and bloggers. It includes the editorial staff of any journal which didn’t bend over to accommodate McIntyre, including Nature and GRL which are accused of interfering with, delaying, and obstructing McIntyre’s publications.

Spy Story

The book concludes with speculation about the underhanded meaning of the emails stolen from the Climate Research Unit (CRU) in the U.K. It’s really just the same quote-mining and misinterpretation we’ve heard from many quarters of the so-called “skeptics.” Although the book came out very shortly after the CRU hack, with hardly sufficient time to investigate the truth, the temptation to use the emails for propaganda purposes was irresistible. Montford indulges in every damning speculation he can get his hands on.

Since that time, investigation has been conducted, both into the conduct of the researchers at CRU (especially Phil Jones) and Mike Mann (the leader of the “hockey team”). Certainly some unkind words were said in private emails, but the result of both investigations is clear: climate researchers have been cleared of any wrongdoing in their research and scientific conduct. Thank goodness some of those who bought in to the false accusations, like Andy Revkin and George Monbiot, have seen fit actually to apologize for doing so. Perhaps they realize that one can’t get at the truth simply by reading people’s private emails.

Montford certainly spins a tale of suspense, conflict, and lively action, intertwining conspiracy and covert skullduggery, politics and big money, into a narrative worthy of the best spy thrillers. I’m not qualified to compare Montford’s writing skill to that of such a widely-read author as, say, Michael Crichton, but I do know they share this in common: they’re both skilled fiction writers.

The only corruption of science in the “hockey stick” is in the minds of McIntyre and Montford. They were looking for corruption, and they found it. Someone looking for actual science would have found it as well.


581 Responses to “The Montford Delusion”

  1. 251
    David Horton says:

    #244 Yes, and then having to admit that there wasn’t, after all, an error.

  2. 252
    jo abbess says:

    @dhogaza
    Re: Comment #236

    Thanks for your recommendation for Judith Curry. May I politely ask that we politely campaign for her to read Mann (2008).

    I can’t recall if I have read this paper in full before, so I just gave myself the task. It took me precisely 3 minutes to find with Google (via Lubos Motl’s “Hockey stick is revived, alive, and well” post funnily enough) and precisely 20 minutes to read :-

    https://www.cfa.harvard.edu/~wsoon/Mannetal08-PNAS-d/Mannetal08-Sep2-PNAS-2008-Mann-0805721105.pdf

    Admittedly, there are some things I would need to look up to confirm I understand them, but the basic thrust of the paper is clear.

    I like this little dig at the denier-sceptic-contrarians who appear to be tree ring obsessed : “It is intriguing to note that the removal of tree-ring data from the proxy dataset yields less, rather than greater, peak cooling during the 16th–19th centuries for both CPS and EIV methods…contradicting the claim…that tree-ring data are prone to yielding a warm-biased ‘‘Little Ice Age’’ relative to reconstructions using other high-resolution climate proxy indicators.”

    The conclusions are excellently well caveated – nobody could accuse Mann et al. of over-egging the pudding-case :-

    “Conclusions : We find that the hemispheric-scale warmth of the past decade for the NH is likely anomalous in the context of not just the past 1,000 years, as suggested in previous work, but longer. This
    conclusion appears to hold for at least the past 1,300 years…from reconstructions that do not use tree-ring proxies, and are therefore not subject to the associated additional caveats. This conclusion can
    be extended back to at least the past 1,700 years if tree-ring data
    are used, but with the additional strong caveats noted. When
    differences in scaling between previous studies are accounted
    for, the various current and previous estimates of NH mean
    surface temperature are largely consistent within uncertainties,
    despite the differences in methodology and mix of proxy data
    back to approximately A.D. 1000…Conclusions are less definitive for the SH and globe, which we attribute to larger uncertainties arising from the sparser available proxy data in the SH. Given the uncertainties, the SH and global reconstructions are compatible with the possibility of warmth similar to the most recent decade during
    brief intervals of the past 1,500 years…”

    So that should be clear enough then, shouldn’t it ? The reference period that ended in roughly 2000, the chances that temperatures in the past were of the same order as the reference period (at least in the Northern Hemisphere), and the fact that the more recent time period after the year 2000 are likely (IPCC definition) to be anomalous on the warm side. Southern Hemisphere data sparse (obviously : as humans weren’t there really to monitor things, and there haven’t been so many data gathering environmental expeditions in the Global South), temperature threshold could have been punctured on the high end a few times in the last millenium and a half.

    Why should Judith Curry find tricky problems with those conclusions ? They all seem pretty clear, well-caveated and unthreatening to me.

    I have in the past read a little of Collide-a-Scape, but I found it rather sapping of the energy. Don’t know if I should put it back on my own special homegrown Internet Browser home page where I have Hypertext Links to all the other news sources and web logs I like to keep up with.

    By the way, I have a special “True Science” link, powered by Google, but filtering out some of the major denier-sceptics :-

    http://www.google.co.uk/search?hl=en&tbo=1&tbs=qdr:d&q=climate+change+-site:climatechangefraud.com+-site:prisonplanet.com+-site:climatescienceinternational.org+-site:climateaudit.org+-site:wattsupwiththat.com+-site:climate-skeptic.com+-site:climatechangefacts.info+-site:friendsofscience.org+-site:climatedepot.com+-site:skepticsglobalwarming.com+-site:joannenova.com.au+-site:drroyspencer.com+-site:pielkeclimatesci.wordpress.com+-site:rogerpielkejr.blogspot.com+-site:junkscience.com+-site:sepp.org+-site:icecap.us+-site:newsbusters.org+-site:weatheraction.com+-site:spectator.co.uk+-site:register.co.uk+-site:timesonline.co.uk+-site:site:worldclimatereport.com+-site:melaniephillips.com+-site:jamesdelingpole.com+-site:jennifermarohasy.com+-site:globalwarminghoax.com+-site:canadafreepress.com+-site:thegwpf.org&btnG=Search&aq=o&aqi=&aql=&oq=&gs_rfai=

    Sadly, Google doesn’t take more words than this in a Search.

  3. 253

    #168 Judith Curry
    #190 dhogaza I second that.

    Judith, just to clarify your stance. You are basing your ‘scientific’ opinion on what you read at “both RC and CA”.

    Have you, or have you not properly read the scientific papers related to your opinions as expressed?

    If you have not, and as you have said, you’ve only “tried to follow the debate by reading journal articles and posts”, and I think “tried” is your operative word; then from a scientific point of view, are not then your opinions more in the ‘less likely’ or ‘very unlikely’ category of confidence, from an objective or qualitative argument perspective?

    In other words, your perspectives on this matter seem to be more a part of the smoke, or shadows of the smoke? I wonder what Plato would say about your perspective. . . shadows generated by shadows that originated from shadow generators, that relied on shadows, not the light in the back of the cave?

    #185 Judith Curry

    Waste of time though it may be . . .

    You’re saying Gavin is committing a logical fallacy in the very same paragraph you commit a huge factual fallacy?

    YOU presented the POINTS as if they were YOUR opinion. To claim otherwise is to claim misrepresentation by inference.


    A Climate Minute: The Natural CycleThe Greenhouse EffectHistory of Climate ScienceArctic Ice Melt

    ‘Fee & Dividend’ Our best chance for a better future – climatelobby.com
    Learn the Issue & Sign the Petition

  4. 254

    #193 Peter Webster

    Judith Curry is so far away for actual holistic reason in her points and logic that it is truly dumbfounding, mind-boggling, ‘jaw-dropping’, gobsmacking, and bamboozling, with the probably possibility of hoodwinking as well.

    If you can’t see this, and you have worked with her if that is you in her papers, then, huh? When the opinion she presents (from her presented posts no matter from whom she derived them, and then presented then as her perspective) is so far from the actual science, she loses credibility.

    She ignores facts, context, and relevance in order to support a meme that is so entrenched in the shadows as to be void of relevance.

    There is a real and tangible reality to the fact that ‘people’ are ‘bitching’ about ‘science’ using ‘nonsense’.

    That is ‘not’ an irrelevant point.


    A Climate Minute: The Natural CycleThe Greenhouse EffectHistory of Climate ScienceArctic Ice Melt

    ‘Fee & Dividend’ Our best chance for a better future – climatelobby.com
    Learn the Issue & Sign the Petition

  5. 255

    #201 Peter Webster

    Your insistence to relegate the argument to ‘new depths’ is reminiscent of many arguments we have seen in RC before, especially on this issue, hash, rehash, hash, rehash, ad infinitum it seems.

    http://www.ossfoundation.us/projects/environment/global-warming/myths/the-hockey-stick

    This seems to have a motive. . . i.e. to present posts her eon RC that can then be used on deinalist blogs to say RC does not look at “alternative opinion” (even though those alternative opinions have no relevance to the reality that is behind the argument, that of global warming is happening, human-caused, and at this point, irrefutable. . . unless you have a legitimate refutation??? I’d love to see that!!!).

    This of course is bizarre on your part. Maybe because you really didn’t know that all these alternative arguments have been long debunked and the denial world merely makes up new irrelevant arguments in order to show that there is still debate?

    Or maybe you do know, and really don’t understand?

    Which is it?

    I ask this because if it is not one of those, then there are few other ‘alternatives’ for your ‘opinion’ other than you don’t know what you are talking about regarding these contexts and their respective relevance. Either that or you should simply claim naiveté.

    I don’t mind claiming naiveté because I know I don’t know everything and in fact am far form it by orders upon orders of magnitude. That does not mean I can’t reasonably understand various arguments points between science and rhetoric which is entrenched in “alternative opinion”.


    A Climate Minute: The Natural CycleThe Greenhouse EffectHistory of Climate ScienceArctic Ice Melt

    ‘Fee & Dividend’ Our best chance for a better future – climatelobby.com
    Learn the Issue & Sign the Petition

  6. 256
    jo abbess says:

    @JohnMashey
    Re: Comment #235

    Thank you so much for your links, your analysis and the evident hard labour of your 185 page report.

    I did not read your report in full when it was linked on DeSmogBlog. I have scanned a few parts of it now. I was not aware that you had done so much work on this problem. It’s very good.

    I first learned about the Science of Global Warming when I studied undergraduate Physics in the early 1980s (no mere pup, me). It was during this time that I also first encountered the basic sceptical position, and read how it could be refuted with several lines of evidence. For a period of about a fortnight, I was confused. I well remember the buzzing in my mind, trying to make sense of what was information and what was disinformation.

    What helped my internal resolution was that I had gone through that exact process of delineation of fact from fiction during my teenage years over the claims of the “creation scientists” (creationists). I was emmeshed in a fairly closed Evangelical Christian community, so it was particularly hard to break the shell there, composed of years and years of doctrinal teaching and the dripfeed of the assertion of theological authority.

    I finally settled on the evidence and disgarded the scepticism on both Evolution (helped by accepting the Geological Age of the Earth) and Global Warming, and I recall that I decided it was the choice of language to lay out their case that helped confirm who to trust and who to ignore.

    You can’t help reacting emotionally to certain styles of writing, regardless of the content, or whether you agree or disagree with it.

    The Climate Change sceptics create a duststorm of irrelevancies, and blind the unprepared seeker after truth. It is always right and proper to go back to basics, swat away the flies, find your centre and calmly read the documentation at the core of the so-called debate.

    But it is also right and proper to not be tricked into circling back to basics all the time. Progress has to be made. Ways forward have to be forged. I remember the attention I paid to a paper by Tom Wigley I believe – must have been in the late 1980s – on ocean heat content having risen since the 1950s. I remember thinking, this is the nail in the sceptical coffin.

    But no, the denier-sceptics have continued to stir up their mini-tornados, using first the research magazines, then the newspapers, and now the Internet to confuse and de-rail as many people as they can, including scientists.

    But the Science is now more robust than ever. And what we need are more people who can communicate it. And more people who can defend it from attrition. And more people to study it, learn it, know it. The mainstream Media have proved incapable of communicating Climate Change. I have been appalled that The Guardian newspaper held a public meeting on “Climategate” (the stolen and badly misinterpreted e-mails saga) and invited Steve McIntyre and Doug Keenan onto the platform. Their position should not be “rehabilitated”. Their views are not providing “balance”. With this kind of psychological failure on behalf of the most open-minded British newspaper, websites such as RealClimate are invaluable as a corrective.

    We don’t want to have to follow a strategy of “war is peace” – waging an assault on the village in order to save the village. We need peaceful means to counteract the cynical denier-sceptics. The undecided public, and partially educated academics, and popularity-seeking politicians – they all need to hear the evidence and the analysis. We need to bypass the denier-sceptics, and undermine falsehoods in the minds of others, through the best means possible. This is a propaganda war, but we are not the propagandists – however we are coerced into strategies to de-support the propaganda. It’s a tricky position. Probably best addressed by “accentuating the positive” – talking more about the discoveries of Science and including basic summaries of why this unseats sceptical opinion. John Cook at SkepticalScience.com is good on this.

  7. 257
    Chris O'Neill says:

    Jean S:

    In my view MBH9X does not provide any evidence for _any_ hypothesis concerning past temperatures. And BTW, even a broken clock is right twice a day.

    So MBH9X had a 2 in 1,440 (minutes in a day) chance of being right and it just happened to fluke it. Sure, I believe you.

  8. 258

    Please read Gavin’s last inline response to Dr. Curry on comment #168. Then read it again. This is what the public needs to hear. Connect with people’s emotions and they relate. (D-A-T-A is a four-lettered word to many.) Thank you, Gavin.

    Thank you Tamino for your superb take-down of another truth-slayer. We need to keep pushing back against these dishonest people even if it means getting down into the mud with them sometimes.

    Tamino’s blog links to a superb book that you should all have in your back pocket. The book by Gant Foster is titled Noise: Lies, Damned Lies, and Denial of Global Warming and is a steal at $12.95 US. I posted a lengthy review of the book on my blog.

  9. 259
    Deech56 says:

    jo abess, nice posts. Besides the published PNAS paper, there is a wealth of supplemental information at the PNAS site and at Mann’s PSU site, available here. You will also want to read their Science article, which will show where the state of their published material. This article is especially important in light of the charge that MBH got rid of the Medieval Warm Period.

  10. 260
    dhogaza says:

    When the opinion she presents (from her presented posts no matter from whom she derived them, and then presented then as her perspective) is so far from the actual science, she loses credibility.

    Think of it as credibility seppuku.

    Her utter lack of credibility at this point is, after all, self-inflicted.

  11. 261
    Lloyd Flack says:

    I think the question of what the effect of doing an eccentric Principal Component Analysis is should be addressed.

    Principal Component Analysis is a translation of the data so the new origin is the mean and then a rotation so that the first axis is in the direction which explains the largest amount of variance, the second axis is the axis orthogonal to the first that explains the largest amount of the remaining variance, the third is the axis orthogonal to the first two that explains the largest amount of what remains and so on. There are as many principal components as there are original variables. The purpose is data reduction. You hope that the first few principal components explain. Most of the variance.

    Now if you choose an origin other than the mean and then perform a singular value decomposition as in Principal Component Analysis then what you end up with is a translation and a rotation of the principal component scores.

    Now if I recall correctly MBH98 and MBH99 fed the results of the Principal Components Analysis into a regression. Now if you perform a regression on the original independent variables and on any translation or rotation of them you get the same fitted values. That is a a regression on the original data, on the true principal components and on the eccentric principal components will all give the same fitted values.

    Now we would not use the full set of principal components. It would defeat the point. A subset explaining most of the variation would be used. If we have chosen enough components to explain most of the variance then the values fitted to the selected components will be not that much different from those fitted to the whole data set. In fact we hope that the smaller principal components are mostly noise and so the values fitted to the selected components are more accurate than those fitted to the original data set.

    Now if we perform an eccentric Principal Component Analysis the our first component will be nearly in the direction between the mean and the chosen center for the eccentric PCA. The next few will be roughly a rotation of the first few principal components. That is the first few components of the eccentric analysis will be approximately a rotation of the first few principal components plus an offset. Not quite. With one component partially wasted on the offset the eccentric analysis might need one more component but the fitted values in a regression on it should be almost the same as those from the true PCA. And this appears to be what has happened.

    If I have misremembered the papers could someone correct me. And if there is anything wrong with my argument could one of the other statisticians commenting here correct me.

    Yes, performing an improperly centered analysis was a mistake and it is a mistake to continue defending that choice. But it really made little difference to the results.

    And as plenty of others have said MBH98 and MBH99 were the first multi-proxy temperature reconstructions. They are not the state of the art. If someone continues attacking their flaws and avoids dealing with later work, and there have been plenty of later reconstructions, then we have reason to be suspicious of the attackers. It looks like attacking easier trgets and avoiding the real issues.

  12. 262

    Okay, I updated my Hockey Stick page yet again

    http://www.ossfoundation.us/projects/environment/global-warming/myths/the-hockey-stick

    In an apparently worthless effort to put the argument to rest I have proposed a

    ‘Conclusion About the Conclusion of the Conclusion’ regarding the ‘Hockey Stick’ controversy and included a circular reference to the final conclusion argument so that there is less chance that further non-controversial subjects presented as controversy can be entered into the chain of conclusions. . .

    It is my hope that this reasonably addresses the platos cave problem whereas denialists are using ‘shadows generated by shadows that originated from shadow generators, that relied on shadows, not the light in the back of the cave’. By shining a light on the shadows, maybe we can reduce the shadow intensity, or maybe I’m dreaming?

    Whew. I’m tired. . . wait, . . .wait, I’m fading, where’s my. . .

    oops, sorry someone turned on a light and my shadow diminished, glad I noticed that or I might have thought that I had disappeared when in fact I never left, only my shadow was diminished.

    Well, that solves it then. Now we can move on to more productive things. . .


    A Climate Minute: The Natural CycleThe Greenhouse EffectHistory of Climate ScienceArctic Ice Melt

    ‘Fee & Dividend’ Our best chance for a better future – climatelobby.com
    Learn the Issue & Sign the Petition

  13. 263
    Eli Rabett says:

    Some odd scientifical points.

    Geoff Wexler says in 224

    In physics, the choice of basis set is usually of little importance.It is just a matter of efficiency and saving machine time.

    This is not true in any realistic system (e.g. chemistry). In most systems the scaling with number of particles is non-linear, so you soon find that you need some combination of infinite memory, infinite processors or infinite speed. The later, at least is limited by the speed of light. And oh yes, convergence is also not linear so you really do need to start with a good basis set. There is an old saying (ok Eli thought of it thirty years ago) that in theory the calculation works.

    Hank Roberts in 236 quotes from an article on flooding wrt to the Iowa dam break. There was an article in Science(?) about a year or two ago that said that sea level rise would be measurably higher if all of the water held back by dams would have flown into the sea without our building.

  14. 264
    adelady says:

    Had a thought about floods and storms. Do insurance companies have a common database for records of storms, fires, floods in various regions to use in actuarial calculations? Or do they all maintain their own?

    I would have thought the reinsurance bodies at least would want some way of checking whether companies were using reliable information when estimating risks and setting premiums.

  15. 265
    mike roddy says:

    It’s been interesting to see something rarely observed here at Realclimate- hardheaded scientists trying to figure out why the deniers say the things they do. We’ve heard about denial “ideology”, psychology, inability to discern evidence, greed (an important one), libertarianism, and a few other things.

    Be careful here. Maybe it’s best to observe the deniers’ statements and actions and simplify their motivations thusly: this is the dark side, baby, whatever fetid well it sprang from. If they win, the consequences will be more horrifying than anything we’ve experienced since the near human extinction event of about 110,000 years ago, when there were about 10,000 of us left.

    You scientists are our most important line of defense, and it’s been heartening to see Schmidt, Tamino, Mandia, Abraham, and others showing some fight, each with his own personal style. When you just dawdle over data, and not fully call them out on their lies, they play games and disrespect you- as does much of the public. If ever a situation called for relentless anger, this is it. The public arena has an element of theater to it. Keep it up, and resist the temptation to just be the footnotes.

  16. 266
    adelady says:

    Mike, I’m inclined to think it’s the other way around. It’s non-scientists like me and lots of others who ought to maintain the rage.

    The scientists can go for it in terms of detailing facts and pointing out the errors (I’m also polite) of contrarians and innocent blunderers into the wrong room. But it’s not the scientists who should be the front-runners – some individuals are excellent in that role, but the first, prime requirement is that they should do good scientific work.

    Part of that scientific work is disseminating the information, but once it’s in the public arena, it’s down to citizens to drive the message more broadly. And drive it home. It may make the fingers bleed using a hammer to drive a very large nail into extremely hard wood, but this is one case where we should do it.

    Let’s face it, many of us are going to have bleeding hands filling sandbags by flooding rivers, and that *won’t* be an analogy, so we might as well get started.

  17. 267
    ThinkingScientist says:

    RE: #186

    Gavin, sorry for the delay in replying but I had to cut my hedge.

    In #186 I said

    “From a stationary stochastic process the MBH98 algorithm detects a non-stationary signal that is then attributed to CO2 forcing (the “hockey stick”). I think that clearly states the problem with the algorithm of MBH98: just how does it do that and still get described as robust?”

    You answered:

    [Response: Attribution is a completely different issue, and is for another day. Why the climate signal is the way it is requires a whole other set of machinery and is not related to picking out the climate signal itself. - gavin]

    I was not asking a question about attribution, so let me re-phrase it so it is clearer:

    From a stationary stochastic process the MBH98 algorithm detects a non-stationary signal: just how does it do that and still get described as robust?

    [Response: I have no idea what this question means. 'Robust' means that the signal retrieved doesn't depend excessively on the method used to get it. The signal any reconstruction method is trying to get is the signal of what actually happened in the climate system. The climate system is not a stationary stochastic process. There is a real history of the climate. It is imperfectly recorded in multiple archives. These archives can be corrupted with non-climatic noise, but by putting a lot of them together we hope to be able to figure out the common signal in time and space since the non-climatic noise will not be correlated across all proxies. The methods one develops for this should be able to handle a reasonable amount of non-climatic noise and still pull out the correct signal. - gavin]

    RE: #250 Lazar states:

    “Gavin is right. MM2005 did not “model a stationary process”, they modelled a non-stationary process as a stationary ARFIMA process and unusually did not attempt any validation statistics for their model. The autocorrelation of the ‘blade’ segment of ‘hockey-stick’ shaped proxies is significantly higher than the rest of the series, this biases their estimates of autocorrelation parameters because their model assumes a stationary autocorrelation structure, making their simulated series unrepresentative of most of the length of hockey-stick series, as can be seen in this graph of lag-1 autocorrelation coefficients.”

    The graph you showed has no attribution and no units on the y-axis, but
    notwithstanding this there are several problems with your argument based on this graph:

    (a) Some of the detrended proxies have a higher lag 1 autocorrelation than the original – this seems rather strange.
    (b) Is the same de-trend used for each proxy and are these the tree ring proxies of MBH98 and MM2005? And why do you show only proxies 60 – 70 on the graph?
    (b) It is not stated how the data are detrended other than the caption for the line at the bottom which is apparently derived after a spline detrend of the whole series. I assume that you think that ergo the non-stationary trend is correctly removed. It makes no difference whether a non-stationary component is estimated by linear regression, polynomial functions, splines, smoothing or any other approach that might be proposed: ANY such decomposition of a signal into a non-stationary trend plus an autocorrelated residual is entirely arbitrary.
    (c) You state “The autocorrelation of the ‘blade’ segment of ‘hockey-stick’ shaped proxies is significantly higher than the rest of the series”. Not according to your graph it isn’t. The “detrended” 1880-1981 line has a lag 1 autocorrelation for series 60 of 0.62 as opposed to 0.65 for the original series – a trivial change when compared to the spread of lag 1 autocorrelation values for the 10 series you show which vary from 0.65 to nearly 0.80. As the 1880-1981 interval contains the contentious “hockey stick” signal it would seem this is the one to consider. Even looking at the full detrend line the effect is small and unlikely to adversely affect a stationary simulation whichever was used.
    (d) Having a slightly higher magnitude of autocorrelation would slightly increase persistence but does not turn a stationary simulation into a non-stationary simulation. I think your graph rather neatly confirms whatI said earlier about WahlAmman2007: they assert but don’t demonstrate this point. Your graph demonstrates the effect is small and therefore trivial.

    There is a further point I would make concerning this graph and the significance of a trend which I think would further reinforce what I am arguing but for that I need to know what units the Y-axis autocorrelation is being measured in. Perhaps you can tell me? Also, if you and Gavin are arguing that the concern is over a non-stationary trend then you should be showing graphs of the autocorrelation vs lag for multiple lags, not just the first lag. Have you got a graph showing multiple lags?

    Finally, with regard to your statement “MM2005 did not “model a stationary process””. On the contrary, the following directly from From MM2005 (my italics):

    “We carried out 10,000 simulations, in each case obtaining 70 stationary series of length 581 (corresponding to the 1400–1980 period). By the very nature of the simulation, there were no 20th century trends, other than spurious ‘‘trends’’ from persistence.”

    [Response: The question is whether the persistence in those series is representative of the non-climatic noise in the proxies. It is not. - gavin]

  18. 268
    John Mason says:

    Tamino,

    Thanks for writing this piece. It provides a very useful reference point that deserves circulation. From now on whenever someone robotically posts “the hockeystick is broken” I’ll put a link to here! They might not follow it but others will :)

    Cheers – John

  19. 269
    dhogaza says:

    Lloyd Flack:

    Yes, performing an improperly centered analysis was a mistake and it is a mistake to continue defending that choice.

    What I see are people for the most part defending the *results*, not the novel PCA approach taken by Mann (who after all has abandoned it in his subsequent papers).

    But it really made little difference to the results.

    Because this is what’s really important, the results stand regardless, and subsequent work by Mann and many others have led to an entire team’s worth of hockey sticks.

    It looks like attacking easier trgets and avoiding the real issues.

    The point is solely to discredit Mann and his colleagues. Find a nit to pick in an old paper, regardless of its significance, scream “Piltdown Mann” as you accuse them of fraud, do your best to destroy their reputation, and then … political inaction is achievable. It’s a good plan, and has worked quite well, but obviously there’s nothing about science or adding to our understanding of the world in that plan.

  20. 270
    Fred Windsor says:

    I am a layman trying to get my head around this Hockey stick thing. I have been following the climate science debates on various blogs for a while but am not entirely clear. I wonder if I might ask a few questions, in particular about the suggestion that the algorithum (is that the right word?) that Mann used will produce a hockey stick if red noise is used instead of data. My questions are these;
    Is the above true?
    What is red noise (in simple terms please)
    Does it always produce a hockey stick for every run with red noise, or for only some?
    Does the blade bit of the hockey stick always appear at the end of the run or can it appear in the middle of the run?
    Is the blade of the hockey stick the same size with the red noise as it is when the data is used.

    Thank you for taking the time to answer. I have asked the same questions at Bishop Hill to get the view of the other side.

    Fred

    [Response: Red noise is produced by a kind of stochastic process that has more variability at longer time scales than at shorter time scales. Compared to 'white noise' which has variability at all time scales, red noise has the high frequencies damped. It arises naturally if you put in random forcing to a physical system with some memory. The MBH algorithm is made up of multiple steps and this are often ignored by the critics. The first step is a data preparation step - what do you include and how so that you get a good representation of the data. The second step is look for whether the data correlates with known patterns of climate variability that we can see in the instrumental record. The third step is to use those correlations to try and extend that variability back in time. And the fourth step is to check that in the reconstruction works for a period that you didn't use in step 2 ('the verfication interval'). Almost all of the talk about MBH revolves around a single issue in step 1.

    In that step, there was an attempt to take a set of very regionalised tree rings in North America and extract only the key patterns of common variability from that so that when the data was put in to step 2, there wasn't an overweighting of the importance of these proxies compared to other scattered around the globe. To do that, MBH used a technique called PCA (principle components analysis). The various ways this was and could be done are discussed in the top post. The results of this PCA step then go on to be used in step 2. Regardless of how you do that step (as long as you are consistent), the same information goes into step 2, and so claims that the total MBH algorithm produces hockey sticks out of red noise are false. However, the centering of the PCA analysis (the period over which the input data is normalised - see above) does affect the ordering of the patterns coming out of the PCA. If you were only interested in the first pattern (which MBH are not), then there would be an affect. But the PCA is being used here simply to condense down the actual information in the N. American network, not to decide what is the single most important one. The blade part of the HS comes from instrumental data and so is unaffected by anything MBH or any other reconstruction method does. - gavin]

  21. 271
    Walt Bennett says:

    Gee,

    2010 and we’re still talking about the hockey stick.

    And we’re on pace to shatter the 2005 record.

    Is anybody laughing out loud yet?

    Let me put it another way: Who else realizes that it’s over?

    Mama Earth, she’s a-goona warm-a a whole-a lotta.

    NO MATTER WHAT WE DO.

    Now – Who wants to have the next conversation?

    (Or shall we dither about for another 20 years and blow the chance to adapt?)

  22. 272

    RE #119 &

    Tamino: “You get a hockey stick with standard PCA, in fact you get a hockey stick using no PCA at all. Remove the NOAMER PC1 and Stahle series, you’re left with a hockey stick. Remove the Gaspe series, it’s still a hockey stick.”

    How many times do you think you can play this leave-one-but-only-one-out -game and still people would buy the rotten argument?

    It’s like this for global climate we need a global average. As it is I imagine the tree ring data are sparse. You wouldn’t want to leave out too much data.

    First one has to understand average (mean). It is X1 + X2 + X3 + . . . + Xn, divided by n. You could leave out one or a few (depending on the size of n) of those Xs, the results probably won’t be compromised or compromised much — assuming that data were not cherry-picked out, but deleted for some other random reason unrelated to biasing the dataset.

    Seems to me the whole denialist thrust is to totally bias all the data to fit their desired results.

    I also have the same desired results — that global warming is not happening, and we have nothing to worry about (which I assume is everyone’s desired result) — but I want the truth about it, not some false construction. I’m not afraid to face the truth, no matter how bad it is.

  23. 273
    mike roddy says:

    Adelady, I see your point, and there’s truth in it.

    The problem here in the US is that journalists are afraid to counter deniers like McIntyre with the scorn that they deserve because most media outlets are owned by right wing corporations or depend on auto and gas advertising- or both. Reporters, even the few good ones,
    self censor, and lay spokespeople are not likely to be granted an audience to skewer the denier fantasists in MSM. The English press allows more taking them to task, but look how Monbiot choked over Climategate.

    As some here know, I and others have let it fly in blog magazines (google 14 most heinous climate villains), which reached a large youth and alternative press audience. We need to reach other niches, too, which people like me are too outspoken to do successfully.

    Scientists will have more success speaking bluntly about denier lies because they are the most credible, and more likely to see their remarks appear in the major media outlets, especially on TV. That’s where frothing at the mouth deniers like Morano have had success, but there are scientists around who can defeat them. First they have to show up, though, and not just sign another petition or scientific organization fact sheet.

  24. 274
    Geoff Wexler says:

    #263

    Thanks for the comment Eli.

    My remark was an example of a simplification which suffers under close scrutiny.

    the scaling with number of particles is non-linear,

    Early band structure calculations were solutions of the one particle problem and for those there was usually a choice between quite different basis function sets all of which were equivalent but some of which might have been too slow to bother with. In nuclear physics the choice between the alpha particle model (for nuclei like that of carbon) and the shell model was somewhat analagous but closer to your chemical examples.

    More substantive:
    #261 concludes that it is a mistake to continue defending the use of non centered analysis. I may try to understand better some time, but it will not be a high priority. Perhaps Tamino might comment?

    Science is full of approximations; so is numerical analysis. I am looking forward to a chapter entitled “mistakes in approximation-theory which make no difference”.

  25. 275
    Geoff Wexler says:

    re : previous comment.

    I seem to have completely mangled the italics.

  26. 276
    Edward Greisch says:

    Headline at Climate Progress: “Hockey Stick fight at the RC Corral – Schmidt to Curry: “In future I will simply assume you are a conduit for untrue statements rather than their originator.”"

    The NYT opinion page is now on our side:
    http://www.nytimes.com/2010/07/25/opinion/25friedman.html?_r=1&ref=opinion

    Thanks 252 jo abbess for the link to the paper! Hockey Stick Lives!

    I smell the beginning of the end for fossil fuels. Now do whatever you can for the greenest politicians you can find for this November. And call your Senators a lot.

  27. 277
    J Bowers says:

    Re. 252 Jo Abbess

    Try this Google Custom search engine as well: Warming101 Search Engine

    http://www.google.com/cse/home?cx=001477428771176214757:adhq_mnia_y&hl=en

  28. 278
    ThinkingScientist says:

    RE: #267

    Gavin, you are arguing in circles :-)

    WahlAmman2007 clearly state in Section 4:

    “To generate “random” noise series, MM05c apply the full autoregressive structure of the real world proxy series. In this way, they in fact train their stochastic engine with significant (if not dominant) low frequency climate signal rather than purely non-climatic noise and its persistence.”

    Their criticism is too much persistence

    In #133 you agree with them by saying:

    #133:
    [...This is actually quite clearly discussed in Amman and Wahl (2007) (section 4), where it is shown that the 'noise' that MM2005c used actually contains a fair bit of signal.

    In #267 you now say:

    [Response: The question is whether the persistence in those series is representative of the non-climatic noise in the proxies. It is not. - gavin]

    The effect of slightly increasing the correlation of the first lag but running the simulation as a stationary series is to slightly elongate the typical length scale of the simulated process (Isaaks and Srivastava, p196 et seq show this very elegantly). If WahlAmman2007 assertion about a climatic autocorrelation being present invalidates the MM2005 test of MBH98 then it can only follow that the algorithm used to generate MBH98 must somehow only work because the (unknown)length scale of the underlying background climatic and non-climatic autoregressive processes are just the right length for the algorithm and in any other circumstance (such as used by MM2005) the algorithm fails. The argument proposed by WahlAmman2007 is therefore absurd and would only have merit if MM2005 had used a non-stationary simulation – which they did not.

    [Response: While I'm perfectly aware you think that I am terribly confused, you are wrong. ;) There are obviously circumstances in which any correlative method (such as MBH or any other method that has been used) will fail. No-one is claiming that any method will work regardless of what the input data is! The question that we have to assess is how well they work given the input data that there actually is. To give an example, let's imagine that all of the proxy data have non-climatic effects that cause millennial long trends that have nothing to do with what happened to temperatures or climate. Let's further suppose that the high frequency components are absent or have relatively small amplitude. In such cases, you will get spurious correlation on the century scale between proxies even if there is no climate signal. This will produce erroneous structures in the reconstruction. Now, no-one thinks that tree ring records have millennial scale non-climatic trends (their problem is precisely the opposite, that the multi-century scale climate trends might be damped), the same is true for ice cores etc. So what is the persistence of the non-climatic noise? Since the actual proxies have a sample auto-correlation that depends on the actual climate signal as well as the non-climatic noise, using the sample auto-correlation overestimates the persistence due to the non-climatic part. It is not that climate related auto-correlation invalidates correlative methods (it doesn't !), but that over-estimating the non-climatic persistence gives you an overly pessimistic view of the method skill under real world circumstances. - gavin]

  29. 279
    Doug Bostrom says:

    Lloyd Flack produces a concise and useful summary of the most important conclusion we may draw from Montford’s book, unintentional though it may be:

    …MBH98 and MBH99 were the first multi-proxy temperature reconstructions. They are not the state of the art. If someone continues attacking their flaws and avoids dealing with later work, and there have been plenty of later reconstructions, then we have reason to be suspicious of the attackers. It looks like attacking easier targets and avoiding the real issues.

    Move on, or surrender credibility. Most people will make the wise choice, the fringes of the bell curve will continue nattering away about an obsolete analysis published some 12 years ago.

  30. 280
    John Mashey says:

    One can easily get confused with all the back-and-forth with the innards of the statistics. A good way to maintain sanity is to remember two of John Tukey’s famous quotes:
    “―The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be
    be extracted from a given body of data.”

    “―Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong
    question, which can always be made precise.”

    For context as to why I quote Tukey, see the 1-page A.10.4 in PDF I mentioned earlier @ DeSMogBlog. I wish he were still with us, to comment on all this.

    re# 256, jo … thanks for the kind words.

  31. 281
    Leonard Evens says:

    At one time I had a good understanding of how Principal Components worked, at least for a collection of data points in R^n. But I’ve never really understood how it is supposed to work for a time series.

    Can anyone recommend a good text which goes into such matters? Don’t spare the mathematics. I’ve taught college level mathematics for almost 40 years, including the linear algebra used to find the eigenvectors.

    I do remember that not everyone thought that diagonalizing the appropriate symmetric matrix was the best way of detecting the most relevant factors explaining the data. It was my impression that there were other ways to proceed which made more sense conceptually, but that everyone used principal components because it was feasible computationally, whether or not the results made sense.

  32. 282
    Jimbo says:

    A new paper in press in Journal of Climate by Jason Smerdon et al from the Lamont Doherty Earth Observatory documents errors in some previous pseudo-proxy studies by Mann and et al.
    Erroneous Model Field Representations in Multiple Pseudoproxy [pdf]

    Rutherford et al. have a response here. [pdf]

    They also assert that the errors have been corrected in subsequent studies. And yet, Rutherford et al. continue to show the wrong NH mean temperature simulated by the ECHO-G model – compare figure 1a in the manuscript by Rutherford and Figure 5b in Smerdon et al 2010. It is obvious that these error have not be corrected.

    [Response: Actually, the only thing that is obvious is that some are quick to spread falsehoods and innuendo before seeking the actual facts. Anyone can confirm for themselves that the issue raised by Smerdon for the GKSS ECHO-G simulation was indeed fixed in Rutherford et al (2008). That is in fact precisely what panel 'a' and panel 'b' in that paper demonstrate. As it happens, Scott Rutherford accidentally copied the wrong panel from that paper (the incorrect ECHO-G series of panel 'a' rather than the correct ECHO-G series of panel 'b') for the purpose of preparing Figure 1 in the latest response. This response has now been fixed to include the correct version of the figure, and that corrected version has been submitted (and is what is available at the link you provided above). Sorry to have to dampen your excitement. -mike]

    [Response: p.s. this topic is henceforth considered o.t. -mike]

  33. 283
    Bernie says:

    Gavin:
    I found your response to ThinkingScientist (#278) thought provoking. I look forward to the extension of the discussion,

  34. 284
    Fred Windsor says:

    Gavin

    Thank you for taking the trouble to reply. I will have to study the issue some more I think.

    Fred

  35. 285
    sod says:

    very good post Tamino. very good comments by gavin.

    i have been watching the comments by Judith Curry over on the blog of Keith Kloor.

    she got the facts wrong multiple times, often because she didn t take the time to check her sources. she made an enormous number of simple logical errors.

    and for some strange reasons, during debates about how to “bridge the gap between the sides in this debate, she came to the conclusion that “WattsupWiththat” and right wing think tanks are valuable sources.

    in short, i think her reply is a typical example of lack of information and the bad sources she now uses on climate sciences issues.

  36. 286
    David B. Benson says:

    I’m speechless; dumbfounded.

  37. 287
    Lazar says:

    #267 ThinkingScientist,

    “The graph you showed has no attribution”

    My calculations using the MBH98 NAITRDB proxies.

    “and no units on the y-axis,”

    Dude correlation is unitless.

    “(a) Some of the detrended proxies have a higher lag 1 autocorrelation than the original”

    Nope. The dashed black line represents each series after detrending, and is always lower than the non detrended solid lines.

    “(b) Is the same de-trend used for each proxy”

    If you mean ‘by using the same method’, yes.

    “and are these the tree ring proxies of MBH98 and MM2005?”

    See above.

    “And why do you show only proxies 60 – 70 on the graph?”

    70 is the number of series. As the lag-1 coef reduces, you start getting series which don’t have the characteristic blade, and the lines converge. They are irrelevant to the points I am making.

    “(b) It is not stated how the data are detrended other than the caption for the line at the bottom which is apparently derived after a spline detrend of the whole series.”

    I can’t remember which specific algorithm was used. Spline detrends are fairly standard tho.

    “ANY such decomposition of a signal into a non-stationary trend plus an autocorrelated residual is entirely arbitrary.”

    Nope. There are tests for non-stationarity. The ‘hockey-stick’ series are non-stationary. Making a series stationary by detrending or differencing prior to estimating parameters is a standard procedure. My purpose was to illustrate the effects that removing the trend has on the autocorrelation estimates. By your same logic, you could argue that McIntyre’s decision to treat all variance, even the trending non-stationary component, as ‘noise’ is arbitrary.

    “(c) You state “The autocorrelation of the ‘blade’ segment of ‘hockey-stick’ shaped proxies is significantly higher than the rest of the series”. Not according to your graph it isn’t. The “detrended” 1880-1981 line has a lag 1 autocorrelation for series 60 of 0.62″

    You’re looking at the solid red line. The period 1880-1981 is censored but series are not detrended.

    “as opposed to 0.65 for the original series – a trivial change when compared to the spread of lag 1 autocorrelation values for the 10 series you show which vary from 0.65 to nearly 0.80.”

    The point is that the blade sections of 1880-1981 inflate the autocorrelation estimates, and that the inflated estimates are not representative of the majority of each ‘hockey’ series. And that by using the inflated estimates M&M created modelled series which had autocorrelation structure which was *stationary* and *generally too high* compared to the series being modelled which have *generally lower* and *non-stationary* autocorrelation structure. M&M’s model is misspecified. And the differences matter. If you graph and look at the simulated series most of them don’t even look like tree-ring series, much less any supposed noise process.

    “As the 1880-1981 interval contains the contentious “hockey stick” signal it would seem this is the one to consider.”

    Yes. You’re also looking at the lowest difference you can find. Series #67-#70 have a larger difference of about 0.05.

    “Even looking at the full detrend line the effect is small and unlikely to adversely affect a stationary simulation whichever was used.”

    Either you’re handwaving, or have you done the calculations? I did the calculations, ran the Monte Carlo benchmarking thing, a while back. Trouble is the results are on an old comment thread at Tamino’s place, and the post is no longer accessible, so I’m working from memory. But yeah, detrending does make a *big* difference, enough to change the conclusions of M&M regarding MBH98 failing significance.

    “(d) Having a slightly higher magnitude of autocorrelation [...] does not turn a stationary simulation into a non-stationary simulation.”

    Of course. I’m not arguing that a stationary model will produce a non-stationary series, nor is Gavin. *That’s not the point*.

    “if you and Gavin are arguing that the concern is over a non-stationary trend then you should be showing graphs of the autocorrelation vs lag for multiple lags, not just the first lag. Have you got a graph showing multiple lags?”

    Why?
    And why don’t you produce such a graph, if you think that it matters?

    “Finally, with regard to your statement “MM2005 did not “model a stationary process””. On the contrary, the following directly from From MM2005 (my italics):

    “We carried out 10,000 simulations, in each case obtaining 70 stationary series of length 581 (corresponding to the 1400–1980 period). By the very nature of the simulation, there were no 20th century trends, other than spurious ‘‘trends’’ from persistence.””

    The process (the real series) they are *modelling* are not stationary. The output of their model *is*.

  38. 288
    Stan Palmer says:

    I know that you won’t publish this but the amount of abuse for Dr. Curry that you are allowing to be published here is counterproductive to the cause of climate change. I have great difficult in understanding just what you think that this abuse accomplishes. Your blog and climate scientists in general already have a reputation of arrogance and being dismissive of contrary evidence. The abuse heaped on Dr. Curry here will only reinforce those views and further discredit the cause that you are trying to foster.

    AGW is a very important issue that has the potential for catastrophic consequences. It deserves a more thoughtful debate than that exhibited here.

  39. 289

    Got a new essay up on my site, if anyone’s interested:

    http://bartonpaullevenson.com/MyView.html

  40. 290
    Judith Curry says:

    In case anyone has missed it, Steve McIntyre has posted on this issue over at climateaudit http://climateaudit.org/2010/07/25/the-team-defends-paleo-phrenology/. It would be interesting for RC to rebut McIntyre’s points, which are far more detailed and documented than the points i made in my review of Montford’s book.

  41. 291

    Stan Palmer,

    It’s hard to debate thoughtfully with people who bring up the same discredited issues again and again and again and again and again until your head wants to explode. Especially when you carefully explain what they got wrong and they completely ignore it.

    I’m sure Judith Curry is a fine human being in many ways. I’m equally sure that she is dead wrong about AGW, dead wrong that the deniers need to be taken seriously by science, and dead wrong that she is doing good rather than harm.

  42. 292
    Carmen S says:

    My respect for Dr. Curry continues to grow, as does my amazement at the number of closed minds here.

  43. 293
    Geoff Wexler says:

    Re : #272

    I also have the same desired results — that global warming is not happening,

    Sorry to be tediously repetitive, but as you and we both know, they could not have achieved this desired result by breaking the hockey stick. Just in case some people from the media are reading this and misunderstand your point.

  44. 294
    BB says:

    I’m very pleased that there is back-and-forth going on about this topic from various folks that are in-the-know.

    Some of it seems to be the anti-Mann folks harping on a small part of (everyone else’s) argumentation, with that side hesitant to directly concede any points, for fear that it gives the former group traction.

    I’m pleased to see folks like Judith Curry contributing here, because it keeps the conversation moving, rather than just everyone agreeing with eachother all day. I see that Bishop Hill is conducting comments of their own (as well as MM providing his own salient commentatry on CA). It’s too bad everyone can’t just hash it out in one location so the rest of us can watch.

    Perhaps, in the end, it’s all about driving traffic and eyeballs to one site vs another. Though I must say some of the redundant vitriol (in any site) can’t be very inspiring.

  45. 295
    Lazar says:

    #186 ThinkingScientist,

    “MM2005 model a stationary stochastic process but the MBH98 algorithm somehow detects what you describe as a non-stationary forcing attributed to CO2. Let me say that again so we can be clear:

    From a stationary stochastic process the MBH98 algorithm detects a non-stationary signal”

    The MBH98 algorithm detects correlations between tree-rings and temperature, it does not detect “non-stationarity”. *Any* correlational analysis may detect spurious correlations between stochastic stationary noise processes and a deterministic non-stationary signal. That is a basic problem with correlational analysis, why do you expect MBH98 would be any different? That’s why everyone does significance testing — which is the issue that M&M were addressing, not that MBH98 detected a ‘significant’ correlation between a stationary noise process and a non-stationary deterministic signal, but that the chance of the algorithm doing so was underestimated. M&M created a noise model to show the effects of decentered PCA on Monte Carlo significance benchmarking, and that’s why their super-modelled noise matters, because the modelling choices change the benchmark estimates and the significance (or not) of the MBH98 1400-1450AD step. Which is why M&M assuming the whole series are noise matters, and why their model being misspecified matters. That’s the issue in a nutshell.

  46. 296

    Stan Palmer @288,

    Could you be specific about what constitutes the “abuse” of Dr. Curry on this forum? Has anyone posted anything you actually find abusive, or are you equating disagreement with abuse?

  47. 297
    Doug Bostrom says:

    Stan Palmer says: 25 July 2010 at 5:24 PM

    The abuse heaped on Dr. Curry here…

    The narrative arc of this thread is plain to see, published in black and white and thus it’s hard to understand the basis of your complaint.

    The tone here turned darker when Dr. Curry volunteered an unsupported “you’re all wet” remark herself, then when challenged to substantiate her remarks abandoned calm discussion and began swerving into empty rhetoric in place of substantive discourse, employing terms such as “Hockey Team” to describe a research community, “shenanigans” and “joke” to describe editorial functioning at Stephen Schneider’s journal, as well as crossing a conspicuous line by implying deceptive “cherry-picking” of statistical methods on the part of Mann et al. Very little mentioned of actual science as opposed to process by her, notably, and that apparently wrong. It’s ok to be humbly or circumspectly wrong but being arrogantly wrong has a way of producing blow-back.

    Shorter version: Read carefully and you’ll see that Dr. Curry dictated the cues on tone and volume.

    Dr. Curry has chosen to adopt the arguably offensive propaganda of a fringe interest group and employ it as a substitute for rational talk, so why are you waving your Dutch Finger at RealClimate?

  48. 298
    Geoff Wexler says:

    #288

    You have run together some quite different points. You will see that I only partly agree.

    1. That RC is arrogant and dismissive of contrary evidence.

    Your evidence for this is based on (a) its reputation (b) this thread.

    (a).As I see it some people want to discourage others from visiting this site. Still others are naive.They don’t realise that much of the so-called evidence is manufactured and has been recycled in lots of different ways. If is not genuine evidence, dismissing it quickly is not arrogant.

    (b). This thread. I thought that Tamino,Gavin and Mike have actually dealt with the evidence not dismissed it. If you think otherwise, you yourself have given no argued examples.

    2. “Abuse of Judith Curry will only reinforce… ”
    Perhaps that may be true, but Gavin and the other RC people have not been guilty of that. Perhaps you should read their comments again. I would describe their comments as forceful and critical not abusive.

    Some commentators have criticised JC’s behaviour. Why shouldn’t they ? This too is not abusive, especially if they back up these criticisms.JC is entitled to come back and defend herself.

    Yet other commentators have become so angry that they have attacked her personally. Some of this is a perfectly understandable response to JC’s unjustified attacks on RC. But it is a dangerous game.

    Here I think you may have a point. It is hard to concede anything to people who are being increasingly hostile, much easier to agree with some new admirers. I have seen this happen before in other areas.

  49. 299
    chek says:

    [edit - please stay on topic and be substantive]

  50. 300
    Paul Tonita says:

    Carmen S:

    Seriously? All she did was stroll through here and make snide remarks, followed by basically copying her homework off the book jacket in an attempt to raise objections to Tamino’s treatment of rubbish claims.

    What is there to respect in that? It’s not scholastically commendable, and certainly not open minded.


Switch to our mobile site