RealClimate logo


The CRU hack: Context

Filed under: — gavin @ 23 November 2009

This is a continuation of the last thread which is getting a little unwieldy. The emails cover a 13 year period in which many things happened, and very few people are up to speed on some of the long-buried issues. So to save some time, I’ve pulled a few bits out of the comment thread that shed some light on some of the context which is missing in some of the discussion of various emails.

  • Trenberth: You need to read his recent paper on quantifying the current changes in the Earth’s energy budget to realise why he is concerned about our inability currently to track small year-to-year variations in the radiative fluxes.
  • Wigley: The concern with sea surface temperatures in the 1940s stems from the paper by Thompson et al (2007) which identified a spurious discontinuity in ocean temperatures. The impact of this has not yet been fully corrected for in the HadSST data set, but people still want to assess what impact it might have on any work that used the original data.
  • Climate Research and peer-review: You should read about the issues from the editors (Claire Goodess, Hans von Storch) who resigned because of a breakdown of the peer review process at that journal, that came to light with the particularly egregious (and well-publicised) paper by Soon and Baliunas (2003). The publisher’s assessment is here.

Update: Pulling out some of the common points being raised in the comments.

  • HARRY_read_me.txt. This is a 4 year-long work log of Ian (Harry) Harris who was working to upgrade the documentation, metadata and databases associated with the legacy CRU TS 2.1 product, which is not the same as the HadCRUT data (see Mitchell and Jones, 2003 for details). The CSU TS 3.0 is available now (via ClimateExplorer for instance), and so presumably the database problems got fixed. Anyone who has ever worked on constructing a database from dozens of individual, sometimes contradictory and inconsistently formatted datasets will share his evident frustration with how tedious that can be.
  • “Redefine the peer-reviewed literature!” . Nobody actually gets to do that, and both papers discussed in that comment – McKitrick and Michaels (2004) and Kalnay and Cai (2003) were both cited and discussed in Chapter 2 of 3 the IPCC AR4 report. As an aside, neither has stood the test of time.
  • “Declines” in the MXD record. This decline was hidden written up in Nature in 1998 where the authors suggested not using the post 1960 data. Their actual programs (in IDL script), unsurprisingly warn against using post 1960 data. Added: Note that the ‘hide the decline’ comment was made in 1999 – 10 years ago, and has no connection whatsoever to more recent instrumental records.
  • CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on the understanding it wouldn’t be passed on to third parties. Those restrictions are in place because of the originating organisations (the various National Met. Services) around the world and are not CRU’s to break. As of Nov 13, the response to the umpteenth FOI request for the same data met with exactly the same response. This is an unfortunate situation, and pressure should be brought to bear on the National Met Services to release CRU from that obligation. It is not however the fault of CRU. The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).
  • Suggestions that FOI-related material be deleted … are ill-advised even if not carried out. What is and is not responsive and deliverable to an FOI request is however a subject that it is very appropriate to discuss.
  • Fudge factors (update) IDL code in the some of the attached files calculates and applies an artificial ‘fudge factor’ to the MXD proxies to artificially eliminate the ‘divergence pattern’. This was done for a set of experiments reported in this submitted 2004 draft by Osborn and colleagues but which was never published. Section 4.3 explains the rationale very clearly which was to test the sensitivity of the calibration of the MXD proxies should the divergence end up being anthropogenic. It has nothing to do with any temperature record, has not been used in any published reconstruction and is not the source of any hockey stick blade anywhere.

Further update: This comment from Halldór Björnsson of the Icelandic Met. Service goes right to the heart of the accessibility issue:

Re: CRU data accessibility.

National Meteorological Services (NMSs) have different rules on data exchange. The World Meteorological Organization (WMO) organizes the exchange of “basic data”, i.e. data that are needed for weather forecasts. For details on these see WMO resolution number 40 (see http://bit.ly/8jOjX1).

This document acknowledges that WMO member states can place restrictions on the dissemination of data to third parties “for reasons such as national laws or costs of production”. These restrictions are only supposed to apply to commercial use, the research and education community is supposed to have free access to all the data.

Now, for researchers this sounds open and fine. In practice it hasn’t proved to be so.

Most NMSs also can distribute all sorts of data that are classified as “additional data and products”. Restrictions can be placed on these. These special data and products (which can range from regular weather data from a specific station to maps of rain intensity based on satellite and radar data). Many nations do place restrictions on such data (see link for additional data on above WMO-40 webpage for details).

The reasons for restricting access is often commercial, NMSs are often required by law to have substantial income from commercial sources, in other cases it can be for national security reasons, but in many cases (in my experience) the reasons simply seem to be “because we can”.

What has this got to do with CRU? The data that CRU needs for their data base comes from entities that restrict access to much of their data. And even better, since the UK has submitted an exception for additional data, some nations that otherwise would provide data without question will not provide data to the UK. I know this from experience, since my nation (Iceland) did send in such conditions and for years I had problem getting certain data from the US.

The ideal, that all data should be free and open is unfortunately not adhered to by a large portion of the meteorological community. Probably only a small portion of the CRU data is “locked” but the end effect is that all their data becomes closed. It is not their fault, and I am sure that they dislike them as much as any other researcher who has tried to get access to all data from stations in region X in country Y.

These restrictions end up by wasting resources and hurting everyone. The research community (CRU included) and the public are the victims. If you don’t like it, write to you NMSs and urge them to open all their data.

I can update (further) this if there is demand. Please let me know in the comments, which, as always, should be substantive, non-insulting and on topic.

Comments continue here.

1,074 Responses to “The CRU hack: Context”

  1. 551
    Seth Pinto says:

    Dr. Schmidt, you had mentioned in the original post that you would post the figure relating to the PCM runs underlying the IPCC AR4.

    http://www.realclimate.org/index.php/archives/2009/11/the-cru-hack/comment-page-18/#comments
    Comment #871

    The figure’s original posting has been removed. Where are you going to post the figure? I apologize if I have missed it as there is so much to look through. Can I see the model runs elsewhere in detail (I believe covering the recent decade?) besides the figure in the AR4? I know the models are not designed for short term use, however paleoclimatology, in its current state, offer no real comparison. Is a thirty year run appropriate?

    I know you are very busy trying to answer so many questions so I don’t expect an answer in the near future. I will keep looking for the figure in the meantime. Your efforts in light of the recent news are appreciated by many. Thank you for your time.

  2. 552
    RaymondT says:

    Gavin, Thanks for your patience in answering my questions. For a given run what is the material balance error in the CO2 and water concentration when summed over all the grid blocks after running the model for a 100 year period ? And how is this material balance error different for each run ?

    [Response: Not quite sure what you’re asking. The difference in the forcing from a CO2 change in different models is about a 10% uncertainty (3.5 to 4.0 W/m2 or similar). The difference in the net water vapour/lapse rate feedback in different models is also small. So the divergence in sensitivity and outcome isn’t dominated by either of those two effects. It’s mostly cloud feedbacks that control the divergence, and they are less well constrained. – gavin]

    [Response: Sorry, still not clear. Are you asking whether the models conserve water? Yes. To machine accuracy. – gavin]

  3. 553

    Blair Dowden #527:

    “I would like to see links to the actual e-mails so I can judge for myself. Let me see the “worst” ones in their proper context.”

    You “ain’t” going to get them. The criminals cherry picked them and didn’t release…mmmm maybe 90% of them, right?…they only released maybe 10% if even that, right?

    So what’s in the other 90% or so? If my memory serves me right after having personally and privately talked with some of the senders/recipients over the years, some would show a sense of urgency over future damaging climate change effects that would make your hair turn white…

  4. 554
    EconRob says:

    Release all the data and all the models. Let everyone, even big oil and king coal, take a crack at it. What is there to hide? If the skeptics play games like the proponents do it will only weaken their case.

    Trying to argue it out here is a waste of time.

    A co-worker once got a theory in his head on what moved stock prices. He build model, collected data, and refined and refined until the backtesting was perfect. He sent the output to me. I said fine, send me the models and data sets so I can check it out. He said he would not because they were proprietary. He still wanted my help selling is “product.” That really is the situation the CRU people are in. They simply have to toss open ALL the data, models and code. If it is correct it will stand if not it will not.

    BTW, his model apparently did not account properly for LIBOR spread gaps circa 2008. LOL.
    Google LTCM if you want another example of too much faith in proprietary models.

  5. 555
    Jim Galasyn says:

    J. Bob, what’s important is the ratio of multi-year (old) ice to first-year (new) ice, shown here. Perennial ice used to have an average age of about 10 years but now averages about 3 years.

    J. Bob also says: So if I were to draw a straight line from the fall of 1979 (0. mkm) to the fall of 2008 (-0.75 mkm), I would be hard pressed to say Arctic will disappear very soon.

    Of course I’m sure you’re aware that you can’t just draw a line between endpoints and end up with a statistically meaningful slope. When you compute a proper trend, by including all the data points, the trend is strongly negative, as shown here and here.

    This is why Dr. Mark Serreze, director of The National Snow and Ice Data Center, says that “You are probably looking at ice-fee summers by 2030. I’d call that a death spiral.”

  6. 556
    Joseph says:

    This is quote-mining plain and simple, isn’t it? If you have a sufficiently large body of email exchanges, you are bound to find something with the appearance of being dodgy.

    I’ve seen this before, in the autism science world. See here.

  7. 557
    Ted says:

    Dear Gavin: Thank you for your comments. Let me clear up mine. You’re right that Deeming did not express a desire to get rid of the MWP (Medieval Warm Period) … what I meant to express was that Deeming testified that someone else had expressed this to Deeming in an email. Sorry for the confusion.

    This statement – about “getting rid of” the MWP – was later attributed to Jonathon Overpeck (“Peck”), which – as you pointed out – he has refuted, particularly in #1206628118. However, the email I cited (1054736277.txt) was, rather, a different email that referred to Peck’s comment as follows: “…addresses a good earlier point that Peck made w/ regard to the memo, that it would be nice to try to “contain” the putative “MWP”…”

    So, it was some memo that Peck commented on that I thought would be useful to produce, along with his actual comments. Especially since, taken on its face, Peck and others seem concerned with undermining the argument that the MWP had higher temperatures than today.

    [Response: Read the chapter (p 468, box 6.4) as it came out instead of hypothesising. – gavin]

  8. 558

    Black Norway rat: Unless the judge, as he should, laughs this off, GISS will be brought to a halt by disclosure proceedings. This is not a pretty sight, even though the lawsuit will eventually be tossed.

    BPL: THAT’S THE POINT. Climate science is saying something that threatens them, so they are trying to shut down climate science. Hacking East Anglia was the act of desperate people who know they’re losing–but that is NOT necessarily good news for our side. It means they’re willing to try anything now. I would saddened, but not even slightly surprised, if someone took a shot one or more leading climate scientists–probably Mann or Hansen. And I don’t mean verbally.

    We’re looking at the diversion of about A TRILLION DOLLARS OF INCOME from established industries to new ones. I imagine the established companies will do a lot before they’ll give that up.

  9. 559
    VagabondAstronomer says:

    I’ve found a site that posts the emails by date. Not going to post the URL here; in my opinion, I feel dirty for perusing them in the first place. In the end, however, I feel that, this is much ado about nothing, except for those who feel it necessary to cut-n-paste so that it matches their world view (they do the same with religion, incidentally).

  10. 560
    tharanga says:

    Gavin: “It’s harder to warm an ocean than it is to cool it.”

    That’s a hysteresis I didn’t expect. For a radiative forcing of +X W/m^2, ocean warming be slower than the ocean cooling under a RF of -X W/m^2? Why – something related to thermal exchange with the deep ocean, or other circulation patterns?

    [Response: The ocean is stratified (lighter water on top), so if you make the surface water lighter (by making it wamer) you increase stratification (and decrease mixing), while cooling it makes it denser and so increases mixing. – gavin]

  11. 561

    Another denier: I am a mathematician by background, now working as a consultant engineer in the oil industry.

    BPL: WHAT a SURPRISE!!!

  12. 562

    What I’m wondering is when most of you people are going to realize you’re dealing with a bunch of very angry, ignorant and illiterate borderline fascist bible thumping teabaggers who are right on the verge of subverting the established government of the United States of America, again, for another eight years. These people are going to make the tenure of George W. Bush and Dick Cheney look ‘quaint’. Is that the kind of America you people want?

    If not, speak up. It’s not just about climate science.

  13. 563

    SE: How about the authors of the emails stop complaining about the “evil hacker in the pay of the “evil conspirators of industry” and acknowledge that the emails sound bad.

    BPL: Because they only “sound bad” to conspiracy-theory lunatics.

  14. 564
    Rob says:

    393
    [Response: CVS. why? – gavin]

    Configuration management is SW meta data, most often it is more interesting and revealing (both ways of course) than the code itself. A good way to understand the evolving of SW is to look at the comments/changes and branching of it.

    I understand your code is released somewhere, is it released as a snapshot of the latest version solely or is the entire CVS repository released? Do you have a link?
    Thanks!

    [Response: Snapshots. – gavin]

  15. 565
    dhogaza says:

    Why not just hand all the raw data over to the likes of climate audit’s Steve because if its solid then he will not find anything amiss.

    How many times does it need to be said that CRU doesn’t have the right to do so for all the data (it was clearly stated in the FOI rejection, posted here earlier), and that McI needs to go to those who supply the data if he wants to get it before CRU’s effort to get the right to distribute it bear fruit?

  16. 566
    EconRob says:

    Blow back.

    Some of this is also the public’s frustration with academia in general. Parents know that college costs go up but it is doubtful the results improve – where does this money go. Then we have guys like Ward Churchill and obvious credential fraud. Most parents helping their kids get into and pay for college know that AA unlevels the playing field and is blatant discrimination against Asian kids. Then you have PhDs and experts all over the place wrong on macro and micro issues (Larry Summers, Ben Bernanke, etc.) all the time. The current credit crises can be seen as a major failure of the wall street quants. I for one am amazed guys like Van Jones graduated from Yale Law. How many times do you hear suchandsuch was higher/lower than EXPERTS were predicting.

    People are getting weary of “pointy” head know-it-alls just being wrong after rigid certainty.

    Now they hear that the AGW crew are hiding things. I’m telling you the AGW proponents have to open it up. All data and all models and if this delays the agenda so be it.

    Saying AGW is settle science just sounds arrogant. Even if it was true. Much of the major scientific thought over the ages has been reversed. Not saying AGW is or is not, but the track record is not good and the perception is worse.

  17. 567
    Marco says:

    I just HAVE to respond to Mike M (#546 at this moment):

    The issue at hand is not that Steve McIntyre is a sceptic. The issue is that he is obfuscating. Yes, he has found mistakes before, great. But he’s taken those mistakes and enlarged them to superhuman proportions. Sometimes he has also claimed mistakes (see the whole Briffa issue) where there essentially were none, and resulting in allegations of fraud uttered by his cheering crowd. He *could* have been a valuable asset to climate science (and science in general) if he had a constructive attitude. He doesn’t.

    Do remember that the mistake he found in the GISTEMP series is STILL widely reported (by skeptics) as evidence that Jim Hansen (and Gavin Schmidt by extension) don’t know what they are doing, and that none of their results are to be trusted.

  18. 568
    dhogaza says:

    BTW, I did ask one of those scientists how prevalent it is to use the terms “trick” and “hide” in an innocuous manner when attempting to define a study, and I got an answer that said, “Who are they trying to sell a used car too?”.

    Perhaps that was a telling comment?

    Franky, I suspect that you’ve misrepresented the statement or aren’t telling the truth.

    Tricks of the trade are common from physics to carpentry.

  19. 569

    Another denier: Being an agnostic on this issue which i believe makes me a neutral observer

    BPL: That’s like saying being an agnostic on whether the Earth is round or evolution happened makes you a neutral observer.

    Listen and learn: AGW denial is pseudoscience on a par with believing aliens built the pyramids. Capiche?

  20. 570
    Rob says:

    Hi again Gavin,
    479 by Loren is not commented.
    Me also think he has an interesting point. Could you comment?

  21. 571

    Another denier: Release all the data and all the models.

    BPL: Gavin et al. — Please, plesae, block any post in the future that tells anyone to “release all the data and all the models.” This has been answered about forty times now in two threads, and they’re still doing it. Don’t give these SOBs a platform! They’re still pushing that “scientists are concealing data!” meme. It’s a crock and anyone competent in the field knows it.

  22. 572

    Am I missing something in the von Storch story?
    http://coast.gkss.de/staff/storch/

    Or is he criticizing researchers for contacting Scientific journals and telling them that they think their publishing system needs improvement because they are publishing crap? (or is it that he thinks they focused on person?)

    While he think it is fine that he made a lot of fuss about the same thing in a journal that he worked for which finally made him quit it?

  23. 573
    RaymondT says:

    Gavin, I mean the numerical material balance error that occurs because of the numerical discretizations of the PDE’s used to solve the continuity equation. I work in the numerical modelling of enhanced oil recovery processes in which the material balance error is very important. The maximum MBE you want is approximately 1 %. To reframe my question, suppose that you had an error of 1% weight fraction in your water vapour concentration what range of a temperature increase would that correspond to in your different runs after 100 years ?

    [Older Response: Not quite sure what you’re asking. The difference in the forcing from a CO2 change in different models is about a 10% uncertainty (3.5 to 4.0 W/m2 or similar). The difference in the net water vapour/lapse rate feedback in different models is also small. So the divergence in sensitivity and outcome isn’t dominated by either of those two effects. It’s mostly cloud feedbacks that control the divergence, and they are less well constrained. – gavin]

    [Newer Response: Sorry, still not clear. Are you asking whether the models conserve water? Yes. To machine accuracy. – gavin]

    [Response: All of our discretisations use a flux form for solving the equations, thus the conservation constraints are built in. There is no mass balance error in the sense you describe. – gavin]

  24. 574
    Bernie says:

    dhogaza says:
    25 November 2009 at 12:13 PM
    How many times does it need to be said that CRU doesn’t have the right to do so for all the data (it was clearly stated in the FOI rejection, posted here earlier), and that McI needs to go to those who supply the data if he wants to get it before CRU’s effort to get the right to distribute it bear fruit?”

    You just have to stop digging. Obstruction is obstruction. http://omniclimate.wordpress.com/2009/11/24/willis-vs-the-cru-a-history-of-foi-evasion/

  25. 575
    David Gordon says:

    Following up from #543

    >[Response: The more trees the better. The later period (post 1990) when there are only 12 would not be as well characterised as the earlier period. But Briffa’s comment on his website (google for the cached version) shows the results are very similar even when you add in more recently collected material. Note that we don’t need tree rings to know that the temperatures have warmed since 1990. – gavin]

    I would like a direct answer please. Is a sample size of 12 tree rings sufficient to meet the threshold required to produce high-confidence and statistically significant results in this field?

    >[Response: There isn’t just one reconstruction. There are many, and some are completely independent (Oerlemann’s for instance). See the NOAA page for some examples. – gavin]

    Reviewing Oerlemann’s reconstruction, he is quite straightforward about a +/- 20% uncertainty level, which appears to make his results a rough guess at best. Also, a little background research shows that he has been working with the IPCC since nearly 1990, which does not seem to qualify under any reasonable interpretation of the word “independent” – he appears to be a member of the same professional-social circle of the authors whose work is in question. The concern here being confirmation bias, where individual researchers working in collaboration with each other tend to reinforce each others’ biases. This is why the independence of verification is important, to be able to confidently rule out such bias when ascertaining the validity of the result. So again, can you identify any actually independent – as in having no direct professional and/or personal connection – source that verifies the reconstruction advanced by the UEA CRU as the basis for claims of present decade (1998-) and predicted future warming?

    >[Response: What extraordinary claim do you think is being made? And of course, the literature is full of papers testing the reconstructions. None of them are making ‘extraordinary’ claims – they are simply trying to piece together a coherent history of climate over time. See Chapter 6 in IPCC AR4 for some more discussion. – gavin]

    The main extraordinary claim is self-evident, the claim that human activity is the cause of ahistorical global warming, and that this warming will continue as projected by UEA CRU. The scope of the political and economic changes proposed on the basis of this claim are so broad as to encompass the activities of the entire human race. The concern with the literature is that the same small group of individuals, and the same data and the same methodology whose validity is in dispute, and that the conclusions on the 1998- and future projected warming, when references are traced through to their origins, seem to lead right back to the elements in dispute. Self-reference cannot meet any reasonable standard of independent validation (please tell me now if you disagree with this statement, as if you do, we have a fundamental disagreement as to the nature of scientific inquiry itself which cannot be resolved through this question and answer session).

    The second extraordinary claim concerns the amplitude (and existence?) of the Medieval Warm Period, which appears to contradict anthropological data on the same subject. For example, evidence of wineries at latitudes presently too cold to grow wine appears to contradict the assertion that present temperature levels have no equal over the past thousand years.

    We have two quite extraordinary claims and they appear to be backed up, ultimately, by a) a study with insufficient sample size to be a valid basis from which to draw firm conclusions on global temperature; and b) a dendrochronology method the validity of which is disputed, and the dispute yet unresolved. And rather than standard scientific validation we have evidence that contradictory studies are responded to by political retaliation and not scientific rebuttal.

    Do you understand how bad this looks from the point of view of an engineer?

    [Response: With all due respect you have things very confused. Nothing depends on the 12 trees in the Yamal record. Toss them out completely if you aren’t happy with them. The instrumental temperature records are plenty enough to demonstrate the warming since then without recourse to trees. You don’t like dendroclimatology at all? Fine, toss that too. It just isn’t that key to anything important (see here). And I think you may be a little confused about CRU does – they chiefly collate one version of the instrumental temperature data. They don’t make projections, and they don’t have one of the main climate models. Your statement about vineyards in the medieval period is very out of date as well. In fact, your whole argument neglects the fact that CRU are just a very small part of the climate science establishment, and every single one of their important contributions has been replicated by independent groups around the world. – gavin]

  26. 576
    Ron R. says:

    Anand Rajan KD said @ 7:22 AM “So when I question the powers of a vague international body to pass laws that impact the whole world, you come back with a personal angle – something toxic dumped in my backyard.

    Certainly, since the macro is merely the micro on a larger scale. If one can dump toxic waste anywhere they please without the regulations you apparently despise then why not your own back yard?

    ” I think the question I raised initially (why should the globe not warm up at all?) requires a better response from the climatologist community, than just scare tactics.”

    Consider that the average global temperatures since the last ices age was about 5 degrees C. cooler. That’s not much. Just 5 degrees since the ice was a half mile high or so in Yosemite Valley. Ok, reverse that, and add a few degrees, how much warmer will that make the earth?

    “Now my question is: Why should your perspective (a fawning worshipping one) be the lens through which we see the world we both live in? And pass laws which impact me?

    “Live”, or the desire to, is the operative word here.

    “I believe the earth is just a blue ball with some green stuff stuck on it and humans crawling all over. Feeling miserable all the time. And looking for something to strike out at. That’s is my perspective.”

    Dr Mengele would be proud.

  27. 577
    Kyle Tingle says:

    Gavin,

    I did not read all the posts from both threads, but probably a few hundred from the various days. Most of the comments have to do with the science and more technical than a lay person such as myself to agree to or refute. I will trust the professionals.

    It is interesting that two camps are clearly identified –
    True Believers – usually publicly funded through government and university grants – pushing ACC as bringing on catastrophe.
    Deniers – accused of being funded by “Big Oil” or “Big Coal” arguing there is no ACC and the 10 year hiatus on warming is a large enough sample to “prove” ACC doesn’t exist.

    It seems the truth is in between – from what I understand, the earth has been warming since the last ice age 10-12,000 years ago. There was the MWP and subsequent cooling and warming has been the general trend for the last 700-800 years. I hope that is close to the general trends.

    Science does show that man has put CO2 into the atmosphere at alarming rates the past couple centuries, but there should have been some natural variations as well, volcanos, forest fires, the Indonesian peat bogs buring, etc. Both camps seem to scream “all or nothing” when it comes to ACC.

    The leaked emails do not refute the science, as much as it shows how protective those scientists, by denying FOI requests, were to preserve their point of view (necessary for employment and grant funds to keep coming in) instead of allowing others to recreate the results. This creates a distrust in the process. From the postings of Barton Paul Levinson and Thomas Lee Elifritz, they would prefer not having independent review of the original data as well, attacking the messenger. No TLE, I am not a “teabagger” or denier.

    Exaggerations such as “An Inconvenient Truth,” “Day after Tomorrow,” and the cover of Al Gore’s new book (with Cuba gone, the hurricane at the equator, and the hurricane by Florida spinning the wrong way) only give strength to opponents that ACC in only hype and doesn’t exist. Boring, gradual effects don’t sizzle for the media. I don’t think the average person believes the disasters will happen, as technology is going to continue to evolve and pollution will slow down as a matter of technological advancement. And most people don’t want to increase their taxes, food costs, utility costs and suffer the decrease in standard of living that comes with it.

    I am sure if the emails of “Deniers” were leaked, they would be spinning scientific evidence to refute the affects of increased CO2 and how to get that side published. I’m betting they would not open up either.

    What would be nice, is that all data be transparent for review and recreation of results. I always learned that was key in the scientific method. Then the reality between natural and ACC would be in the open and even the deniers would have a difficult time spinning that there is no ACC.

  28. 578
    manacker says:

    BPL (563): Because they (the hacked emails) only “sound bad” to conspiracy-theory lunatics

    Wake up, BPL. Sticking your head in the sand won’t do it.

    Many posters here (for example EconRob 566) see that the exposed arrogant approach of some AGW scientists, manipulating the data, blocking dissenting views and being less than transparent has cost the AGW movement dearly. As ER wrote “the track record is not good and the perception is worse”.

    This event has been a game changer. It is no longer possible to get by with “the science is settled” rhetoric. More openness and public scrutiny will now be required.

    Those scientists that accept the new rules will survive and succeed. Those that stick their heads in the sand will not.

    Max

  29. 579
    JCS Bsc Msc says:

    Gavin,

    I have repeatedly tried to post comments on your website asking you to respond to the following statement:

    If the first rule of science is to question everything, and another fundamental rule is that no hypothesis can be proven true, regardless of imposing the precautionary principle, why is the first rule and another fundamental rule being discarded, and any SCIENTIST (SKEPTIC) vilified or or censored for practicing what can only be considered good science.

    [Response: The simple act of asking questions doesn’t make for good science. Here’s an example. Gravity is pretty well accepted and yet has some fundamental questions still outstanding. How good is the science involved in asking why a bird flying undermines Newtonian physics? And then asking it again? And again? And again? On the contrary, good science is about asking interesting questions that push the science forward, not simply jumping up and down and screaming ‘uncertainty!’ That’s fine once, maybe twice, but people will be more interested in constructively reducing that uncertainty (which always exists) than in paying attention to someone who simply repeats truisms. – gavin]

    I am a Climate scientist with a degree in applied science wildlife biology and a masters in climate change and sustainability and I am not convinced by anything I have read, seen, studied or experimented that there is a definitive correlation between CO2/Greennhouse gases and climate variability.

    Also I have read much of this information from this hack/screw up/whatever and I clearly see what i would consider malpractice and unethical collusion. PArticularly in the case where advice is given in ways to avoid taxes.

    I believe in Climate Variability, I also believe that humans as a whole cause irreparable environmental harm to the planet, however I am skeptical of the hypothesis that is Anthropogenic Climate Change and believe that more research and a more open and debatable approach needs to be undertaken to achieve real results in understanding this. Why is this wrong and why are so many other skeptics with the same opinion vilified and persecuted. Why have you censored more than 6 previous posts I attempted to put up on this topic.

    Can you not see how this topic risks the credibility of science as a whole!!!!!!

  30. 580
    encs says:

    We’re looking at the diversion of about A TRILLION DOLLARS OF INCOME from established industries to new ones.

    Am I the only one old enough to remember the FRAM commercials? “You can pay me now, or you can pay me later.”

  31. 581
    Ron R. says:

    I guess I find it utterly ironic, Anand, that we would be beside ourslves with glee if we found the tinyest bit of protolife on another, otherwise sterile planet while here on earth we are rapidly causing the decimation whole ecosystems. Forests and species turned into mere numbers in some fatcat’s swiss bank account, groundup on a one way trip to the dump.

    http://www.well.com/~davidu/extinction.html

    “Where do we stand in our efforts to achieve a sustainable world? Clearly, the past half century has been a traumatic one, as the collective impact of human numbers, affluence (consumption per individual) and our choices of technology continue to exploit rapidly an increasing proportion of the world’s resources at an unsustainable rate … during a remarkably short period of time, we have lost a quarter of the world’s topsoil and a fifth of its agricultural land, altered the composition of the atmosphere profoundly, and destroyed a major proportion of our forests and other natural habitats without replacing them. Worst of all, we have driven the rate of biological extinction, the permanent loss of species, up several hundred times beyond its historical levels, and are threatened with the loss of a majority of all species by the end of the 21st century.” -Peter Raven, past president of AAAS, the American Association for the Advancement of Science.

    http://atlas.aaas.org/index.php?sub=foreword

  32. 582
    Dale says:

    Do you imagine that the people who are suing Gavin et el are the same who complain about fivolous law suits

  33. 583
    Mark Million says:

    To whom it may concern,

    In regards to sea-level rising. How can one tell, for certain, the actuall hiegth of the sea; when one takes tidal activity into account? Or can it be accuratly measured at all?

    -Mark Milion

    [Response: You need to average over the tidal cycles, but you can do that using the satellite altimeters (see http://www.sealevel.colorado.edu ) and long tide gauge records. – gavin]

  34. 584
    wildlifer says:

    #565
    It appears perhaps that McI just wants data he’s told he can’t have. I mean, what has he done with the Briffa data he said he wanted and didn’t have, until he had to admit he did have it? Anything?

    It seems to be all about the appearances he can create. He was asked why he hadn’t audited any “skeptic” papers (on Curry thread mirror site) and he replied he was just concerned with IPCC-referenced papers. I was dumbfounded. Here are these papers that allegedly refute some of the IPCC papers, but they’re not germane to the IPCC?

  35. 585
    SE says:

    “563
    Barton Paul Levenson says:
    25 November 2009 at 12:12 PM

    SE: How about the authors of the emails stop complaining about the “evil hacker in the pay of the “evil conspirators of industry” and acknowledge that the emails sound bad.

    BPL: Because they only “sound bad” to conspiracy-theory lunatics.”

    Or not. Depending on which conspiracy theory you are invested in.

    IE A person that views the world in black and white which does not allow for any opinions in between and blanket denies all views from the other extreme and demands that they be silent.

    Sounds like intolerant religion to me.

    Which says nothing about the science. Yet stops people from looking at it closer.

    Does that Sound like a real consensus building strategy BPL?

    My “spin” is that the release of the emails has done much good for the science even if individuals are attacked by their own words written in private. I find it highly doubtful that anyone in the UK is going to lose their job over them. Though that may not be completely the case in the US.

    People are now paying attention. Seems like a big waste of time at this particular point in time to evangelize over spilled milk.

  36. 586
    Rob Heron says:

    SE: How about the authors of the emails stop complaining about the “evil hacker in the pay of the “evil conspirators of industry” and acknowledge that the emails sound bad.

    BPL: Because they only “sound bad” to conspiracy-theory lunatics.

    RH: They only DON’T “sound bad” to global warming lunatics. And I thought the skeptics were the deniers.

  37. 587
    Sloop says:

    Reply to #323 “Layman Lurker”

    First, Dr. Schmidt and others have covered ad nauseum the “data access” red herring on this blog and I’m sure elsewhere. I’m not going to repeat them.

    Instead, I’ll respond as someone who has professionally evaluated, funded, and utilized environmental science and monitoring projects.

    First, yes, I don’t think SteveM is qualified to pursue his ‘arguments’, not because he doesn’t possess adequate skills in statistics (although clearly many scientists would argue that.), and not because he fallaciously portrays himself as the noble contrarian expert, the lone gunman standing up against cattle rustlers. Credible scientists who articulate and pursue contrarian views to accepted theory abound in science. IT’S HOW SCIENCE WORKS. I’m working on the policy and research implications of the arguments of a group of contrarians in estuarine science right now.

    Furthermore, contemporary science values more than ever work that is interdisciplinary and pursued by multiple scientists, or teams of scientists; that is always looking for alternative explanations, for errors in models, incorrect or obsolete methods, faulty sensor measurements, poorly understood boundary conditions, incorrectly applied statistical methods or data analysis, etc. Contemporary science is now so complex that opportunities abound for such kinds of errors; so it is only professional practice to be constantly on the lookout for them. In estuarine science we’re still fighting over the meaning of chlorophyll measurements! Our best and only tool: The more heads looking carefully and honestly at the problem, the unknown, the trend etc. the better. Simple but true. That kind of teamwork is one thing I look for in assessing scientific research for management and funding. As an evaluator of science I always look for the contrarian view in order to make a more informed judgment about where policy, management, science communications, and research funding programs should go. I’m always hoping to find and learn from the type of scientist SteveM portrays himself as. Finding such a person or team is very valuable to me.

    If SteveM were truly interested in and cognizant of the science, he would have long ago utilized a very different approach to working with the community of climatological research and numerical modeling, understanding that the quality of his thinking and arguments is for the most part all that is required to join the Republic of Science.

    I, in the context of biophysical science research, don’t need or wish to ascribe nefarious intent to his motivations. He may be smart, sincerely convinced he’s right, and that he is pursuing a noble cause. Yet he could still be, and is, totally wrong. Has he found a few “gotcha’s” in the statistics? Sure. Is that contrarian science? No.

    In the context of social sciences and policy analysis; his motivations and communications strategies would be of great interest to me.

    I feel bad for folks, scientists, engineers, laypeople, ideologues, etc. That trumpet this man’s campaign. You’ve been sold down the river.

    So here’s what I propose for someone without damaged credibility who wishes to pursue SteveM’s arguments legitimately. Put together a proposal to pursue the lines of research that result logically from his criticisms. If that proposal entails a new assessment of met and climate proxy data logs and metadata information, etc., that would be great and given all his rhetoric, necessary. SteveM’s successor would have to develop partnerships with the scientists who’ve done the proxy research that SteveM has repeatedly defamed; but given the hell they’ve been put through (and it does come with the territory given the socio-political consequences of their findings) they’d possibly be happy to go back to the basics; especially since funding for the time needed to do this re-validation would be provided (see below).

    O and absolutely this new team should suggest reviewers to me of their forthcoming proposal. I’ve received and valued many negative reviews from reviewers suggested by the writers’ of proposals I’ve worked on and funded. That doesn’t mean I won’t find other reviewers, but I always welcome suggested reviewers as it makes it easier for me to delve into scientific discipline/community when evaluating a proposal. Over the years I’ve developed all sorts of “tricks” to efficiently and accurately assess the state of a scientific discipline and how a particular proposal relates to it.

    Once I’ve collected say five written reviews, I’ll give the research team the opportunity to review them (reviewer names expunged) and submit a written rebuttal. Then I’ll assemble another team of qualified reviewers who will look at the proposal, the reviews, and the rebuttals. I’ll ask them to give me a thumbs up or a thumbs down on funding the proposal; if a thumbs down I’ll ask them to provide recommendations on how the proposal can be improved sufficiently to make it worth funding. Then I’ll take all that and sit down with executive management professionals who work on environmental issues, both from government and external to government, and I’ll ask them to review the conclusions of all the independent reviewers and my own analysis of the proposal. Once I have their thumbs up for funding, I’ll go out and help the team find the funding for the proposal. Because if this proposal successfully executed means that federal and state governments won’t have to worry about global atmospheric/ocean/climate change in the future as much as that “cabal of fanatics at GISS, CRU”, etc., are telling us I would be incredibly grateful professionally and personally. It would be well worth me devoting the next ten years of my professional life to funding such a proposal, if it ever could be produced. And I do know how to cultivate research funding too so my services would be of value to your team.

    Finally, would this be considered special treatment of the proposal: an overly harsh, unfair review? No. This is exactly the process undertaken for the millions of dollars worth of research I have been involved in funding, even if that proposal is asking for as little as $50,000.

    Fair enough?

  38. 588
    SecularAnimist says:

    David Gordon wrote: “The classic saying goes ‘extraordinary claims require extraordinary proof’.”

    Which is by far the worst thing that Carl Sagan ever said, since the question of what constitutes an “extraordinary” claim is entirely subjective, as is the question of what constitutes “extraordinary” proof.

    For the obstinate and ignorant deniers of anthropogenic global warming who laughably call themselves “skeptics” while they uncritically and unquestioningly believe everything they hear from Rush Limbaugh or read on a right-wing blog, there is of course NO amount of evidence that is sufficiently “extraordinary” to establish the reality of anthropogenic global warming. They’ve been told by the corporate-sponsored, so-called “conservative” propaganda machine that AGW is a “liberal hoax”, and that is “settled science” as far as they are concerned.

  39. 589
    heko says:

    First of all, thank you for putting so much of your valuable time to responding here.

    RE: [Response: Both papers were cited in the report – MM either refers to McIntyre and McKitrick (2005, E&E) (Ch6), or more likely McKitrick and Michaels (2004) (Ch. 2), and the other paper was Kalnay and Cai (2003), also cited (Ch. 2). No redefinition apparently took place, which is unsurprising. – gavin]

    I may have missed something in these long threads and I’m not thoroughly familiar with the IPCC process, so pardon me for probably asking the obvious or repeating a question:

    1) Why exactly did prof. Jones state “he can’t see either of these papers being in the next IPCC report”?

    2) Why did he try to “keep them out somehow” in the first place?

    For simply bad science, the reasons should be obvious, but for the other paper prof. Jones writes that he himself urged the authors to send the article.

    Thanks in advance!

    [Response: The space in the IPCC reports is limited, and so I doubt you want the whole thing taken up with showing that particular papers are wrong. However, where there are serious (if often flawed) points that need to be covered in order to give a balanced picture, then they will be discussed (as they were). This process is very widely spread out amongst the authors, reviewers, re-reviewers etc. and so individuals don’t have that much influence regardless of their initial feeling. – gavin]

  40. 590
    Alan Millar says:

    “Of course, the e-mail you are using to describe this as “rubbish”, is describing OBSERVED CHANGES IN SOLAR ACTIVITY.

    If it’s not changed sufficiently to explain warming, then it’s not the source of observed recent warming.

    Apparently you want to replace observation with your belief that “it is all solar”.”
    dhogaza”

    You have totally missed the point. I am not saying the content of the e-mail is rubbish.

    I am saying that if you believe in AGW you cannot attribute the 1910-1940 warming to Solar activity and that it was the failure of the said scientists to point this out that is the problem. It has been conveniant for certain sections of the scientific community to let this meme go unchallenged.

    Alan

  41. 591
    Kevin says:

    Hate to add to the workload here, but I am sure it is a simple question.

    When looking at many of the AGW graphs, they are often expressed as an anomaly, +, -. I have had trouble however figuring out what the “zero point” is to which the data are being compared against. Is if from some particular year or set of years that the Zero level is being determined?

    Thanks for anyone who can help! (Gavin is doing enough already!)

    Kevin

    [Response: It’s usually a 30 year period climatology – 1951-1980 in GISTEMP, 1961-1990 in HadCRU for instance. The trends are insensitive to this baseline though, and you can readjust very easily. – gavin]

  42. 592
    Alan Millar says:

    548 Tamino
    “He confuses the “1940s blip” with the temperature increase from 1910 to 1940 — because he really doesn’t have a clue what he’s talking about. But he still feels qualified to conclude that climate science has “unresolved crucial issues.”
    His statement about “muttering Solar and Aerosols as the answer” reveals that either he’s ignorant of the influence of the volcanic lull, or he chooses to omit mention of it because he’s deliberately trying to deceive people”

    I did not mention the 1940 ‘blip’ problem, a separate issue altogether. I mentioned that part of the e-mail that refers to the 1910-1940 ‘problem’.

    I am aware that some people refer to an alleged lull in vocanic activity during that period rather than directly mentioning aerosols. However, they and you are indicating that meant aerosols were low during that period. That is unless you are postulating another way that lack of volcanic activity can cause global temperatures to rise.

    However, aerosols did not fall during that period, they rose and quite sharply.

    I am sure you know that volcanoes are not, by a long way, the only source of aerosols in the industrialised 20th century.

    [Response: Actually you aren’t quite right about the aerosol issue. If you look at the Greenland ice cores, the peak of black carbon deposition is the 1930s (McConnell et al). – gavin]

    You, Tamino, cannot explain the problem away with this sort of hand waving. The e-mail shows that certain scientists recognise that the problem is currently unresolved.

    Alan

  43. 593
    David B. Benson says:

    EconRob (566) — Here is some of the settled part, from BPL.

    Barton Paul Levenson:
    1. CO2 is a greenhouse gas (Tyndall 1859).
    2. CO2 is rising (Keeling et al. 1958).
    3. The new CO2 is mainly from burning fossil fuels (Suess 1955).
    4. Temperature is rising (NASA GISS, Hadley CRU, UAH, RSS, etc.).
    5. The increase in temperature correlates with the increase in CO2 (60–76% for temp. anomaly and ln CO2 for 1880-2007). See
    http://bartonpaullevenson.com/Correlation.html
    More graphs are found here:
    http://chartsgraphs.wordpress.com/2009/08/13/excel-chart-misrepresents-co2-temperature-relationship/#more-1687

  44. 594
    Dan says:

    “Much of the major scientific thought over the ages has been reversed. ”

    No. It is progressed, not been reversed. Science is a method of making hypotheses, gathering and analyzing data, making conclusions based on those data, subjecting the analysis to peer-review, and making new hypotheses. When new data comes forth that provides new conclusions, that is not “reversing”.

  45. 595
    cmb says:

    It has occurred to me that these emails, whatever else they may be, also comprise an easily accessible record of denialist abuses. I’ve been having fun with this fact in other fora.

    One example: http://tinyurl.com/y8f73m4

    I’m convinced that, in more capable hands than mine, this event and its widespread coverage represent an opportunity to chronicle these abuses for the world. =)

  46. 596
    Sven Errantius says:

    “What Draconian laws and what quasigovernmental body would that be? I’m not aware of any.

    Comment by Barton Paul Levenson — 25 November 2009 @ 7:10 AM

    Try the ministerial proposal that Her Majesty’s Government impose individual carbon rationing starting in 2020, with rations shrinking to a few kilos a day in a generation,

    Last year, 350 and related UNEP quangos began publicity campaigns demanding the global imposition of a zero carbon regime as well. As limiting annual CO2 rise to 1 ppm works out to a global daily carbon ration under a Kg, don’t hold your breath awaiting transparency from Copenhagen on calling carbon prohibition what it is.

    Inconvenient Youth or Bill Mckibben’s good intentions notwithstanding, should enforcement outstrip technical progress, and alternative energy fail to take up the kilotonne a year slack of carbon intensive heating, lighting, and food transport, heroic carbon cuts could indeed lead to draconian outcomes.

    Especially if , a propos of Jef Masters concern true believers succeed in

    ” waging a disinformation campaign against the science saying that their product is dangerous.”

    Some may be object that alternative power may not materialize in time, and if it fails to , others may object that freezing to death in the dark is not an acceptable price to pay for slowing global warming by ten microdegrees a day. However, these are mere details of enforcement , and we should therefore celebrate the carbon footprint of the UK’s six million electronic surveillance cameras as a worthwhile investment in catching carbon scofflaws.

    Perhaps GISS should follow the CRU’ssocially responsible example by welcoming more overhead cameras to provide real time email surveillance today, and deter carbon tax evasion tomorrow.

  47. 597
    Mike Donald says:

    #518

    After a few hits of BPL the average smart alec sceptic must feel like a tugboat against the Bismarck.

    BTW comment #540 (Weather underground) I’ve got to copy/paste that elsewhere – such resonance with my own thoughts.. Why thunk up bon mots when cleverer peeps do the hard work for you? It does put it into context though. This most ferocious vilification of scientists :-

    “probably the world’s most pervasive and best-funded disinformation campaign against science in history”

    Is a phrase I’ll remember for sometime.

    Still, despite the artistry displayed, there’s a school of thought that whackamole and projecting future doom and gloom aint the way to go.

    http://www.guardian.co.uk/environment/cif-green/2009/nov/23/climate-denial-strateg

    and indeed there’s a solar panel plant near Detroit no less.

    http://www.nytimes.com/2009/11/24/opinion/24herbert.html?_r=1&scp=1&sq=detroit%20solar%20&st=cse

    So maybe “a future of hope” – tough call I know – might persuade a few rather than entrench beliefs. Call me naive…

  48. 598
    Phil M says:

    Back to the old ‘hiding the divergence’ chestnut
    – up until this CRU scandal erupted, I hadn’t really appreciated that ‘divergence’ was a well known issue
    – I *had* noticed that the ends of the proxy data plots didn’t seem to follow the real temps very well, but since the plots usually seem to have nice up-ticks to them, and end in 1980 or 1960 or thereabouts, I assumed, I am sure like a lot of people, that, had these plots continued, they would probably track the actual temps.
    – actually, I assumed they would follow the actual temps, because that is how you guys do a proxy plot – you tune the last bit to the real temps, so everything looks believable.
    – but now I realise that even the little up-tick at the end is an product of the artificial up-lift added by the fortran code, which is in the FOIA data.
    – without the blending operation in the code, the end bits of the proxy plots would droop embarassingly downwards.

    So, although you maintain that the ‘divergency problem’ is well known, and indeed, discussed in the Nature article, and in AR4, when one looks at the plots it’s easy to miss it.

    So this is why I think it is right to focus on the phrase ‘hide the decline’.

    – I would request that when hockey-sticks are generated in future, the ‘hide the decline’ code is removed, and so we can see what the actual proxy results are, and not the blended ones.

    Also, in order to make things clearer still, I think you should refrain from plotting the real-temps in large black lines or red lines over the proxy data, as this only serves to draw one’s eyes’ attention away from what the proxy data is actually doing.

    – there are plenty of places where we can see what the real temps do, so it doesn’t help transparency to have them plotted over the proxy data.

    [Response: All the data series are available separately, you can put them into gnuplot or Excel yourself. Smoothing timeseries near the end is always tricky, but this figure shows what difference the technique makes. – gavin]

  49. 599
    Ray Ladbury says:

    Biff Larkin asks: “Gavin, someone, above, asked whether you were doing any of this moderating during your working hours for the US government.. It’s a fair question, and as you probably know, now the subject of a Freedom of Information Act Request.”

    You know, Gavin, since, like most scientists, I am sure you are pretty much always working on your research–or at least thinking about it, I suggest that you save time by making monitoring of Biff’s contributions a REALLY LOW priority. I guarantee it will not subtract from the information on the blog.

    Oh, and Biff, I’m on Thanksgiving Break.

  50. 600
    Hide the Decline says:

    Hello, I am wondering if you would ask Michael to respond the this article.

    http://wattsupwiththat.com/2009/11/25/climategate-hide-the-decline-codified/

    [Response: What does it have to do with Mike? This is code related to the MXD tree ring series from, I would guess, Tim Osborn. No-one here was involved with that at all. The decline of which you speak is not hidden at all (Briffa et al, 1998, Nature). – gavin]