RealClimate logo


CRU Hack: More context

Filed under: — gavin @ 2 December 2009

Continuation of the older threads. Please scan those (even briefly) to see whether your point has already been dealt with. Let me know if there is something worth pulling from the comments to the main post.

In the meantime, read about why peer-review is a necessary but not sufficient condition for science to be worth looking at. Also, before you conclude that the emails have any impact on the science, read about the six easy steps that mean that CO2 (and the other greenhouse gases) are indeed likely to be a problem, and think specifically how anything in the emails affect them.

Update: The piece by Peter Kelemen at Columbia in Popular Mechanics is quite sensible, even if I don’t agree in all particulars.

Further update: Nature’s editorial.

Further, further update: Ben Santer’s mail (click on quoted text), the Mike Hulme op-ed, and Kevin Trenberth.

1,285 Responses to “CRU Hack: More context”

  1. 501
    manacker says:

    Hank Roberts

    You are slipping into ideological polemic (487) rather than reasoned debate. It’s not an area I want to get into.

    Openness and transparency in climate science is a fall-out of Climategate, like it or not.

    It will not go away by ignoring it or rationalizing why climate science is doing good for mankind and should therefore be above scrutiny.

    The public (who pays for climate research with taxpayer funding) has a right to know anything about climate science that it wants to know, including independent audits of the data and data handling process, if they so desire.

    Those are the facts here.

    Max

    Max

  2. 502
    manacker says:

    BPL (497)

    See my post 498 in response to Martin 486.

    Max

  3. 503
    Tuomo says:

    Given that this is a comments thread to a CRU-email context post: Here’s a question about what exactly “Mike’s Nature trick” is. By Mike’s Nature trick I mean what exactly was done by Jones that he referred by that term.

    If you read the web, you see a lot of claims. Some obviously wrong, some more intriguing. My question to you: Are the following claims true?

    – Briffa’s proxy data is valid as a temperature proxy only if it correlates with the instrumental temperatures.

    [Response: Basically true. You could also use other independent temperature proxies in earlier times. – gavin]

    – In Briffa’s original proxy data series, the proxy series correlates positively with the instrumental series before 1940, but there’s no positive reliable positive correlation in the post 1940 data.

    [Response: Not quite. The divergence happens post 1960. – gavin]

    – (Based on the email correspondence and code samples, someone could guess that) Jones deleted the post-1960 values of the Briffa series, replaced them with some materially different values.
    – The Jones’s materially different values are most likely instrumental temperature measurements or other values selected to be close to the instrumental measurements
    – Jones then smoothed the spliced series.

    [Response: This is a misreading. The only goal was a smoothed blend of the proxy and instrumental data to indicate the long term and recent changes without it being too cluttered. The post-1960 data in the Briffa reconstruction isn’t relevant to that. But smoothing requires some decision about what to do with the end point problem (in this case starting in 1935 since it was a 50 year smooth). Jones used the instrumental data so that values from 1935 to 1985 are a blend of the proxy and instrumental data. I’m not quite sure what the criterion was at the 1999 end point of the instrumental period. – gavin]

    – The end result was a “proxy” series that looked like an accurate reconstruction of late 20th century temperatures when compared to the instrumental measurements.

    [Response: No. The end result was described as the proxies and instrumental record.- gavin]

    – The results were published in World Meteorological Organization WMO-No. 913 (http://www.wmo.ch/pages/prog/wcp/wcdmp/statemnt/wmo913.pdf)

    [Response: The smooth was used in that brochure. – gavin]

    My question to you are which ones of these claims are true and which are not? I guess with more resolution I’d like to know which claims are demonstrably wrong (proven innocent), which claims could be true but that there’s not credible evidence for (not proven guilty), and which claims are demonstrably true (guilty).

    What is of particular interest to me is whether the output of the “highly artificial” “fudge factor” code was ever used in any published papers. I have a co-author that uses “if 6==9” as debugging toggle, and code sections in those parts do not (at least intentionally) make it to our papers. He argues that this might be a similar case.

    [Response: No. There was a draft, but it doesn’t seem to ever have been published, and is very clear about why and how this was done. – gavin]

  4. 504
    manacker says:

    BPL

    For the IPCC comparison of temperature trends covering short-term periods (some less than 30 years) with longer-term periods to attempt to demonstrate an acceleration in warming see AR4 p.104 FAQ 3.1

    The curve could have been drawn with all points starting at the beginning of the 20th century, proving that the first 40 years had a warming trend of almost twice the entire 100 year.

    Short term trends in a cyclical record can always be picked to show a greater rate of increase than the longer trend.

    Max

  5. 505
    Hank Roberts says:

    Bruce Williams — you’re confusing three different pointers I suggested in my response to you at (450) above (5 dec. at 9:05pm)

    I recommended you look at the first link under Science on the right hand side of the page — that would be this page, the page that you’re reading right at the moment.

    You can also look up the infrared astronomy work, it’s easy to find.

    You can also click on that search I suggested, which searches realclimate for the keywords.

  6. 506
    Tuomo says:

    Barton Paul Levenson says to Lubos Mokras: “Could some smart scientist explain, how can nature distinguish between man made and nature made CO2?”

    Nature can’t. (And I am not talking about the journal.)

    However, a person analyzing and modeling data has to draw that divide. A system has both exogenous and endogenous variables. Exogenous variables are not causally impacted by the system state.

    Endogenous variables are determined by the system. Co2 should be modeled as having both components. Usually, it’s a good modeling choice to consider co2 that is quickly released by humans as exogenous. Usually, it’s a good modeling choice to consider the co2 that is absorbed by the oceans endogenous.

    The endogenous part reflects the system equilibrium path, and empirically inferring any causality from that part in a complex system is difficult. In contrast, the exogenous variation in co2 is easier to use in establishing causal relationships.

  7. 507
    dhogaza says:

    _The Mote in God’s Eye_ was pretty good though.

    Unfortunately, the world has largely passed Pournelle by … think of it as …

    Evolution in action :)

  8. 508
    Martin says:

    Max (498),

    No, we really do not need to “wait and see”. It doesn’t matter. Even if your putative 8-yr trend was significant (a point you still haven’t answered, as far as I can tell), we KNOW the long-term trend is deadly. We don’t live in this pretend world of yours in which we’re utterly mystified about what the next decades will bring.

    –Martin

  9. 509

    Would anyone know if the IPCC climate computer models were anything other than just statistical, as opposed to, say, models based on chemical and physical interactions?

    [Response: Yes. Because you wouldn’t be able to download the code, read and understand it, and then replicate the results. – gavin]

  10. 510
    Tom Dayton says:

    Lubos Mokras, 6 December 2009 at 9:39 AM, for details beyond Barton Paul Levenson’s reply to you about how we know CO2 addition is from humans, try these sources:

    To start, read The Global Warming Debate’s Chapter 7: Attributing Mankind.

    Then see SkepticalScience’s Human CO2 is a tiny % of CO2 emissions.

    Followed by SkepticalScience’s Are humans too insignificant to affect global climate?.

    And then SkepticalScience’s Comparing CO2 emissions to CO2 levels.

    And the ever popular It’s volcanoes (or lack thereof).

  11. 511
    stevenc says:

    BPL, we really aren’t disagreeing. I was just trying to come up with a way of explaining what happens in such a way as to avoid confusion over the 2nd law. I still prefer my preventing cooling over warming explanation since that makes it clear it is not violating the 2nd law. Perhaps if I used the pot burner and lid comparison it would be better. The burner warms the pot while the lid prevents the heat from escaping. The lid does not warm the pot it mearly prevents the pot from cooling. Does this make it more clear? I obviously failed previously or we wouldn’t still be discussing it.

  12. 512
    Don Shor says:

    Max: “But there is really no point in denying that it has cooled since the 21st century started, because that is what the physically observed and reported record shows.”

    BPL (to Max): “You do not have enough data to conclude that. A “trend” has to be statistically significant, and your regression is not. There has been no significant cooling; the data is sufficiently flat that we don’t know if the real trend is up, down or sideways. We simply don’t know. And the main reason we don’t know is that 8 years is too short to prove anything.”

    Ok, what has been the trend 1979 – 2008? How does it compare with the warming experienced in, say, the first half of the 20th Century?

  13. 513
    Phil. Felton says:

    Re #466
    Bruce Williams says:
    6 December 2009 at 11:28 AM
    Ref #452

    Phil Felton
    Are these graphs empirical or calculated? And who produced them?

    I calculated them using LInepak, they’re accurate.

  14. 514
    barry says:

    Manacker:

    The sources for all 4 temperature records are clearly shown on the chart I posted.

    As in: http://farm4.static.flickr.com/3622/3464289034_9e57f541b7_b.jpg

    Forgive me if this is woefully ignorant, but isn’t your link to ‘GISS’ data actually NCDC (NOAA) data?

    NCDC monthly data = ftp://ftp.ncdc.noaa.gov/pub/data/anomalies/monthly.land_ocean.90S.90N.df_1901-2000mean.dat

    GISS = http://data.giss.nasa.gov/gistemp/tabledata/GLB.Ts+dSST.txt

    I think you understand that 8 years is to short to establish a climate trend, and I believe you are aware that 8-year flat or cool trends are present throughout the various temperature series that show overall warming. What we are currently observing is nothing very unusual per se WRT a warming climate. Other than that we are unable to forecast interannual variability, which no one disputes, I’m not sure what point you are trying to make.

    The comparison 1979 1998 trend with 1979 2009 trend is a more robust method to assess the climatic trend changes, if any, of the last 10 years or so, than excising data previous to a more recent point in time, don’t you think? The signifigance values are much higher for the former method after all.

    (Trying to bring this back to topic….)

    It is somewhat ironic that you insist on access to *all* data, yet are not inspired to use as much as is available already to make an analysis. I note that this is a recurring theme in other respects on the data issue.

  15. 515
    Doug Bostrom says:

    Max is in “reasonable” mode here, I see.

    Max, elsewhere:

    “As such, it [the IPCC report] is full of agenda driven pseudo-scientific exaggerations and distortions, which all go in the direction of making its “pitch”.

    “Forget the alarmist junk science sites like RealClimate and read the many original studies out there, if you really want to know what’s going on.”

    “Real Climate is selling AGW hysteria. ”

    Topped by the hysterically ironic:

    “Stick with factual argumentation, not silly name-calling.”

  16. 516
    barry says:

    Cheers to you, Gavin, for dealing with all these posts and responding to queries and challenges, even repeating the same information for scrollers. Your patience is a great boon to the matter (although I think your unedited first-retorts to some of the posts might be entertaining).

    Are there any amongst the RC contributors that could relieve you for a while, or are you the best choice for referee?

  17. 517
    Rod B says:

    I really hate to jump in this briar patch again, but it’s just too numerous to ignore. manacker made a claim, backed up with linear regression and data from four major temperature sources, that global temperatures have cooled in the 21st century. He didn’t say (IIRC) what statistical significance this had; nor how it fit or didn’t fit into longer term temperature trends; nor if this is “significant” overall; etc; etc; etc. The gyrations and tap dancing a bunch of you do in trying to deny the obvious (no more no less) because you wish it weren’t is astounding (not to mention entertaining…)

  18. 518
    ZZT says:

    >> All raw data should be published.

    >Absolutely. And we should live in a utopian social-ist universe where >scientists have unlimited funding, there is no proprietary data, and we >have invented a time machine, so we can transport a few terabytes of disk >back to 1980, when the cost of enough disk to store all the raw data would >have been in the millions of dollars.

    Err…you seem to be forgetting that storage costs decreases at an impressive rate. The tape storage of the 1980s could/should/probably were moved to disk in the 1990s. No need for a time machine, no need for angst.

    Presumably the cost of storage would be vastly less than the cost of collecting the data in the first place and I am sure that all those grant applications had ample provision for IT resources.

    Even Jon Stewart and his writers noted that only allowing the ‘value added data’ to survive was undeniably unconvincing.

    Could it be that there are a new set of deniers – those that deny that the CRU crew could ever make a mistake?

    From what one can see of the CRU data stewardship, coding, posturing, and prevarication it is hard to imaging that an open source effort with a linux web server wouldn’t have been a better investment for the human race.

    Unless of course it was all political in the first place.

  19. 519
    Chris Colose says:

    Bruce Williams,

    I’m not sure where multiplying a global emissivity change by the total greenhouse contribution by CO2 is supposed to get you. Maybe it’s me, but I cannot even see why this would make sense. By the way, the total CO2 contribution to the greenhouse effect is more on the order of 20% than 4%, but this is complicated by overlap effects and the current atmospheric composition. Note first of all that the forcing of CO2 is logarithmic and therefore the effect of doubling of CO2 at modern concentrations is substantially less, than say, removing all of the CO2 from the atmosphere. The temperature change that results from a forcing of 4 W m-2 (or approximately 2x CO2 under the terrestrial climate regime and realistic initial concentrations) can be approximately given as:

    {∂σ T_eff4/∂T_sur}-1 or 4σT_eff3

    where we assume the emission and surface temperature are linearly related, and please note that you need to use the effective temperature (~ 255 K) since this is where radiation balance is set, not the surface temperature. This results in a climate sensitivity of about 0.27 K (W m-2)-1, or around 1 K for a doubling of CO2. This is treated a bit more realistic in models, although the results are basically the same, with very small differences in results.

    This is purely considering the fact that the area under the Planck curve must increase to compensate for the “bite” taken out of the spectrum by the extra CO2, as the planet now radiates at a colder temperature until equilibrium is established. To get the IPCC range of sensitivity, you need to include other things, such as the increase in water vapor of the atmosphere, global albedo changes, and the like. Although it can be said with high confidence that this substantially (at least doubles or triples) the no-feedback case, the error bars are much larger, and simple formulas don’t really work given the complexity of the feedbacks involved, so the evidence is primarily built off of past climate behavior.

  20. 520
    Tobias says:

    Here’s my five cents…

    What I can see happened with the hack and what I read from the posts …leads above all to one conclusion… the whole issue of “fighting global warming” and “saving the planet” is completely besides the point, not to mention over the top.

    Human beings are not up to the task of climate science when it comes to reliable predictions… and, above all, should be thrown into the next best dungeon for any such attempts as geoengineering.

    What’s even worse is that this whole ridiculous debate about CO2 being at the heart of our troubles is just covering for things in the world that are truly of high impact to the welfare of human and all other beings on this planet.

    The hacked emails clearly show climate science as what it is today …a scientific hoax… a political instrument to cover up for the actual system failure in our, especially western, societies… which is the monetary system… and all the shit, corrupt science included, that comes along with it.

    And all you “experts” trying to counter cite, re-reference and refute the arguments won’t do anything about that.

    The Bilderbergs, Rothchilds, Rockefellers and Goldman-Sachs of this world… are truly having a fun time watching all you people debate matters completely besides the point… while war is waging in the middle east and the coastlines of the Philipines are being privatized and nobody seems to care.

    If anything… global climate is a matter of engergetic input output relations… in other words… in theory, there is nothing beneficial to solar energy other than less emissions (ignoring any waste produced during manufacture & maintenance)… all that energy production, no matter from which source, is bound to aggregate and turn into heat upon consumption.

    And that is simply why CO2 is quite a stupid measure… if anything, it should be Joule. Same goes for any engery source we try to bind and make available on the surface upon which we crawl and the atmosphere that surrounds it.

    Even if I were to take any of this debate for a serious matter, fighting global warming is as much fruitless as adopting to changing climate is not.

    There have always been refugees around the world and obviously for quite other reasons than climate. I find this whole debate, above all to be humongously ignorant, and all you proponents of “save the planet by saving the climate” to be rather polemics.

    Sure, burning fossil fuels should be restricted and abolished, but not to “fight warming”. Above all it is a ridiculous waste… as, especially for oil, there are so many uses, which we restrict by burning it… and let’s not forget the emissions and ecological mess through sunken tankers and the kind of terraforming natural habitats through mining.

    Saving energy… is a brilliant idea. Quite a reasonable proposition… but “fighting climate change” …anyone, are you out of your minds?

    I believe, scientists around the world, not just when it comes to “climate science” are completely overrated, if not much too little discussed in terms of the implications that come along with all these technologies as devlopped on the grounds of scientific achievements.

    This debate is about one thing: Stop, you guys! We have real matters to discuss. People are starving and being fed junk and given junk in relief to control their lives ever more severely… the ultra-rich in this world are having a good laugh about all you people saving the planet, while they are tightening the chains around your necks and ankles ever more tighly.

    I am not a proponent of such nomenclature… but looking at all these matters, I can truly understand why some people get so outraged that they start calling you environmental nazis. Do you want to know why? Or are you just angry or ignorant about me even mentioning this? The reason why is, because you are so much endulged into your own state of mind and into the high priesthood of climate science that you are ignorant to the real pains and sufferings all around you, mere bystanders, possibly with a bulkload of useless ideals and hypocrises.

    And, believe it or not… I am not sponsored by anyone but rather speaking out of my own tiny mind. Also I have no anti-environmentalist agenda whatsoever, the opposite is the case. I love permacultrue and find it to be much neglected by all of us industrialised junkees.

    You guys have to learn to be a bit more buddhist with respect to your false matters and a bit more Marthin Luther King with respect to matters of real concern… such as: When does anyone start ask for true democracy? …for people voting on matters, not on political colours or camps? Vor people being able to veto government regulations that the majority of us would NEVER agree to.

    But I guess, if this whole debate were a matter of democracy, climate science would rather have to fear the voive of the people, as it would surely cast its relevance into nothingness.

  21. 521
    RaymondT says:

    Gavin, What is the percentage (roughly) of climatologists who work on the determination of the radiative forcings in the climate models?

    [Response: Very hard to say. There are hundreds (thousands?) of people working on radiative transfer, aerosols, solar variability – models, theory, remote sensing etc. – and that is out of 10s of thousands. – gavin]

  22. 522
    Ken W says:

    Re: 499
    “This is inherent in human beings, … Are their people in the world that never outgrow this?”

    So, are we to assume (from your post) that humans are therefore NEVER responsible for ANYTHING bad? If not, then why even post such a silly thing?

    Another common human tendency is the tendency to dismiss long-term threats (e.g. certainly these little enjoyable cigaretts won’t hurt me).

    Another common human tendency is to think we understand commplex issues without doing any serious research. That’s very common when non-scientists dismissively ignore mountains of scientific evidence built up over decades.

  23. 523
    RaymondT says:

    Gavin, The reason I asked about the % of climatologists working on the determination of the radiative forcings is that the consensus argument is often used in public policy decisions. My understanding is that the roughly 2,000 scientists who said that “The evidence for global warming was unequivocal and it was very likely that humans play a dominant role in that warming” are not climatologists but either scientists such as biologists who observe changes in the climate which are consistent with the model predictions or scientists and engineers who are looking for mitigation solutions. Since the earth sciences are so complex and since research demands that problems be sub-divided in narrow disciplines I am wondering how someone for example working on the increase in malaria due to climate change can assert with 90% confidence that climate change is occurring ? Since the global warming theory depends so heavily on the numerical modelling of the climate it seems that the people who can best have an opinion on the certainty of climate change would be those climatologists who work on the selection of the radiative forcings used in the climate models. How can someone for example working on the measement of the ocean temperatures assert with 90% certainty that climate change is occurring ?

  24. 524
    Jim Galasyn says:

    Such a delicious turn: Were Russian security services behind the leak of ‘Climategate’ emails?

    …The server is believed to be used mainly by Tomsk State University, one of the leading academic institutions in Russia, and other scientific institutes.

    Computer hackers in Tomsk have been used in the past by the Russian secret service (FSB) to shut websites which promote views disliked by Moscow.

    Such arrangements provide the Russian government with plausible deniability while using so-called ‘hacker patriots’ to shut down websites.

  25. 525
    RaymondT says:

    Gavin, I realized that I have fallen in the trap of confusing climate change and anthropogenic climate change (AGW). Here is my corrected message

    The reason I asked about the % of climatologists working on the determination of the radiative forcings is that the consensus argument is often used in public policy decisions. My understanding is that the roughly 2,000 scientists who said that “The evidence for global warming was unequivocal and it was very likely that humans play a dominant role in that warming” are not climatologists but either scientists such as biologists who observe changes in the climate which are consistent with the model predictions or scientists and engineers who are looking for mitigation solutions. Since the earth sciences are so complex and since research demands that problems be sub-divided in narrow disciplines I am wondering how someone for example working on the increase in malaria due to climate change can assert with 90% confidence that AGW is occurring ? Since the global warming theory depends so heavily on the numerical modelling of the climate it seems that the people who can best have an opinion on the certainty of AGW would be those climatologists who work on the selection of the radiative forcings used in the climate models. How can someone for example working on the measurement of the ocean temperatures assert with 90% certainty that AGW is occurring ?

  26. 526
    Fred Staples says:

    Numerous other sources, Silk, 50. One of teh best documented is the Central Engand Temperature record.

    Tamino’s fina paragraph reads :

    “The rate of warming in CET since 1980 is 0.05 +/- 0.02 deg.C/yr, or half a degree C per decade. If this trend continues, then by mid-century CET will have increased by a substantial amount, another 2 deg.C. This will bring CET to heights unknown for at least 350 years, probably several thousand years, and in all likelihood warmth not seen since humans inhabited the British Isles”

    From 1940, 7 decades, the trend is just 0.13 degrees per decade.

    Since 1659, no single year has averaged 11 degreesC. Of the 44 years with temperatures at 10degreesC or above, 22 were before 1945.

    Is his comment justified?

  27. 527
    Timothy Chase says:

    Re Manacker’s 479, Part I of II

    This is in response to manacker’s 479.

    In 438, Transparency and Complexity Revisited, Part I of V, I asked:

    What do you think was “uncovered” by “Climategate”? Do you have some specific correspondence in mind? I strongly suspect that any letter that you might bring up at this point and points that you might “reasonably” choose to draw from it have already been addressed, within the past few threads. In fact I strongly suspect that they have been addressed several times.

    manacker responded in 479:

    The leaked emails have uncovered very little in legally incriminating evidence, if any at all.

    … and yet a crime was committed. By those who broke in and stole all the material, material which they held on to not so that it could be examined in detail but so that it could be released when it would have the maximum impact upon Copenhagen.

    Please see:

    E-mails alleged to undermine climate change science were held back for weeks after being stolen so that their release would cause maximum damage to the Copenhagen climate conference, according to a source close to the investigation of the theft.

    Climate e-mail hackers ‘aimed to maximise harm to Copenhagen summit’
    Ben Webster, Times Online, December 3, 2009
    http://www.timesonline.co.uk/tol/news/environment/copenhagen/article6941880.ece

    (Hat-tip to DeepClimate)
    *
    manacker wrote in 479:

    They have shown an arrogant inner cabal of highly influential climate scientists.

    As I stated earlier in 438,”Vague allusions establish little more than a lack of the command of the facts.”

    I would argue that they show a group of individuals who are used to being attacked by industry-financed “skeptics” who often have an ideological axe to grind, scientists who may at times be understandably defensive.

    Regarding the industry-financing:

    If you add it all up, the fossil fuel industry outspent the environmental groups by $36.8 million to $2.6 million in the second quarter, a factor of 14 to 1. To be fair, not all of that lobbying is climate change lobbying, but that affects both sets of numbers. The numbers don’t even include lobbying money from other industries lobbying against climate change, such as the auto industry, U.S. Chamber of Commerce, etc.

    The Manufactured Doubt industry and the hacked email controversy
    Dr. Jeff Masters’ WunderBlog
    http://www.wunderground.com/blog/JeffMasters/comment.html?entrynum=1389

    And the attacks aren’t simply in print or in the UK.

    Please see:

    An alleged series of attempted security breaches at the University of Victoria in the run-up to next week’s Copenhagen summit on climate change is evidence of a larger effort to discredit climate science, says a renowned B.C. researcher.

    Andrew Weaver, a University of Victoria scientist and key contributor to the Nobel prize-winning work of the Intergovernmental Panel on Climate Change, says there have been a number of attempted breaches in recent months, including two successful break-ins at his campus office in which a dead computer was stolen and papers were rummaged through.

    Attempted breaches show larger effort to discredit climate science: researcher
    Megan O’Toole, National Post, Dec. 3, 2009
    http://www.nationalpost.com/m/story.html?id=2300282

    (Hat-tip to DeepClimate)
    *
    manacker wrote in 479:

    They [the CRU emails] point to some massaging of data to make things work out in favor of the point being proven…

    Do you havew something specific in mind?

    I had stated in 438:

    I strongly suspect that any letter that you might bring up at this point and points that you might “reasonably” choose to draw from it have already been addressed, within the past few threads. In fact I strongly suspect that they have been addressed several times.

    Is this why you have chosen not to be specific?
    *
    manacker wrote in 479:

    As an example see my post 427 of 5 December 2009 2:24pm, which shows that there are bugs and errors in the CRU programming.

    I am a programmer. To say that there are bugs or errors in code is a virtual tautology. However, I would ask whether there is any indication that this resulted in a single figure being published that was wrong. A single point in a graph. At what stage was the code? Was it actually be used or was it merely in the developmental stage? Was it being adapted for a purpose for which it wasn’t intended, and was it abandoned before it actually got used?
    *
    manacker responded in 479:

    To the point of “transparency”, I am simply telling you that “Climategate” has resulted in the fact that a higher level of transparency and public scrutiny by independent auditors who do not “have a horse in the race” will now be demanded of climate science than has been the case so far, i.e. the “rules of the game” have changed.

    Those who would set themselves up as “independent auditors” of climatology are anything but “disinterested” or qualified for that matter. (Please see for example: 1, 2, 3, 4, 5.)

  28. 528
    Timothy Chase says:

    Re Manacker’s 479, Part II of II

    To audit something you must first understand it.

    I suggested as much here:

    Now lets look at some of the things which novices at least might matter with regard to climatology: land and satellite temperature measurements. What do these measurements depend upon — which assuming the novice understood – would want articulated and all in only place neatly wrapped up with a bow? So that they could replicate the results, reproduce the study, understand the results and their significance?

    Transparency and Complexity Revisited, Part IV of V, comment 441

    … and then stated with regard to the surface temperature records:

    what about the height at which the temperature is taken? If the thermometer is lower it will be closer to the skin temperature. What about the direction of the wind? The shape and constitution of the terrain? This may matter. Was the thermometer in the shade? Did they switch thermometers? What color was the thermometer? What time of day was it? These things may or may not matter to you, but they may matter a great deal to someone else. And these are just some of the aspects one might wish to consider regarding the measurement of just land temperature.

    ibid.

    However, if you don’t trust the results from Hadley CRU there are plenty of other temperature records that show virtually the same thing.

    For example, tempearture trends from the surface:

    Global Temperature from GISS, NCDC, HadCRU
    January 24, 2008
    http://tamino.wordpress.com/2008/01/24/giss-ncdc-hadcru/

    This would appear to suggest that any coding errors that Hadley may have made did not make it into the their calculations regarding trends in temperature. But if you have a problem with surface measurements there are always satellites — which show essentially the same trends:

    ToughStations.org: Pouring Salt On Climate Critics “Contaminated” Wounds
    17 Sep 2007
    http://logicalscience.blogspot.com/2007/09/toughstations.html

    Of course if you were to wish to independently audit the satellite records, as I indicated before (441) here are a few of the questions you would have to consider:

    What about satellites? The orbit will most certainly matter. Has it decayed? What about the time of day that it was over a specific point? What direction were the instruments pointed? How were the instruments constructed? What physical principles were they relying upon? How often the satellite gets tested? Are such tests “hands-on”? What algorithms did they use for compensating for orbital decay — assuming they weren’t in the position to put a new satellite in orbit and validate that orbit each year? What of the quality of the materials that are used in the construction of the instruments — and the instruments used in the validation of the satellite’s instruments?

    It seems that to critique the experts you would have to become one yourself. Indeed, it appears you would have to become an expert in all relevant areas. How are you to audit that? Demand that they articulate and neatly package for you every detail? As I indicated comments 438 through 442, we are dealing with the same sort of problem that Frederich A. Hayek identified with the attempt to replace the economic coordination of a decentralized free market with a centrally-planning.
    *
    manacker wrote in 479:

    Those climate scientists that accept the new rules will be better off for it, particularly if they have no “dirt” to hide. Those that stonewall, whitewash or try to cover up will not do so well.

    “No ‘dirt’ to hide.” This reminds me of a quote, “Those who have nothing to hide have nothing to fear.” Some might attribute it Joe McCarthy, others to J. Edgar Hoover (who its said blackmailed presidents), still others have attributed it to Joseph Goebbels, and still others to “1984.” Any thoughts on its origin?

    Judging from the use that the emails have been put to, the demands for such “transparency” are rooted in the desire for material from which to quote-mine (taking passages out of context — see comment 333) which even in principle is impossible, where any serious attempt to achieve it would bring the scientific enterprise to a grinding halt. No level of detail would satisfy Young Earth Creationists that evolution has taken place, and no level of detail would satisfy libertarians or conspiracy nuts that the global warming is taking place and that it is as serious as mainstream science indicates that it is, and no level of detail would suffice to get fossil fuel interests to admit as much.

    As suggested above by the correspondences between different temperature records, science is self-auditing. But it goes well beyond the temperature records. It includes the melting glaciers that have been providing over a billion people with water in Asia alone, the droughts in Australia, the falling water levels in US southeast, the melting of the Arctic Icecap, the collapse of iceshelfs in Antarctica, ice loss in Greenland, West Antarctica and now East Antarctica. There are many independent lines of evidence for global warming, for the fact that we are causing it and for the fact that it is something we should choose to deal with now. And the justification for a conclusion supported by multiple independent lines of evidence is generally far greater than the justification that it would receive from any one line of evidence considered in isolation. Over time, different lines of investigation result a form of auditing far more powerful and exacting than “climate skeptics” could ever hope to honestly impose.
    *
    manacker wrote in 479:

    Sure, there are major differences between Watergate and Climategate.

    But there are also similarities.

    As I indicated in 442:

    … “Climategate” reminds me a great deal of Watergate — inasmuch as it revolves around the attempt to dig up some dirt — or at least material which could be twisted and made to appear questionable — by means of an act of theft. The biggest difference in this case (so far) appears to that the victims are being “put on trial” rather than the perpetrators of the crime.

    … this is one point on which we are in agreement.

  29. 529
    Rob says:

    Barton Paul Levenson@414

    Thank you very much, now it all makes sense.

  30. 530
    Rob says:

    Bruce@485
    It’s due to the alleged positive feedback. The scientists have asserted that cloud feedback are positive and significantly so.

    That’s it.

    [Response: Huh? Read the IPCC report. – gavin]

  31. 531
    Martin says:

    dhogaza (506),

    I believe “Evolution in action” is a quote from Oath of Fealty — another good book, but not even close to the imagination displayed in Mote in God’s Eye.

    Cheers,

    Martin

  32. 532
    Tuomo says:

    I wrote: “What is of particular interest to me is whether the output of the “highly artificial” “fudge factor” code was ever used in any published papers.”

    [Response: No. There was a draft, but it doesn’t seem to ever have been published, and is very clear about why and how this was done. – gavin]

    Thanks a lot. Do you have a pdf of that draft that also shows the figures? Your link is to a html version (http://74.125.93.132/search?q=cache:www.cru.uea.ac.uk/~timo/papepages/pwosborn_summertemppatt_submit2gpc.pdf) that doesn’t show the graphs and the figures are important to resolving this matter. Also, since I can only see captions, which figure is that code segment producing?

  33. 533
    Rob says:

    Here and there in this thread, glacier retreat have been discussed.

    Question:
    Most often people seem to link glacier retreat to drought. How is it so? The amount of precipitation and hence the feed to the rivers and streams would be the same wouldn’t it? Maybe the flood during spring wouldn’t be that high, instead the flow would be more evenly distributed over the year? Would that cause drought?

    [Response: Snow pack provides a big reservoir of water in mountain regions, and allow for significant water flow in the summer when rain is low and evaporation high. Thus a reduction of snow pack could lead to summer water shortages. – gavin]

  34. 534
    t_p_hamilton says:

    Tuomo said:”Barton Paul Levenson says to Lubos Mokras: “Could some smart scientist explain, how can nature distinguish between man made and nature made CO2?”

    Nature can’t. (And I am not talking about the journal.)

    However, a person analyzing and modeling data has to draw that divide.”

    Utterly and completely wrong. One merely needs to read up on the carbon cycle and realize that it is RATES that need modeling separately, not the actual origin of individual CO2 molecules. Equilibrium is rate into atmosphere = rate leaving atmosphere. Disturbing one of these with an a new term added will reach a new equilibrium. This is why half of the CO2 we burn stays in the atmosphere (human rate into atmosphere leads to absorbed by the oceans at half the human rate ). This explains why the oceans are getting more acidic. The rate of carbonate precipitation will catch up again so that rate in = rate out, in thousands of years.

  35. 535
    Tuomo says:

    I wrote: “What is of particular interest to me is whether the output of the “highly artificial” “fudge factor” code was ever used in any published papers.”

    [Response: No. There was a draft, but it doesn’t seem to ever have been published, and is very clear about why and how this was done. – gavin]

    I read the paper. Here’s what I believe is the relevant section, on page 21:

    “Warm-season temperature reconstructions with extended spatial coverage have also been developed, making use of the spatial correlation evident in temperature variability to predict pasttemperatures even in grid boxes without any tree-ring density data. The calibration was undertaken on a box-by-box basis, and each grid-box temperature series was predicted using multiple linear regression against the leading principal components (PCs) of the calibrated, gridded reconstructions described in section 4.4. The PCs were computed from the correlation matrix of the reconstructions, so the calibration was in effect removed and similar results would have been obtained if the PCs of the raw, gridded density data had been used instead. The only difference is that the calibrated data with the artificial removal of the recent decline were used for the PCA. Using the adjusted data avoids the problems otherwise introduced by the existence of the decline (see section 4), though all reconstructions after 1930 will be artificially closer to the real temperatures because of the adjustment(the adjustment is quite small until about 1960 – Figure 5c). Tests with the unadjusted data show that none of the spatial patterns associated with the leading PCs are affected by the adjustment, and theonly PC time series that is affected is the leading PC and then only during the post-1930 period. Inother words, the adjustment pattern is very similar to the leading EOF pattern, and orthogonal to theothers, and thus only influences the first PC time series.”

    I don’t find the argument particularly convincing as far as the substance of this hypothetical experiment goes. But that’s not the point. The point is that I think this is conclusive evidence that the “very artificial” “fudge factor” code is _not_ fraudulent.

  36. 536
    t_p_hamilton says:

    Fred Staples makes some statements and asks a question:”Numerous other sources, Silk, 50. One of teh best documented is the Central Engand Temperature record.

    Tamino’s fina paragraph reads :

    “The rate of warming in CET since 1980 is 0.05 +/- 0.02 deg.C/yr, or half a degree C per decade. If this trend continues, then by mid-century CET will have increased by a substantial amount, another 2 deg.C. This will bring CET to heights unknown for at least 350 years, probably several thousand years, and in all likelihood warmth not seen since humans inhabited the British Isles”

    From 1940, 7 decades, the trend is just 0.13 degrees per decade.

    Since 1659, no single year has averaged 11 degreesC. Of the 44 years with temperatures at 10degreesC or above, 22 were before 1945.

    Is his comment justified?”

    Yes. See http://tamino.wordpress.com/2008/04/28/central-england-temperature/ and search for answers to someone coincidentally named Fred Staples.

  37. 537
    Tuomo says:

    Barton Paul Levenson says to Lubos Mokras: “Could some smart scientist explain, how can nature distinguish between man made and nature made CO2?”

    Tuomo said: “Nature can’t. (And I am not talking about the journal.) However, a person analyzing and modeling data has to draw that divide.”

    t_p_hamilton says: “Utterly and completely wrong. One merely needs to read up on the carbon cycle and realize that it is RATES that need modeling separately, not the actual origin of individual CO2 molecules. Equilibrium is rate into atmosphere = rate leaving atmosphere. Disturbing one of these with an a new term added will reach a new equilibrium. This is why half of the CO2 we burn stays in the atmosphere (human rate into atmosphere leads to absorbed by the oceans at half the human rate ). This explains why the oceans are getting more acidic. The rate of carbonate precipitation will catch up again so that rate in = rate out, in thousands of years.”

    Aren’t you overreacting to semantics? Reason why separating between endogenous and exogenous release of co2 is useful for analysis is precisely because the mechanism by which the co2 then operates is exactly the same, irrespective of the source. (Hence my comment “Nature can’t.”) If it weren’t so, then you couldn’t take the exogenous releases, see what impact they have on the system, and draw more general conclusions about how the variables relate to each other in the system.

  38. 538
    David Bailey says:

    I’d like to return to the HARRY_README file. I can’t begin to understand why he had to infer the format of the data files he was using – at one point he has to decide whether longitude is stored -180 – 180 or 0 – 360!!!

    Even if he was reconstructing existing results, it seems incredible that the data was just stored as arrays of bare numbers with no documentation as to how they were to be interpreted.

    What confidence do we have that the original analysis did not stumble on the same problems – possibly with erroneous results.

    If taxpayers have paid good money to record the data, is it too much to document the format of the results!

  39. 539
    Don Shor says:

    Rob: “Question:
    Most often people seem to link glacier retreat to drought. How is it so? The amount of precipitation and hence the feed to the rivers and streams would be the same wouldn’t it? Maybe the flood during spring wouldn’t be that high, instead the flow would be more evenly distributed over the year? Would that cause drought?”

    [Response: Snow pack provides a big reservoir of water in mountain regions, and allow for significant water flow in the summer when rain is low and evaporation high. Thus a reduction of snow pack could lead to summer water shortages. – gavin]

    For a good overview of the impact of climate change on the western US, and particularly California water supplies, go to the Pacific Institute web site http://pacinst.org/
    and download the pdf Climate Change and California Water Resources.

    The overall assumption is that more water would fall as rain, less as snow, and that snow-melt in the spring and summer, which fills our reservoirs, would be reduced. Earlier runoff can be more difficult to capture and store. Some water districts would have increased water available, some would have less. Ground water would be presumably less affected, though in some areas agricultural irrigation is a factor in groundwater recharge.

    To many this is primarily an engineering problem. California voters will be facing a $10 billion bond issue on the next ballot that would include more reservoir storage, various conservation requirements, and the possibility of a major conveyance structure for north-south water.

  40. 540
    RaymondT says:

    Gavin, I work in a research center modeling enhanced oil recovery. When we apply for grants from the government we are asked about the environmental impact of the process in terms of how much CO2 is emitted. We also have a branch which looks at the sequestration of CO2 in oil fields and in aquifers. Most of the researchers working in oil recovery or CO2 sequestration are not familiar with climatology. When the director is interviewed he simply asserts that “if the projections of the IPCC are correct we are heading for a catastrophe”. What I am getting at is that AGW is assumed to be a fact however those doing the research on mitigation (especially the modelers) for example are not aware of the reasons for the science behind the IPCC forecasts. The same applies to some biologists studying the assumed effect of AGW on the spread to insect pests. However, these scientists are part of the IPCC consensus. My question to you is: to what extent are the catastrophic scenarios predicted by the IPCC biasing the science of climatology ? Does a researcher in climatology have better chances of getting a grant accepted if he assumes that AGW exists ?

    [Response: First of all, the ‘scenarios predicted by the IPCC’ aren’t predictions, they are projections based on ‘what if’ economic scenarios and and state of the art physical science. Whether they are catastrophic is in the eye of the beholder. IPCC makes no claims about whether they are catastrophic. With respect to grants, I am sure that anyone starting out there grant proposal full of factual errors (e.g. “we don’t know whether AGW exists”) will make reviewers raise their eyebrows. However, if someone were to write “I have an idea for looking into this or that unresolved issue, and it is important for our understanding of climate sensivity”, they will have just as good a chance at funding as anyone else — probably better. Finally, in response to the quesiton “does IPCC bias the science” the answer is simple: IPCC *reflects* the science. That’s how it was designed, and that is how it works.–eric]

  41. 541
    Hank Roberts says:

    > The amount of precipitation and hence the feed to the
    > rivers and streams would be the same wouldn’t it?

    No. You can look this stuff up for yourself. For example, as I can guess some key words that may relate to your question:

    http://www.google.com/search?q=lomborgh+climate+change+drinking+water

    leads to, among other useful resources:

    http://delayedoscillator.wordpress.com/2009/11/12/on-water-climate-change-lomborg-and-getting-your-facts-right/

    http://sciblogs.co.nz/crikey-creek/2009/11/12/bjorn-lomborg%E2%80%99s-op-ed-sense-and-non-science/

  42. 542
    manacker says:

    Timothy Chase (527/528)

    Your long plaidoyer sounds more like it’s coming from a defense attorney rather than a computer programmer.

    You are sounding quite defensive – although I have made no claims of wrongdoing.

    Your statement that the only “crime” was in releasing the stolen data does not necessarily fit UK (or US) laws protecting whistle-blowers, but (not being a lawyer either) I’ll leave that up to others to decide.

    The claim of sloppy computer programming came from a BBC interview with a computer programming engineer, entitled: “CRU’s programming ‘way below expected standards’”
    http://news.bbc.co.uk/2/hi/programmes/newsnight/8395514.stm

    Some quotes:

    “…some climate scientists are suggesting to government that the situation is potentially so damaging that they’d like to see the UK global temperature record re-analysed from scratch to clear the air. That might not be such a bad idea, given some of the fresh concerns about the quality of the programming in question.”

    “John Graham-Cumming is a software engineer; he is not a sceptic on climate change but he is shocked by what he’s seen in the programming.”

    Dr. Cumming stated,

    “If you look at the work that was done here in the alleged CRU files…it is not clearly documented, there is no audit history of what’s happened to it, so it would be below the standard you’d expect in any commercial software.”
    He tells of bugs and errors in the programming language resulting in lost data without any warning to the end user.

    When asked the question of whether he would be comfortable “betting billions or trillions of dollars on this software”, he states that he would not,

    “because it is not obvious what it is doing and why it’s doing it, and that needs to be made clear”.

    That is just one expert’s opinion, of course.

    Max

  43. 543
    John Peter says:

    Composed this for who you gonna call # 122, #123 (Dec 6 4:12.13 PM) but was closed out. I’ll try here

    Jim Hanson just now (12/7 12:30 PST)in a call-in interview on CNN took questions from the public.

    A farmer lady was very upset by “all this global warming talk” since her crops are freezing. Jim told her Global Warming and Climate change were longer term than today’s temperature and referred her to his new book “Storms of our Grandchildren” – to be released tomorrow for a further explanation.

    Answering the questions about East Hadley, Jim said that it was unfortunate that they did not make their data readily available. He pointed out that there was much other data and he had always made his available.

    Jim handled the questions well -all the questions I heard – Jim answered coolly, calmly and truthfully. Follow his lead, admit the poor judgment of the East Hadley folk and move on. That’s my recommendation and I’m sure Kelemen would agree.

    Gavin a second point, can we get a review of Jim’s new book or something on RC. Good publicity and interesting thread.

    John

  44. 544
    manacker says:

    Barry (514)

    Believe you will see that the links cited are the ones to the data plotted.

    I have agreed with you, “8 years [almost 9 years now, Barry] is too short to establish a climate trend”.

    The point being made is that we do not know whether or not the current cooling is the beginning of a longer trend or not. Some scientists apparently believe that this is the case, but it is, of course, anyone’s guess.

    You do not know the answer to that question any more than I do (or Phil Jones, James E. Hansen, either, for that matter).

    There have been no periods of comparable cooling of this length since 1976, when the late 20th century warming cycle started. If you can show me periods of this time length in the HadCRUT record, which show linear cooling of 0.1°C, please do so.

    The more serious complication comes from the recent Met Office explanation that “natural variability” (a.k.a. natural forcing factors) have been strong enough to more than offset record increases in CO2 concentration (expected to raise temperature by 0.2°C) to result in a cooling of 0.1°C.

    This raises the question: if these natural forcing factors were strong enough since the end of 2000, why could they not also have been a significant cause for the late 20th century warming cycle (or for the largely unexplained early 20th century warming cycle, for that matter)?

    But that is all speculation.

    The only thing that is unequivocal is that it has cooled since the end of the 20th century, no matter which record you look at.

    And that was the point I made, which you are unable to refute.

    Max

  45. 545
    Hank Roberts says:

    Rod B says: 7 December 2009 at 12:0 AM

    > …manacker made a claim, backed up … He didn’t say
    > (IIRC) what statistical significance this had; nor …
    > if this is “significant” overall; etc; etc; etc.

    Exactly. That’s what he does. Pretty picture, vague handwaving, no credible analysis. It’s like billboard advertising, pictures for people who don’t have time to read.

  46. 546
    RaymondT says:

    Thanks Eric for your reply: “With respect to grants, I am sure that anyone starting out there grant proposal full of factual errors (e.g. “we don’t know whether AGW exists”) will make reviewers raise their eyebrows.”

    I don’t see why stating “we don’t know whether AGW exits” should be a FACTUAL error since the role of CO2 on global warming has not been isolated from the multi-decadal variability. For example Mojif Latif asks “How much did internal decadal variability contribute to the warming during the recent decades?”.

    [Response: Those are very clearly different statements. The latter is interesting, the former, not so much. – gavin]

  47. 547
    David B. Benson says:

    Here is a good way to view the temperatures this decade:
    http://tamino.wordpress.com/2009/12/07/riddle-me-this/

  48. 548
    SecularAnimist says:

    manacker wrote: “The only thing that is unequivocal is that it has cooled since the end of the 20th century, no matter which record you look at. And that was the point I made, which you are unable to refute.”

    That’s an outright lie, which has been repeatedly “refuted” on this very thread, and you know it. That means you are repeatedly, deliberately, knowingly lying — and, you are lying to people who know you are lying, and you know that they know you are lying. Which seems a strangely futile thing to do.

  49. 549

    ZZT #518: in 1980, an open real 2400ft tape stored about 5MB. It may have cost an order of magnitude less than disk, but a complete record of a year’s climate data would have been a lot of tapes. Of course these numbers improved over the decade but the exponential improvement in computer technology makes capabilities in today’s world a poor basis for looking back 30 years.

    As an example, I looked up the CPU and memory specs of a $5-million top of the line Cray from 1975, extrapolated Moore’s Law through today and arrived at a price tag of under $50. Then I looked at the spec of my bottom of the range cell phone, and it’s actually a tad better than predicted.

    In terms of 1980s research budgets, tape would not have been much of a help. Hundreds of thousands of dollars instead of millions.

  50. 550

    Rod B: I really hate to jump in this briar patch again, but it’s just too numerous to ignore. manacker made a claim, backed up with linear regression and data from four major temperature sources, that global temperatures have cooled in the 21st century. He didn’t say (IIRC) what statistical significance this had; nor how it fit or didn’t fit into longer term temperature trends; nor if this is “significant” overall; etc; etc; etc. The gyrations and tap dancing a bunch of you do in trying to deny the obvious (no more no less) because you wish it weren’t is astounding (not to mention entertaining…)

    BPL: Apparently you are as much of a statistical illiterate as he is. For the 1,835th time:

    8 years doesn’t mean anything, because the rise in temperature is highly irregular. You need at least 30 years.

    The 8-year dip is not statistically significant. That means the error bars include zero. The real trend could be up, down, or sideways. NOT ENOUGH INFORMATION TO KNOW.

    If you’re impressed that Max did a statistically meaningless analysis because the line he got sloped a bit down and to the right, go for it, but don’t expect anyone who’s ever taken a statistics course to take your seriously. IT’S STILL WARMING. DEAL WITH IT.