Are the CRU data “suspect”? An objective assessment.

As an example, we extracted a sample of raw land-surface station data and corresponding CRU data. These were arbitrarily selected based on the following criteria: the length of record should be ~100 years or longer, and the standard reference period 1961–1990 (used to calculate SAT anomalies) must contain no more than 4 missing values. We also selected stations spread as widely as possible over the globe. We randomly chose 94 out of a possible 318 long records. Of these, 65 were sufficiently complete during the reference period to include in the analysis. These were split into two groups of 33 and 32 stations (Set A and Set B), which were then analyzed separately.

Results are shown in the following figures. The key points: both Set A and Set B indicate warming with trends that are statistically identical between the CRU data and the raw data (>99% confidence); the histograms show that CRU quality control has, as expected, narrowed the variance (both extreme positive and negative values removed).

CRUobject

Comparison of CRUTEM3v data with raw station data taken from World Monthly Surface Station Climatology. On the left are the mean temperature anomalies from each pair of randomly chosen times series. On the right are the distribution of trends in those time series and their means and standard errors. (The standard error provides an estimate of how well the sampling of ~30 stations represents the full global data set assuming a Gaussian distribution.) Note that not all the trends are for identical time periods, since not all data sets are the same length.

Conclusion: There is no indication whatsoever of any problem with the CRU data. An independent study (by a molecular biologist it Italy, as it happens) came to the same conclusion using a somewhat different analysis. None of this should come as any surprise of course, since any serious errors would have been found and published already.

It’s worth noting that the global average trend obtained by CRU for 1850-2005, as reported by the IPCC (http://www.ipcc.ch/pdf/assessment-report/ar4/wg1/ar4-wg1-chapter3.pdf), 0.47 0.54 degrees/century,* is actually a bit lower (though not by a statistically significant amount) than we obtained on average with our random sampling of stations.

*See table 3.2 in IPCC WG1 report.


References

Clayton, H. H., F. M. Exner, G. T. Walker, and C. G. Simpson (1927), World weather records, collected from official sources, in Smithsonian Miscellaneous Collections, edited, Smithsonian Institution, Washington, D.C.

Conrad, V. (1944), Methods in Climatology, 2nd ed., 228 pp., Harvard University Press, Cambridge.

Jones, P. D., and A. Moberg (2003), Hemispheric and large-scale surface air temperature variations: An extensive revision and an update to 2001, Journal of Climate, 16, 206-223.

Peterson, T. C., et al. (1998), Homogeneity adjustments of in-situ atmospheric climate data: a review, International Journal of Climatology, 18, 1493-1517.

Thompson, D. W. J., J. J. Kennedy, J. M. Wallace, and P. D. Jones (2008), A large discontinuity in the mid-twentieth century in observed global-mean surface temperature, Nature, 453(7195), 646-649.

Page 2 of 2 | Previous page

242 comments on this post.
  1. Thomas Hobbes:

    It would be fairly simple to compare the data if the CRU had not, inexplicably, killed the link to the data set.

    Would the argument here be more compelling if the specific allegations of the deniers with respect to the New Zealand and Australian measurements (Darwin et al) be explained.

    [Response: The point is that individual stations are being cherry picked. An honest assessment would pick sites at random, as we have done. It is of course possible that some stations have problems that CRU didn’t catch. Picking on those isn’t objective.–eric]

  2. Tom Fuller:

    Eric, I’m in the middle of writing a column for Examiner.com. (If you don’t know me, no worries, as I’m not a major player in the debate–but I am usually quite unsympathetic to ‘your side’ and have been very direct about it in the past.) If I include in my column that you and Real Climate say that the warming trend over the past 150 years is 0.47 degrees Celsius per century, my readers (well, at least those who comment regularly) will go berserk. I confess that this surprises me as well, given the regular predictions of much steeper rises. Before I spread the good news from Ghent to Aix, and my more skeptical commenters start singing ‘ding dong the witch is dead,’ could you maybe put this figure into context that would show why this rate of warming should not put the entire controversy to rest?

    [Response: Tom: this is the *average* rate over the *entire* last 150 years. It’s taken directly from the IPCC report. To claim it means anything other than that would be totally dishonest of you. (Do you know what the word “exponential” means, by the way?)–eric]

  3. Carl:

    Could you kindly indicate which 33 stations are in Set A and which 32 stations are in Set B? Many thanks!

    [Response: I’ll add a link to the locations. The point, however, is that you can do this sort of experiment yourself. It’s simple, we’ve provided the links, and there is no “code” necessary expect a pen and paper (or, if you like, Microsoft Excel).–eric]

  4. Dave C:

    This was very helpful. Thanks!

  5. KnockJohn:

    Nice to see a return to the science. And indeed a very interesting couple of graphs. The thing is, there are a lot of stations out there, and reports about Darwin, Nashville, New Zealand, Orland CA and Crand Canyon (although not CRU UEA stations) seem to make quite a compelling argument that sometimes the homoginization steps, unlike in your arbirarily chosen sets here, lead to flat raw data giving warming value added data.

    Now then, there are a lot of people interested in climate data, many of whom have a mathematics background. If people could collectively survey each and every dataset, raw versus homoginized, surely we could reach a point where the true condition of the full dataset could be assessed.

    surfacestations managed this with actual station sitings – in the USA surely volunteers could be recruited and station data could be emailed out and results calculated / graphed up and sorted regarding UP / FLAT / DOWN for further research.

    Just a thought.

    KJ

    [Response: Sure, but this would be repeating what CRU, GISS, etc. have been doing for years and years. If you want to reinvent the wheel, go ahead. Oh, while we’re at it, let’s redo the epidemiology on smoking and cancer. Until that’s done, let’s all take up smoking. After all, who can trust the corrupted peer-reviewed literature in leftist journals like the New England Journal of Medicine?–eric]

  6. Forlornehope:

    Is there an explanation for the difference in the set B data between 1850 and 1900?

  7. Donald Brown:

    Thanks, this is very helpful. I have not heard any skeptic yet claim that the contentious data proves that that the warming described by the published temperature trend data has not occurred. Am I missing something, or should not the burden of proof at some point be on the skeptics if there is a reasonable scientific basis for concern? Ethics would have the burden of proof shift to those who want do continue potentially harmful behavior to them once it is established that there is reasonable scientific concern that harm to some people could occur without their acceptance. It seems to me that this controversy is missing a crucial question, namely given what we know for sure, who should have the burden of proof since continuing potentially dangerous behavior has consequences.

  8. David Wright:

    Ghosts Carbon monoxide gas in my home might be harmful, but that fact alone does not shift the burden from you to prove to me that it they are there.

    ;)

  9. Mesa:

    Questions:

    1. The CRU data is the simple average of the fully adjusted (?) data of the stations you are using in each of the two examples? [I assume your data is a simple average as well, but unadjusted in any way].

    2. It’s interesting that the average GHCN adjustment over time from 1910-present shows a .25 C warming as Roman M and other have shown. I wonder why this doesn’t show up in this analysis? In other words, you would expect the CRU data to be about .25 C above the unadjusted data, for a simple average if it tracked GHCN. Any thoughts? IE your selected station samples show much less adjustment than an average sample from GHCN from 1910-present for some reason…

    Thanks for the analysis…..

  10. Slioch:

    So, am I correct in understanding that the raw samples provide an average temperature trend from 1850 to 2005 of 0.60 and 0.53degC/century (+or-) whereas the CRU/IPCC reported figure was 0.45degC/century?

    At the crude newspaper level of sound-bites, that seems to me to be a useful counter to the ignorant charge that CRU have been over-egging the global warming pudding. Have I got that right?

    [Response: Note my correction — the CRU number is actually 0.54. I was looking at the wrong part of the table in IPCC. Either way, there is no significant difference. It would not be correct to say that the CRU data have a smaller trend than the raw data, but it would be equally wrong to say CRU is bigger. And in general CRU gives smaller trends than other compilations.–eric]

  11. Charlie T:

    If there is a bias suggested by your results, it is that the raw data shows an even colder past (SetB).
    However Phil Jones’ recent presentation indicates that a global cooling trend in the raw data was turned into a warming trend by his adjustments.
    Look at the homogenisation slide in:
    http://www.cgd.ucar.edu/cas/symposium/061909presentations/Jones_Boulder_june2009.ppt
    Maybe it is your selection of only the long records that has caused the problem?

  12. MarkB:

    One possible correction: “0.47 C per century” trend for 1850-2005 I believe applies to the northern hemisphere. Globally, it should be 0.42. This is from the decadal trends listed on page 248 of the IPCC chapter linked.

    For Tom Fuller:

    CRU decadal trends:

    1850-2005: 0.042 per decade (0.42 per century)
    1901-2005: 0.071 per decade
    1979-2005: 0.163 per decade (NCDC and GISS are a bit higher)

    [Response: That’s including oceans. We were looking at land-only data, and number is 0.54.–eric]

  13. David Sittenfeld:

    Thank you for this analysis. It is well-explained and quite clear, and exactly what is needed right now.

  14. Marco:

    Eric, you’re being too kind on those referring to ‘problems’ with measurements in e.g. New Zealand and Australia (Darwin in particular).
    The New Zealand issue is based on “we don’t understand the procedures, and all references to the literature explaining the procedures will be duly neglected”. See for example:
    http://hot-topic.co.nz/nz-temps-more-stations-no-adjustments-still-warming/
    and references therein.

    The ‘Darwin-issue’ is even worse: Willis Eschenbach is the one doing the analysis (that should be a big red flag already), indicating he knows corrections need to be made(!), decides not to do them because he can’t find which and why…and then insinuates fraud because others *do* make corrections!
    http://scienceblogs.com/deltoid/2009/12/willis_eschenbach_caught_lying.php

  15. Adam Gallon:

    Are these raw data taken from the GHCN network?
    If so, what adjustments have they undergone?
    Would you agree that it’s necessary to have a good idea of what has occured prior to the 1850 start date?
    For instance the Uppsala record http://www.kolumbus.fi/tilmari/gwuppsala.htm shows pronounced variations, with the rise from the LIA period showing periods of warming much greater than those seen currently – and with lower CO2 levels.
    Do you also support the requests that the CRU provides the methodology behind its manipulation of raw (Or part-cooked already?) data?
    I wonder how much of this will be cut?

  16. Kevin McKinney:

    Update on another dataset: NCDC global results for November are in.

    Highlights:

    Most categories come in at 5th-6th warmest ever;
    Warmest Southern Hemisphere November & austral spring ever;
    Warmest global UAH lower & midtrop ever;
    Very warm November for North America generally;
    Central Asia was quite cold indeed;
    And, yes, the stratosphere is still cooling, by the looks of it.

    http://www.ncdc.noaa.gov/sotc/?report=global&year=2009&month=11&submitted=Get+Report

  17. eric:

    I’m signing off at this point.

    Hopefully those with the energy to do so can correct the apparently good-faith misconceptions that get submitted. I strongly recommend ignoring the rest.

    –eric

  18. yggdrasil:

    Gavin please, let’s not nit-pick, I am sorry I mixed up popular technology with PM. Here is the link again, with a link to the WT that pointed it out. Could you please link me to a similar list from your side, I want to put the two side by side and see what I can make out of them.

    http://washingtontimes.com/news/2009/dec/11/the-tip-of-the-climategate-iceberg-55941015/

    http://www.populartechnology.net/2009/10/peer-reviewed-papers-supporting.html

    [Response: Look up the reference list for AR4. But you are completely wasting your time. Just look at the list you’ve posted – half are from an un-peer-reviewed journal (E&E) whose editor prints whatever she feels like. The other range from the absolutely kooky (G&T, Chilingar, Miskolczi) to the completely mainstream. There are a number of takedowns of the list around – read them first. – gavin]

  19. Hangtime55:

    How then did the Administrators at RealClimate know that the 61 megabyte archive was the property of Phil Jones at the CRU when RealClimates said in a statement on November 17 , 2009 ” . . . We were made aware of the existence of this archive last Tuesday morning when the hackers attempted to upload it to RealClimate, and we notified CRU of their possible security breach later that day . . . ” ? ? ?

    RealClimate stated that ” hackers ATTEMPTED to upload the file . ‘Attempted’ in the English language is defined as : to make an effort to do, accomplish, solve, or effect without success ? So ………………………..?

    [Response: They uploaded the file and attempted to put out a post saying so. They did not succeed. – gavin]

  20. Kevin McKinney:

    #8 (David Wright) Are you saying that I have a responsibility to monitor the safety of your home? (Or RC does?) That’s news indeed–I would have thought the burden was all yours.

    Say, who’s responsible for my place? It would be good to know!

    Of course, we all live on this planet (don’t we?), so I suppose it’s our shared responsibility to ensure its safety either way.

  21. MarkB:

    Re: #12

    Thanks. I saw the 0.047 per decade number in the IPCC table and made an assumption.

  22. yggdrasil:

    By AR4, do you mean the UN Assessment Report? Does that in fact represent the best climate science? Just checking because I really want to get the best if I can. Thanks for the tip, I will be sure to exclude the EE articles from my final report.

  23. Christopher Hogan:

    Simple, brilliant, and easily understood. A very nice piece of work.

    A good companion piece would be NOAA’s analysis responding to criticism from Anthony Watts/Heartland Institute that most US weather stations were suboptimally sited. NOAA took all 1200+ stations and took the 70 that Watts et al judged to be “good or best”, and showed that both sets of data gave essentially the same surface temperature trend. The .pdf of the NOAA analysis is here:

    http://www.ncdc.noaa.gov/oa/about/response-v2.pdf

  24. Ken W:

    Adam Gallon (15)
    “Would you agree that it’s necessary to have a good idea of what has occured prior to the 1850 start date?”

    What happened prior to 1850 has absolutely nothing to do with the point of this post. This post demonstrates that the homogenization of the instrumental temperature data by CRU does NOT create any false warming trend when compared to the raw data. Eric & Kevin have now (not that anyone who follows the literature needed to be convinced) demonstrated that our surface temperature has indeed been warming.

  25. Andy Gates:

    Yggdrasil, the UN IPCC Fourth Assessment Report, aka AR4, was state of the art in 2007. Since then things have moved on a bit, and some of the authors produced an interim report titled Copenhagen Diagnosis which updates the science.

    You can get IPCC AR4 here: http://www.ipcc.ch/
    You can get the Copenhagen Diagnosis here: http://www.copenhagendiagnosis.com/

    Oh and: good post Kevin/Eric, I’ve been looking for exactly this analysis. :)

  26. tharanga:

    As an aside, the Italian biologist was looking at GHCN homogenisations for the NCDC, not CRU. So his analysis is giving context to the Darwin hubbub, rather than any CRU controversy. That said, I haven’t read Jones’ papers; perhaps CRU uses the GHCN adjustments? I don’t know.

  27. AC:

    I would be interested to hear your thoughts on this comparison of your graph of a representative sample of HADCRU3 stations, to the graph presented in IPCC CH9 pg 182:
    flickr comparison

    IPCC certainly shows what looks like an “exponential” curve, but the HADCRU3 data you pulled does not.

    A couple of other basic questions:
    Is the HADCRU3 data adjusted for urban heat?
    Can we get to the HADCRU3 data anywhere to do this exact same analysis against data for the last 40 years only?

    Thank you.

  28. Russell Seitz:

    Since reconstructing the palaeoclimatic record is a two pipe problem par excellence, it’s inspiring to see Eric at last showing a little backbone in soldarity with the poor SOB’s who brave Siberian mosquitoes, avalanching tropical bergschrunds, rabid bats and ravenous polar bears en route to collecting raw physical data ,

    Not to mention being confronted by raving Fox reporters on their return,

    I suggest Eric commence with Arrhenius’s old favorite, the cigars of the Copenhagen firm of E. Nobel

  29. Martin Vermeer:

    Mesa #9:

    1. The CRU data is the simple average of the fully adjusted (?) data of the stations you are using in each of the two examples? [I assume your data is a simple average as well, but unadjusted in any way].

    As I understand it, the CRU data is the average of the grid boxes corresponding to the members of either set A or set B that are being compared to. And yes, these grid box values were produced by gridding (i.e., re-sampling to grid nodes using some suitable technique) from the fully adjusted global set of station data. And the average of the “raw” station data is indeed just that.

    BTW I’m a bit surprised that you didn’t plot a histogram, for sets A and B, of the differences between each raw station trend and its corresponding CRU grid box trend… much narrower and more centred on zero, and a better metric for, well, how little damage the reductions could potentially do if they were indeed somehow all wrong. Isn’t that close to what ‘gg’ is doing?

  30. Jsc:

    I am concerned “researcher bias” is starting to be a major factor on this website and on “skeptic” websites as well. So much of the analysis seems to be an “us vs them” affair that is is hard for me to avoid thinking about researcher bias.

    By the way, I am not implying a bias in the research, but rather a bias in the decision on whether or not to pursue / publish the results. This article is a good example. Would it have been trumpeted if it had shown significant problems with the CRU dataset? The researcher decides whether or not to pursue a topic (this is equally true on the skeptic side btw), and the researcher decides if the research is ready for publication.

    Having done research myself, I know it is all to easy to stop when you get an answer that validates your view and continue on when something is funny.

    I’m not sure what to do, but the “us vs. them” mentality is troubling.

    JSC

  31. Paul Klemencic:

    Somewhat off-topic, but topical:

    I see John Tierney has an article and a post on his blog at the NY Times discussing McKitrick’s idea of linking a carbon tax to a atmospheric temperature measurement. He quotes Gavin and Eric in his post. There are several problems with a “McKitrick Temperature Tax” , but I am intrigued by a “McKitrick Global Heating Tax”. I put this comment on Tierney’s blog (version here corrected slightly for grammar):

    I think the idea of having a carbon tax linked to a measurement of global warming is a terrific idea. If we can identify a metric that responds quickly to changes in the planetary energy budget, then this is the best way to reward efforts to reduce global heating. And when the planet stops heating, this system will reward the decision makers who forecast the leveling off of heat buildup on earth.

    The problem is that McKitrick has proposed a rather esoteric measurement to tie the carbon tax to. He and the skeptics, and Gavin Schmidt, all seem to agree that atmospheric and surface temperature measurements suffer from a great deal of natural variability, or measurement issues depending on which camp you want to believe.

    A second problem: Increasing temperatures in the atmosphere lags increasing heat energy on earth by decades, and likely centuries . The oceans, land areas, glaciers and ice caps are acting as heat sinks that absorb over 95% of any energy imbalance for the planet, Until the ice melts off, and the oceans heat and stop absorbing the excess energy in the planetary budget, the atmospheric temperature won’t increase as rapidly as the heat is building up. And if we eliminated GHG emissions tomorrow, the atmospheric temperature could continue to rise for a century! Clearly we need a faster response mechanism.

    What is needed is a metric that more directly measures the planetary energy imbalance, and approximates the heat buildup on our planet.

    Fortunately, the leading scientists seem to agree that ocean heating is the key parameter, with ice melt an important contributor. Dr. Trenberth has published a series of papers showing the planetary energy imbalance, and both Dr. Hansen and the skeptic Dr. Roger Pielke Sr. agree that ocean heating is the key parameter.

    So tie the carbon tax to sea level rise, which is a very robust and well known measurement. Sea level rise is due primarily to thermal expansion of the sea water and melt of land based ice sheets and glaciers, and thus accounts for over 92% of the planetary energy imbalance. SLR can be measured by satellites accurately within 3 mm, roughly the annual rise seen over the last 15-20 years. There is less variability than atmospheric temperature measurements.

    When the planet stops heating, as the skeptics claim is happening now, then SLR will stop, or slow to a much slower rate. The amount of heat being absorbed and ice melt associated with SLR at 3 mm per year is enormous compared to atmospheric air heating. If the skeptics are correct, the “McKitrick planetary heating tax” would fall to zero within several years, probably 5-10 years at most, and Dr. McKittrick will get the credit for the largest tax cut in history.

    If temperatures were used instead of SLR, the McKitrick tax could continue to rise for over a century before responding to the lack of heating or cooling of the planet. Clearly SLR is a better metric to base a carbon tax.

    i suggest you contact Gavin Schmidt again, and propose a carbon tax based on SLR, using this well known measurement from the University of Colorado (the most recent academic home of the Pielkes).
    http://sealevel.colorado.edu/current/sl_ib_ns_global.jpg

    Kudos to Austin Vidich in a comment posted at midnight last night who suggested SLR, and to many of the posters who quickly identified the lag problem with atmospheric temperatures.

  32. Mark Lindeman:

    JSC, better to light a replicable candle than to curse the meta darkness. That’s the main point of the post, in my view.

  33. David B. Benson:

    Jsc (30) — It is us who know and use the scientific method versus them which don’t.

    Paul Klemencic (31) — Actually, the time to near equilibrium from a large forcing is more like a dozen centuries, not just one. Admittedly, the lattr part of the change is rather small and slow.

  34. AC:

    Mark Lindeman – but what was replicated? Certainly not the graphs of instrumental temperature presented by the IPCC:
    temperature comparison: IPCC vs. HADCRU3

  35. thomas hine:

    Looks like the U.S. was able to “teleconnect” with the “real” global trend afterall. I’ve been waiting for an analysis like this for a long time, thanks! Where’s the blade?

  36. Giorgio Gilestro:

    Very elegant and very clear, guys and thank you for the link to my post (yes, I am an Italian who lives in Wisconsin actually, but I am on the job market so God knows where I am going to end up… If you can use sleep scientists at NASA, I’d run.)

    I hope the reader will graps immediately that the goal of these analysis is not much to add pieces of scientific evidence to the discussion, because these tests are actually simple and nice but quite trivial. The goal it is really to show to the blogoshpere what kind of analysis should be done in order to properly address this kind of issue, if one really wants to. I believe there is so much data out there on the internet that amateur climatologists in the blogoshpere would actually be able to really contribute to the debate if they only would put paranoia and ideology aside.

  37. NikFromNYC:

    Four quesions:

    (a) Why not do the full set of unbroken records instead of an “arbitrary” sampling of only a third of them?

    (b) Explain your selection of A vs. B series. Why break it into two? You claim both A and B have statistically identical trends vs. raw yet B clearly shows a difference whereas A does not.

    (c) Please list the full range of data sets out there (USHCN, GHCN, GISS, etc.) for which raw/adjusted are available and explain which is associated with CRU and why. Which global average plots are associated with which raw data sets? I have not seen a simple illustration of such relationships and data links do not explain it in simple fashion.

    (d) Take a look at my identically scaled overlay of what are claimed to be the adjustments to (1) GHCN (global) vs. (2) USHCN (United States) over time. Since GHCN is evidently associated with CRU then if GHCN is exonerated, what implication does this have for the far greater-value adjustments to USHCN?:

    http://i46.tinypic.com/6pb0hi.jpg

  38. tharanga:

    JSC: If there were such significant problems with one of the datasets, such as CRU, somebody would have noticed by now. Do you think this is the first time somebody is bothering to test the methods?

    Further, I’d say that we shouldn’t always expect raw data to match the processed data so well. After all, you wouldn’t bother doing the processing unless it made some improvement (though yes, we do see fewer outliers above, so even here we see improvement). So for some regions and individual data series, homogenisation would be important.

  39. Martin Vermeer:

    Jsc #30: I think you got this backwards. Kevin and Eric picked a piece of public-outreach low hanging fruit (You do agree public outreach by scientists is a good thing, don’t you?), using data files they had lying around and a few dozen lines of their favourite scripting language, in order to prove to the hilt something they already bloody well knew to be the case, as anyone familiar with the subject does. You call this ‘research’? Hey, they do this kind of thing with their left foot before breakfast ;-)

  40. Charly Cadou:

    After reading some of the Harry_readme.txt comments, one wonders whether those who use CRU gridded data (eg. TS 2.1)for precipitation-runoff modeling do so at their own risks.

  41. Torvald:

    #31 It would be a better idea to connect that carbon tax to the current ppm levels of CO2.

  42. Completely Fed Up:

    “Further, I’d say that we shouldn’t always expect raw data to match the processed data so well”

    Why?

    If you interpolate an image correctly (trilinear mapping, for example), you can count up the intensities of Red, Green and Blue average over all pixels, and find that you get (within the limits of IEEE floating point accuracy) the same number before interpolation as you did after.

    The point of such careful and complex (as opposed to the simple “nearest neighbour” method) to interpolate is so that there is NO CHANGE in the overall numbers.

  43. DB:

    If the CRU number is 0.54° of warming per century and half of the warming is tied to things other than CO2: namely black carbon, methane, land use changes, UHI effects, and various CFCs, then carbon dioxide is only responsible for 0.27°? That is really mind boggling that the big bad bully of C02 is only good for a quarter of a degree. Am I missing something here?

  44. DB:

    C02 is only half of the constituents of the warming. We can’t blame all of that .0.54°/century on carbon dioxide. So half of .54° = 0.27° of warming due to C02 over a century?

  45. trrll:

    JSC, frankly, the likelihood that this analysis could have come out differently is basically nil, because their are multiple research groups analyzing such climate data, so there is no way that one group could be “cooking the books” in some way without a discrepancy showing up. For that reason, an analysis like this is almost certainly unpublishable–it is hard to a publication for belaboring the obvious. I don’t think the point of this post was to convince the deniers, anyway. Anybody who believes that CRU, GISS, etc. are all engaged in a grand conspiracy has doubtless already dismissed RealClimate as co-conspirators, so why would they believe that the raw data randomly sampled just because RealClimate says so?

    The key point here is that the data is readily available for anybody who is genuinely interested in temperature trends or who is concerned about the possibility of temperature adjustments introducing bias, and it provides an example of how to go about it. This is not sophisticated science, just random sampling that anybody who has taken a basic statistics course would understand. The remarkable thing, really, is the apparent total lack of interest of climate science critics/auditors in doing this kind of basic analysis. One cannot help but suspect the motives of those who focus on criticisms of cherry-picked individual stations, or who insist that the validity of the enterprise cannot be evaluated without analysis to every scrap of data and code used by climate scientists for their own analyses, but who cannot be bothered to do this kind of analysis using unbiased sampling techniques. Or perhaps they have done it, but have chosen not to report it?

    [Response: Well said. My emphasis added.–eric]

  46. Steven van Heuven:

    Hi all,

    I’ve coincidentally tried a somewhat comparable exercise yesterday. Downloaded raw and adjusted GHCN data. Then wrote a MATLAB script that reads the data, selects all WMO stations, selects the measurement series that are present in both datasets, determines the differences between them (i.e., the ‘adjustment’ or ‘homogenization’), bins the adjustments in 5-year bins, and plots the means and std’s of the data in the bins. Not surprisingly, both for the global dataset and the European subset this shows near-neutral adjustments (i.e., no “cooling the old data” or “cooking the recent data”). Additionally, the script shows the deviation from the 1961-1990 mean of each measurement series (both raw and homogenized). Strong warming in the most recent decades is absolutely obvious in both datasets. Here’s a link to the resulting PDF for Europe:

    RESULTS-EUROPE.pdf

    If you want to try it yourself (data + simple script + example output):

    GHCN-QND-ANALYSIS.zip

    I’m not a climatologist (although I am a scientist, and have performed QC on environmental data – which I guess puts me squarely on the Dark Side of the debate on AGW). Yet I’ve done this analysis in 4 hours, without any prior knowledge of the GHCN dataset. What this shows, in my opinion, is that anyone who claims to have spent yeeeaaaars of his/her life studying the dark ways of the IPCC/NOAA/WMO/etc., and still cannot reproduce their results or still cannot understand/believe how the results were obtained, is full of sh#t… Thanks for implying the same, above (if I may read it that way). :)

    Keep it up.
    Steven.

  47. jeff id:

    Eric,

    Do you know if CRU uses homogenized GHCN or do they do their own process?

    Ryan did a post at tAV which takes a brute force look at homogenization. It’s not particularly damaging to any case either way but it’s interesting.

    http://noconsensus.wordpress.com/2009/12/15/3649/

    My own opinion is that the homogenization may very well be acceptable. It would be nice to see it clearly spelled out why certain decisions are made. –I have read several papers on it and without someone to answer a few questions, it’s very difficult to figure out.

    [Response: Jeff, I’m no expert on the homogenization process. I’m sure this information is available and others reading these posts will be able to help. Ok, I’m really signing off now. Real work to do, not to mention it is supposed to be holiday time of year.–eric]

  48. AC:

    Steven – genuinely curious – why do you think your Europe graph shows no warming mid-century, while Eric’s graph does? Is this a pure anomoly? Or does it represent some difference between the raw dataset that Eric uses and the v2.mean dataset from GHCN? GHNC feeds into GISS? and GISS and HADCRU3 match pretty closely if I recall.

    Do v2.mean and the World Monthly Surface Station Climatology, 1738-cont use the same method for identifying stations? It would be fascinating to compare these two give the differences in the graphs you produced vs. Eric’s.

  49. ZZT:

    Question – how do we know that these stations are not simply measuring the growing proximity of asphalt and air conditioning units with time?

  50. Barton Paul Levenson:

    I’ve posted a new page to my climate web site which shows, as clearly as I could put it, why you need 30 years or more to find a climate trend (and thus why we CAN’T say “it’s been cooling since 1998!”):

    http://BartonPaulLevenson.com/30Years.html

  51. Theo Hopkins:

    There was a link earlier to the Washington Times.

    Oppps.

    Being a Brit I, at first, mistook this for the Washington Post.

    Is this intended?

  52. t_p_hamilton:

    DB has learned that there are other greenhouse gases, and has learned that the globe has warmed over the past century, 0.5 degrees total, askin what is the big deal?. Next up: learning that the rate of CO2 warming has accelerated since 1970 to 1.7 degrees for the next century unless something is done. Learning that even if CO2 were to stop being emitted tomorrow, the temperature will continue to rise because energy absorbed > energy emitted.

    The following paper on forcings of various atmospheric components has a familiar name (if too technical, read the news blurb in the same journal)

    Shindell, D.T., G. Faluvegi, D.M. Koch, G.A. Schmidt, N. Unger, and S.E. Bauer (2009) Improved Attribution of Climate Forcing to Emissions. Science 326, 716 – 718 doi: 10.1126/science.1174760

  53. t_p_hamilton:

    ZZT asks:”Question – how do we know that these stations are not simply measuring the growing proximity of asphalt and air conditioning units with time?”

    Are the asphalt and AC units sneaking up slowly on the thermometers?

  54. Tom Fuller:

    If you would like to comment on the article I wrote, you may find it here: http://www.examiner.com/examiner/x-9111-SF-Environmental-Policy-Examiner~y2009m12d15-Global-warming-by-the-numbers

    Thank you for your response and the useful information.

  55. dhogaza:

    There was a link earlier to the Washington Times.

    Oppps.

    Being a Brit I, at first, mistook this for the Washington Post.

    Is this intended?

    While I can’t speak to intention, I’ll point out that the Post drifts closer and closer to the Times with each passing week … it is no longer the paper you probably grew up thinking it was.

  56. Ken W:

    ZZT (49):

    The asphalt/air conditioning myth is dealt with in a link already posted above. Here it is again:
    http://www.ncdc.noaa.gov/oa/about/response-v2.pdf

    Here are 2 more good links on the subject:
    http://www.realclimate.org/index.php/archives/2007/07/no-man-is-an-urban-heat-island/comment-page-11/
    http://www.realclimate.org/index.php/archives/2004/12/the-surface-temperature-record-and-the-urban-heat-island/

  57. John P. Reisman (OSS Foundation):

    #31 Paul Klemencic

    The problem with the McKitrick tax is that it puts the solution behind the initiation of the problem. Sort of like trying to stop a forest fire after the trees have all burned down. It makes no sense of course, but that’s McKitrick’s brain in action.

  58. Feli:

    ZZT asked
    Question – how do we know that these stations are not simply measuring the growing proximity of asphalt and air conditioning units with time?

    Like this station perhaps?
    http://www.zamg.ac.at/histalp/images/stationmode/T01_SON_year.png
    http://www.helios.at/cfm/galerie/index_detail.cfm?galerie_id=218 (no need for air condition there)

    The National Weather Services have a lot more station data then those integrated in the worldwide grids and they have scientists analysing these data. I guess they would realise, if most of their own stations show cooling while the GISS (or CRU) data in the region show warming.

  59. David B. Benson:

    Barton Paul Levenson (50) — Great!

    For the purist, maybe alos add a version using an ARMA(1,1) version to handle the autocorrelation; I’d certainly find that of interest!

  60. DB:

    RE: Post 52 by t_p_hamilton:

    Wow, one day it is 6°, the next day it is 3° , and today by your estimation the warming will be around 1.7° over the next century? Does this include the .5° already arrived at, which by the way, only half is due to C02, and probably less if you figure in solar activity? Does your 1.7° degree figure include variables such as solar cycle 25, which by all estimations is forecast to be the weakest in many years? I agree that 1.7° is probably a fairer number than 6 degrees. However one can’t help but remember when Michael Crichton was lambasted for his book, that has oddly turned out to be prescient. I include the post on ‘State of Confusion’ that was posted on Real Climate by Gavin on 13 December 2004.. Makes you wonder why the book was attacked so vigorously here on Real Climate.. But I digress. In any event, here is the link to the post in the archives here on Real Climate, and both the posts and comments are well worth a read.

  61. DB:

    http://www.realclimate.org/index.php/archives/2004/12/michael-crichtons-state-of-confusion/#comments

  62. dhogaza:

    The problem with the McKitrick tax is that it puts the solution behind the initiation of the problem. Sort of like trying to stop a forest fire after the trees have all burned down. It makes no sense of course, but that’s McKitrick’s brain in action.

    It’s just another dodgy attempt to forestall meaningful action for another couple of decades.

  63. Joe Enscore:

    Truly..truly…the kind of remarks such as “the Post drifts closer and closer to the Times with each passing week…” nonsense leads more and more people to believe that politics is inherent in the AGW position. Stick to the science and leave out the politics. The more you do this, the more impetus you give to the skeptics.

  64. Jim Bouldin:

    Now that’s my kind of analysis! Thanks.

    we extracted a sample of raw land-surface station data and corresponding CRU data
    Meaning, the CRU grid cells were spatially co-located with the stations?

    I don’t see much difference in variance between station and CRU data, but a big difference in kurtosis. Any ideas as to why?

  65. David B. Benson:

    DB (61) — It is rather that all the othr forcings, both up and down, approximately cancel out over long enough time, leaving but ln(CO2):
    http://bartonpaullevenson.com/Correlation.html
    Under the assumption that everything cancels out, the Arhennius formula gives an “immediate” warming of close to 2 K for 2xCO2; this generally agrees with an eventual warming (equilbibrium climate sensitivity or ECS) of about 3 K, which is IPCC AR4’s “most likely” value for ECS.

  66. richard:

    Curious as to the reaction to the DOE “Litigation Hold Notice,” sent to all DOE employees. This notice demands that no climate science and or CRU related data including personal recordings, notes, emails, phone records etc. be deleted or destroyed.

    Is DOE planning to sue anyone involved with CRU?? And do they have jurisdiction?? Quite dodgy it seems.

  67. Charles Copeland:

    JSC (#30) writes:

    I am concerned “researcher bias” is starting to be a major factor on this website and on “skeptic” websites as well. So much of the analysis seems to be an “us vs them” affair that is is hard for me to avoid thinking about researcher bias.

    My only cavil with this statement is the use of the present tense (“is starting to be a major factor…”). I would say that publication bias has LONG been a major problem. The fact that the term itself goes virtually unmentioned at RealClimate is itself of its pervasiveness, in that nobody here seems to have reflected very much on the issue (only two hits at RC, one in 2005 and one in 2007).

    I know that Patrick Michaels is a persona non grata here but whatever about his shortcomings I think his chapter on the subject in his recent book ‘Climate of Extremes’ is a must-read for RC’s editors. It’s Chapter 7 ‘Pervasive Bias and Climate Extremism’ and it includes one core argument that I at least consider (at this writing anyhow) irrefutable. It’s that in the absence of bias “each new research finding stands an equal probability of making the predicted effects of global warming on “environmental and health damages” “better” or “more severe”.

    But this does not appear to be the case, as Michaels demonstrates by classifying articles on climate change published in ‘Science’ and ‘Nature’ as indicating the impact of global warming to be ‘better’, ‘neutral’ or ‘worse’. The ratio better to worse is approx. 8:1, which suggests that climate scientists have been appallingly optimistic in the recent past (a slur which they do not accept) or that there is a tendency to ignore optimistic or neutral outcomes today: after all the null hypothesis is no news, and the positive outcome is even less news than the null hypothesis. Besides, the null hypothesis and null funding tend to go hand in hand.

    Instead of shooting from the hip, I would suggest that commenters take the trouble of reading the chapter in full — it really is worth the effort and my summary is far too brief. Indeed I would suggest that RealClimate (to demonstrate their scientific integrity and enhance their credibility) orchestrate a wide-ranging and no-holds-barred debate on the issue.

    [Response: This argument is simply nonsense. It implies that we knew exactly the severity of the climate change problem years ago and all subsequent work is just details. In fact, it is much more likely that the we underplayed the severity of the problem, and are now coming to realise all the issues. No ‘even balance’ between ‘better’ or ‘worse’ would be expected. Instead, the balance determines the direction of increasing or decreasing concern. – gavin]

  68. cougar_w:

    #30

    You don’t much understand the scientific method.

    There is no “them”, but only “us”

    It is not opinions, it is observations.

    cougar

  69. Doug Bostrom:

    Notice how all progress at Copenhagen has (not) been brought to a halt via the release of the CRU information.

    http://www.nytimes.com/2009/12/16/science/earth/16forest.html?hp

    Hah-hah! Somebody somewhere is not very happy right now. They thought they had carborundum, instead it turns out they stole jello powder. Poured into the gears, vanishes without a trace, without even a last hurrah.

    Sorry, contrarians, the juggernaut of improvement has plenty of momentum now, more than you can ever hope to arrest.

    It’s pretty telling that whatever party staged the CRU stunt was so tightly wound up into their own strange perspective, they actually thought a few emails were going to bring the house down.

    Perhaps if less time were spent skulking, muttering about conspiracy theories and going though other people’s garbage, the motley crew of freaks behind the CRU theft could concentrate on attempting to construct a coherent and robust hypothesis that integrates and explains away all or at least most of the many indications we’ve seeing that our planet is a little too small to suffer arbitrarily large, random adjustments and still emerge with an unscathed film of air, water and regolith.

  70. cougar_w:

    #49 “how do we know…”

    That is an excellent question!

    To put it another way: How do we know that this planet is not simply measuring the growing proximity of asphalt and air conditioning planets with time?

    Huh? What about that? Smart guys.

    [sorry, couldn’t help it]

    cougar

  71. t_p_hamilton:

    DB has a fair question: Why did I say 1.7 degrees for recent trend?

    From wikipedia http://en.wikipedia.org/wiki/File:Satellite_Temperatures.png, “Each record is plotted as the monthly average and straight lines are fit through each data set from January 1982 to December 2009. The slope of these lines are 0.187°C/decade, 0.163°C/decade, and 0.239°C/decade for the surface, UAH, and RSS respectively.”

    DB also correctly states that figures such as 2-6 degrees are given for the NEXT century (assuming different emission scenarios, feedback, sensitivities, etc). The two degrees on the lower end of the scale are not too inconsistent with that, and is the lowest we can expect if we do nothing.

    The University of Wisconsin has an applet to play with. I put in freezing CO2 emissions at the current rate, temperature went up 0.7 degrees but had turned down by 2060. Making CO2 go to zero immediately, the temperature turns down immediately! This applet assumes a middle of the road climate sensitivity of 3 degrees for doubling CO2.
    http://carboncycle.aos.wisc.edu/index.php?page=carbon-budget-tool

  72. cougar_w:

    SF Examiner is a waste of pulp. I’ve read stuff on the editorial page that would gag a maggot. Just went over there and read Tom Fuller’s piece; condescending rubbish.

    What makes me flip is these armchair upper crust toadies weighing in on science research like it were a horse race; the equivalent of “Well see here, I collect stamps from around the world which is a very exacting science and this is what I think about climate change…” just sounds silly after a while.

    Leave the science to people who know what the scientific method is, at least.

    cougar

  73. Jay K.:

    DB is attempting the same end-around that has been attempted by so many for so long, that is implying that past rates of increase are typical and recent ones are not, therefore warming is….well, something.

    But a single answer to your questions, DB, would come in the form of: “Do you understand the positive feedback systems that have/will/are kicking into gear and that have the potential to cause massive warming?” You seem to be ignoring those and going for some low hanging fruit that has little to do with what is being discussed.

    The general consensus of warming since 1850 seems to be in the range of .48 deg C to .7 deg C. Eric has shown, here, that the CRU datasets, unadjusted, show .54 deg C, and has also said that the CRU data is typically lower than other datasets. So your concern appears more in the realm of concern trolling than actual intellectual rigorous questioning.

  74. jonesy:

    Do the GISS and NCDC temperature charts (from 1880 on) use the same station raw data as CRU, or do they use different stations?

  75. lgp:

    The article states “we extracted a sample of raw land-surface station data and corresponding CRU data.”

    Is the raw raw, or has it been “adjusted” prior to comparision to CRU using “concensus” approaches to UHI and other techniques to decrease older raw temperatures? If so it’s not RAW data. Showing that one can introduce the same type of biases into the raw data prior to comparision to CRU only shows that this analysis is not independent of CRU.

    [Response: Try reading the post again, slowly. What we did is what we said we did, plain and simple. If you want more details, go to the links we provided, which, yes, are the raw data. You are really trying hard to find fault where there isn’t any.–eric]

  76. Ron R.:

    Guys, all this argument about datasets and math, station location and the causes of climate change (and ALL the acronyms that people seem to love) never ever seem to end. They just go on and on and…

    And that’s fine, that’s science, we need that but it’s all Greek to the average Joe/Jane and gives the impression that nothing is really known for sure. After all these guys are still arguing about it!

    If I may, we could use a page or another site containing the actual-concrete-visual-evidence of climate change. A visual page, like before and after photos of deglaciation, tundra thawing, the loss of ice in the arctic (and the opening up of the northwest passage) and calving in the antarctic. Maps showing the change in range and migration of species from the native habitats. Rising sea levels (Tuvalu for example). Graphs of the earlier arrival of spring. Graphs of the rise in CO2. Lots of before and after stuff. You know what they say about a picture being worth a thousand words, and that the camera doesn’t lie.

    Maybe this could be a page that people can contribute to on a regular basis, kind of like the recent Data Sources page.

    People also need to understand that perhaps 95% of the basis for Climate Change is known, and perhaps 4% unknown. Then there is the 1% which we don’t know that we don’t know bit (the random factor in every situation).

    Right now when one clicks on a link on a subject to the right he comes up with a list of previous posts that mention the subject somewhere therein which is not really that helpful if one is looking for encyclopedic info on the topic.

    Just a thought.

  77. John P. Reisman (OSS Foundation):

    #76 Ron R.

    I started collecting images here

    http://ossfoundation.us/projects/environment/global-warming/myths/images

    sections like

    http://ossfoundation.us/projects/environment/global-warming/myths/images/glacier-retreat

    and

    http://ossfoundation.us/projects/environment/global-warming/myths/images/arctic

    I think are helpful. i will try to concentrate more on this. If you, or anyone has image suggestions let me know and I will look at adding or making new sections

    http://ossfoundation.us/contact-info

  78. Frank Johnston:

    Dear Ron R. :

    You have an excellent point. For those of us who glaze over at tables and graphs, Gavin Schmidt has co-authored an excellent book of images and photographs that show the reality of climate change.

    http://www.realclimate.org/index.php/archives/2008/01/our-books/#Schmidt09

    http://www.realclimate.org/index.php/archives/2008/01/our-books/

    seeing is beleiving.

    yours
    Frank Johnston

  79. Scott A. Mandia:

    Sorry this is a bit OT:

    John Stewart’s Daily Show tackles the “debate”. Very funny as usual.

    http://www.thedailyshow.com/full-episodes/257989/mon-december-14-2009-sigourney-weaver

  80. Doug Bostrom:

    “It’s Chapter 7 ‘Pervasive Bias and Climate Extremism’ and it includes one core argument that I at least consider (at this writing anyhow) irrefutable. It’s that in the absence of bias “each new research finding stands an equal probability of making the predicted effects of global warming on “environmental and health damages” “better” or “more severe”.

    Here’s an experimental analogy you probably should not attempt.

    Select 10 of your favorite, flammable objects. Your house, your car, your piano, your television set, your sofa, et al; find a good variety of things that are made differently and to various purposes yet all have in common that they burn fairly readily once ignited.

    Now, one by one, set each object ablaze. Observe carefully, record the results.

    How many samples out of 10 extinguish themselves?

    If you then publish results showing that your samples of possessions each burned fairly thoroughly, to the point of destruction, does that mean you’re “pro-fire”, biased?

    Or, pick the same samples, smash each with a hammer, wait to see which ones reassemble themselves. Once you’ve become bored, ask yourself: Are you pro-entropy?

    What an absolutely idiotic argument for Michaels to make.

  81. Steve Bloom:

    NASA put together a special page of visualization links for the Copenhagen conference. It could stand to be more user-friendly, but there’s an awful lot of good material.

  82. Ray Ladbury:

    Fine guys, let’s talk about Darwin:

    http://scienceblogs.com/deltoid/2009/12/willis_eschenbach_caught_lying.php

    Now go wash the egg off your face and we can get back to science.

  83. Edward Greisch:

    I wouldn’t want to advocate anybody doing anything wrong or criminal, but I am so tired of hearing about the denialist hack……. I’d bet [no not really wager] that if somebody in some country that doesn’t have internet laws cracked into the emails of the denialists, some really juicy stuff would be found. Perhaps the British Attorney General can subpoena emails? How is the criminal investigation going? Will the court record be published?

  84. Dan e Bloom:

    I was talking here about polar cities to save survivors in 2500 AD and you all thought I was nuts. What do you think now? We are getting closer to that time….

  85. jonesy:

    I have a very challenging question. Who is Eric? The name is not on the contributor page.

    [Response: Second page. – gavin]

  86. J. Bob:

    Enclosed are few illustrations of long term temperature anomaly averages. This was done to compare how temperature data results can change as more start ions started to record temperature. It also starts to give some perspective of comparing early temperatures with current ones. The secondary purpose was to have a set of long term temperatures to compare against sunspot and proxy data.

    The 1st compares the E. England data from 1659 to 2008 with the Hadcet data from 1850 to 2008. In the case the E. England data was “anomalized” to be compatible with the Hadcet data. The “anomalization consisted of computing the average from the time period of 1961 to 1990. This average was then subtracted from the data set’s raw data to arrive at the “anomaly”.
    http://www.imagenerd.com/uploads/lt-temp-1650-2008-1-Rxrdy.gif
    The upper figure is the raw data, while the lower figure shows the result of a 40 year Fourier filter. Note the downturn of the English data and the apparent flattening of the Hadcet data. Also the slope of the 350+ data appears to be flatter the 150+ year of the Hadcet.

    However this is only one station, so let’s take a look at stations with records starting before 1750. The figure below show the “anomalized” data from 4 early groups: E. England, DeBilt (Netherlands), Uppsalla and Berlin ( the last two courtesy of http://www.rimfrost.no/ ), compared to the Hadcet. In this case each station was “anomalized” first and then the average was computed (ignoring missing data, not the best way to merge data, but a start)
    http://www.imagenerd.com/uploads/lt-temp-1750-2008-4-EyvXd.gif
    Again the longer term data shows a flatter trend line then the Hadset. The higher end on the Ave4 data is a little skewed at the end due to the Uppsalla data being about 1 degree higher then the Debilt, and Berlin data, and ~1.5 above the English data. Also it shows a gradual rise in temperature up to about 1965 when it seems to rise rapidly. The question is why? With all the industrialization prior to that time, why the sudden upswing on that data set?

    The next figure is a repeat of the above, only using data from stations which started recording temperature prior to 1800 (as gathered from Rimfrost plus the English and DeBilt data). This included Paris, Geneva, Edinburgh, Prague, etc.
    http://www.imagenerd.com/uploads/lt-temp-1800-2008-14-9ZSv8.gif
    With this data set, the rapid rise of temperature begins about 1850 and is greater then the Hadcet.

    The next set is the “anomalized” data from stations which started recording prior to 1850 ( including St. Paul Mn (Fort Snelling) and NYC).
    http://www.imagenerd.com/uploads/lt-temp-1850-2008-27a-UtBGD.gif
    Here the rise after ~1870 is not so dramatic, but it does show a sudden rise after the late 1800’s. Another thing is that the later date record include a note that some of the later data has the note GISS attached to it, so it may not be clear when the original raw data begin and the GISS adjustments were made.

    The last figure is a average of 50 stations with recording lengths beginning prior to 1900. These include New Zealand, Siberia, Beijing, Malta, Shanghai, Australia, St. Helena, etc.
    http://www.imagenerd.com/uploads/lt-temp-1900-2008-50a-PhLn0.gif
    At this point it can bee seen that smoothed lines of the Ave50 and Hadcet start to resemble one another a bit closer. The 40 year MOV was also included to compare it and the 40 year Fourier filter. It shows the Fourier goes out to the ends of the data, instead of being “cut off”.

    So in summary one could make the case that there seems to be a slowing down of the temperature after some decades of rising, for whatever reason. The other is that in spite of the ups and downs of temperature oscillations the long term rise (from mid 1700’s) is not as bad as looking as the last ~150 years.

    Time to call it a night. Temperature now –5 F., tomorrow will be a good day for some skiing.

  87. ZT:

    Many thanks for the links on asphalt not producing warming. I am surprised to read that the heat capturing effects of asphalt are so easily dismissed as myth – there must be some effect – as anyone who walks across a parking lot in the summer would attest. But perhaps this is indeed vanishingly small. However, I think that the records do show that stations which are not moved and are not in developed areas (and therefore do not have an increase of asphalt around them per unit time) show a lower warming rate. Has anyone investigated this – or do the statistics not permit a conclusion on such small data sets?

  88. Steve Fish:

    Comment by Charles Copeland — 15 December 2009 @ 6:30 PM:

    You say– “..in the absence of bias “each new research finding stands an equal probability of making the predicted effects of global warming on “environmental and health damages” “better” or “more severe”.

    In addition to what Gavin and Doug Bostrom have said:

    1. The published research by climate scientists is not involved in predicting the effects of warming; instead they estimate the amount of the change or, for example, the amount of ice melting. This is not a subtle difference from “environmental and health damages.”

    2. In areas of research in which the experimental and null hypotheses are found with equal frequency are those in which the effect being studied is very weak or non existent so (usually in studies with a large n) random effects throw the results one way or the other. An example of this is the research concerned with the relationship between cell phone usage and cancer where there are quite a few studies finding small positive or negative relationships. An example of a limiting case might be a dozen studies of the relationship between drunk driving and automobile accidents. Do you expect that the null and experimental hypotheses would have equal probability?

    3. Finally, especially in a relatively mature field, most studies research some component of a subspecialty within the area. In the climate sciences, for example, there might be a study of a new tree species that might be useful for a long term temperature proxy, or a new method for reconciling spatial and methodological discontinuities between temperature monitoring stations may be tested, or a new way of measuring back radiation of different infra red radiation frequencies could be explored in order to identify the differential effects of the different green house gasses.

    For short, given that you have represented Patrick Michaels accurately, he doesn’t know what he is talking about. This concept is very simple, but if you wish me to read the chapter you will have to send it to me because I am not buying it.

    Steve

  89. Ian George:

    Checking raw data against trend maps, I came across the following.
    Using the the av max temp raw data for Lismore, NSW, Australia, I found there had been changes when the data was used to create the official long-term temp and anomaly maps. The early raw data records appear to be ‘dumbed down’ as follows.
    Temperatures prior to 1940 show a discrepancy of between 0.4c to 1.0C.
    Temperatures from 1940 – 1979 show a discrepancy of about 0.3C.
    Temperatures from 1980 are consistent with the raw data.
    Thus when the max temperature and the anomaly graphs are produced they both show a continuous warming from 1910. When the raw data is plotted, there is no warming apparent. Some examples are:-
    1919 – raw data av was 27.4C Dropped to 26.7C for official graphs.
    1940 – raw data av was 26.4C Dropped to 26.1C for official graphs.
    1980 – raw data av was 26.1C No change at 26.1C for official graphs.
    2002 – raw data av was 26.5C No change at 26.5C for official graphs.

    Also the mean temps for this station at NASA gistemp show the same pattern. After the raw mean temps are ‘homogeneity adjusted’, the early entries are adjusted down 0.5C and the later entries are hardly adjusted at all. This results in a warming trend in temps which would not be there before the adjustment.

    After the famous ‘blink’ map, it makes it hard for an innocent bystander like myself to believe the warming is as bad as you say it is. And now with the emails and the programming notes, it makes it harder to believe as it confirms what I have been looking at.
    It’s not what you, the western public, our governments, etc think of these emails, etc. It’s what China, India et al make of it all.

    [Response: What on earth are you talking about? The raw data are what we plotted in this post! Don’t rely on other people’s plots though — do the exercise we did and look at the original data. It is not that hard.–eric]

  90. Ron R.:

    John P. Reisman, Frank Johnston (et al :-)) it’s not my site nor my call. It’s just a suggestion.

    Currently if I want to find visuals on these things I have to hunt all over the net. Maybe there’s a site that I am not aware of with the visuals. On thinner ice is one. I think the European Space Agency has a shot of the northwest passage. I suspect that there is a lot out there and it sould be gathered.

    Just looked at jour site John. Nice start. A good first page to send newbies.

  91. David Wright:

    An analysis here claims to show average adjustments as a function of time. If that plot is correct, you would expect no difference in adjusted and unadjusted trends for the period 1850-now, because the adjustments are negligible at both ends of that time period.

    However, because of the significant downward adjustment near 1900, you would expect a significant difference in adjusted and unadjusted trends for the period 1900-now. You would expect the adjustment to increase the trend by about 0.25C/c, about a 50% difference.

    You would also expect a significant difference for the last ~30 years, because there appear to be about 0.1C downward ajdustment of temperatures in the 1970s relative to the present. Thus a large part of increase in the “recent” trend above the long-term 0.5C/c trend would be attributable to adjustments.

    Of course, that doesn’t mean the adjustments are wrong. It just means they are relevant. Do you claim that the cited adjustment plot is wrong, i.e. you would also find no significant difference between adjusted and unadjusted trends for the periods 1900-now and 1975-now? Or do you acknowledge that the adjustments are important for those periods, and simply claim they are trustworthy?

  92. Norman:

    Because of the Internet, I am able to look at temperatures around the world in an recording station. I have been checking out towns like Fairbanks Alaska, Yellowknife Canada, and Yakutsk Russia as samples.

    The AGW theory claims that the Arctic region will experience the greatest degree of warming first.

    So far these locations have been average temperatures or below. A few days they are slightly above. My question is why? What is causing the cooler temperatures in the Arctic regions this season. I understand there can be temporal fluctuations, I just don’t understand using AGW theory why these places are now cooler than previous recordings. What climate forces are cooling the North this winter?

  93. tharanga:

    Re 89:

    This sort of thing (picking some random station, not understanding why certain adjustments were made, and then wildly extrapolating to the entire data set) seems to be the flavour of the moment.

    Giorgio’s code produces a list of stations with the trend introduced by the GHCN homogenisation for each (which isn’t used by GISS, by the way, people).

    Pretty much for every station like Darwin there’s also a station where adjustments introduced a cooling trend. Perhaps I should start a blog, so I can hype up these cooling stations, pretend the homogenisation procedures aren’t published, and finally accuse people of the fraud of hiding the Incline?

    p.s. eric, you keep promising you’ll sign off. Feeling a bit addicted to the nonsense of the comments?

  94. Arie Brand:

    Offtopic but worthwhile: Monbiot versus Plimer. The “Artful Dodger” exposed.See:http://www.abc.net.au/lateline/content/2009/s2772906.htm

  95. Doug Bostrom:

    “It’s not what you, the western public, our governments, etc think of these emails, etc. It’s what China, India et al make of it all.”

    If one pokes one’s head out of the enormous polka-dotted ass of the contrarian pantomime horse, one can see that China et al are responding to climate change w/enormous financial, political and diplomatic capital investment, lending a priority to the issue that imposes real opportunity costs in other areas. In other words, conclusions have been drawn, minds are made up and courses of action set.

    The train has left the station; few of the people responsible for addressing this issue on the policy level give a rat’s ass about a rabble of cranks on the Internet left standing on the platform of history, yammering on and shadowboxing their war as witless conscripts of the pointy end of the spotty fake horse.

    All this desperate tilling and hoeing and sifting in search of supposed hidden footprints of conspiracy left in temperature records is:

    Choose any/all
    )Pathetic
    )Amusing
    )Pathological
    )Degenerate
    )Pointless

  96. caerbannog:

    (#91)

    An analysis here claims to show average adjustments as a function of time. If that plot is correct, you would expect no difference in adjusted and unadjusted trends for the period 1850-now, because the adjustments are negligible at both ends of that time period.

    Is it reasonable to conclude from that plot that the differences between adjusted and unadjusted *temperatures* should be at a maximum during the 1900-1920 (eyeball approximation) period? If so, I see no evidence of that in RC’s raw vs. adjusted plots.

  97. Ian George:

    From 89
    Response
    ‘What on earth are you talking about? The raw data are what we plotted in this post! Don’t rely on other people’s plots though — do the exercise we did and look at the original data. It is not that hard.’

    That’s what I’m talking about. The data received by NASA from Australia had already been ‘corrected’ and they have adjusted it again – down prior to 1980 for the Bureau in Australia and then again downward for earlier data by NASA.
    Later data reads as per the raw data. They’re the official figures I’ve quoted, nobody else’s.

  98. John P. Reisman (OSS Foundation):

    #90 Ron R.

    There is an image in the Arctic section of the Northwest passage

    http://ossfoundation.us/projects/environment/global-warming/myths/images/arctic/20080811_Figure4.jpg/view

    Though now it has been open three years in a row.

  99. Completely Fed Up:

    And then Ian George would require the calibration tests of the thermometers. And the manufacturing report. And the history of the calibration team (they could be in on the conspiracy too!) and if any of that’s missing, well, obviously, this proves that the data is made up…

  100. Ian:

    I had thought that since Climategate RC had become more inclusive but the response to post#5 shows all the old patronising and condescending attitudes that characterised RC. There are many scientists, of whom I am one, who know enough about climate change to know they don’t know enough. There are reports that suggest temperature data from Darwin is dodgy and it is a legitimate question to ask if it is true the temperature measurements in Antartica are from only one station . Most do not understand why the homegenisation of raw data almost always results in an increase rather than not. Addressing these questions without the arrogant overtones would be really appreciated by many

    [Response: How about dropping the endless repetition of non-facts? The adjustments are actually very close to Gaussian centered on zero in the GHCN procedure, and yet the blogosphere only ever focuses on the outliers on one certain direction. And no, the measurements from Antarctic (used by GISTEMP for instance) don’t just come from one station. Why do you automatically believe people who have been shown over and again to be misrepresenting the true situation? – gavin]

  101. Knut Witberg:

    When you write emails like the leaked ones, you become automatically a suspect, are you still pounding that question? And please stop the “out of context” argument – there is ample context.

    But from there to draw any conclusions about the CRU estimates of the temperature developement, is not wise. That we agree on. But bear in mind that it is due to lack of openness that we have got into this situation. I believe that the CRU estimates are not too far off, but now they need to be investigated.

    However, there is much more to that discussion, for instance what is the exact objective of the measurement? What is it exactly that CRU measures? What would we want it to be? For instance, should the measurement include the effects of urbanization? extensive change of land use? large area irrigation? etc

  102. Barton Paul Levenson:

    Charles Copeland: I know that Patrick Michaels is a persona non grata here but whatever about his shortcomings I think his chapter on the subject in his recent book ‘Climate of Extremes’ is a must-read for RC’s editors…

    BPL: Michaels egregiously lied to Congress by showing only the highest of Hansen’s three 1988 scenarios in a chart and claiming that his “prediction” (singular) had been falsified by subsequent events. That makes him an unreliable source of information. Period.

  103. Barton Paul Levenson:

    DEB: I was talking here about polar cities to save survivors in 2500 AD and you all thought I was nuts. What do you think now?

    BPL: I think you’re nuts.

    Sorry, I couldn’t resist. But A) in 2500, there won’t be a North Pole, and B) the survivors won’t have the capital, infrastructure and functioning economy enough to build a domed city in Antarctica.

  104. Barton Paul Levenson:

    ZT, the Urban Heat Island effect has been known and compensated for for a long, long time. Here are some references if you’re interested:

    Hansen, J., Ruedy, R., Sato, M., Imhoff, M., Lawrence, W., Easterling, D., Peterson, T., and Karl, T. 2001. “A closer look at United States and global surface temperature change.” J. Geophys. Res. 106, 23947–23963.

    Parker, DE. 2004. “Large-scale warming is not urban.” Nature 432, 290.

    Parker, DE. 2006. “A Demonstration That Large-Scale Warming Is Not Urban.” Journal of Climate 19, 2882-2895.

    Peterson, Thomas C. 2003. “Assessment of Urban Versus Rural In Situ Surface Temperatures in the Contiguous United States: No Difference Found.” J. Clim. 16(18), 2941-2959.

    Peterson T., Gallo K., Lawrimore J., Owen T., Huang A., McKittrick D. 1999. “Global rural temperature trends.” Geophys. Res. Lett. 26(3), 329.

  105. Anne van der Bom:

    Jsc
    15 December 2009 at 2:29 PM

    Would it have been trumpeted if it had shown significant problems with the CRU dataset?

    It’s worse. Being a climate scientist, Eric is of course in on the conspiracy. Do you think he would have bothered even starting the analysis, knowing beforehand that the data was cooked?

    You start insinuating wrongdoings by Eric, based on a figment of your imagination. Don’t you think you have to draw a line somewhere?

    Having done research myself, I know it is all to easy to stop when you get an answer that validates your view and continue on when something is funny.

    Oh my, you really think this is the first time a climate scientist does an analysis on the temperature record.

    This blog post is a simplified repetition of what climate scientists have done many, many times before in a much, much more rigorous way. This blog post is meant as a simple excercise that any Excel jockey could repeat at home. Don’t pretend it is more than it is.

  106. Charles Copeland:

    Gavin, Doug, Steve,
    Thank you for your comments, much appreciated. I’ve just discovered that Michaels has published an article on the same subject in the journal Energy and Environment (2008, Vol. 19 No 2). I’ve applied for a free electronic copy via my employer and will forward it to you, if you’re interested, as soon as I receive it (hopefully this afternoon or tomorrow). The title of the article is “Evidence for “publication Bias” Concerning Global Warming in Science and Nature“.
    More detailed reply later, time permitting.

  107. Bob Arning:

    #67:

    But this does not appear to be the case, as Michaels demonstrates by classifying articles on climate change published in ‘Science’ and ‘Nature’ as indicating the impact of global warming to be ‘better’, ‘neutral’ or ‘worse’. The ratio better to worse is approx. 8:1, which suggests that climate scientists have been appallingly optimistic in the recent past

    If I read his table 7.1 correctly, 8:1 would be the ‘worse’ to ‘better’ ratio, not the reverse.

    here

  108. Ray Ladbury:

    The “Michaels criterion” for bias is absurd. It would be met only if science were changeless. Moreover, it depends on which particular quantity or phenomenon one is considering. CO2 sensitivity is currently estimated to be roughly 3 degrees per doubling–and this has been the case for a very long time now.

    Charles Copeland says, “I’ve just discovered that Michaels has published an article on the same subject in the journal Energy and Environment (2008, Vol. 19 No 2).”

    And really, how could one refute one’s own argument more effectively than by publishing in that cesspit of a journal?

  109. Scott A. Mandia:

    Take a look at Figure 4 on my page below:

    http://www2.sunysuffolk.edu/mandias/global_warming/global_warming_misinformation_urban_heat_island.html

    It sure looks like the “satellite only” measurements from UAH (run by skeptics Chrosty and Spencer) and RSS very closely match the actual surface measurements.

    A few more examples here:

    http://www2.sunysuffolk.edu/mandias/global_warming/global_cooling.html

    Charles: Energy and Environment pretty much publishes anything but mostly contrarian viewpoints. By many standards it is NOT considered to be peer-reviewed. See the link below and scroll down to the E&E section:

    http://greenfyre.wordpress.com/2009/11/18/poptarts-450-climate-change-denier-lies/

    Even the editor has claimed that the journal has a political agenda and isn’t really a science journal.

  110. Dale:

    Since at least the time of Galileo the right wing has been behind the eight ball as far as science is concerned. It’s a fact that during WW2 the OSS, a pre curser to the CIA went nuts trying to find researchers with the “Right Stuff,” who were not of the political left. They didn’t do very well. Today as then, liberals dominate scientific research. Maybe not because their smarter but because they’re more likely to think outside the box?

    If we we’re to take all the detractors posting here and all other like minded individuals and put them all in one world, would we have been able to develop a vaccine for polio? Would we have been able to be successful with the genome project? Would we have been successful in our attempt be develop the atom bomb which ended WW2? I wonder.

  111. Carl:

    (#3): Eric – thanks, I get the point that anyone can confirm the trend with available data. Since I was going to try to do just that over Christmas break, I thought it would be nice to start by replicating these results just to confirm that I’m doing things correctly.

  112. Kevin McKinney:

    Re: 92:

    And Arctic amplification of warming has been observed in spades. However, it’s least marked in winter, since there is virtually no insolation at the highest latitudes during that season.

    No insolation=no greenhouse effect.

    Positive anomalies for the Arctic for this summer were quite eye-popping–2.5 C +, if I recall correctly.

    You can access a bunch of relevant info here:

    http://data.giss.nasa.gov/gistemp/graphs/

  113. Giorgio Gilestro:

    77, 78. I like the idea of the pictures. It may appeal to a lot of people. Is there a collection of beautiful pictures big enough to make a screensaver?

  114. john:

    so I guess my question now is: if there is no statistically significant difference between raw and processed data, why process/correct it at all? why not leave the data that has no problems alone and only process the ones with clear gaps, errors, etc.?

  115. AMac:

    I recommend that readers look at the first two graphs in Ryan O’s post at the Air Vent, referenced by Jeff Id in comment #47 (15 Dec 09 at 4:15pm) supra.

    http://noconsensus.wordpress.com/2009/12/15/3649/

    These graphs ought to highlight some important areas of agreement between the AGW-Consensus crowd (e.g. here at RealClimate) and the Skeptics (e.g. over at ClimateAudit).

    The red traces in those two graphs depict the count of stations with long-term records, by Ryan O’s selection criteria (he plausibly claims that many other cuts would give the same general shape). The post-1990 decline in the count of GHCN stations with long-term temperature series is truly shocking. As the stakes in Climate Change become higher, data quantity nosedives. Huh?!?

    Everyone ought to agree on the urgency of a few simple measures.

    Raw data should be collected and made public in as transparent a fashion as possible. As much metadata (siting, history of adjustments, photo of site, etc.) as possible should be included. These sorts of databases are essential for identifying and quantitating real trends.

    The many station locations dropped from the GHCN between 1990 and ~2005 should be salvaged. Happily, it seems that in many cases, data collection continued past the dropoff point; the lapse was in collation. Most of those records could be backfilled.

    For those stations that were physically abandoned, most could be reclaimed, and observations re-started in 2010. The addition of metadata would allow factors such as increased urbanization to be taken into account.

    This latter initiative wouldn’t help today or tomorrow. But what if “the science isn’t settled” ten years from now? A much-expanded set of long-term records would be a huge plus, as far as improved understanding of climate.

    Funding of such projects by NOAA, NSF, WMO, and similar agencies would seem penny-wise and pound-wise to me.

    [Response: See here. – gavin]

  116. Nick O.:

    #92 – Norman, for a recent comment on the idea of ‘Arctic cooling’, have a look at Jeff Masters’ blog and the references cited therein, on the subject of the emergence of the ‘Arctic Dipole':

    http://www.wunderground.com/blog/JeffMasters/archive.html?year=2009&month=12

    NOTE: you need to look at the second article in the list for December, the first being on movement rates of glaciers, hot off the press from AGU!

    #110 – Dale, just for your interest, I am myself a geoscientist and have a right of centre political perspective, and I also support the climate science i.e. find it well argued and credible. I think that we should if possible be very careful not to label those persuaded by the science as being only of a liberal or left wing persuasion, as the argument then gets dumped into a simplistic left vs right split, which will get us nowhere.

    I also try to ‘think outside the box'; indeed, most of my modelling, meta-modelling and experiment design work involves trying to tackle big problems in new ways …

    # Eric – very good post, makes the point very neatly.

  117. Completely Fed Up:

    “if there is no statistically significant difference between raw and processed data, why process/correct it at all? ”

    Because there IS a difference in detail.

    5 gallons distributed around 50 bottles with a varying amount in each totalling 5 gallons doesn’t make a difference to how much water there is, when asking “has anyone spilt water and messed up our experiment”, but it DOES become important that the variation be kept when you go on to do the experiment: “which bottle gets drunk first, and does the volume of drink in it make a difference?”

  118. Bill DeMott:

    My area of expertise provides raw, unmanipulated temperature data from sites around that world that is completely independent of the surface air temperature data record. The American Society of Limnology and Oceanography (ASLO) has just published a special issue of Limnology and Oceanography titled: “Lakes and Reservoirs as Sentinels, Integrators and Regulators of Climate Change. A large majority of the papers are available as “open access” at the ASLO website (authors pay an optional fee for open access).

    http://www.aslo.org/lo/toc/vol_54/issue_6_part_2/index.html

    Very high quality data for the last 20-40 years show strong warming of the surface waters of large lakes from California, Washington state, Michigan, Sweden, Switzerland, Sweden, Germany, Switzerland, central Africa and Siberia and other locations (reviewed byAdrian et al. 2009). If the global air temperature data were somehow biased, flawed or even fraudulent, this should be obvious from the data on lake temperatures. However, the agreement is excellent. Since many of the lake data are from northern latitudes, the rates of warming in lakes are, for the most part, considerably higher than the means for global air temperatures.

    A paper on the physics of warming in Lake Tanganyika, (the second or third largest lake in the world by volume) documents warming since 1913 and shows, the close relation of its temperature anomaly with local air temperature, which in turn is closely linked with southern hemisphere and global air temperature anomalies (Verburg and Hecky 2009).

    Temperature per se is only a small part of the papers presented in the special volume. Lakes are sensitive to changes in climate and this is clear from strong signals in water chemistry and food chains that are apparent from recent warming as well as the record of changes over thousands of years from sediments. For example, my paper shows how changes in temperature stratification has indirectly altered the food chain of a large lake on the Italian-Swiss border (Manca and DeMott 2009).

  119. Jryan:

    Objectivity is indeed “picking sites at random”… not sifting through the data for strong correlations.

    Also, can Eric Steig be considered an objective evaluator?

    [Response: It doesn’t matter whether I’m objective. The methods are what’s objective here. If you don’t believe me, do it yourself.–eric]

  120. Bill DeMott:

    “It’s not what you, the western public, our governments, etc think of these emails, etc. It’s what China, India et al make of it all.”

    I strongly doubt that leaders in China are concerned about the right wing blogs of the English speaking world. If you know anything about Chinese culture, you would know that teachers, scientists and engineers are held in much greater esteem than in the US. This may be why Chinese have placed so much emphasis on math and science in their educational system. These attitudes, coupled with an authoritarian polical system, is allowing the Chinese adopt new findings in science and technology very quickly.

    One substantial advantage in the west, is that western scientists are probably faster to accept new ideas that are contrary to those of earlier scientists and professors. In my experience, scientists in China, Japan and even in continental Europe show more respect for tradition and are more reluctant to disagree with the publications of their teachers.

  121. Scott A. Mandia:

    #118: Bill DeMott:

    Thanks. I will likely be adding some of this data to my Modern Day Climate Change page.

  122. meteor:

    Hi gavin

    In your response 100, did you see that the data of NASA Antarctic stations are not updated after “for the best”,1992?
    In french we say : “donner des bâtons pour se faire battre”

    [Response: a) not NASA stations, b) not the only source of data, and c) On peut s’amuser à chercher un poil aux oeufs, mais ca sert à rien. – gavin]

  123. JosephG:

    Just a commentary on your argument to prove your case.

    (I am skeptical of global warming but I’m also skeptical that there isn’t global warming. I’m generally looking for something convincing either way and generally walk out empty-handed because I don’t find the evidence convincing enough. I do think there is a lack of transparency on the pro-global-warming side which I find unfortunate, and the row over the emails has only reinforced this perception)

    Your article is essentially in response to this quotation you cited:

    “Trying to prevent skeptics from seeing the raw data was always a questionable strategy, scientifically.”

    in response to which you wrote:

    “The implication is that something secretive and possibly nefarious has been afoot in the way data have been handled, and that the validity of key data products (especially those produced by CRU) is suspect on these grounds. This is simply not the case.”

    There are problems with the argument you chose to use, which is to show how statistically that the warming trend is identical with or without data correction for a subset of all weather stations.

    1. You haven’t indicated which weather stations you selected. One of the things that has been revealed in this controversy is that “random” and “cherrypicking” are sometimes intertwined (on either side of the issue). Basically, because you’re not providing which weather stations you selected, it’s impossible to actually replicate what you’ve done.

    2. The link to the CRU is dead-ish. There’s no data there, the reader is redirected to the Univeristy’s website.

    3. That the trends are identical for a subset of weather data used for the global average temperature does not disprove that there might be something wrong with the way that each weather data is used, nor does it disprove that there might be something wrong with the way that all weather data is combined into the HADCRU global temperature index.

    4. I work in building energy simulation, and one thing I know is that urban heat island effect is poorly understood even today. We use weather files in simulating how buildings will behave, but a big wild card we don’t account for is how such buildings will behave in the actual locale in which the building will be in (which is usually in a dense urban location). I’m at pains to figure out how a known, transient, and poorly understood phenomenon can simply be “compensated out” in all weather station data. Your figures show that essentially urban heat island isn’t compensated for in the weather stations you selected, however the CRU has stated that te global index is in fact compensated for it. If they’re indeed not compensated for, I can’t see how global temperatures can possibly be correct. If they’re compensated elsewhere, then the study you have done does not disprove that their homogenisation is flawed or perhaps nefarious.

    5. Your argument doesn’t quite constitute a complete explanation as to why some CRU outsiders are trying to get information from CRU insiders (i.e. the quote you were responding to in the first place) and why those CRU insiders are doing to do everything *not* to give it to the CRU outsiders. Everyone’s first interpretation of someone acting evasive is that this someone has something to hide, and the hacked emails (if genuine) reveal that they are actively being evasive with their data. If it’s of no consequence, I can’t understand why they would be so intent on keeping it to themselves.

    PS: I’m not trying to make a polemic out of this, I’m just honestly trying to understand what’s going on here and resolve conflicting perceptions I have, so please bear that in mind if you do intend to respond.

    [Response: The point of the post is that anyone can do the same analysis. I don’t need to provide you with any data for you to do this. Go to the web, get the raw data and the CRU data, and compare them. You’ll get the same results we did. I gave you the links, the math is trivial. Why make it complicated when it isn’t?–eric]

  124. Giorgio Gilestro:

    if there is no statistically significant difference between raw and processed data, why process/correct it at all?

    Because at the single station levels differences can be still significative and important, either in warming direction or cooling direction. Given that all climate model now try to do a better job at prediction local variations, to start with accurate data is imporant.

  125. Neil Pelkey:

    Verberg and Hecky oddly do not cite Tierney et. al. Science Vol. 322, No. 5899, pp. 252-255, 10 October 2008. This would have shown their data to be the the normal range for the last 60,000 years.

  126. BlogReader:

    #99 And then Ian George would require the calibration tests of the thermometers. And the manufacturing report.

    Which should be included in any report.

  127. SecularAnimist:

    Barton Paul Levenson wrote: “… in 2500 … the survivors won’t have the capital, infrastructure and functioning economy enough to build a domed city in Antarctica.”

    Which suggests that it would be prudent for those who currently have the wealth and power to command such resources to get started on building their nuclear-powered, climate-controlled domed cities now.

    Of course, those cities will only be able to house a tiny fraction of the Earth’s population. Perhaps the “top one percent”.

    Seems like a good place for fossil fuel corporation CEOs to invest some of the hundreds of billions of dollars in profit they expect to gain from several more decades of business-as-usual consumption of their products.

  128. Hank Roberts:

    > Everyone ought to agree on the urgency of a few simple measures.
    > …
    > [Response: See here. – gavin]

    That stated the need; lest someone leap on that 1999 statement, it’s worth noting that the new system to address those needs, the CRN, went operational in 2004:
    http://www.publicaffairs.noaa.gov/releases2004/jan04/noaa04-004.html

    They’re running in parallel; the historical network, the updated historical sites, and the new CRN stations — always smart with new systems: http://www.ncdc.noaa.gov/crn/observations.htm

  129. Ken W:

    ZT (87):
    “I am surprised to read that the heat capturing effects of asphalt are so easily dismissed as myth – there must be some effect – as anyone who walks across a parking lot in the summer would attest. But perhaps this is indeed vanishingly small. ”

    The myth is not that asphalt traps heat (it does), the myth is that asphalt has biased the temperature readings and produced a false warming trend. As the links demonstrated, the dataset handlers are able to properly account for asphalt creap and produce a quality dataset.

    The problem with some of the AGW deniers (e.g. those who keep claiming the dataset is biased) is that they seem to think that scientists are too stupid to recognize potential issues like asphalt creap and account for it.

  130. Ron R.:

    Someone has the idea:

    http://www.globalwarmingimages.net/index.html

    But where are the pictures?

    Here’s an example of what I’m talking about. You can clearly see the before and after.

    http://thrivingoceans.org/index.php/2009/05/27/coral-bleaching-and-macroalgae-infiltration/

    Another:

    http://snf.fra.affrc.go.jp/English/wwwsupl/supl1j1.jpg from http://snf.fra.affrc.go.jp/English/wwwsupl/supl1.htm

    Another:

    http://sites.google.com/site/vulnerabilityadaptation/_/rsrc/1240428658864/elyce/Coral%20Bleaching.jpg from http://sites.google.com/site/vulnerabilityadaptation/elyce

    I found these on Google images (don’t know anything about their possible copyrights: http://tinyurl.com/yfxbxqt

    Steve Bloom #81: Nice but the focus doesn’t seem to be quite there.

    Giorgio Gilestro #113: “77, 78. I like the idea of the pictures. It may appeal to a lot of people. Is there a collection of beautiful pictures big enough to make a screensaver?

    Gavin has a book but AFAIK there’s not an online version.

    Would be a fun project for someone (not me), even funded research?

  131. ZT:

    I asked another question on asphalt – was it rejected or inappropriate?

    Many thanks for the links on asphalt not producing warming. I am surprised to read that the heat capturing effects of asphalt are so easily dismissed as myth – there must be some effect – as anyone who walks across a parking lot in the summer would attest. But perhaps this is indeed vanishingly small. However, I think that the records do show that stations which are not moved and are not in developed areas (and therefore do not have an increase of asphalt around them per unit time) show a lower warming rate. Has anyone investigated this – or do the statistics not permit a conclusion on such small data sets?

  132. Ken W:

    John (114):
    “if there is no statistically significant difference between raw and processed data, why process/correct it at all?”

    The more data the better. This is generally true in all fields of science, not just climate science. If data, even though it may not be perfect, adds to a better understanding of a field of study it would be foolish to throw it away or ignore it. It would also be foolish to not use valid methods to improve the quality of the data, when there are known issues that can easily be corrected.

    In my own experience (not climate related), I’ve had to go back and correct large subsets of measurments because of some mistake in the calibration process. Without doing so the entire experiment (quite expensive) would have been worthless. But using a simple (and demonstratably correct) method, the data was saved and the end result was a successful experiment and increased knowledge about the process.

    While excellent approximations of global warming trends can be determined from just 100 well placed measuring stations, such a limited dataset wouldn’t be at all helpful for regional climate study. And such a limited set of measurements would also be attacked by deniars claiming “they’re hiding data that might prove it’s actually cooling”.

  133. Mark A. York:

    Revkin to leave NYT.
    http://www.cjr.org/the_observatory/revkin_taking_nyt_buyout.php

  134. DVG:

    You folks should use the WSJ opinion piece by Mike Hulme as one of your blog postings. I know this is off-topic, but then my comments rarely make it through anyway (about 1 of 5), so I thought this might be a good way to communicate my suggestion to you. The articles is at:

    http://online.wsj.com/article/SB10001424052748704107104574571613215771336.html

  135. pp:

    AMac says:
    “The post-1990 decline in the count of GHCN stations with long-term temperature series is truly shocking. As the stakes in Climate Change become higher, data quantity nosedives. Huh?!?”

    Just a wild guess on my part, but could that not simply be a result of the collapse of the USSR? I believe many, many things in the former Soviet Union states lost much of their funding after that event, presumably Met services were no different?

  136. meteor:

    thanks gavin

    “je ne chercbe pas de poil aux oeufs’ I’m not a denialist…(you can catch a glance on my site if you want)
    But I prefer your second link.

  137. cyclox:

    The pushback continues. Someone needs to go point-by-point through this

    http://www.dailyexpress.co.uk/posts/view/146138

    [Response: They have. – gavin]

  138. ICE:

    @121,132

    Wow, Gavin knows french expressions that we (= french people) don’t ! ;)

    [Response: On peut toujours apprende quelque chose de nouveau…. – gavin]

  139. t_p_hamilton:

    Gavin, in your link to antarctic stations, I noted not a single urban site. What are you guys hiding?

  140. John P. Reisman (OSS Foundation):

    #110 Dale

    I agree with Nick O. I’m politically Centrist but personally conservative. Science itself is agnostic. We all live in one world and the development of the polio vaccine was unaffected by the fact that their are two dominant political parties in America.

    The polio vaccine was developed by a guy in Pennsylvania pretending he was the polio virus and wondering what in the human body might harm him. It was a thought experiment that did not worry about the politics of left or right but the battlefield within the human body.

  141. Mark A. York:

    RE: Fuller’s article. The Examiner.com is not the San Francisco Examiner. It’s a nationwide vanity opinion site with no editorial input at all and 0 credibility in all regards.

  142. Jason O'Connell:

    [Response: The point is that individual stations are being cherry picked. An honest assessment would pick sites at random, as we have done. It is of course possible that some stations have problems that CRU didn’t catch. Picking on those isn’t objective.–eric]

    At what point does the sample size of “picked cherries” become large enough that it ceases to be “cherry picking?”

    [Response: Sure, but this would be repeating what CRU, GISS, etc. have been doing for years and years. If you want to reinvent the wheel, go ahead. Oh, while we’re at it, let’s redo the epidemiology on smoking and cancer. Until that’s done, let’s all take up smoking. After all, who can trust the corrupted peer-reviewed literature in leftist journals like the New England Journal of Medicine?–eric]

    Are you implying that the relationship between human-produced CO2 and global climate is identical to the relationship between habitual smoking and cancer/heart disease?

  143. ZT:

    Thank you for your comment Ken. I am absolutely only interested in learning – not denying. I am not an expert in this field – as I am sure that you can tell.

    If urbanization is not effecting thermometer readings – why is the land temperature increasing faster than the ocean? Surely, if the global temperature was increasing – would not both land and sea increase at the same rate?

    If there is a growing divergence between land and sea (?) that would imply either that the land and sea are not in thermal equilibrium (I would have thought that they are because wind is pretty effective in cooling and warming) or that the land measurements capture additional effects.

    Comments?

    (This is a question raised by Tom Wigley, of course):
    http://www.eastangliaemails.com/emails.php?eid=1067&filename=1257546975.txt

  144. AC:

    [Response: a) not NASA stations, b) not the only source of data, and c) On peut s’amuser à chercher un poil aux oeufs, mais ca sert à rien. – gavin]

    Just out of curiosity, have you actually clicked on those station links? Those are some of the saddest looking graphs I’ve ever seen – multiple years missing, etc.

    You originally said:
    [And no, the measurements from Antarctic (used by GISTEMP for instance) don’t just come from one station. Why do you automatically believe people who have been shown over and again to be misrepresenting the true situation? – gavin]

    Do you think you’ve shown that there are multiple contintental antarctic stations with good data sets?

    I haven’t found any on either of these links.

  145. alantrer:

    Well there we have it then. Homogenization adds no statistically relevant value in deriving the global mean temperature.

    Do the science a favor by presenting a simpler argument using just the raw data.

  146. Mal Adapted:

    #95 Doug Bostrom — a most gratifying excoriation of the denier army! Diplomacy and tact are manifestly ineffective on that crowd. Your blunt but elegant language may not have any more effect on your target, but it impressed me mightily. My profound admiration and thanks 8^)!

  147. trrll:

    “so I guess my question now is: if there is no statistically significant difference between raw and processed data, why process/correct it at all? why not leave the data that has no problems alone and only process the ones with clear gaps, errors, etc.?”

    There are a couple of answers to this. The simple one is, “Because scientists are obsessive about being as accurate as possible.” It is very common to see data corrections that do not alter the conclusions. I’ve had students ask me, “Why should I have to redo the analysis if you expect that it won’t alter the conclusions?” The answer being, “Because this one you used was wrong; this one is right.”

    A more subtle explanation is that you don’t know whether correction/normalization of the data will make a difference until you do it. But it is an error to make your decision of what analysis method to use contingent upon the data. Although we tend to associate cherry-picking of data with denialists, it is an error that all scientists must guard against. If you allow yourself to change your mind about whether to use a particular data manipulation once you have seen the result, or if you apply a correction in some instances but not others, you open the door to the possibility of selecting (perhaps unconsciously) the data manipulation that gives the results that you would like to see. After-the-fact analyses such as the one presented by Eric are routinely done because we like to have an idea of how large the impact of data normalization on our conclusions actually is. While we endeavor to choose the most appropriate analysis method for our data, we’d like the data to be robust enough that the conclusions are not dependent upon how the data is corrected or normalized. If adjustments to data make a critical difference, then we are going to worry a lot more about those correction/normalization methods, or look for alternative data that can validate our conclusions without requiring the same correction/normalization.

  148. lgp:

    [Response: Try reading the post again, slowly. What we did is what we said we did, plain and simple. If you want more details, go to the links we provided, which, yes, are the raw data. You are really trying hard to find fault where there isn’t any.–eric]

    Isn’t finding fault what peer review is all about? When the authors use the term “objective”, then set out to prove the “concensus” then it’s not objective. That you find the same answer as CRU only prove’s Pielke’s judgement that Phil Jones’ assertion that CRU isn’t independent of GISS et al false.

    Rather than picking a random set and applying the same “concensus” corrections as CRU does, an “objective” analysis would be to pick the most pristine sites (those with the least UHI correction required, those with the least TOB correction required, etc…in otherwords those with the least “Anthropogenic” fiddling). If you got the same answer then, that would “objectively” demonstrate that the “Anthropogenic” biases are not being introduced in the temperature trend.

    In your graphs you show the similar trends as CRU, but in the distributons (the figures to the right) the means are close, but the widths of the distributions are not. Why not? Sounds like something a peer reviewer would be quick to ask about.

    When do you plan to submit this work to a journal, and when you do, please keep us posted with the peer review comments as it works it’s way through the peer review process.

  149. John P. Reisman (OSS Foundation):

    #139 AC

    I think you’re missing the point. Add the total internal forcing + blackbody and relative constants + pre v. post industrial forcing components (i.e. GHG levels and capacities imposing on internal heat trapping of atmosphere) + natural variation (short and long) + atmospheric lifetime of Co2 + etc. = relatively clear picture of current and expected warming which translates to infrastructure and capacity.

    By looking at the big picture, one can find ways to see reasonably that data gaps pose an interesting problem but insignificant in consideration of the overarching AGW picture.

    Worrying about a few needles in the haystack is less significant when ones main concern is understanding that it is a haystack.

  150. Jiminmpls:

    This is totally OT, but check out the comments on this article on the Daily Tech blog (which George Will cites as a “scientific” source)

    http://www.dailytech.com/Farmers+Say+Global+Warming+Fix+is+Full+of+Manure+Promise+Big+Cuts/article17153.htm

    Truly scary stuff. I can only imagine what you all go through on a daily basis.

    Keep up the good work and please BE CAREFUL OUT THERE!

  151. Doug Bostrom:

    Interesting background on CRU fizzle here:

    http://www.newscientist.com/article/mg20427393.600-battle-for-climate-data-approaches-tipping-point.html

  152. Jon H:

    63: “Truly..truly…the kind of remarks such as “the Post drifts closer and closer to the Times with each passing week…” nonsense leads more and more people to believe that politics is inherent in the AGW position. Stick to the science and leave out the politics. The more you do this, the more impetus you give to the skeptics.”

    The WaPo’s shift right has been longstanding, and not just on climate issues. They’ve been pandering to climate skeptics lately, but they’ve been pandering to neocons for far longer. Stories that questioned the threat posed by Iraq got stuffed inside the paper instead of front-paged.

  153. Gerry Morrow:

    Eric, good article, but it leaves a few dangling questions surely. The questions are why, if the data and adjustments were reasonable and verifiable, did the CRU repeatedly refuse to tell McIntyre where it was? Secondly they are now saying they’ve lost the original data, but always assuming they had, why didn’t they tell him they’d lost it at the first request? Finally, why the threats, joking or otherwise, to delete the data if it was going to get into McIntyre’s hands? It seems an awful lot of trouble to go to if you are convinced your data is spot on and clean as a whistle.

    If Muir Russell is half the investigator I think he is he’ll be asking these questions because, whether you like it or not, your work is being used to decide policies which will affect everyone in the world if implemented and there are some serious gaps in the logic as to why McIntyre was given what we Brits call “the bum’s rush”.

  154. CM:

    OK, stupid question: why did they split the data into two sets? To make something stand out more clearly?

  155. Bruce H. Foerster:

    Can someone please explain to me why the major source of greenhouse gases is rarely, if ever mentioned in any commentary regarding climate change? There is much talk about renewable energy, better insulated buildings, energy efficient lighting, etc. but there is not much dialogue on the biggest villain of all.

    The largest source of greenhouse gases is from the raising of livestock for human consumption which, according to very reliable source ( Worldwatch Institute ) accounts for most ( 51% ) of all green house gas emissions!

    Are we that addicted to consuming animal flesh that we choose to not even discuss this issue? Is the meat industry lobby so powerful that they can continue to hide and down play this now well know fact? Why are people still eating meat when it is so very bad for their health and the health of their planet when there are so many delicious veggie alternatives?

    The fact is that if everyone on our planet changed their diet from an animal to a plant based diet we could eliminate MOST of the greenhouse gases entering our atmosphere which are quickly condemning virtually all living beings on this planet to certain death as the planet will soon no longer be able to support human, animal or plant life.

    I feel that we must focus on what is clearly the largest emitter of greenhouse gases if we have any hope at all of saving our world! Bottom line: we must stop eating meat immediately or our destiny will be sealed! Please read this article and do everything possible to ensure that our planet survives the biggest threat we have ever known!

    http://www.independent.co.uk/environment/climate-change/meat-creates-half-of-all-greenhouse-gases-1812909.html

  156. Hank Roberts:

    > alantrer 16 December 2009 at 12:20 pm
    > there we have it …
    > Homogenization adds no … value

    Where did you get that? Are you joking to the skeptics?

    When I search on the words you use I find many references on why and how data is worked over to make it usable.

    Here’s one example:

    http://www.gfdl.noaa.gov/bibliography/related_files/jrl0301.pdf

    p.224 VOLUME 16 JOURNAL OF CLIMATE
    Temporal Homogenization of Monthly Radiosonde Temperature Data. Part I: Methodology July 2002

    ABSTRACT
    Historical changes in instrumentation and recording practices have severely compromised the temporal homogeneity of radiosonde data, a crucial issue for the determination of long-term trends. Methods developed to deal with these homogeneity problems have been applied to a near–globally distributed network of 87 stations using monthly temperature data at mandatory pressure levels, covering the period 1948–97. The homogenization process begins with the identification of artificial discontinuities through visual examination of graphical and textual materials, including temperature time series, transformations of the temperature data, and independent indicators of climate variability, as well as ancillary information such as station history metadata.

    To ameliorate each problem encountered, a modification was applied in the form of data adjustment or data deletion. A companion paper (Part II) reports on various analyses, particularly trend related, based on the modified data resulting from the method presented here.

    Application of the procedures to the 87-station network revealed a number of systematic problems. The effects
    of the 1957 global 3-h shift of standard observation times (from 0300/1500 to 0000/1200 UTC) are seen at many stations, especially near the surface and in the stratosphere.

    Temperatures from Australian and former Soviet stations have been plagued by numerous serious problems throughout their history. Some stations, especially Soviet ones up until 1970, show a tendency for episodic drops in temperature that produce spurious downward trends.

    Stations from Africa and neighboring regions are found to be the most problematic; in some cases even the character of the interannual variability is unreliable. It is also found that temporal variations in observation time can lead to inhomogeneities as serious as the worst instrument-related problems.
    ——–

    Just an example worth a look for anyone who is starting off by eyeballing the raw data files — which is what these people did. You could do worse than to follow their exact example, documenting everything you see and what you do about it. In fact you might get a publication out of it!

  157. Charles Copeland:

    @cyclox, No 134 (‘The pushback continues’)

    Cyclox, I’ve had a brief look at the Daily Express’s ‘100 reasons why’ and, yes, most of it is the usual rubbish. But one shouldn’t over-egg the pudding.

    One example: Reason 46 runs “The IPCC alleges that “climate change currently contributes to the global burden of disease and premature deaths” but the evidence shows that higher temperatures and rising CO2 levels has helped global populations”.

    Michael Page’s refutation in the New Scientist (“50 reasons why global warming isn’t natural”) states:

    Incorrect. Excessive heat during summers is already killing more people than are being saved by milder winters.“, with a link to another New Scientist article entitled “Global warming will increase world death rate”. This article in turn refers to a contribution to the journal “Occupational and Environmental Medicine” with the heading “Temperature, temperature extremes, and mortality: a study of acclimatisation and effect modification in 50 US cities”.

    There is no reference in the abstract of the peer-reviewed OEM article to the ‘world death rate’ and one can hardly extrapolate from 50 US cities to the 7 billion inhabitants of the entire planet. Frankly I have no idea as to whether the net death rate (lives saved due to milder winters minus lives lost due to hotter summers) is positive or negative but Page’s counter-argument certainly doesn’t convince, since it is based on a totally miscontrued interpretation of an original peer-reviewed article.

    There are enough sound arguments to defend the facts of AGW. Why add spurious ones?

  158. Rando:

    92 and 116: Take a look at today’s temps in Alert (-7C) and Eureka (-11C) in Nunavut, way up north in northern Canada. They’re forecasting +5 and rain in Iqaluit on Friday – that’s on Baffin Island. The Arctic sea ice is going to start melting if that keeps up. Mind you, it’s -31C over here in Yellowknife today, so you could just say….interesting weather we’re having today, isn’t it?

  159. Louise D:

    #134. I’ve just looked at the Daily Express article again, it’s so bad it’s difficult to read all 100 reasons “why clinate change is natural”, but nearly half bear no relation to the title – such as ‘Despite the 1997 Kyoto Protocol’s status as the flagship of the fight against climate change it has been a failure’. The sad thing is that many people in Britain read this paper and are more likely to belive what it says than beleive any science based arguments. the refutation of the first 50 points is in the New Scientis which I would guess has a much smaller readership Just one more examlpe of the Express’s reasoning; point no.89 “It is a myth that CO2 is a pollutant, because nitrogen forms 80% of our atmosphere and human beings could not live in 100% nitrogen either: CO2 is no more a pollutant than nitrogen is and CO2 is essential to life.”

  160. David B. Benson:

    Bill DeMott (118) — Thank you.

  161. rb:

    In response to #92, how long have you been collecting data?

    Here are some temperature data (F) for a few widely spaced northern North America locations:

    Town 1970s 1980s 1990s 2000-2006
    Eureka, Nunavut -4.9 -3.7 -1.8 -.5
    Norman Wells, NWT 21.1 22.2 22.7 23.4
    Nome, Alaska 25.5 27.9 a 28.3
    Churchill, Man. 18.9 19.6 20.1 21.4
    Goose, Bay (NFL) 30.7 31.3 31.0 33.4

    a-incomplete data

    Do you see any trends? Although I haven’t updated since 2006, the values for 2000-2009 aren’t going to change significantly from 2000-2006.

    If you look at the 2009 average data up to this point for towns in northern North America and northern Europe, the vast majority have temperature averages above the long-term averages.

    Roger Blanchard

  162. Andy Gates:

    @134 cyclox, it took me 90 minutes to rip all hundred apart – it’s a fun project for a quiet evening. I analyzed the flaws in that “100 reasons” – classing them fairly arbitrarily – and fully three-quarters are ideological, or straightforward lies, or smears. The rest is a smattering of crank references, misquotes, tired canards, junk science and the like.

    http://andygates.livejournal.com/273199.html

    Their method, this “100 reasons” is the standard Gish Gallop: lots and lots of guff, which takes time to take apart, so in a limited setting you get 100 presented, maybe a couple refuted, and 98 “what ifs” left to confuse the public. But it’s all ideologically driven, there’s no damn science in it anywhere. It’s “100 lies we want you to believe”.

  163. kasphar:

    ZZT@ 49

    This might be an example of what you are talking about.
    In my town there are two weather stations within 300m of each other. One is a manual w/s and the other an automatic w/s. The MWS is near a tarred road with buildings close by. The other is in the middle of a grassed oval with some buildings 60-70m away but only on the western side.
    The MWS can record maximum temps up to 1.0C higher than the AWS.
    Go to
    http://www.bom.gov.au/climate/averages/tables/cw_058063.shtml

  164. TH:

    Looks like December will be the coldest on record in the US, after the third coldest October.

    “Whoever undertakes to set himself up as judge in the field of truth and knowledge is shipwrecked by the laughter of the Gods.” – Albert Einstein

  165. Thor:

    # 100
    [Response: …. The adjustments are actually very close to Gaussian centered on zero in the GHCN procedure…. -gavin]

    Does this Gaussian distribution have a constant average value over time?

  166. Completely Fed Up:

    “Homogenization adds no statistically relevant value”

    Uh, you misspelled “bias”.

    Homogenisation adds no statistically relevant BIAS.

  167. Ray Ladbury:

    Alan Trer says: “Well there we have it then. Homogenization adds no statistically relevant value in deriving the global mean temperature. Do the science a favor by presenting a simpler argument using just the raw data.”

    Well, except if they didn’t make corrections you’d claim it was all the Urban Heat Island or some other artifact. Remind me again why scientists should give a tinker’s damn what you think?

  168. Completely Fed Up:

    #125:

    “#99 And then Ian George would require the calibration tests of the thermometers. And the manufacturing report.

    Which should be included in any report.”

    Why? The thermometers are made at a factory.

    They are sold.

    Do you have a thermometer?

    Do you have the tech spec for it?

    No, I doubt you have.

    And so I guess you’ll say merely because they aren’t available (despite not being raw data or source code or models or any of the hundred other things you’ve decided MUST be made available) that this PROVES that the data is wrong.

    Tell you what.

    Make your own observing network (pay for it yourself, I don’t want my taxes to pay for your insanity). With hookers and blac k jack if you like.

    Then while we work with what our current network shows (that we need to avoid CO2 production) you can go on and build up the case for the evidence that there’s nothing to worry about.

    Please. Remember to keep all station logs, all emails on any subject, all invoices (you may have paid a company with biases on AGW for their kit) and all other information we’ll discuss when you’re ready to report.

    Then use that data (if you can PROVE it’s right) to show we should go back to Oil/Coal which, since we haven’t burned any, is still sitting nicely under ground.

  169. Ray Ladbury:

    DVG@131, As my puppies are now dogs and fully housebroken, I now have no need of the Wall Street Journal. Thanks all the same.

  170. lgp:

    correcting my typo correction (again)

    in post my post awaiting moderation, I have a typo

    That you find the same answer as CRU only prove’s Pielke’s judgement that Phil Jones’ assertion that CRU isn’t independent of GISS et al is TRUE.

    should read

    That you find the same answer as CRU only prove’s Pielke’s judgement that Phil Jones’ assertion that CRU IS independent of GISS et al is FALSE.

    [Response: What? GISS has nothing to do with the analysis in this post. Or are you simply saying “both GISS and and CRU used temperature data from the same planet?”–eric]

  171. aH1GH3Rpower:

    How do we know that the data your using to back up the CRUs hasnt been tampered with aswell? Such as in the New Zealand incident http://blogs.news.com.au/heraldsun/andrewbolt/index.php/heraldsun/comments/climategate_making_new_zealand_warmer. Also im sorry but this talk of not having time to reinvent the wheele with only 60 some testing stations doesnt seem like enough to claim to be an objective assesment of the planets temp trends. And with the policy solution of classifying Co2 as a polutant and taxing it? The gas that every plant form on this planet uses to breath, which every animal atleast indirectly uses to eat, and who inturn we eat just seems a little irresponsible.

  172. jonesy:

    [Response: What? GISS has nothing to do with the analysis in this post. Or are you simply saying “both GISS and and CRU used temperature data from the same planet?”–eric]

    So do you mean CRU and GISS (and NOAA) all use the same station raw data, but their temperature results are independent because they all process it independently?

    [Response: Bingo! (Actually, I doubt they use exactly the same raw data — no doubt there are some differences — but the overlap is obviously going to be huge).–eric]

  173. Paul:

    The results of the GW debate have become intimately tied to what is potentially the largest international treaty for at least a generation. To pretend at this point that the science has not been politicized strikes me as quite naive.

    http://healthjournalclub.blogspot.com/

  174. Ray Ladbury:

    TH says “Looks like December will be the coldest on record in the US, after the third coldest October.”

    Uh, dude, ever look at a map? Did you know there are whole countries outside the US?

  175. Doug Bostrom:

    aH1GH3Rpower says: 16 December 2009 at 7:02 PM

    “How do we know…”

    Check under your bed first, that’s my advice. Who knows what’s going on down there?

  176. jonesy:

    [Response: Bingo! (Actually, I doubt they use exactly the same raw data — no doubt there are some differences — but the overlap is obviously going to be huge).–eric]

    Thanks, I think that’s information that I and a lot of people are confused about.

    So, if anyone wants to find the “raw data” for a certain station, it’s my understanding that some is publicly available but some is proprietary. Is that correct? Is the publicly available data all at one site, or at various sites?

  177. wildlifer:

    @171 Old refuted nonsense:
    http://scienceblogs.com/deltoid/2009/11/new_zealand_climate_science_co.php

  178. sHx:

    DEB(#84) said, “I was talking here about polar cities to save survivors in 2500 AD and you all thought I was nuts. What do you think now? We are getting closer to that time….”

    BPL(#103) replied, I think you’re nuts.

    Sorry, I couldn’t resist. But A) in 2500, there won’t be a North Pole, and B) the survivors won’t have the capital, infrastructure and functioning economy enough to build a domed city in Antarctica.

    Sorry, I couldn’t resist either. I am very curious to know why BPL, a “SF action adventure” writer (according to his website) and a frantic commenter on this blog, be so pessimistic about the possibilities for human habitation by 2500 AD.

    A) Why will there not be a North Pole by 2500 AD? Is it because it will float on dwindling ice floes and eventually go extinct like polar bears? Will the gyroscopes only show the South Pole by 2500 AD? The questions are cheeky but fair to ask, considering the hyperbolic claim that brought them forth: that the AGW will make the North Pole disappear! No, BPL, “polar cities” in the North won’t be established on the pole, but on the vast, uninhabited polar lands around it. And the North Pole isn’t going anywhere soon, climate change or not. Are you nuts?

    B) Why shouldn’t a domed city in Antarctica be possible? A civilian population centre -most probably a tourist resort for the fabulously rich and eccentric- is already well within the realms of the present day capital and technology. In fifty or a hundred years, towns and and cities with hundreds of domes may spring up, especially if valuable resources like gold and diamond are found. The existing Antarctic Treaty is written on ice, not on stone. And like Rome, Antarctic polar cities won’t be built in a day.

  179. David Horton:

    #164 TH – “Looks like December will be the coldest on record in the US, after the third coldest October.” Yeah, uh, record HOT November temps in Australia. This is the southern hemisphere, you will recall, where we are coming into another frighteningly hot summer with bushfires, yet again, burning ferociously. Guess it’s cold up north, eh TH, who’d a thunk it?

  180. Phil Scadden:

    aH1GH3Rpower – and do you realise that “NZ incident” was trumped up garbage easily refuted by NIWA. (See hot-topic.co.nz for more discussion). See here
    http://www.realclimate.org/index.php/archives/2009/12/cru-hack-more-context/comment-page-20/#comment-149237 for more on Hokitika.

    And for goodness sake – if you think you can make global warming go away by making unfounded attacks on the station record, how then do you explain sealevel rise (sealevel.colorado.edu), or the satellite MSU-LT trends? And the problem with CO2 is rapidly releasing Gtonnes of carbon sequestered over millions of years. The natural CO2 cycle isnt relevant. If your stance is that you doubt the science because you find suggested solutions (liking taxing carbon so carbon alternatives are sought instead) unpalatable, then please suggest a better way to reduce CO2 in the atmosphere. I’ll back any solution that actually creates a reduction but judging facts on the basis of your biases about solutions seems intellectually dishonest.

  181. Dale:

    John #137, for the most part I agree. The consertive Roscoe Bartlett is at forefront of peak oil in the congress and is a strong supporter of the science of AGW. When I refer to right wingers I’m talking about people who have perverted the meaning of a conserative and turned it into reactionary. These are people who are guided by idiology and if facts fly in the face of what they believe they simply put their fingers in there ears, close their eyes and yell, “Honey, turn on Rush….quick!” They are people who view the world through a soda straw and are up to their hips in dogma and don’t pocess two cents worth of pragmatism. They have the least amount of scientific credentials yet they want to be the ones who tell us what the science says. If Inhofe and all like minded people were all in one world, would one of them contemplate themselves as a virus?

    By the way, you’re one of the people I always read and have great respect for your thoughts.

  182. Steve Fish:

    Comment by trrll — 16 December 2009 @ 1:18 PM:

    Thanks! You describe how I was trained, and how I trained my students. Those outside of science sometimes have a hard time understanding this embedded ethic.

    Steve

  183. Molnar:

    When you reorder some of the items on that Daily Express list, it comes out a bit funny…

    “9) Leaked e-mails from British climate scientists – in a scandal known as “Climate-gate” – suggest that that has been manipulated to exaggerate global warming

    42) The Met Office asserts we are in the hottest decade since records began but this is precisely what the world should expect if the climate is cyclical”

    So the recent warming has been exaggerated, unless the climate is cyclical, then it wasn’t…

    “17) The science of what determines the earth’s temperature is in fact far from settled or understood.

    7) The 0.7C increase in the average global temperature over the last hundred years is entirely consistent with well-established, long-term, natural climate trends.

    22) There is strong evidence from solar studies which suggests that the Earth’s current temperature stasis will be followed by climatic cooling over the next few decades”

    Obviously, we don’t know what determines the temperature of the earth, except that the recent warming is caused by “well-established, long-term, natural climate trends” and the world will cool in the future because of the sun.

    Right.

  184. VagabondAstronomer:

    TH…
    Cite sources please. Also include other MS globally.

  185. dhogaza:


    Bingo! (Actually, I doubt they use exactly the same raw data — no doubt there are some differences — but the overlap is obviously going to be huge)

    The information I’ve read is that the GHCN database includes all the available data that’s not restricted by distribution agreements. This is what GISTEMP uses, and represents about 95% of the data used by CRU, the other 5% being the data subject to such agreement and which McIntyre et al have been screaming about being “hidden” for all of these years.

    Is the publicly available data all at one site, or at various sites?

    As you’ve probably gathered from my post, one site, along with a whole lot more.

    As you can see, the data’s been very effectively kept secret and hidden from view on a public US government website.

    It’s a wonder that the denialists haven’t been telling this to people with the same vehemence they’ve screamed “hidden data! hidden data!”, isn’t it?

  186. dhogaza:

    Oh, Jonesy, and as far as that other 5% that’s restricted and used when creating HadCRUT (but not NASA GISTEMP), the data can be had from the individual countries, though unfortunately often for a fee.

    See, McIntyre et al have been asking for restricted access data for free that in some cases CRU had paid for, rather than going to the source and paying for it themselves. Understandable, who wants to pay for data? And not all of the restricted data has been paid for, but still – you do get the point, I hope? Asking people to give you something they don’t have the right to give you is just fine. When rejected, screaming “fraud!” “misconduct!” and the like is not.

  187. John MacQueen:

    “The implication is that something secretive and possibly nefarious has been afoot in the way data have been handled, and that the validity of key data products (especially those produced by CRU) is suspect on these grounds.”

    Not so at all.

    The implication that there could be honest mistakes made with no nefarious intent is also there, which would render the products suspect on those grounds as well.

    [Response: Except that this would be even more unlikely, since the mistakes mistakes made by CRU would have to be nearly identical to the mistakes made by NOAA, GISS, and the Japanese compilation, since all the data sets agree very very well with one another. That’s even more impossible to believe than a conspiracy!–eric]

  188. Steve Fish:

    Comment by CM — 16 December 2009 @ 2:07 PM:

    Eric chose data on the basis of the length and completeness of data sequences. One might complain that his specific choice had a bias. Splitting the data into two sets is a further test of random selection. A problem would be indicated by a difference between the two sets. This is a common strategy. Think of it as similar to splitting a card deck for shuffling.

    Steve

  189. Norman:

    #112 Kevin McKinney says:
    16 December 2009 at 9:07 AM

    “And Arctic amplification of warming has been observed in spades. However, it’s least marked in winter, since there is virtually no insolation at the highest latitudes during that season.

    No insolation=no greenhouse effect.”

    My understanding of the “greenhouse effect” was that it was not determined by solar radiation but would be more pronounced in the Arctic region. Using the Stephan-Boltzmann Constant, even a cold arctic surface will still emit IR (cold maybe by our standards but still very warm compared to absolute zero). The carbon dioxide in the Arctic atmosphere will absorb this radiation, warm, and in turn emit some radiation back to earth to replace some of the energy it lost.

  190. Norman:

    #161 rb says:
    16 December 2009 at 3:37 PM
    In response to #92, how long have you been collecting data?

    Thanks for you response and information. I am only starting to look at northern data the last month or so to get a “feel” for the Northern climate. Your data set does indicate the region has experienced warming. My question was what was causing the current cooling I have been observing.

    I am ignorant of the climate forces working up there. If greenhouse theory makes sense for you data set but I was wondering why it looks like it is cooling. I may be using too small of a data set.

  191. Norman:

    #158 Rando says:
    16 December 2009 at 2:50 PM
    92 and 116: Take a look at today’s temps in Alert (-7C) and Eureka (-11C) in Nunavut, way up north in northern Canada. They’re forecasting +5 and rain in Iqaluit on Friday – that’s on Baffin Island. The Arctic sea ice is going to start melting if that keeps up. Mind you, it’s -31C over here in Yellowknife today, so you could just say….interesting weather we’re having today, isn’t it?

    Do you have any theories as to what is going on to cause this effect. Very cold or normal for Yellowknife but much warmer in very North Canada. Is some warm ocean current causing this?

  192. Steve Fish:

    Comment by lgp — 16 December 2009 @ 1:24 PM:

    You are incorrect regarding peer review. Reviewers look to see if the study authors used the appropriate data and analysis to support their findings. It is more complicated, but these are the basics.

    The point of the original post was to demonstrate what critics, such as YOU, can do to test the veracity of the research. If you don’t trust the findings of climate researchers, all your unsubstantiated complaints mean zilch when you can test them yourself to demonstrate your point. Get busy.

    Steve

    [My emphasis. –eric]

  193. Ron R.:

    John P. Reisman, while I did look at your first link before I finally got the time to look over your arctic and glacier pics. So far they are the best I’ve seen. I especially like the glacier page. That’s pretty much what I’m talking about. Good job. Now if we can do that for the other effects as well we’ll have something!

    For some reason there is a large blank space on your pages before the info begins. Also when I clicked on some of them on the arctic page a page came up that said “502 bad gateway”.

    On the northwest passage here’s ESA’s 2007 view.

    http://www.esa.int/esaCP/SEMYTC13J6F_index_1.html

  194. Cardin Drake:

    This is an interesting post and cuts to the heart of the matter. I think at this point in order to regain credibility, CRU will have to demonstrate exactly what you are trying to prove–that the adjustments to the raw data make sense and will withstand scrutiny. I have not seen anybody who has taken issue with this analysis on any of the skeptical blogs yet.
    However, they have pointed out that there is a significant difference between the raw and adjusted numbers on NOAA’s published USHCN data. Currently it is about .5 F, which is large portion of the 1.3 degree anomaly.
    That is hard to reconcile with the CRU data since you would expect the adjustments to be similar.

    [Response: I’m not “trying to prove” anything. I’m showing people how to take a look at the data for themselves and make an assessment about whether CRU is full of cheaters or the deniers are full of it. And CRU doesn’t need to do anything, since any fool can do the analysis we did. There’s a simple reason none of the skeptical blogs have taken issue with what we did: they know we’re right. Indeed, the fact they are ignoring it tells you something, doesn’t it? I thought this post would actually be the end of the discussion, but evidently, I was naive.–eric]

  195. Terry:

    I seem to remember sometime ago Gavin implying that one should only need about 100 high quality temperature stations to adequately characterise the global temperature. Why then dont they take that advice and simply ditch the contaminated sites, and use only pristine sites. I would expect that there are at least that number on the globe. The whole issue of adjustment and homogenization errors surely then disappears.

  196. John:

    Has anyone written a rebuttal of the following document by Christopher Monckton? http://scienceandpublicpolicy.org/images/stories/papers/originals/Monckton-Caught%20Green-Handed%20Climategate%20Scandal.pdf

    I’m particularly interested in any answers to the supposed “endpoint fallacy” since it seems rather absurd but I can’t put my finger on why.

    Thanks in advance.

  197. ZT:

    Thanks for the link Kasphar – I’ll have a look.

    Any chance for an answer on Tom Wigley’s question about the possible divergence of land and sea temperatures?

    On the original analysis – an interesting statistical test might be to examine the similarity of the two sets of stations before and after processing.

    This would indicate whether value adding could have the effect of increasing, decreasing, or keeping constant the statistical properties of the two sets of data. The hypothesis being that referencing various stations in the homogenization yields to correlation across the data.

    This would might help to strengthen the argument. Currently a skeptic might claim that the homogenization processing introduces a common bias into all series.

    Are the original data available for these particular stations?

  198. kdk33:

    Surface station reconstructions are suspect. Period. Too many stations, too many corrections, too many data splices, too many complicated averages, too many people outsmarting themselves – the noise is bigger than the signal.

    Stick to the satellite data. It’s been warming some. Big deal.

    But why? – that’s the question.

  199. Ian:

    I don’t think this is inappropriate given the title of this thread. but Do you have any comment on the allegations from Russia that CRU are very selective in their choice of weather stations in Russia?

    [Response: Since the only substantial differences appear to be in the 19th Century, the issue is very likely to be connected to instrument changes/metadata changes that the IEA analysis doesn’t look at at all. This is just bandwagoning and the flinging around of baseless accusations by a right-wing think tank. – gavin]

  200. Joseph Sobry:

    155 Cattle, meat etc.
    Please see http://en.wikipedia.org/wiki/Cattle#Population
    The world cattle population is estimated to be about 1.3 billion head.[1]. India is the nation with the largest number of cattle, about 281,700,000 or 28.29% of the world cattle population, followed by Brazil: 187,087,000, 18.79%; China: 139,721,000, 14.03%; the United States: 96,669,000, 9.71%; EU-27: at 87,650,000, 8.80%; Argentina: 51,062,000, 5.13%; Australia: 29,202,000, 2.93%; South Africa: 14,187,000, 1.42%; Canada: 13,945,000, 1.40% and other countries: 49,756,000 5.00%.[51] Africa has about 20,000,000 head of cattle, many of which are raised in traditional ways and serve partly as tokens of their owner’s wealth.

    Please remember that cows in India are scared, sorry, I mean sacred.
    Cattle in India not only eat grass and other plant matter but serve other environmental needs I will not describe here. I do not know but I think that cows are not eaten by humans in India. One wonders what happens to them after uselessly pumping CO2 and methane in the air for a lifetime.
    Also the Masai herders in Africa are very attached to their cattle.
    There are probably other people on the planet with similar penchants.

    The number of Bison in the USA was estimated at ~60,000,000 when brave hunters shot practically all of them in a geologically (catastrophically for the bison) very short time mostly from the safety of trains. It would be interesting to see if there was a drop in methane or CO2 levels in the Greenland Ice Record once this valiant task was completed.

    While it is rightly recommended not to eat too much beef or any other meat for rational dietary and health reasons that is no reason to end all production of meat which is a very valuable source of food for most humans. Indeed there are a lot of humans who would fare much better if there was some meat in their diet.

    A more reasonable (much smaller) amount of meat in northern hemisphere human diets is certainly recommended and would put a lower ceiling on the number of cattle and other animals raised for food not to mention the amount of adipose tissue in the general population in said hemisphere.

    In addition one should be careful what one wishes. Grazing animals and grasslands live in a close symbiosis. It would not surprise me to learn that uncropped grasslands produce more methane and CO2 than the symbiotic grass and grazers.

    One would also think that cattle and other grazing animals were and are part of the normal carbon cycle.
    With the exception of some aspects of feedlot practices for poultry and pigs and for cattle (usually inflicted towards the end of an animal’s life) the conversion of plants into animal tissue and activity is perfectly natural . That is what we humans as mammals and omnivores do directly by eating greenery or indirectly by eating meat.

    Furthermore the total conversion to a plant food diet for the human species may cost more in CO2 and methane than an appropriate amount of meat eggs dairy and other animal products in the diet.It is my understanding that fleets of refrigerated trains trucks and airplanes are currently used to supply fresh lettuce, fruits and other plant food to the millions in the cities all over North America. This is done especially during the winter when no local produce is available. I can scarcely believe that his is not also the case in Europe and other parts of the world. I am trying to grasp how much more transport will be needed to supply all of humanity with a plant food only diet.
    It would be much better if the people would eat the locally produced animal products during the winter/spring and the locally produced fresh plants during the summer/fall. After all that was the old practice just a few decades ago and going back many hundreds of years.

    I honestly think we should keep focused on fossil fuels or fossil carbon.
    The consumption of fossil fuels does a lot more damage to our health, our environment and our planet than any CO2 or methane exhaled in any manner or direction by any mammal or other animal for that matter.

  201. thingsbreak:

    RE: “This is just bandwagoning and the flinging around of baseless accusations by a right-wing think tank”

    Gavin, I surely hope you’re not somehow suggesting that the “skeptical” blogs are conflating a CATO Institute Senior Fellow’s/anti-regulation think tank’s opinion with that of the entire nation of Russia- oh, no wait, that’s exactly what they did. Carry on.

  202. Ken W:

    ZT (143):
    “If urbanization is not effecting thermometer readings – why is the land temperature increasing faster than the ocean? Surely, if the global temperature was increasing – would not both land and sea increase at the same rate?”

    You can clearly see the land warming faster than the ocean in this GISS dataset plot:
    http://data.giss.nasa.gov/gistemp/graphs/Fig.A4.lrg.gif

    This is exactly what one should expect, because it takes so much more energy to raise the temperature of water than land or air. That’s why inland climates experience more extreme temperature ranges than coastal climates. Even though there is wind and constant exchanges of energy between the surface of the ocean and the atmosphere, things aren’t in an equilibrium state. There are many complex factors (e.g. evaporation which loses heat energy into the atmosphere, circulation patterns of both wind and water, melting glaciers pouring cold water into the oceans, etc.) which will also effect the measured ocean surface warming.

    There actually is an urban warming effect. Temperatures are warmer in urban areas than surrounding rural areas (another human factor in climate change), but as has been shown in previous links they don’t bias the computed global surface temperature. Those urban effected temperature measurements (i.e. the ones photographed next to air conditioners or surrounded by blacktop) are either eliminated or corrected using nearby rural measurements.

  203. dhogaza:

    This is just bandwagoning and the flinging around of baseless accusations by a right-wing think tank

    From a country (Russia) banking a large part of its economic future on increasing sales of oil and natural gas to Europe.

  204. Ken W:

    kdk33 (198):
    “Stick to the satellite data. It’s been warming some. Big deal.
    But why? – that’s the question.”

    Primarily because of increased atmospheric CO2 from human burning of fossil fuels.

    http://www.skepticalscience.com/Empirical-evidence-that-humans-are-causing-global-warming.html
    http://www.skepticalscience.com/How-do-we-know-global-warming-is-still-happening.html

  205. DAS:

    @ #49 ZZT: “Question – how do we know that these stations are not simply measuring the growing proximity of asphalt and air conditioning units with time?”

    Wow, you’ve hit on something there alright(roll eyes)…more asphalt + more AC equipment = higher readings…DUH! The sea level actually only exists to play the part of a giant mercury thermometer: more heat and pollution from industry, human activity, and energy usage causes level to rise.

  206. Dave C:

    Is anybody else amused by the fact that kdk33 (#198) seems to think that by employing a bit of sophistry, hand-waving and no data at all he can refute the findings of an entire field of research?

  207. Kevin McKinney:

    Norman, you may be right in your comments, but nevertheless the amount of energy “coming in” to the climate system in the highest latitudes in winter is mostly due to physical transport–air masses & ocean currents. The absorption of IR you describe is–subject to the correction of someone who understands this in depth, which I don’t claim to–much de-emphasized in winter as compared with summer.

    I’m pretty darn sure that the seasonal differential in warming is empirically well supported. This quote is from a story on a 2003 paper, but I think there’s more recent data as well, could I but take a bit more time to search:

    “Most importantly, temperatures increased on average by 1.22 degrees Celsius per decade over sea ice during Arctic summer. The summer warming and lengthened melt season appears to be affecting the volume and extent of permanent sea ice. Annual trends, which were not quite as strong, ranged from a warming of 1.06 degrees Celsius over North America to a cooling of .09 degrees Celsius in Greenland.”

    http://www.nasa.gov/centers/goddard/news/topstory/2003/1023esuice.html

    Interesting and recent, but not, sadly, quite to the point we’ve been discussing:

    http://www.nature.com/ngeo/journal/v1/n11/full/ngeo338.html

    Claims to show human attribution of the polar warming.

    On the other hand, this paper says I’m completely wrong:

    http://www.agu.org/pubs/crossref/2009/2009GL040133.shtml

    Says that cloud forcings create more amplification in the winter than in summer. How you reconcile that with the 2003 paper, I’m not sure. (This layman is headed for bed, before he looks even more foolish.)

    Must read more about this. . .

  208. Terry:

    Gavin, I dont wish to be a pest, but what is the official line on my question @195. From a purely scientific data QA perspective it surely makes sense. Cheers

    [Response: 100 would do it – if you knew they were perfect. Since you don’t, you need redundancy so that you can check for outliers and jumps etc. However you don’t need 1000s of stations to get a good estimate of the global trends – which is why it doesn’t much matter how you cut it, you end up with the same thing. But, the more the merrier when it comes down to pinning down regional patterns – they do require more data overall. – gavin]

  209. DVG:

    Ray Ladbury@169: Wow, religious conviction certainly runs deep.

  210. John P. Reisman (OSS Foundation):

    #193 Ron R.

    Sorry about the 502 messages. The site is pretty busy and when the ram overloads it shuts down because it is on a shared server. There is a cron job that reboots it after 5 minutes and I am actively looking for a new host for the site that gives me more ram headroom.

    I uploaded the 2007 image of the passage so now there is 2007 and 2008.

    If you are interested in helping build new image sections contact me:

    http://www.ossfoundation.us/contact-info

    I can use all the help I can get.

  211. Cardin Drake:

    re:194
    Eric,
    This post is a good starting point. But CRU can’t just bunker down and wait for all this to go away. Any fool can do this analysis, but I’m afraid that CRU will have to start from their own raw data and show step by step how they got to their adjusted data before they will have any credibility. And if you think about it, that is not too much to ask of them.
    Legitimate questions remain. For the NOAA data set, it doesn’t make sense that the adjustments wind up .5 degrees higher than the raw data. Intuitively, the UHI effect would make you expect that the adjustments would net out lower, or at least neutral.

  212. Terry:

    Re Gavin @208, Sure the more the merrier and redundancy is a good objective, but my question still remains as to why use contaminated sites that need correction, if there are enuf uncontaminated ones already available. I also understand that the latest CO2 satellite data indicates that regional effects are now clearly very important, but again surely there must be sufficient clean sites to do the job.

  213. Edward Greisch:

    84 Dan e Bloom: By 2500 we will either be extinct or living on Mars, asteroids, the moons of Jupiter, the moons of Saturn etc. Polar cities are still a non-starter. Your time scale is off by a multiple of 10. Things are going to happen 10 times as fast as you imagine. Maybe faster.

  214. Alan of Oz:

    Great post, simple logic and usefull links.

  215. CM:

    Steve Fish (#188), thanks for the explanation. I’m in the humanities, and don’t know much about actually working with numbers. I comfort myself that at least I’m aware of my limitations, unlike some (most?) of the R script kiddies suddenly out there doing private investigations of weather station records and FIX_ME notes. This kind of post, with guidance on how they could at least make a meaningful junior science-fair project out of it, is a very good idea.

  216. sHx:

    For the love of all things green and cool, could someone who is in-the-know tell me what is the percentage of the data the CRU used for its climate modeling that is NOT covered by the confidentiality agreements? This question will be moot in six or seven months’ time since the UK Met service announced that they’ll be seeking permissions from other Met services in order to release all of its raw data, but it is extremely annoying to see a few evidently non-climate scientists seeking mileage out of the debate by bandying the figures of 95 and 98 percent.

    More than two weeks ago, in another thread, I took the issue with a commenter named Marco, in the context of the legitimacy of the FOI requests made by Steven McIntyre, when Marco claimed that “the confidentiality agreements covered only 2% of the data, which carries very limited extra information”. Marco later claimed that the figure of 2% was “notably” in McIntyre own request for the FOI. He did not provide any citation to substantiate his claim.
    http://www.realclimate.org/?comments_popup=2019#comment-145458
    http://www.realclimate.org/?comments_popup=2019#comment-145510

    And today we have dhogaza saying at #185:

    The information I’ve read is that the GHCN database includes all the available data that’s not restricted by distribution agreements. This is what GISTEMP uses, and represents about 95% of the data used by CRU, the other 5% being the data subject to such agreement and which McIntyre et al have been screaming about being “hidden” for all of these years.

    and at #186:

    Oh, Jonesy, and as far as that other 5% that’s restricted and used when creating HadCRUT (but not NASA GISTEMP), the data can be had from the individual countries, though unfortunately often for a fee.

    See, McIntyre et al have been asking for restricted access data for free that in some cases CRU had paid for, rather than going to the source and paying for it themselves. Understandable, who wants to pay for data? And not all of the restricted data has been paid for, but still – you do get the point, I hope?

    I skip the blindingly obvious questions of “where did you read the figure of 95%?” and “do you want me to believe that Steve McIntyre would not spare a few lousy dollars to buy the missing data from the gazillions that he was supposed to be receiving from the fossil fuel industry?” I skip these questions because this time I want to hear from climate scientists, not from cheap propagandists.

    What is the percantage of data used by the CRU that is NOT covered by the confidentiality agreements? What is the weighing of the ‘missing bits’ on the CRU’s climate models? Has Phil Jones ever made an attempt to release the data not covered by confidentiality agreements AND provide the names of those Met services that sold the rest?

    I am not interested in the code for the climate model, or Jones’s right to scientific vainglory, or whether there are other sources proving the AGW. My questions are only about the CRU data because specific figures of 95% and 98% are being spread around, without any citation, as the percentage of the freely available CRU data, and attacks are made on others on this basis. The answers to these questions are important to me not because I intend to replicate the CRU’s climate science in my bedroom on my Pentium 4 PC but because I want to assess the credibility of various claimants like Phil Jones, Steve McIntyre, Marco and dhogaza.

  217. Barton Paul Levenson:

    Bruce H. Foerster: The largest source of greenhouse gases is from the raising of livestock for human consumption which, according to very reliable source ( Worldwatch Institute ) accounts for most ( 51% ) of all green house gas emissions!

    BPL: Maybe 51% of methane emissions, but the vast amount of artificial CO2 emitted is from burning fossil fuels.

  218. Completely Fed Up:

    “From a country (Russia) banking a large part of its economic future on increasing sales of oil and natural gas to Europe.”

    More like

    From a company mounthpiece rather like the American Petrolium Institute, the Cato Institute or Heartland Institute.

    There are fewer differences between Russians and the West than the US are generally comfortable with. Unfortunately, there are fewer differences between Russians and the West than many USians would hope for.

  219. Pekka Kostamo:

    #195 Terry:
    Perhaps a good starting point is to take a look at the international monitoring networks, as presented by the World Meteorological Organization (WMO). With some patience, you find details of the various station networks as well as the multiple parameters used in monitoring the climate.

    http://www.wmo.int/pages/prog/gcos/index.php?name=ObservingSystemsandData

    The climate observation stations are, of course, just a subset of the World Weather Watch Global Observations System, an overview of which is conveniently available at:
    http://www.wmo.int/pages/prog/www/OSY/GOS.html.

    Further pages of the WMO give details of data collection subsystems, the messaging and measurement standards etc., developed over the 150 years that a global effort to predict the weather has existed.

    It is a huge system, serving great many purposes both locally and internationally. Climate monitoring is just one aspect of it.

  220. Completely Fed Up:

    sHx prvides wisdom:

    “A) Why will there not be a North Pole by 2500 AD? Is it because it will float on dwindling ice floes and eventually go extinct like polar bears?”

    Nope, because the north pole is an ocean area with ice on top.

    I would not think thinning floating ice is a good place to build your heated city.

    And by 2500 there won’t be any ice to build on there anyway.

  221. Completely Fed Up:

    “Does this Gaussian distribution have a constant average value over time?”

    In the same way as a FFT does.

  222. Anne van der Bom:

    ZT
    16 December 2009 at 12:45 PM

    If urbanization is not effecting thermometer readings

    I would interpret Kens words “the myth is that asphalt has biased the temperature readings” as: “the myth is that asphalt has biased the temperature record“.

    Urbanization IS affecting thermometer readings, but it is accounted for in the final temperature records.

    Problem with many blog scientists weighing in on climate change is that they want to overcorrect so the positive trend goes away.

  223. Barton Paul Levenson:

    sHX,

    You caught me in careless phrasing. I meant, of course, “there will be no north polar ice cap in 2500.”

    As the the capabilities of human society in 2500–I, personally, expect human civilization to collapse almost completely within the next forty years. I doubt it will be back up in time to build self-sustaining Antarctic cities in 2500.

  224. Barton Paul Levenson:

    Joseph Sobry,

    The Indians don’t eat the cows because in sowing and harvest seasons, the cows pull the plows. There’s usually a good reason behind a seemingly irrational custom.

  225. Jiminmpls:

    #164 TH – Yes, in parts of the US, October was very cold indeed – and November was the 3rd warmest on record. Now December is starting off cold again. Western ski resorts had their earliest openings in 40 years in October, but were pretty much shut down for the big Thanksgiving holiday. Gosh, sounds like extremes in weather to me – which is exactly what those science guys have predicted.

    OTOH, globally, the pattern is quite different. October was the sixth warmest on record and November was the fourth warmest on record. Sept-Nov was the fourth warmest on record and Jan-Nov was the fifth warmest on record.

    Check out this site: http://lwf.ncdc.noaa.gov/sotc/

    It’s very easy to understand and you can start basing your opinions on factual information rather than bullsh1t.

  226. Snorbert Zangox:

    Gavin,

    The article that I read appears to be in an English language version of a Russian newspaper. I do not know if that newspaper is associated with “a right-wing think tank” or not. However, that article says “The data of stations located in areas not listed in the Hadley Climate Research Unit Temperature UK (HadCRUT) survey often does not show any substantial warming in the late 20th century and the early 21st century.” That implies that more than 19th century data are affected.

    http://en.rian.ru/papers/20091216/157260660.html Scroll down to “Russia affected by Climategate”

    [Response: Well, journalism is in trouble in Russia too… The relevant figure from the IEA report is here which show the difference between the their ‘all station’ index and the HadCRU index for Russia as a whole. Since there was no check for inhomogeneities or jumps in the IEA index, it doesn’t stand up to much scrutiny, but even if it were fine, the differences in the 20th C trend are small. The whole ‘someone made adjustments/screened stations therefore fraud!’ line of argument is getting very old. – gavin]

  227. Ray Ladbury:

    DVG @209, Isn’t it interesting that creationists, climate change denialists and even smokers all immediately jump on the religious line like that. Maybe it is because they would never think of actually looking at the evidence and drawing conclusions based on that.

    The Wall Street Urinal used to be a decent paper. Now I wouldn’t line a birdcage with it. If I want news from an economic perspective, I read The Economist–and I do read it weekly. They at least acknowledge physical reality–unlike the Urinal where spin is supreme.

  228. Lynn Vincentnathan:

    This is a great post.

    I teach criminal justice research methods, and some of the basics are the same for all sciences, so I sometimes use issues in climate science as examples. I just finished teaching hypothesis testing — which is very difficult…it sounds like some double negative, cukoo way of doing things to novices…, as in “why would anyone want to establish a null hypothesis, when it’s the research hypothesis we’re interest in.”

    Anyway, I told them how climate denialists are always assaulting the scientists with “Have you considered that increasing solar activity may be warming the earth, or X, Y, or Z natural factors.” And the scientists reply, of course we have. That’s our null hypothesis. We only reject it when we are confident that the data can no longer be explained by those natural factors; there is a .05 or less probability that those natural factors can explain the data. ((Note that conscientious laypersons were not sitting around waiting for the null to get down to .05 probability, which happened in 1995, but were busy well beforehand reducing, reusing, recycling, going on alt energy, AND saving money in the process.))

    Anyway, in research methods there are all sorts of issues of validity (did you set your watch to the new time zone) and reliability (does your watch keep good time) — e.g. statistical conclusion validity, which requires enough data (years of temp findings) to reach statistical significance. Then there is internal validity, construct validity, external validity, criterion-related validity (proxies, etc), and so on.

    And it seems what the scientists were doing in those emails highlighted by the denialists was in a nutshell addressing and correcting problems relating to such validity and reliability threats. And, of course, laypersons with agendas can be easily convinced that sounds cukoo.

    Shame on the real scientists amongst the denialists who work against scientific understanding.

    [Response: Thanks. Very thoughtful and interesting.–eric]

  229. Patrik:

    Dear scientists,

    What is the main difference between Set A and Set B?
    Because Set A shows exactly what many sceptics are saying:
    That the 1930:s was just as “hot” as present day.

    [Response: A and B are two random samplings of a small fraction of the data so you wouldn’t expect them to match the global average perfectly. Some parts of the globe may have been as warm as present day; indeed, that is exactly what the data suggest.–eric]

  230. John P. Reisman (OSS Foundation):

    I summarized ‘CRU Data’ & ClimateGate based on the wonderful RC work from the 4 posts.

    http://www.ossfoundation.us/projects/environment/global-warming/myths/climategate

  231. Kevin McKinney:

    From Serreze et al, 2009:

    “As the climate warms, the summer melt season lengthens and intensifies, leading to less sea ice at summer’s end. Summertime absorption of solar energy in open water areas increases the sensible heat content of the ocean. Ice formation in autumn and winter, important for insulating the warm ocean from the cooling atmosphere, is delayed. This promotes enhanced upward heat fluxes, seen as strong warming at the surface and in the lower troposphere. This vertical structure of temperature change is enhanced by strong low-level stability which inhibits vertical mixing. Arctic amplification is not prominent in summer itself, when energy is used to melt remaining sea ice and increase the sensible heat content of the upper ocean, limiting changes in surface and lower troposphere temperatures. Loss of snow cover contributes to an amplified temperature response over northern land areas, but this temperature change is not as pronounced as over the ocean.”

    http://www.the-cryosphere.net/3/11/2009/tc-3-11-2009.pdf

    Comiso 2006 has a good discussion on satellite observations of polar warming/amplification (though not that much on the seasonality issue):

    http://141.161.23.43/arctic.pdf

    On the DIY front, this NOAA page lets you create your own timeseries based on various data. I created this series based on NCEP reanalysis for latitudes north of 75. It clearly shows the amplified winter warming in the form of rapidly warming minima; the highs increase a bit too, but much, much less.

    http://tinyurl.com/yao8xv8

    Hmm, all this corrected information tastes a lot like crow. . . ;-)

  232. Ron R.:

    John P. Reisman, I atempted to contact you re: the pics but your contact page came back telling me that my email address is invalid. It’s not. Can you check that everything’s working right? Or if you want to leave your email address here I can contact you.

  233. alantrer:

    Re: # 156
    The context of Eric’s post was surface station temperature records (CRUTEMP3v). My comment was within that same context. I imply nothing with regard to radiosonde records.

    Re: # 166
    If it were a spelling error I feel certain my spell checker would have flagged it. Perhaps you meant semantics? I confess I don’t have one of those checkers. I’m clearly exposed to human error there.

    Re: # 167
    Reducing complexity of an argument reduces objections to it. In this case using the raw values would eliminate a large portion of the skeptics’ objections. As demonstrated, A’ as input achieves the same result (statistically) as A. If using A rather than A’ would reduce objections then it would make sense to use A.

    However this whole argument is moot. In # 194 Eric claims his post was not an attempt at a proof. It was simply a demonstration of how one could take a look at the data themselves. And in that respect there appears to be plenty of activity under way.

    [Response: A slight correction: Any thinking person can see they are wasting their time trying to get a different answer than I did. That’s the point of doing random sampling. The chances of getting a different answer than I did with the full data set is .01 percent. So, no, I didn’t prove anything, but I did demonstrate it with very very high confidence.–eric]

  234. Ron R.:

    I was wrong Steve Bloom #81. You’re right that NASA page has some good stuff on it, thanks.

    John, here’s a couple of direct links. I’d use the earliest and latest years available for comparison. BTW, they have larger sizes and animation available.

    On the Arctic:

    http://svs.gsfc.nasa.gov/vis/a000000/a003500/a003563/still_seaIce1979_0921.noGraph_web.png 1979
    http://svs.gsfc.nasa.gov/vis/a000000/a003500/a003563/print2008SeaIceEarthDatesSequence.1933_web.png 2008

    Wow!

    On the Greenland ice sheet melt:

    http://svs.gsfc.nasa.gov/vis/a000000/a003400/a003475/GreenlandMelt_less3.1979_web.png 1979
    http://svs.gsfc.nasa.gov/vis/a000000/a003400/a003475/GreenlandMelt_less3.2007_web.png 2007

  235. Silk:

    “What is the main difference between Set A and Set B?
    Because Set A shows exactly what many sceptics are saying:
    That the 1930:s was just as “hot” as present day. ”

    Not from my eyeballing.

    Some years in the 1930s were hot, but (eye-balling, because I don’t have the data) the last decade is clearly warmer on average than any previous decade.

    And eyeballing again, the trend is clearly upwards.

    Neither of which would please the sceptics.

  236. Ron R.:

    Five-Year Average Global Temperature Anomaly

    http://svs.gsfc.nasa.gov/vis/a000000/a003500/a003596/StillImages.1881_web.png 1881-1885

    http://svs.gsfc.nasa.gov/vis/a000000/a003500/a003596/StillImages.2004_web.png 2004-2008

  237. Snorbert Zangox:

    It does not look as if the inclusion of all stations changes the slope of the temperature/time curve for the last 30 years of the 20th century. I would like to see the UAH data for the same areas overlain on the curve that you linked. However, the inclusion of all stations for the full 145 years of temperature record would change the slope of the overall line significantly. (by about half if my eyeball estimate is close) This is because the full network data set shows higher temperatures for all years up until 1960.

    Also, keep in mind, if the contention of the Russians and others are correct, that CRU added temperature to the data during the first half of the 20th century, the slope of the overall line fall even more.

    I think that the approach of demonstrating that the putative changes did not happen by displaying selected data sets and by demonstrating that the differences are small will fail. I think that the only way to put this to rest is to return to the source data, display all of the code and repeat, publicly, derivation of the CRU adjusted data temperature set.

    [Response: If, when you say “this will fail” you mean that many people will continued to believe what is staring them in the face, yes, you are probably right. The question is whether it is worth anyone going through many years of work to find out something that is already known. Very few, if any scientists, are going to bother, since they know they are talking to a crowd of people that are apparently incapable of grasping the most basic statistics. If you mean that *you* don’t believe the results shown here address the issue, then I’m not sure what to say. Again: We did not show ‘selected’ data sets. We show randomly-selected data sets. There is a huge difference.–eric]

  238. ZZT:

    Many thanks for all the answers. Can anyone do a comparison of the A and B sets before the temperatures have been adjusted? If both sets show the same trend and statistical properties before adjustment, this would be quite convincing. If the adjustments are essentially random, they should introduce no net effect.

  239. Many Volcanos:

    I’m not a climate scientist but I do have some expertise with data. If I’m understanding this correctly climate scientists are trying to infer/predict future climate change based on only a few hundred years of sketchy data? Isn’t the earth 4 billion plus years old and has had weather basically since it’s had oceans? I’m not sure how any conclusions can be made with such a limited sampling. To say the Global Warming debate is concluded based on these findings seems premature…

  240. Kevan Hashemi:

    I don’t recall anybody is suggesting that CRU’s analysis of their own station data set is faulty. I’m a skeptic, and I applied a simpler version of their analysis to their data and obtained the same trend. So I’m not sure what criticism this article answers.

    Some people claim that CRU picked only sites that showed warming in Russia, rejecting 75% of the available stations. Other people claim that urban heating is a systematic error in the entire global set. My friends and I have been concerned about the dramatic changes in the number of weather stations available in the twentieth century, which you can see graphed against the CRU trend here. We even show that disappearing stations exhibit their own pronounced upward trend.

    We would very much like to hear your answers to these concerns about the CRU record. I’d like to see you show the plot of the number of weather stations versus anomaly and discuss with us why you believe the station number does not introduce a systematic error.

  241. Completely Fed Up:

    “based on only a few hundred years of sketchy data?”

    Why is this wrong?

    After all, engineering bases itself off the written record for engineering works rather than the 30,000 year record of human tool use (not to mention tool use by nonhumans!).

    Please justify calling it sketchy.

  242. eric:

    Comments are now closed.

    I get the last word.

    It really amazes me that some people asked “how do we trust your analysis of the CRU data that we don’t trust”? This is really grasping at straws, folks.

    But the point is: you don’t have to believe us. Go look at the data yourself.

    My favorite comment, #46:

    “What this shows, in my opinion, is that anyone who claims to have spent yeeeaaaars of his/her life studying the dark ways of the IPCC/NOAA/WMO/etc., and still cannot reproduce their results or still cannot understand/believe how the results were obtained, is full of sh#t…”

    Also, thanks to John Reismann for his excellent summary:

    http://www.ossfoundation.us/projects/environment/global-warming/myths/climategate