RealClimate

Climate science from climate scientists...

You are here: Home / Archives for Gavin

about Gavin Schmidt

Gavin Schmidt is a climate modeler, working for NASA and with Columbia University.

The CRU hack: Context

by

This is a continuation of the last thread which is getting a little unwieldy. The emails cover a 13 year period in which many things happened, and very few people are up to speed on some of the long-buried issues. So to save some time, I’ve pulled a few bits out of the comment thread that shed some light on some of the context which is missing in some of the discussion of various emails.

  • Trenberth: You need to read his recent paper on quantifying the current changes in the Earth’s energy budget to realise why he is concerned about our inability currently to track small year-to-year variations in the radiative fluxes.
  • Wigley: The concern with sea surface temperatures in the 1940s stems from the paper by Thompson et al (2007) which identified a spurious discontinuity in ocean temperatures. The impact of this has not yet been fully corrected for in the HadSST data set, but people still want to assess what impact it might have on any work that used the original data.
  • Climate Research and peer-review: You should read about the issues from the editors (Claire Goodess, Hans von Storch) who resigned because of a breakdown of the peer review process at that journal, that came to light with the particularly egregious (and well-publicised) paper by Soon and Baliunas (2003). The publisher’s assessment is here.

Update: Pulling out some of the common points being raised in the comments.

  • HARRY_read_me.txt. This is a 4 year-long work log of Ian (Harry) Harris who was working to upgrade the documentation, metadata and databases associated with the legacy CRU TS 2.1 product, which is not the same as the HadCRUT data (see Mitchell and Jones, 2003 for details). The CSU TS 3.0 is available now (via ClimateExplorer for instance), and so presumably the database problems got fixed. Anyone who has ever worked on constructing a database from dozens of individual, sometimes contradictory and inconsistently formatted datasets will share his evident frustration with how tedious that can be.
  • “Redefine the peer-reviewed literature!” . Nobody actually gets to do that, and both papers discussed in that comment – McKitrick and Michaels (2004) and Kalnay and Cai (2003) were both cited and discussed in Chapter 2 of 3 the IPCC AR4 report. As an aside, neither has stood the test of time.
  • “Declines” in the MXD record. This decline was hidden written up in Nature in 1998 where the authors suggested not using the post 1960 data. Their actual programs (in IDL script), unsurprisingly warn against using post 1960 data. Added: Note that the ‘hide the decline’ comment was made in 1999 – 10 years ago, and has no connection whatsoever to more recent instrumental records.
  • CRU data accessibility. From the date of the first FOI request to CRU (in 2007), it has been made abundantly clear that the main impediment to releasing the whole CRU archive is the small % of it that was given to CRU on the understanding it wouldn’t be passed on to third parties. Those restrictions are in place because of the originating organisations (the various National Met. Services) around the world and are not CRU’s to break. As of Nov 13, the response to the umpteenth FOI request for the same data met with exactly the same response. This is an unfortunate situation, and pressure should be brought to bear on the National Met Services to release CRU from that obligation. It is not however the fault of CRU. The vast majority of the data in the HadCRU records is publicly available from GHCN (v2.mean.Z).
  • Suggestions that FOI-related material be deleted … are ill-advised even if not carried out. What is and is not responsive and deliverable to an FOI request is however a subject that it is very appropriate to discuss.
  • Fudge factors (update) IDL code in the some of the attached files calculates and applies an artificial ‘fudge factor’ to the MXD proxies to artificially eliminate the ‘divergence pattern’. This was done for a set of experiments reported in this submitted 2004 draft by Osborn and colleagues but which was never published. Section 4.3 explains the rationale very clearly which was to test the sensitivity of the calibration of the MXD proxies should the divergence end up being anthropogenic. It has nothing to do with any temperature record, has not been used in any published reconstruction and is not the source of any hockey stick blade anywhere.

Further update: This comment from Halldór Björnsson of the Icelandic Met. Service goes right to the heart of the accessibility issue:

Re: CRU data accessibility.

National Meteorological Services (NMSs) have different rules on data exchange. The World Meteorological Organization (WMO) organizes the exchange of “basic data”, i.e. data that are needed for weather forecasts. For details on these see WMO resolution number 40 (see http://bit.ly/8jOjX1).

This document acknowledges that WMO member states can place restrictions on the dissemination of data to third parties “for reasons such as national laws or costs of production”. These restrictions are only supposed to apply to commercial use, the research and education community is supposed to have free access to all the data.

Now, for researchers this sounds open and fine. In practice it hasn’t proved to be so.

Most NMSs also can distribute all sorts of data that are classified as “additional data and products”. Restrictions can be placed on these. These special data and products (which can range from regular weather data from a specific station to maps of rain intensity based on satellite and radar data). Many nations do place restrictions on such data (see link for additional data on above WMO-40 webpage for details).

The reasons for restricting access is often commercial, NMSs are often required by law to have substantial income from commercial sources, in other cases it can be for national security reasons, but in many cases (in my experience) the reasons simply seem to be “because we can”.

What has this got to do with CRU? The data that CRU needs for their data base comes from entities that restrict access to much of their data. And even better, since the UK has submitted an exception for additional data, some nations that otherwise would provide data without question will not provide data to the UK. I know this from experience, since my nation (Iceland) did send in such conditions and for years I had problem getting certain data from the US.

The ideal, that all data should be free and open is unfortunately not adhered to by a large portion of the meteorological community. Probably only a small portion of the CRU data is “locked” but the end effect is that all their data becomes closed. It is not their fault, and I am sure that they dislike them as much as any other researcher who has tried to get access to all data from stations in region X in country Y.

These restrictions end up by wasting resources and hurting everyone. The research community (CRU included) and the public are the victims. If you don’t like it, write to you NMSs and urge them to open all their data.

I can update (further) this if there is demand. Please let me know in the comments, which, as always, should be substantive, non-insulting and on topic.

Comments continue here.

It’s all about me (thane)!

by

Well, it’s not really all about me. But methane has figured strongly in a couple of stories recently and gets an apparently-larger-than-before shout-out in Al Gore’s new book as well. Since a part of the recent discussion is based on a paper I co-authored in Science, it is probably incumbent on me to provide a little context.

First off, these latest results are being strongly misrepresented in certain quarters. It should be obvious, but still bears emphasizing, that redistributing the historic forcings between various short-lived species and CH4 is mainly an accounting exercise and doesn’t impact the absolute effect attributed to CO2 (except for a tiny impact of fossil-derived CH4 on the fossil-derived CO2). The headlines that stated that our work shows a bigger role for CH4 should have made it clear that this is at the expense of other short-lived species, not CO2. Indeed, the attribution of historical forcings to CO2 that we made back in 2006 is basically the same as it is now.
[Read more…] about It’s all about me (thane)!

Muddying the peer-reviewed literature

by

We’ve often discussed the how’s and why’s of correcting incorrect information that is occasionally found in the peer-reviewed literature. There are multiple recent instances of heavily-promoted papers that contained fundamental flaws that were addressed both on blogs and in submitted comments or follow-up papers (e.g. McLean et al, Douglass et al., Schwartz). Each of those wasted a huge amount of everyone’s time, though there is usually some (small) payoff in terms of a clearer statement of the problems and lessons for subsequent work. However, in each of those cases, the papers were already “in press” by the time other people were aware of the problems.

What is the situation though when problems (of whatever seriousness) are pointed out at an earlier stage? For instance, when a paper has been accepted in principle but a final version has not been sent in and well before the proofs have been sent out? At that point it would seem to be incumbent on the authors to ensure that any errors are fixed before they have a chance to confuse or mislead a wider readership. Often in earlier times corrections and adjustments would have been made using the ‘Note added in proof’, but this is less used these days since it is so easy to fix electronic versions.
[Read more…] about Muddying the peer-reviewed literature

350

by

I was quoted by Andrew Revkin in the New York Times on Sunday in a piece about the 350.org International Day of Climate Action (involving events in 181 countries). The relevant bit is:

Gavin A. Schmidt, a climate scientist who works with Dr. Hansen and manages a popular blog on climate science, realclimate.org, said those promoting 350 or debating the number might be missing the point.
“The situation is analogous to people trying to embark on a cross-country road trip to California but they’ve started off heading to Maine instead,” Dr. Schmidt said. “But instead of working out ways to turn around, they have decided to argue about where they are going to park when they get to L.A.”
“If you ask a scientist how much more CO2 do you think we should add to the atmosphere, the answer is going to be none.”

I’ve been told that some readers may have misinterpreted the quote as a criticism of the 350.org campaign itself. This was not the intent and in fact my metaphor wouldn’t have made sense in that context at all. Instead, it was a criticism of people who are expending effort arguing about whether 350 is precisely the right number for a long term target, or whether it should be somewhat higher or lower. Since we aren’t currently headed anywhere near 350 ppmv (in fact we are at 388 ppmv CO2 and increasing by about 2 ppmv/yr), we need to urgently think of ways the situation can turn around. Tapping into the creativity and enthusiasm shown by the 350.org campaigners will certainly be part of that process.

We discussed some of the thinking behind this ‘Target CO2‘ when Jim Hansen and colleagues’ paper first came out, where I think we made it clear that picking a specific CO2 target to avoid ‘dangerous’ climate change is an inexact science at best. The comments by Robert Brulle and Ray Pierrehumbert at DotEarth and James Hrynyshyn also highlight some of that complexity. And I think the suggestions by ‘Paulina‘ for how a tweaked article might have been clearer are very apropos.

However, as the final line in my NYT quote should make clear, personally I think the scientific case not increasing CO2 any further is very strong. Since the planet has not caught up with current levels of concentrations emissions (and thus will continue to change), picking an ultimate target that is less than today’s level is therefore wise. Of course, how we get there is much trickier than knowing where it is we should be going, but having a map of the destination is useful. As we discussed in the ‘trillionth ton‘ posting a couple of months back, how we get there also makes a difference.

In my original email to Andy Revkin, I had actually appended a line:

If you ask a scientist how much more CO2 do you think we should add to the atmosphere, the answer is going to be none.

All the rest is economics.

(and technology, and sociology, and psychology and politics etc.) but the point is that working out how we get there from here is the real challenge and the more people who are aware and involved in developing those solutions the better.

Why Levitt and Dubner like geo-engineering and why they are wrong

by

Many commentators have already pointed out dozens of misquotes, misrepresentations and mistakes in the ‘Global Cooling’ chapter of the new book SuperFreakonomics by Ste[ph|v]ens Levitt and Dubner (see Joe Romm (parts I, II, III, IV, Stoat, Deltoid, UCS and Paul Krugman for details. Michael Tobis has a good piece on the difference between adaptation and geo-engineering). Unfortunately, Amazon has now turned off the ‘search inside’ function for this book, but you can read the relevant chapter for yourself here (via Brad DeLong). However, instead of simply listing errors already found by others, I’ll focus on why this chapter was possibly written in the first place. (For some background on geo-engineering, read our previous pieces: Climate Change methadone? and Geo-engineering in vogue, Also the Atlantic Monthly “Re-Engineering the Earth” article had a lot of quotes from our own Raypierre).
[Read more…] about Why Levitt and Dubner like geo-engineering and why they are wrong

Decadal predictions

by

There has been a lot of discussion about decadal climate predictions in recent months. It came up as part of the ‘climate services’ discussion and was alluded to in the rather confused New Scientist piece a couple of weeks ago. This is a relatively “hot” topic to be working on, exemplified by two initial high profile papers (Smith et al, 2007 and Keenlyside et al, 2008). Indeed, the specifications for the new simulations being set up for next IPCC report include a whole section for decadal simulations that many of the modelling groups will be responding to.
[Read more…] about Decadal predictions

Plimer’s homework assignment

by

Some of you may be aware of George Monbiot’s so-far-unsuccessful attempt to pin down Ian Plimer on his ridiculous compendium of non-science. In response to Monbiot’s request for explanation and sources for some of Plimer’s more bizarre claims, Plimer has responded with a homework assignment that is clearly beyond even his (claimed) prowess. This is quite transparently a device to avoid dealing with Monbiot’s questions and is designed to lead to an argument along the lines of “Monbiot can’t answer these questions and so knows nothing about the science (and by the way, please don’t notice that I can’t cite any sources for my nonsense or even acknowledge that I can’t answer these questions either)”. (Chris Colose and Greenfyre have made similar points). It’s also worth pointing out as Andrew Dodds has done that each question is actually referencing a very well known contrarian and oft-debunked argument, but dressed up in pseudo-scientific complexity.

However, as a service both to Plimer and Monbiot (as well as anyone else interested), we give a quick scorecard on the relevance, actual scientific content (whether the questions can actually be answered) and sources for discussion for each of the, to be charitable, ‘odd’ questions. For relevance, we grade each question on a scale from 0 to 5, 0 being irrelevant to the issue of detection and attribution of 20th Century climate change, 5 being extremely relevant. For scientific content, we rate the reasonableness of the question posed (i.e. does it make any sense at all), from A to F (A being a very well posed question, F making no sense). For sources, we generally point to a paper or discussion that addresses the real issue.
[Read more…] about Plimer’s homework assignment

Sea ice minimum forecasts

by

One of the interesting things about being a scientist is seeing how unexpected observations can galvanize the community into looking at a problem in a different way than before. A good example of this is the unexpectedly low Arctic sea ice minimum in 2007 and the near-repeat in 2008. What was unexpected was not the long term decline of summer ice (this has long been a robust prediction), but the size of 2007 and 2008 decreases which were much larger than any model had hinted at. This model-data mismatch raises a number of obvious questions – were the data reliable? are the models missing some key physics? is the comparison being done appropriately? – and some less obvious ones – to what extent is the summer sea ice minimum even predictable? what is the role of pre-conditioning from the previous year vs. the stochastic nature of the weather patterns in any particular summer?

The concentration of polar expertise on the last couple of questions has increased enormously in the last couple of years, and the summer minimum of 2009 will be a good test of some of the ideas that are being discussed. The point is that whether 2009 is or is not a record-setting or near-record setting minimum, the science behind what happens is going to be a lot more interesting than the September headline.

[Read more…] about Sea ice minimum forecasts

Science at the bleating edge

by

Many of you will remember our pioneering discussion of the Sheep-albedo hypothesis, where we suggested that increasing numbers of sheep would cause a cooling climate because of their impact on the ground reflectivity. Well, new observations have now added significantly more complexity to this seemingly understood situation. It has just been reported that as well as the original sheep-albedo effect, there is now evidence of a second sheep-albedo effect (smaller sheep as the world warms). That would be a amplifying effect in general, but the actual population of sheep concerned come in many colours, so the overall affect is uncertain. (There is even speculation about a third effect (the sheep lifetime albedo impact), which could go either way depending the Lamb Marketing Board advertising campaign).

Yet more evidence that the science is very woolly.

*Title taken from a comment from glen