The entire CRUTEM3 database of station temperature measurements has just been released. This comes after a multi-year process to get permissions from individual National Weather Services to allow the passing on of data to third parties and from a ruling from the UK ICO. All the NWSs have now either agreed or not responded (except for Poland which specifically refused). Since the Polish data is a such a small fraction of the globe (and there are a few Polish stations in any case via RBSC or GCOS), this doesn’t make much difference to hemispheric means or regional climate. These permissions were obtained with help from the UK Met Office (who have also placed the station data on their website in a slightly different format) and whose FAQ is quite informative.
This dataset has occasionally come up in blogospheric discussions.
dhogaza: an awful lot of news outlets just take a press release and change a few words before publishing it. Some don’t even bother with changing a few words….
Others will paraphrase the release, but only the better journalists bother with research and getting actual quotes instead of the predigested press release quotes.
Of course, this sort of laziness pales in comparison to the in~cestuous “reporting” going on in certain parts of the right-wing press.
And that feeble excuse for a spam-filter is still being useless.
OK, so I downloaded the CRUTEM3 data, modified my “really dumb temperature anomaly gridding/averaging program” and ran the CRUTEM3 data through it.
Here is a quick-n-dirty plot of my results: http://img35.imageshack.us/img35/2210/mycrumyghcnnasaghcn.jpg
The plot shows my own GHCN (raw data) and CRUTEM3 results plotted against the official NASA “Meteorological Stations” results.
In case you can’t read the legend on your iWhatever, my own GHCN and CRUTEM3 results are plotted in dark blue and red, respectively. The NASA “Meteorological Stations” results (copied/pasted directly from NASA/GISS) are plotted in yellow.
Now I’m *all* confused — I thought that the temperature data that CRU had been hiding from us all this time was going to show massive *cooling*. ;)
Anyway, the results here are the output of a very straightforward temperature anomaly gridding/averaging procedure that a student intern could code up.
So what’s with the deniers out there? Certainly a few of them have the college-freshman programming skills needed to tackle a project like this.
So, when is one of them going to “step up to the plate”, generate his/her own global-temperature results from the CRUTEM data, and then apologize to Phil Jones?
RE: #9.
On Jones efforts to get the data released before climategate. Some more facts: In the july 2009 time frame when we raised the issue of the confidentiality agreements and CRU responded positively to our 50+ FOIAs by posting the agreements, Jones indicated that he was going to work to get the data released.
On nov 12th 2009, the draft letter to all the countries was exchanged (as an attachment) between thorne and Jones in the very last mail of the climategate stack.
My subsequent FOIA request for all CRU documents between July 2009 and Nov 2009 related to Jones efforts to get the data released, yielded No documents, except for the attachment to the Nov 12th email. The task of getting the countries to comply was apparently turned over to MET. That decision was documented as well in the ICOs recent investigation. Just to be clear on the facts. Jones, I think, did the right thing by turning the matter over to MET. So, to be exactly accurate I think we need to point out that Jones actually didn’t work to get the the data released, at least CRU could produce no documents from the time period July 2009 to Nov 2009, showing any effort. He apparently turned the matter over to MET who got the job done. That’s not a criticism. That just appears to be the way things were handled.
As for SteveMc, re doing the temperature series. That’s never been his interest in the data. That’s been my interest. As, I’ve said since 2007 when we started asking for the data, I do not expect to find anything ‘wrong’ with the data. But, I will interested to see how perfectly I can match their results. I’m very close just using standard GHCN data, but I want to see those last few wiggles disappear:
http://stevemosher.files.wordpress.com/2011/07/verification.png
For some of us this hasnt been about “wrong” data. It’s just been about access TO data. Had folks looked at actual requests, rather than the the blogs we choose to read, they might have found that some of us share a great many of their beliefs. We probably differ on the issue of access to data. I’ll suggest again as I have since 2007 that arguments against total transparency dont play well, and losing these fights is more costly than just complying with requests in the first place.
So …. how’s your temperature reconstruction with the available data going, Mosher?
That was of course a rhetorical question that everybody here already knows the answer to.
The thing is, Steven Mosher, Jones did it and followed through with the goods. Even if it was just to get the jackals off his back, the deed is done.
Now, if I had a whipround, do you think McIntyre would do a temperature series for once?
steven mosher – Out of interest, why didn’t you go directly to the relevant owners of the data to get hold of it?
Perhaps you did? If so, what response did you get?
> arguments against total transparency dont play well
“Information wants to be free marketed” works for Poland.
57 Hank. Good point. In Britain’s case the data isn’t paid for by the taxpayer, it’s paid for by sales of services and product. The UK Met Office is self-sufficient, the opposite to a taxpayer burden. But to do this they have to make profit and pay the owners dividends (the Ministry of Defence in the UKMO example). A libertarian’s dream. How ironic that (generally) it’s libertarians and neoliberals who queue up to condemn such agencies for not giving them something for free. Have they really thought any of this through?
J Bowers – The irony of the situation is highly striking, at least on a superficial level. It appears a group of overwhelmingly libertarian-leaning people have achieved their ends only because a Government regulatory body ruled by-fiat that a set of freely entered contracts should be ignored.
This seems slightly misleading to me. As I understand it, the ability to model ENSO has more to do with the spatial and temporal resolution of the model (and accuracy of the physics in the model over those scales). What climate sensitivity you end up with seems to have more to do with the assumed aerosol forcings, so it seems to me in principle you could have either a low sensitivity good fit to ENSO or a high sensitivity good fit to ENSO. [Maybe Gavin could correct me if my impressions are wrong on this.]
That said, I do think that looking at whether a model can produce realistic short-period climate fluctuations is a necessary prerequisite to comparing that model to short-period trends. Which brings up an OT comment: I think it’s misleading to compare the global mean of models to the global mean temperature… Not all models are created equally, and physics isn’t a democracy. It doesn’t matter what the average of the models say, it’s what the best models say (paraphrase of Dick Feynman there.)
This is great! Finally the station temperature measurements are available… Now all we have to do is convince Poland to fork over the information. Hopefully it won’t take too much longer; this information is gold.
steven mosher – Out of interest, why didn’t you go directly to the relevant owners of the data to get hold of it?
Perhaps you did? If so, what response did you get?
Comment by Paul S — 30 Jul 2011
#####
1. That would not have answered the question people wanted answered.
2. We had good reason to believe that the agreements referred to either did not exist or were being misrepresented.
3. we were told that some small number of nearly 200 countries refused access. We did the logical thing. Before bothering all 200 ( as cru had to do ) we asked CRU.. which countries exactly had a problem releasing data.
RE54:
“So …. how’s your temperature reconstruction with the available data going, Mosher?
That was of course a rhetorical question that everybody here already knows the answer to.”
#####
My own personal goal has never been a reconstruction. ( odd word to use here ) But I’ve done several emulations of CRU’s method. I’m very happy to say that I was able to match their results, nearly perfectly in some cases.A few emails back and forth and the last bits were squared away. If you thought I was an AGW skeptic well you are wrong. For me its more about putting tools and power in people’s hands. Pretty dang simple.
57
““Information wants to be free marketed” works for Poland.”
I think you’ve missed the argument entirely, Hank. The argument has been that there was no benefit to the science in using closed data. In fact, the whole argument was that you get the same answer with confidential data as you do without! Do you get it. That means there is no reason to use confidential data, no excuse for co mingling the two. My argument has been.
1. The science shows that you don’t need the small portion of data that is covered by confidential agreements.
2. If you insist on using this data, then you should take care to not comingle it with open data. That is, keep them separate.
3. Get a document control person in charge of requests for data.
Hank, people who opposed the release of the data have lost all the arguments. On the web, in your mind, you held your own. In the world, you lost. No shame in that.
[Response: There is shame in continuing to misrepresent the arguments and issues. The ‘people who oppose the release of data’ are the NWSs (and their governments – including the UK) who put commercial priorities ahead of openness, not CRU, and not Jones. – gavin]
Thanks for that, Mosher, we all knew that access to the data wasn’t the issue, but it’s good to see you put it in black and white. No surprise that nothing’s really been done with it other than to use this as a stick with which to beat the climate science community over the head.
Yes, I do – it’s called “revisionism”, and I think you’re wasting your time promoting it here.
64 Mosher — “Hank, people who opposed the release of the data have lost all the arguments.”
Not in Poland. Will you somehow force them to admit they’re wrong, or just leave all the hard and patient work to Phil Jones and UEA, while “liberators of data” don’t lift a finger to help, content to just point fingers?
Mosher asserts I’m one of the Polish/climatologist conspirators.
“There’s glory for you.”
Various groups in the private sector have long campaigned for meteorological measurements, especially for the UK and France, to be made available more cheaply, as in the US. It’s nothing particularly to do with climate change, but just that met data is useful for business in a variety of ways. Right now the data is so expensive that almost no-one buys it, which is a lose-lose for everyone.
Does anyone know if the ICO ruling changes anything? Will the UKMO and MeteoFrance still be charging an arm and a leg for their data?
69 Steve Jewson — “Does anyone know if the ICO ruling changes anything?”
Possibly not as the Met Office still pays dividends to its owner, the MoD (the level of dividends that means the MoD can buy more Apache helicopters and the like). You could always start a petition, but you’ll also be up against the Met Office being able to pay for itself through sales and services as a Trading Fund, thus becoming more of a burden on the taxpayer which flies in the face of Lib-Dem policies.
http://epetitions.direct.gov.uk/
Ding ding! Compare to Mosher’s statement above.
And Mosher (not surprisingly) hasn’t come back.
“1. The science shows that you don’t need the small portion of data that is covered by confidential agreements.”
You don’t need the data from Poland if all you care about it global mean temperature. But despite the blogospheric focus on the global mean*, these temperature datasets are used for more than that, and for some of those purposes (for examples, comparing Northern European temperatures against models), one _does_ want the data from Poland.
*One thing I can’t stand is the people who claim they can “duplicate GISS” by making a simple regression model that uses forcing inputs and temperature outputs and get r^2>0.9 over the historical record… for the global mean. I can’t stand their claims because a) it often turns out that their methods would _not_ work out of sample, even just for the global mean, and b) they often make snide comments about “why spend all this effort developing a complex model when I can duplicate it in a few lines of code”, totally ignoring the fact that global mean temperature is only one of a number of key outputs from the data…
#62, steven mosher – ‘1. That would not have answered the question people wanted answered.‘
What was the question? Was this not about gaining access to the data?
#64 – ‘Hank, people who opposed the release of the data have lost all the arguments.‘
Who is it you perceive was opposing release of the data, and what motive would they have for doing so?
this info should be available for all countries so we can work together on fixing it