1934 and all that
Another week, another ado over nothing.
Last Saturday, Steve McIntyre wrote an email to NASA GISS pointing out that for some North American stations in the GISTEMP analysis, there was an odd jump in going from 1999 to 2000. On Monday, the people who work on the temperature analysis (not me), looked into it and found that this coincided with the switch between two sources of US temperature data. There had been a faulty assumption that these two sources matched, but that turned out not to be the case. There were in fact a number of small offsets (of both sign) between the same stations in the two different data sets. The obvious fix was to make an adjustment based on a period of overlap so that these offsets disappear.
This was duly done by Tuesday, an email thanking McIntyre was sent and the data analysis (which had been due in any case for the processing of the July numbers) was updated accordingly along with an acknowledgment to McIntyre and update of the methodology.
The net effect of the change was to reduce mean US anomalies by about 0.15 ºC for the years 2000-2006. There were some very minor knock on effects in earlier years due to the GISTEMP adjustments for rural vs. urban trends. In the global or hemispheric mean, the differences were imperceptible (since the US is only a small fraction of the global area).
There were however some very minor re-arrangements in the various rankings (see data). Specifically, where 1998 (1.24 ºC anomaly compared to 1951-1980) had previously just beaten out 1934 (1.23 ºC) for the top US year, it now just misses: 1934 1.25ºC vs. 1998 1.23ºC. None of these differences are statistically significant. Indeed in the 2001 paper describing the GISTEMP methodology (which was prior to this particularly error being introduced), it says:
The U.S. annual (January-December) mean temperature is slightly warmer in 1934 than in 1998 in the GISS analysis (Plate 6). This contrasts with the USHCN data, which has 1998 as the warmest year in the century. In both cases the difference between 1934 and 1998 mean temperatures is a few hundredths of a degree. The main reason that 1998 is relatively cooler in the GISS analysis is its larger adjustment for urban warming. In comparing temperatures of years separated by 60 or 70 years the uncertainties in various adjustments (urban warming, station history adjustments, etc.) lead to an uncertainty of at least 0.1°C. Thus it is not possible to declare a record U.S. temperature with confidence until a result is obtained that exceeds the temperature of 1934 by more than 0.1°C.
More importantly for climate purposes, the longer term US averages have not changed rank. 2002-2006 (at 0.66 ºC) is still warmer than 1930-1934 (0.63 ºC - the largest value in the early part of the century) (though both are below 1998-2002 at 0.79 ºC). (The previous version - up to 2005 - can be seen here).
In the global mean, 2005 remains the warmest (as in the NCDC analysis). CRU has 1998 as the warmest year but there are differences in methodology, particularly concerning the Arctic (extrapolated in GISTEMP, not included in CRU) which is a big part of recent global warmth. No recent IPCC statements or conclusions are affected in the slightest.
Sum total of this change? A couple of hundredths of degrees in the US rankings and no change in anything that could be considered climatically important (specifically long term trends).
However, there is clearly a latent and deeply felt wish in some sectors for the whole problem of global warming to be reduced to a statistical quirk or a mistake. This led to some truly death-defying leaping to conclusions when this issue hit the blogosphere. One of the worst examples (but there are others) was the ‘Opinionator’ at the New York Times (oh dear). He managed to confuse the global means with the continental US numbers, he made up a story about McIntyre having ‘always puzzled about some gaps’ (what?) , declared the the error had ‘played havoc’ with the numbers, and quoted another blogger saying that the ‘astounding’ numbers had been ’silently released’. None of these statements are true. Among other incorrect stories going around are that the mistake was due to a Y2K bug or that this had something to do with photographing weather stations. Again, simply false.
But hey, maybe the Arctic will get the memo.
10 August 2007 at 6:03 PM
I’ve been perusing some of the skeptic blogs where there is much air-punching and back-slapping. The predominant theme, particularly in the reader comments, is that this new adjustment means global warming hasn’t actually been happening these last 30 years, it was all a NASA glitch. Or to loosely paraphrase Bob Carter, “global warming stopped in 1934″.
They don’t seem to realise the global trend over the last 30 years still shows dramatic warming, both in the US and especially globally. In fact, the trend seems to me a more important statistic than the “top ten warmest years”. What I would be interested in seeing is the change in the global warming trend since 1975. It was around .17C per decade - does anyone have exact figures from before and after?
10 August 2007 at 6:07 PM
It isn’t “much ado about nothing”, it’s an embarrassment. I don’t usually agree with you, but I have always considered you to be a credible and honestly contrary opinion until you made that statement.
Do you think we can now be “99% certain” that 1938 was the warmest year in the last 1000 years, or are we still 99% certain that it was 1998. How certain are we of ANYTHING that Hansen says, now?
The problem with most skeptics isn’t that we don’t have open minds… we do. I know I do at least. The problem is the grotesque hubris we associate with scientists who are championing scientific conclusions that we intuitively know to be MUCH shakier than they are letting on. This only reinforces that.
[Response: Sure it’s embarrassing, but only the end result determines whether it matters or not. This doesn’t for anything important. Would you characterise the blogosphere reaction as proportionate to the 0.03 deg C shift between 1934 and 1998 in the US temps? Perhaps you’d like to point me to any statements that said anything about being 99% that 1998 was the warmest year in the US? Read the Hansen quote above - written in 2001! -gavin]
10 August 2007 at 6:07 PM
They don’t seem to realise the global trend over the last 30 years still shows dramatic warming, both in the US and especially globally. In fact, the trend seems to me a more important statistic than the “top ten warmest years”. What I would be interested in seeing is the change in the global warming trend since 1975. It was around .17C per decade - does anyone have exact figures from before and after?
Yes. 1940-1975 the temperature was falling whilst CO2 was increasing.
Apparently its due to pollution, however, now the evidence is that pollution causes the temperature to rise.
Its been a bad week for AGW.
10 August 2007 at 6:18 PM
Actually to be fair, your statement that “the people who work on the temperature analysis (not me), looked into it and found that this coincided with the switch between two sources of US temperature data.” is incorrect. Steve M pointed out where the error came from in his blog posts and his email notifying GISS of the problem. The GISS people simply confirmed that he was correct.
[Response: Not so. He saw the jump but did not speculate as to the cause. - gavin]
10 August 2007 at 6:22 PM
John:
I have been monitoring the same blogs. Clearly the magnitude of the correction is of no great significance to the overall trend. Some on those sites have become inappropriately elated. But I seriously dispute your assertion that this means the majority of regular contributors believe that this proves global warming isn’t happening. Certainly that was not Steve McIntyre’s position. Where it is dramatically important is in the arguments over access to data and the ability to replicate methods. In this instance, GISS’s error provided Steve McIntyre with a proverbial “smoking gun”.
Frankly, my hope is that this “ado” does in fact lead to a dramatic increase in openness and access to data and statistical techniques and code used to analyze climate data.
10 August 2007 at 6:29 PM
Here is a back-of-envelope calculation of the effect of this on world temp analysis. I’ve posted this a few places now.
It is an 0.3% change to world temp anomaly results after 2000.
0.003C.
The error was only in data for the lower 48 states, and was 0.15C for that data. The lower 48 is about 2% of the earth’s surface. 0.15 x .02 = 0.003C
Global temp change over the last century is 0.8 - 1.1C depending on method. .003C out of 1C (in the range and easy to calculate) is 0.3%
10 August 2007 at 6:30 PM
Gavin: It would be helpful if both graphs used the same Y-axis scale.
10 August 2007 at 6:50 PM
Perhaps you’d like to point me to any statements that said anything about being 99% that 1998 was the warmest year in the US?
I apologize. I appear to have mixed-up my climatologists. I think it was one of your co-bloggers who made that statement.
It should also be noted that most people fully understand that this is US-only data. The manner in which that is pointed out, however, is characteristic of the same sort of “hubris” I mentioned earlier… it’s somewhat dismissive and something along the lines of “This is US only, though. The rest of the world’s data is still almost as good as we thought the US data was a couple of days ago.”
A more healthily skeptical reply would note that it is, indeed, US only, and only affects global temperatures slightly, but, given the fact that the US has the most reliable and well-maintained network, it raises concerns about the quality of data we have been using across the board.
On a related note… is it true that the “margin of error” for the “global surface temperature” is actually larger than the net warming in the 20th century? I’ve seen that claim thrown around and figured it wouldn’t hurt to ask.
10 August 2007 at 6:53 PM
Re: #3 (Nick)
When you say
you are mistaken. As I have pointed out before on this blog, there is no justification for such a claim.
The most revealing aspect of the AGW debate is this: when the “warmers” make a claim, it’s based on serious research and considerable effort, and if an error is found, it’s admitted and corrected ASAP. When “deniers” make a claim (like yours), it’s based on the lack of serious research or considerable effort, and if an error is pointed out, it’s excused away or triggers a scrambling attempt to change the subject.
10 August 2007 at 7:00 PM
As I have pointed out before on this blog, there is no justification for such a claim.
Perhaps he would justify it with the 2 charts in this post.
10 August 2007 at 7:07 PM
Gavin, a query re your link to the global mean (http://data.giss.nasa.gov/gistemp/graphs/Fig.A2.txt). I downloaded that data a few months ago and comparing it to now, the values after 2000 are exactly the same. Is this because the change to global T is smaller than two decimal places? Or has that data not been updated yet?
Bjc, you’re right that most of the blogs themselves aren’t saying global warming isn’t happening, it’s more happening in the readers’ comments. Overall, I found the emphasis is on comparing 1934 to other top ten’s - very little to no mention of the change to the global trend.
[Response: Changes are too small to show. - gavin]
10 August 2007 at 7:28 PM
I think it’s also worth considering that many people have taken this to indicate that any future corrections will, like this one, indicate a lesser trend in 20th century warming (at any locale or globally). But this is clear gambler’s fallacy: You can’t point to the roller and extrapolate what they’ll roll based on what they have rolled.
Worth considering also the seeming desperation of some to associate “audits” with showing that warming is less than we thought. “Audits” can easily show that warming has been more than we thought.
10 August 2007 at 7:34 PM
Run for the ICE. I find this a common meme on the AGW side. When the land record gets a little quake the instinct is to run for the ICE. Actually, it’s pictures of ice, and as we have been told pictures have no value. unless, that is, they are pictures of “nighlights” used by hansen or pictures of Ice, preferably with polar bears, used by Gore.
Consider this. If one makes an error about data input files ( the first thing you check in IV&V) it is not advisable to run to a slippery surface. The artic ice record prbably needs a good audit as well.
10 August 2007 at 7:34 PM
The positive result of this brouhaha: an army of people who up until this week dismissed the temperature graphs, now not only embrace them but embtace them to a point-to-point accuracy on the scale of 1/100th of a degree.
10 August 2007 at 7:44 PM
Just to expand on my last point at 12, (somehow) many see this as indicating that we can’t trust anyone (especially Hansen) to handle the data properly. In fact, if a slight error - the possibility of which already indicated by Hansen 6 years ago - causes us to become Cartesian skeptics about him, then why didn’t the small detections and corrections in the computer code used in models, which are publicly indicated by NASA?
My mother slightly overcooked the garlic bread in the oven tonight; I guess I can never be sure whether she will make the best prepared food for my friends when they come over. How can I know what will happen????!!!
J
10 August 2007 at 7:58 PM
spilgard,
I concur. This whole debacle reminds me of a chess game I had last night with my significant other (gotta love him). We’re both pretty good players, and at first he was winning (took my knight AND bishop; I had no hits), but eventually the tables turned. Near the end he took one of my pieces and declared: “Aha! I’ve got your piece!” to which I replied, “Honey, you used one of your last pieces to knock one of my many; I’m winning”
S. Mosher (comment 13) says AGW’s (I hate polarization) run to the ICE, but no - we simply remind, everyone, and with a little alarm, that the trend is clear.
10 August 2007 at 8:03 PM
The argument from “denialists” has been: We’re not sure about the data because there may be abnormal heating do to the artificial environments close to the temperature stations.
Here we have had literally billions of dollars put into research of climate data, and the only person who catches a glaring error is an outsider.
What if the change do to asphalt or AC units had been more gradual? Should we have any confidence that anyone will be able to, much less want to, find the error?
If we can only detect the anomalies when there is an obvious spike in the data, and the only people that make corrections are “denialist”, doesn’t that shed ANY doubt in your mind on the integrity of the process?
Any at all?
even like . . . one Iota?
[Response: First lesson: don’t believe your own propaganda. - gavin]
10 August 2007 at 8:09 PM
RE 14. we never dismissed the temperature graphs.
1. Believers: accepted them without question.
2. Deniers: ignored them without cause.
3. Doubters: put questions to them.
The doubters were right.
The deniers, like a stopped clock, got lucky.
The believers, ran for the ice.
It strikes me that the belief that Ice melts only because it gets warm is somewhat simplistic. So gavin,
perhaps we could benefit from a lesson in arctic ice formation and decay and the various causes– wind stress, temperature, albedo, soot, salinity, wave roughness, leads, pressure ridges, etc etc etc.
Now I know that shrinking ice has the same propaganda impact as Air conditioners by weather stations, but a little more depth on the Ice issue might be a good idea,
thin ice metaphors and all that.
Oh, Which stations do you use to measure arctic temp?
[Response: Leave the ‘believers’ and ‘doubter’ rhetoric at home. No scientist believes anything in the sense you claim. Nor as I made abundantly plain two weeks ago is anyone who works with real data blind to the problems there are. The ice isn’t just good propagand a, it’s a hard nosed fact that makes fooling around with AC theories of climate change moot. Arctic temperatures are directly measured by the Arctic Buoy program, but in the GISS analysis, the temperatures are extrapolated from nearby land stations. That works quite well, but it would be better to assimilate the buoy data directly - there’s been some discussion I think, but I don’t know what the status is. - gavin]
10 August 2007 at 8:20 PM
steven mosher (#13) wrote:
Well, I think it makes a little more sense for supporters of science to point at the arctic ice as this is something people are likely to understand than it does for climate “skeptics” to point at the Dust Bowl and Great Depression of the 1930s. Do skeptics really want the American public thinking in those terms….?
Don’t worry - I suspect you will be getting a fair amount of data in the next month and a half. Pictures, charts, commentary… the whole nine yards.
As for the global climate, expect a little breather until 2009….
Earth will feel the heat from 2009: climate boffins
By Lucy Sherriff
10th August 2007 15:31 GMT
http://www.theregister.co.uk/2007/08/10/climate_model/
It appears that climatologists are in the process of improving their short-range forecasting.
10 August 2007 at 8:27 PM
There was a bunch of chatter on Slashdot regarding this. I felt the need to point out that there were so many other disparate sources of evidence for global warming that there was really no question, even if said glitch actually was significant:
1) Scientists are witnessing ice shelves in Antarctica falling into the sea.
2) The North Pole is melting so that there will soon be a North-West Passage to which Canada is laying claims.
3) Much of the global warming data does not come from NASA.
4) Ski areas in the Alps are soon to be going out of business.
5) There is glacial melting everywhere.
.
6) Indonesia’s islands are being submerged by rising sea level
I’m sure this group can come up with many more proofs that are independent of the problematic data.
10 August 2007 at 8:29 PM
RE 16. remind me the trend is clear? Clear?
Clear +- a little fuzzy. How clear? Crystal clear?
foggy windshield clear? Tinted glass clear? Leaden glass clear? clear? Trends are Positive or Negative.
Trends have slopes. Those slopes have errors.
Now, I never denied the trend. I question the trend. As I have pointed out on CA I am a confirmational Holist.
Simply, all theory is underdetermined by data. All observation is theory laden. Falsification can always be avoided by appealing to other data ( sea ice, SST, species migration, etc etc etc).
Acceptance of theory has an epistemic component (”fits” the “data”) a pragmatic component ( makes useful predictions) and a social component ( the dreaded consensus)
What I noted is a meme. When the land record is attacked, believers tend to run for the ice. Primarily because it serves as a common sense reference point.
Folk wisdom. Ice melts when it gets warm. It’s rooted in common experience, but I suspect it’s more complicated that the cubes in my G&T.
10 August 2007 at 8:40 PM
RE 18 gavin.
I will gladly leave the believer /denier rhetoric at home. Funny, I have never seen you inline a similar chastisment to people who throw the denier label around.
I think it is an instructive distinction relative to the issue of confirmation bias. Be that as it may. I count you therefore in the doubter crowd:neither as a believer in AGW or a disbeliever. A true skeptic.
Shrugs. ( that is the skeptic handshake)
Which artic land stations?
10 August 2007 at 8:42 PM
12: Worth considering also the seeming desperation of some to associate “audits” with showing that warming is less than we thought. “Audits” can easily show that warming has been more than we thought.
So let’s do them and find out. Is it too hot, too cold or is my poridge just right? This has passed the point of being scientific data. It has become financial data. If someone wants to start taxing me and interfering with my economy based on these data and models, it’s time for full disclosure and accurate accounting.
http://www.giss.nasa.gov/research/briefs/hansen_07/
“…Figures 1 and 2, makes clear that climate trends have been fundamentally different in the U.S. than in the world as a whole…”
Maybe. Or maybe they make it clear that there’s even more of a problem with the data in the rest of the world. Perhaps if we had quality data for the rest of the world, the trends wouldn’t be so fundamentally different?
If you want me to believe in catastrophic global warming, you’ve got to convince me it’s not just a case of Garbage In Garbage Out. So far, I’m not convinced. I’m less convinced today than I was last week.
We need to be confident in the data to believe the trends you show us in the nice pretty pictures. Between this error and photos at surfacestations.org, I’m not confident in the US data, let alone those from the rest of the world. Peer review is for science. The IPCC can do all the peer review they want. That doesn’t cut it for economies. Economies use accounting and audits. Failure to do so leads to situations like Enron and WorldComm. If we just want to do science, stick with peer review. If someone wants to influence economies, it’s long past time for auditors to start poking around.
If NASA doesn’t want to publish source code and all data, at a minimum maybe they should hire an accounting firm to audit climate related data, methods and processes and to issue a public report on the quality they find (and that firm should hire Steve McIntyre ; -). I’m not saying I’d trust that report as much as full and open disclosure, but it would be a start. Both a formal audit and full disclosure would be fantastic!
Oh. And changes to published data should be version controlled, with something akin to release notes.
Opaque or unaudited data should not find its way into the policy debates. Ever.
10 August 2007 at 8:46 PM
Just to expand on my last point at 12, (somehow) many see this as indicating that we can’t trust anyone (especially Hansen) to handle the data properly.
I think the point is that we shouldn’t have to trust someone as in a single person or entity such as Nasa GISS to develope what is in effect policy for our country.
It’s un democratic and un scientific.
A person could make a mistake. Ahem.
Imagine if we were to use only one thermometer to measure the global temp, say Houston.
[Response: Presumably in that case you’re happy that other people are producing global temperature records, for example CRU - William]
10 August 2007 at 8:52 PM
It’s interesting that bit about the Arctic ice retreating to it’s furthest extent already.
We better get some ice cores from that unmelted portion before that record is lost forever. Who knows what revelations it will add to Antactic, Greenland, and assorted glacial ice series. - oh wait we are talking about Arctic ice. Sneaky bit of misdirection by you guys.
10 August 2007 at 8:57 PM
I am wondering if the ocean is rising and flooding out Indonesia, why isn’t it flooding Cape Cod? are they not all at sea level?
10 August 2007 at 8:59 PM
RE 18. one last thing. Gavin, the “Ice” is not a hard nosed fact. The “ice” is observed with an instrument.
Records are collected and processed by code written by humans, subject to error and audit. Not facts. Not raw sensory input. Not facts, interpretations of data collected by instruments, processed by code. This is different from the stuff in my Gin & Tonic. I have not looked at the chain of custody for that “data”. I reserve judgement. The present record of managing data and the lack of outside audit does not inspire confidence. It deepens the need for systematic doubt.
There are issues with the land instrument record, even though the summer heat is a hard nosed fact. And the SST record has instrument issues ( buckets, buoys, didnt you write up something on a booboo made in a recent paper that thought the sea was cooling)
Anyway, as you know from spending a considerable time futzing with data and anomalies and adjustments and renormalizations…
All observations are theory laden.
[Response: Of course. But just like (I think) Tuzo Wilson said “Nothing in geology makes sense except in terms of plate tectonics”, nothing makes sense in current climate change without anthropogenic factors. That’s not one dataset here or one anecdote there. The body of work that demonstrates consistency with the mainstream understanding is huge. And these little hiccups don’t come anywhere close to affecting it. To deny that…. well…. - gavin]
10 August 2007 at 9:03 PM
There is a term in accounting called “immaterial”.
I think we could learn to use it again.
J
10 August 2007 at 9:05 PM
Earth will feel the heat from 2009: climate boffins
By Lucy Sherriff
10th August 2007 15:31 GMT
http://www.theregister.co.uk/2007/08/10/climate_model/
It appears that climatologists are in the process of improving their short-range forecasting.
Interesting. It seems to be inproved just enough to cover the time period right after the next election.
You sure this is a non political website?
10 August 2007 at 9:07 PM
re 25. here tiger. from wiki.
There are two aspects of confirmation holism. The first is that observations are dependent on theory (sometimes called theory-laden). Before accepting the telescopic observations one must look into the optics of the telescope, the way the mount is constructed in order to ensure that the telescope is pointing in the right direction, and that light travels through space in a straight line (which itself is sometimes not so, as Einstein demonstrated). The second is that evidence alone is insufficient to determine which theory is correct. Each of the alternatives above might have been correct, but only one was in the end accepted.
That theories can only be tested as they relate to other theories implies that one can always claim that test results that seem to refute a favoured scientific theory have not refuted that theory at all. Rather, one can claim that the test results conflict with predictions because some other theory is false or unrecognised. Maybe the test equipment was out of alignment because the cleaning lady bumped into it the previous night. Or, maybe, there is dark matter in the universe that accounts for the strange motions of some galaxies.
That one cannot unambiguously determine which theory is refuted by unexpected data means that scientists must use judgments about which theories to accept and which to reject. Logic alone does not guide such decisions.
10 August 2007 at 9:08 PM
29.
It would probably be best to stay out of that discussion. The same thing could easily be said about many other locuters in this discussion, even the “auditors” themselves.
If you do nothing else, interpret the arguments in their bets light.
J
10 August 2007 at 9:15 PM
24.
I agree that we shouldn’t uncritically agree with just one source of scientific analysis. Yet even if we relied solely on NASA, NASA GISS already agreed with McIntyre even before McIntyre decided to audit climate (i.e. before his blog) - in 2001.
Of course, there are other issues where they obviously disagree, but Gavin was quite clear and open about this - certainly not the tactics of a dogmatist that invites you to rely on NASA’s word alone. I think we ought to appreciate that.
J
10 August 2007 at 9:35 PM
Re 27.
Nothing comes close, YET. The point is science is always contigent. Wilson is merely stating this. If we gave up plate tectonics, we would have to redo a bunch of work. It is pragmatically valorized not epistemically more secure. As Quine noted no theory faces facts in isolation. This is why no single fact overturns accepted theory. There is always a balancing act. Facts can be ignored, epicycles created, results questioned. It is a huge undertaking to overturn a theory, and with no viable replacement, highly unlikely.
All the more reason to free the code
No ne denies the consistency of the body of evidence. That is the whole point of theory laden observation and confirmation bias. Surely you have encounter some folks who never saw a negative feedback they liked? Shrugs
10 August 2007 at 9:41 PM
#20, etc on Indonesia. Please don’t just quote lead sentences — too often, as in this case, they are attention getters with zero substance.
The article quoted in no way claims that Indonesia has lost ANY islands due to sea rise.
At least the substance of the article explains what has actually been happening: “The issue has become a hot topic after Indonesia upset neighbouring Singapore recently by banning sand exports to the city state, blaming sand mining for literally wiping some of its islands off the map.”
Why wasn’t the lead about sand exports? Because that’s not an exciting topic like fear of being inundated by ever-rising sea levels.
10 August 2007 at 9:50 PM
Did not Gore say that 9 of the warmest years were in the last decade? The adjustment now indicates that 4 of the warmest years were in the 1930’s and 3 of the warmest were in the last decade or so(for the USA). And you are saying much ado about nothing.
[Response: Global means! And there nothing has changed.- gavin]
10 August 2007 at 10:06 PM
Re: 33,
“It is pragmatically valorized not epistemically more secure.”
No, Wilson said that it doesn’t MAKE SENSE without plate tectonics, not that it would be inconvenient without it. Those are two different claims. Likewise, evolution by natural selection of genes is the only thing that thus far can explain biological change in the distant past, even if we don’t quite understand the gory details of that past.
J
10 August 2007 at 10:21 PM
I agree with Gavin that calling people “warmers” or “deniers” is not productive. What we now know is that there was an error in the GISS record and the error was discovered by Steve McIntyre. GISS was notified and, to their credit, they investigated, found the error and corrected the reocrd. The real question is whether or not there are more errors in the record. My instint tells me that GISS is looking at the data to insure that there are no more errors. IMHO, it would be better if GISS released all their raw data, the rational and code for any and all adjustments, and any other data that is revalant to the the temprature record. This would allow everyone to inspect the data. If there are additional errors, they could be found and corrected. By releasing this data, the confidence in the record could be improved.
10 August 2007 at 10:22 PM
Re #29: The Met Office news release says nothing about the model covering only the time after the next general election (likely to be in 2010), but perhaps there is something in the text of the article (which I haven’t read) linking the model to the British election cycle. Gavin knows more about both of these subjects than I, so perhaps he can comment.
10 August 2007 at 10:24 PM
The last election was in 2005, so the next might be in 2009, but it could be called anywhere from 2007 to 2010, whenever Gordon Brown or his government decide suits them.
I assume you are talking about UK politics after all, as it is a UK site being linked, and the research quoted was done by the Met Office in the UK.
10 August 2007 at 10:36 PM
Does the CRU correct for the urban heat island effect as the GISS does?
Congrats to the GISS in correcting discovered errors.
10 August 2007 at 10:44 PM
I have a related question one the web page it now states:
“Our analysis, summarized in Figure 1 above, uses documented procedures for data over land (1), satellite measurements of sea surface temperature since 1982 (2), and a ship-based analysis for earlier years (3). Our estimated error (2σ, 95% confidence) in comparing nearby years, such as 1998 and 2005, increases from 0.05°C in recent years to 0.1°C at the beginning of the 20th century. Error sources include incomplete station coverage, quantified by sampling a model-generated data set with realistic variability at actual station locations, and partly subjective estimates of data quality problems (4).”
I have never seen a “partly” subjective estimate of data quality issues. How does one integrate this with
objective methods and report a Std.dev.?
and how much of the 2sig band is objective and how much is partly subjective and how was this CI constructed and was this partly objective partly subjective method tested?
Which individuals subjective judgement was used to assess the “data quality” issues and what repeatable methodolgy did they use? Did they employ a rating system?
is it documented? were the individuals who did the subjective assesment instructed in the tendency of subjective assesments to regress to the mean. If more than one individual was used to subjectively asses data quality did those individuals go through a normalization process?
Does partly subjective mean guess?
That page is a treasure trove. Keep digging, then we get to the china station issues.
10 August 2007 at 11:01 PM
Sigh.
If the aforementioned skeptics had taken the time (2 minutes?) to plot both datasets in Excel, they would have found that there is little to celebrate. I guess that this speaks to the immensity of their intellectual dishonesty.
Old CONUS Data:
http://uploader.ws/upload/200708/OLD_CONUS.gif
New CONUS Data:
http://uploader.ws/upload/200708/NEW_CONUS.gif
10 August 2007 at 11:02 PM
The finding seems similar to “one small step for a man, one giant leap for mankind” - but instead, its “one small temperature change but one giant argument for openness and transparency” in data collection and selection methods, sites selected, algorithms and source code availability and so on. Where the goal from all parties is seeking the truth, an open source approach will be helpful in negating both pro- or con- arguments on AGW. Unfortunately, as ClimateAudit has documented, a number of climate researchers have maintained their data, methodologies, algorithms and source code are private. And hence, not tested, vetted nor reproducable by others. That NASA’s GISS had a Y2K error in its data analysis is a powerful argument that secret science is bad regardless of the field or the source.
[Response: What is secret? The algorithms, issues and choices are outlined in excruciating detail in the relevant papers: http://data.giss.nasa.gov/gistemp/references.html - gavin]
10 August 2007 at 11:11 PM
My mother slightly overcooked the garlic bread in the oven tonight; I guess I can never be sure whether she will make the best prepared food for my friends when they come over.
Some people might argue that cooking garlic bread is more of an art than a science.
Some people might argue that adjusting the surface record to account for UHI, etc., is more of an art than a science.
Etc.
10 August 2007 at 11:16 PM
Gavin, would you mind clearing up my question from earlier? Is it true that the margin of error in the calculation of “global surface temperature” is greater than the observed warming in the last 100 years?
That is a point that I keep seeing. Is it true?
[Response: No. The error on the global mean anomaly is estimated to be around 0.1 deg C now, increasing slighty before 1950 or so and a little larger in the 1880s. The global mean changes of around 0.7 deg C and significantly higher. - gavin]
10 August 2007 at 11:21 PM
A minor correction to my previous post (#34): I was unaware of the blog that gavin linked to. I had in mind several blogs listed on digg.com that have been receiving the bulk of the attention today. None of those blogs had graphs of the actual CONUS data.
But now that I know that (at least) one blog went through the trouble of plotting the data in Excel… I’m even more stunned. How desperate does one have to be in order to cling on to such trivial differences?
10 August 2007 at 11:32 PM
Perhaps this would be a good time to remind some people (and I’m sorry if I bore the rest of you by harping on this) that AGW is a prediction, not an explanation. It wasn’t derived by examining past temperature records, noting a rise, and picking on CO2 as the cause. It’s based on the measured increase in atmospheric CO2, combined with knowledge of its properties WRT infrared radiation. (Plus a lot of other factors, of course.) That some pre-industrial year, or series of years, might have been warmer than today is relevant only so far as it reminds us that CO2 is not the only possible driver for temperature.
10 August 2007 at 11:36 PM
Five minutes with this new data set - apply smoothing functions (at varying spans, non-causal filters), then some sinusoidal fits and analysis of residuals - brings doubt to the presence of an underlying US warming trend, let alone any room for anthropogenic effects!
No warming trend in the US - lets hope the global data sets are better verified and validated. This is entirely embarrassing for climate science.
We will need complete transparency in the global temperature records and any ‘adjustments’ made (techniques, methodologies), as clearly climate scientists are not up to the task. Work will need to be verified by the broader scientific community.
regards, Steve
10 August 2007 at 11:48 PM
re 30. Steve Mosher — “observations are dependent on theory”
In the words of Albert Einstein”
“… On principle it is quite wrong to try founding a theory on observable magnitudes alone. In reality the very opposite happens. It is theory which decides what we can observe.”
In post #44 at http://www.kendallharmon.net/t19/index.php/t19/article/4811/
I give an concrete example of this principle insofar as it pertains to temperature measurements by making an analogy with special relativity.
10 August 2007 at 11:57 PM
One has to wonder how many other “anomalies” Mr McIntyre might have come across during his audit, but declined to inform anyone, as they happened to indicate that temperature data had been incorrectly revised too far downward.
11 August 2007 at 12:12 AM
I want to concur with:
35.”We will need complete transparency in the global temperature records and any ‘adjustments’ made (techniques, methodologies), as clearly climate scientists are not up to the task. Work will need to be verified by the broader scientific community.”
Absolutely true. Anything less is poor science.
And in response to the question about ‘trusting Hansen’. :
24.”I think the point is that we shouldn’t have to trust someone as in a single person or entity such as Nasa GISS to develope what is in effect policy for our country.”
Yes. Isn’t the phrase ‘trust but verify’. Look, we didn’t trust Einstein, we ran independent experiments to confirm his theories. That’s science. The data and the internal adjustment algorithms need to be made completely public and transparent so they all can be verified. This is true even irregardless of the public policy implications that make this more consequential than many other scientific issues.
One of the responses to a comment that used the word ‘hubris’ said: “Sure it’s embarrassing, but only the end result determines whether it matters or not. …”
Wrong attitude! A more appropriate response is to consider this an issue of data collection/analysis integrity and quality and thus, even if the change is small, ensure *processes* need to be put in place to make sure there are not other errors lurking in the adjustment algorithms. So not just this but other datasets should be opened up for independent review.
Whether you are a ‘believer’ or a ’skeptic’ in AGW, this should be a point of common agreement: Let us at least get the facts right. Let us build confidence in what the facts are through transparency of data and independently reviewable internals of adjustment algorithms.
“More importantly for climate purposes, the longer term US averages have not changed rank. 2002-2006 (at 0.66 ºC) is still warmer than 1930-1934 (0.63 ºC - the largest value in the early part of the century)”
So recent 5 year temps in the US have been a mere 0.03C greater than in the 1930s? Fascinating.
11 August 2007 at 12:32 AM
Aha! I was writing up a blog entry on this very topic, and then came here and was happy to see what you wrote; it agrees pretty much with what I said. I also looked at the data a bit, too, to show that despite the claims, this doesn’t change a whole lot.
http://www.badastronomy.com/bablog/2007/08/10/is-it-hot-in-here-or-is-it-just-me/
I’m glad I stopped by before posting!
11 August 2007 at 1:08 AM
OK, there is an easy solution to all these complaints.
How about if we double or triple NASA GISS’s budget? Americans complaining about all the stuff that should be done should be in favor of that, and should start writing their Senators and Representatives tomorrow for special earmarks, because they think it’s important to get things right.
Also, express a wish to pay higher taxes for this [hence, non-taxpayers need not apply]. For the surface stations, demand a lot of extra money as well.
I do assume that people complaining about lack of access to everything, have for instance, have explored data.giss.nasa.gov in its entirety, and downloaded the extensive code from www.giss.nasa.gov/tools (everyone reads FORTRAN 90, right?), and the code is fairly short, even if there’s 191MB of data to check over as well. [I’ll admit, I only downloaded the code to look at, I didn’t pore over the data sets.] I assume everyone complaining has the expertise to actually help?
Patrick: can you explain to us your expertise in physics, algorithms, statistics, analysis of imperfect data, software engineering, simulations? You have done most of this stuff professionally, right? [Of course, as an anonymous poster, it will be hard for us to check.]
I don’t want to pick on anyone in particular, but a lot of the complaints in this thread have not seemed very well-informed about realistic methods of doing real science… but seem more likely to impede real work. Sigh, science is hard enough to get right as it is.
11 August 2007 at 2:08 AM
#52
“If there were no warming trend at all, you’d expect the hottest 10 years to be randomly distributed…”
This is an incorrect assumption. You could see a recent warming, without it being part of an ever increasing function - for example, at the rising phase of a sinusoid.
If you bother to fit some single / multiple frequency sinusoidal models to the data and then examine the residual components, I believe you will quickly negate a requirement for an increasing, sustained signal component i.e. a US warming trend
regards, Steve.
11 August 2007 at 2:32 AM
Steve (#48)
“We will need complete transparency in the global temperature records and any ‘adjustments’ made (techniques, methodologies), as clearly climate scientists are not up to the task. Work will need to be verified by the broader scientific community.”
Are you aware of this remarkable device used in the scientific community known as “publishing”? It seems that many of the techniques, methodologies and “adjustments” are found right there, clearly explained in the scientific literature.
As for the global temperature records, did you know that you can get the data for any station in the GISS network here: http://data.giss.nasa.gov/gistemp/station_data/
before or after “adjustments” documented in the “scientific literature” are applied?
I normally advocate complete transparency in science, including open source code and experimental results. But your points in this regard are so silly that they are barely worth replying to.
11 August 2007 at 3:02 AM
Steven Mosher (#21) wrote:
Sounds like Quine to me.
Actually he’s not all bad - at least the bits he gets from Pierre Duhem.
No theory follows with cartesian certainty from any given set of observations. Not bad.
So for example, I may have a theory that the world is older than 10,000 years. I might cite geologic formations and the rate at which they form, perhaps fossils, radiocarbon dating, a series of fossilized trees and there rings, the distribution of species, etc.. But interpreting much of this evidence as evidence for the age of the world would involve other theories. Ok.
Then what about the theory that the world is older than five decades? Well, you may not have been here five decades, so I suppose you could hold the theory that any evidence that the world is older than five decades was simply placed there, perhaps by space aliens. Then if someone claims that they are sixty years old and they know that the world has been here at least that long, well, perhaps they only think they are sixty years old, or perhaps they are lying.
Come to think of it, this is where your “social component” might come in handy - but perhaps we will get to that a little later.
Now what about the claim that the world is at least five seconds old. I certainly think it is, but you may regard this as nothing more than a “theory” which is “underdetermined” by the “data,” where the data consists of the experience of “memories” and anything else which I might claim as evidence for an older world. But at this point we aren’t even talking about observation per se - we are talking about memory.
*
So lets move on to observation.
You write:
So does this mean that when I look at an iceberg floating off, it might actually not be floating away? Does this mean that if I am looking at the temperature displayed by a thermometer I am holding is rising, that the temperature might not be rising? Or does this mean that I might only think that the displayed temperature of the thermometer is rising, or that the thermometer is in my hand - or that I have a hand?
If so, I would begin to wonder whether you are engaging in the philosophy of science rather than in some freshman philosophy bullsession. I would also have to wonder just how desperate the “global warming skeptics” have gotten that they find it necessary to appeal to this kind of reasoning.
*
But lets continue.
You write:
This isn’t the way that I normally hear it.
From what I understand, falsification can always be avoided by appealing to another hypothesis. For example, Newton’s gravitational theory predicted that Mercury would have a particular eplliptical orbit, but even in Newton’s day they could see that its orbit tended to shift relative to what Newton’s theory predicted. An auxiliary hypothesis was introduced, namely that there was another planet by the name of Vulcan which was closer to the sun and exerted a gravitational pull on Mercury, causing its orbit to shift.
Now if one claimed that Vulcan was of such a special nature that there would never be any evidence for it other than the observed precession of Mercury’s orbit, then it would be an ad hoc hypothesis. But supporters of this theory made predictions which could be tested independently of that which their hypothesis was introduced to explain. As such, it was an auxiliary hypothesis, not an ad hoc one. As a consequence, it was a scientific theory at the time.
*
But lets focus on the phrase “appealing to other data.” A little later, you speak of a theory as “fitting the data” and “making useful predictions.” Somehow I think these are all interconnected issues. A theory is justified by the evidence, or as you prefer to call it, “the data.” But a theory is also used to explain the evidence, or alternatively, “the phenomena.” And some theories explain more evidence than others. This makes them more useful. Likewise, some theories make more specific predictions than others and this makes them more useful.
But if someone offers a theory involving Descartes’ demon which they then involk to explain any inconvenient fact, they might find this “useful,” but it wouldn’t be useful in quite the same way. Why? Because it would be consistent with anything. As such, it wouldn’t make any specific predictions. Or to put it another way, it wouldn’t be testable.
However, if someone offers a theory which explains the orbit of the planets and the paths of objects which fall on the earth, one which can be used to make specific calculations of where things will be at any specific moment, this is useful. And if it does so and works under a great many circumstances, explaining a wide variety of phenomena, then it is even more useful. At this point we have good reason for not abandoning it the first time we encounter evidence which tends to disconfirm it. It is reasonable to look for an auxiliary hypothesis.
Why?
Because there exists a large body of evidence which supports that theory. Or alternatively, because that theory explains a great deal of evidence. Much of this evidence is independent of other evidence. Someone investigates the paths of planets in the outer solar system. Someone else investigates the paths of the moons around Jupiter. Someone else studies the inner planets or the orbit of the moon.
Then there are those who drop cannon balls or apples. All of these largely independent lines of investigation examine phenomena which is explained by Newton’s gravitational theory, but they also provide evidence for the theory. And even when some later theory comes along and supercedes an earlier theory, it must be consistent with and account for the evidence which the earlier theory accounted for.
This is in fact the basis for what is known as the correspondence principle between classical mechanics and quantum mechanics, between classical mechanics and special relativity, and between Newton’s gravitational theory and Einstein’s General Theory of Relativity. In a certain sense, while much of the form of an earlier theory is discarded, the material of that theory - consisting of the evidence which justified it and which it accounted for - is preserved in the theory which supercedes it.
*
A hypothesis or theory which is justified by multiple lines of investigation is generally justified to a far greater degree than it would be if it were simply justified by any one line of investigation considered in isolation.
Now the vast majority of the scientific community has accepted the view that:
1. The earth is getting warmer;
2. greenhouse gases are largely responsible for this; and,
3. That what has been raising the level of greenhouse gases are human activities.
You on the other hand are still stuck on (1). Not dogmatically denying it, I understand, but simply doubting it with your healthy, “scientific” skepticism.
So in the interest of science, lets look at the evidence:
1. We have surface measurements in the United States which show an accelerating trend towards higher temperatures.
2. These are temperature measurements being taken by planes and satellites, and they show that the troposphere is warming - just as we would expect.
3. The stratosphere is cooling - just as is predicted by the anthropogenic global warming theory. (Incidently, the latter of these is something which cannot be explained by any theory based upon solar variability.)
4. Measurements of temperatures at the surface of the ocean show that these temperatures are increasing.
5. Measurements of temperatures at various depths show warming as far down as 1500 meters.
6. Measurements of sea level show that it has been rising just as we would expect from thermal expansion.
7. Gravitometric measurements of Greenland and Antarctica which are showing net ice loss in both cases.
8. We can witness sea-ice loss in the Arctic which is dramatically accelerating.
9. We are seeing the acceleration of glaciers in both Greenland and Antarctica, particularly within the last few years. Greenland is no doubt affected by black carbon, but Antarctica is much more isolated.
10. We are witnessing the rise of the tropopause.
11. There is the poleward migration of species - just as one would expect with rising temperatures.
12. There is the increased intensity of hurricanes just as we would expect from rising sea surface temperatures.
13. There is the accelerating decline of glaciers throughout the world with few rare exceptions.
14. There is the rise in temperatures at greater depths in the permafrost.
15. There is the rapid expansion in the last few years of thermokarst lakes throughout parts of Siberia, Canada and Alaska.
16. There are changes in ocean circulation - just as has been predicted by climate models, for example, with temperatures rising more quickly overland.
17. We are seeing the disintegration of permafrost coastlines in the arctic.
18. We are witnessing changes in the altitude of the stratosphere.
19. We are getting temperature measurements from countries throughout the world which show the same trends.
20. When we perform measurements using only rural stations, we see almost identical trends compared to those which we get when we perform measurements with all surface stations.
All of this constitutes evidence for global warming. Some of it constitutes strong evidence for a particular theory of the mechanism by which this warming is taking place. But you would have us discard a conclusion which is based upon such a large body of evidence based upon a fraction of a degree for a particular year for a relatively small region of the globe.
Somehow I doubt a fidelity to science is what motivates you.
11 August 2007 at 4:03 AM
I had written (#19):
papertiger (#29) responded:
Different country - Hadley out of England - notice the UK in the website address. But I suppose they could be part of a global conspiracy. You have to watch those conspiratorial types pretty darn closely - particularly the scientists when they start getting uppety….
As for the Arctic sea, I don’t know its political affiliation for certain, but the blue would seem to be a dead giveaway.
11 August 2007 at 4:04 AM
Gavin, if the US48 corrections indeed were imperceptible in the global record, I would expect the GISS global record to either not change at all or to roughly correspond to the US48 corrections (mainly due to rounding errors since the effect is small).
However diff-ing the GISS global record versions, I find none of the above. E.g. the 2005 US48 correction of -0.3°C is reflected as a +0.01°C in the new global dataset. How come?
[Response: What’s your source for that? The wayback machine version from May 17 2007 is 0.01 degree greater than the current version. i.e. it went down with the correction as you would expect. The data for Fig.A2 doesn’t appear to have been updated yet (and the update stamp confirms that). I’ll ask that they do. - gavin]
11 August 2007 at 4:39 AM
Steve…
“If you bother to fit some single / multiple frequency sinusoidal models…”
Some basics…
Fitting any old set of functions to a given data set is often not very useful. Usually one has some justification for the basis functions one is using what’s yours ? You do realize that given enough terms it is possible to fit any function (data set) exactly…of course such fits are not useful in the context of this issue.
Have you tried using only say 75% of the data set and doing the same fits and then using the other 25% to see if your fits have any predictive power at all ?
11 August 2007 at 4:41 AM
btw; How does this affect the accuracy of Hansens’ models? The ones that were portrayed as so stunningly accurate?
[Response: Not at all. If you recall, the evaluation of those 1988 projections was against the global data, which hasn’t changed. - gavin]
11 August 2007 at 5:27 AM
And what did the 1930s bring the United States? Drought and the dustbowl throughout the midwest. Perhaps Steinbeck’s greatest novel, “The Grapes of Wrath”. Images by folks like Dorthea Lange and Walker Evans that burned into the soul of the country.
“only” 0.03 warmer than a time that brought significant hardship to many, many farmers.
And, hey, keep in mind that we’re really concerned about GLOBAL warming. The current temps are part of a global phenomena, the high US temps in the 1930s weren’t.
11 August 2007 at 6:05 AM
[[If you want me to believe in catastrophic global warming, you’ve got to convince me it’s not just a case of Garbage In Garbage Out. So far, I’m not convinced. I’m less convinced today than I was last week.]]
Then you’re not paying attention.
Global warming doesn’t rely only on the surface temperature station record, though that record is reliable enough. It has been detected in sea surface temperatures (are there urban heat islands and poorly sited temperature stations on the ocean?). It has been detected in boreholes. It has been detected in the record from balloon radiosondes. It has been detected in the record from satellites. It has been detected in melting glaciers, tree lines moving toward the poles, animals migrating toward the poles. It has been confirmed again and again and again and those who doubt simply don’t understand the situation.
11 August 2007 at 6:05 AM
True enough, any curve shape — even a step or a spike — can be reproduced with sinusoids. But what physical process does each sinusoid represent?
11 August 2007 at 6:07 AM
[[I am wondering if the ocean is rising and flooding out Indonesia, why isn’t it flooding Cape Cod? are they not all at sea level?]]
Sea level is not the same everywhere in the world. It is affected by the Earth’s rotation, gravitational anomalies, currents, differences in local salinity, and differences in local temperatures. Sea level on average is rising. That doesn’t mean it’s rising at exactly the same rate everywhere in the world.
11 August 2007 at 6:38 AM
“but, given the fact that the US has the most reliable and well-maintained network, it raises concerns about the quality of data we have been using across the board.”
On what do you base the idea that our network is best? Why is it better than for example that of Sweden?
11 August 2007 at 6:53 AM
Gavin:
The two charts you present are grossly misleading, is there no way to amend them and use the same y-axis.
11 August 2007 at 7:07 AM
43: “Unfortunately, as ClimateAudit has documented, a number of climate researchers have maintained their data, methodologies, algorithms and source code are private.”
“[Response: What is secret? The algorithms, issues and choices are outlined in excruciating detail in the relevant papers: http://data.giss.nasa.gov/gistemp/references.html - gavin]”
How is “outlining” anything equivalent to actually posting the data and code?
[Response: Because a) the raw data are publicly available and b) papers are supposed to contain enough detail to allow others to repeat the analysis. If a paper says ‘we then add A and B’, you don’t need code that has “C=A+B”. - gavin]
11 August 2007 at 7:42 AM
I’m a physician and not a climate scientist, and am a little confused about the points being made here. Are people arguing that the GISS correction suggests that N America is not warming, that the hemispheric or global means are not increasing or what? How do the respondents tie this in with sea temperatures and satelite data. What do they think of the recent correction of the satelite data anomaly? And what then of the measured sea level rises? Sorry it’s just I’m having trouble following the logical thread of this argument and some of the conclusions being bandied around.
11 August 2007 at 8:44 AM
An important statement relevant to this issue was made as part of the recommendations from the CCSP Synthesis and Assessment Report on Tropospheric temperature trends:
http://www.climatescience.gov/Library/sap/sap1-1/finalreport/default.htm
see particularly ch 6 (recommendations)
http://www.climatescience.gov/Library/sap/sap1-1/finalreport/sap1-1-final-chap6.pdf
“To ascertain unambiguously the causes of differences in data sets generally requires extensive metadata for each data set. Appropriate metadata, whether obtained from the peer-reviewed literature or from data made available on-line, should include, for data on all relevant spatial and temporal scales:
• Documentation of the raw data and the data sources used in the data set construction to enable quantification of the extent to which the raw data overlap with other similar data sets;
• Details of instrumentation used, the observing practices and environments and their changes over time to help assessments of, or adjustments for, the changing accuracy of the data;
• Supporting information such as any adjustments made to the data and the numbers and locations of the data through time;
• An audit trail of decisions about the adjustments made, including supporting evidence that identifies non-climatic influences on the data and justifies any consequent adjust- ments to the data that have been made; and
• Uncertainty estimates and their derivation.
This information should be made openly available to the research community.”
“The independent development of data sets and analyses by several independent scientists or teams will serve to quantify structural uncertainty and to provide objective corroboration of the results. In order to encourage further independent scrutiny, data sets and their full metadata should be made openly available. Comprehensive analyses should be carried out to ascertain the causes of remaining differences between data sets and to refine uncertainty estimates.”
Last week, Bush signed a bill on “America COMPETES”. There is a relevant part on the open exchange of data and metadata:
“SEC. 1009. RELEASE OF SCIENTIFIC RESEARCH RESULTS.
(a) Principles- Not later than 90 days after the date of the enactment of this Act, the Director of the Office of Science and Technology Policy, in consultation with the Director of the Office of Management and Budget and the heads of all Federal civilian agencies that conduct scientific research, shall develop and issue an overarching set of principles to ensure the communication and open exchange of data and results to other agencies, policymakers, and the public of research conducted by a scientist employed by a Federal civilian agency and to prevent the intentional or unintentional suppression or distortion of such research findings. The principles shall encourage the open exchange of data and results of research undertaken by a scientist employed by such an agency and shall be consistent with existing Federal laws, including chapter 18 of title 35, United States Code (commonly known as the `Bayh-Dole Act’). The principles shall also take into consideration the policies of peer-reviewed scientific journals in which Federal scientists may currently publish results.”
As a climate researcher, I wholeheartedly support the above principles. In my opinion research scientists (and particularly government research scientists) should not be given any “choice” in this matter if they wish to receive government research funding, publish their research in the peer reviewed journals of the major professional societies, and have their data used in assessment reports.
Yes all this adds to the cost of doing research, and even the COMPETE bill is apparently an unfunded mandate. But it’s a cost we need to accommodate in some way. I have seen too many examples in the climate field where scientists do not want to make their data and metadata available to skeptics such as Steve McIntyre since they don’t want to see their research attacked (and this has even been condoned by a funding agency). Well, in the world of science, if you want your hypotheses and theories to be accepted, they must be able to survive attacks by skeptics. Because of its policy importance, climate research at times seems like “blood sport.” But in the long run, the credibility of climate research will suffer if climate researchers don’t “take the high ground” in engaging skeptics.
With regards to Steve McIntyre and climateaudit. In the early days of McIntyre’s attacks on the “hockey stick”, it was relatively easy to dismiss him as an industry “stooge.” Well, given his lengthy track record in actually doing work to audit climate data, it is absolutely inappropriate in my opinion to dismiss him. Climateaudit has attracted a dedicated community of climateauditors, a few of whom are knowledgeable about statistics and are interesting thinkers (the site also attracts “denialists”). For all the auditing activity at climateaudit, they have found relatively little in the way of bonafide issues that actually change something, but this is not to say that they have found nothing. So taking the high ground, lets thank Steve and climateauditors if they actually find something useful, assess it and assimilate it, and move on. Such actions by climate researchers would provide less fodder for the denialists, in my opinion.
11 August 2007 at 8:55 AM
it is still curious the difference between the US and world readings.In Britain the 1930’s temperatures were only matched very recently. Indeed when I was a boy in the 1960’s older people were always saying how much warmer it was then. It turns out they were right. Why is this and which parts of the world have warmed to give the sharp upward increase in global temperatures?.
[Response: The GISTEMP website allows you to compare any two time slices, so have a look there, comparing say 1930-1935 to 2000-2005. You’ll find that almost everywhere is warmer than then (including Europe), but not the SE US, nor much of the Eastern Pacific. - gavin]
11 August 2007 at 9:25 AM
I may have missed it, but I don’t think anyone has linked James Hansen’s response to this issue.
http://www.columbia.edu/~jeh1/distro_LightUpstairs_70810.pdf
By the way, I tried to contact him to be added to his email list and found that his email has been disabled. Given the vicious attacks on him by right wing blogs and talk radio, I suppose his inbox has been overwhelmed by messages calling for his head. It is sad. He is an outstanding scientist of highest integrity and one of my heros - a man who will speak the truth to power at whatever cost.
11 August 2007 at 9:30 AM
[ Barton: Sea level is not the same everywhere in the world. It is affected by the Earth’s rotation, gravitational anomalies, currents, differences in local salinity, and differences in local temperatures. Sea level on average is rising. That doesn’t mean it’s rising at exactly the same rate everywhere in the world. ]
Odd you didn’t mention that maybe some plates might be sliding lower into the ocean. Exactly how much of a rise is there? And how does it vary across the globe?
11 August 2007 at 9:52 AM
Gavin said:
You might want to rethink that, considering the denial folks seem to need to be led around by the nose.
11 August 2007 at 10:31 AM
Theory laden observation. one of my favorite examples.
from Hansen 2001:
“The strong cooling that exists in the unlit station data in the northern California region is not found in either the periurban or urban stations either with or without any of the adjustments. ”
Rural can’t be cooler. And it can’t be cooler after our perfect adjustment scheme.
Go figure. we are talking about the early 1900s here. and defining urban/small town/rural using 1980 population data and a photo from a satellite.
They continue:
“Ocean temperature data for the same period, illustrated below, has strong warming along the entire West Coast of the United States. This suggests the possibility of a flaw in the unlit station data for that small region.After examination of all of the stations in this region, five of the USHCN station records were altered in the GISS analysis because of inhomogeneities with neighboring stations (data prior to 1927 for Lake Spaulding, data prior to 1929 for Orleans, data prior to 1911 for Electra Ph, data prior of 1906 for Willows 6W, and all data for Crater Lake NPS HQ were omitted so these apparent data flaws would not be transmitted to adjusted periurban and urban stations.”
The ocean is warm, so the cool station record must be wrong. the 5 stations that don’t agree; they must be wrong. And we know they must be wrong because we have records of the temperature of water, taken by guys hauling in buckets of water somewhere in the pacific.
Now, very often on the coast the land tracks the ocean temp. Let’s see where these stations are.
In this region ( they don’t specify the exact extent of the region) they mention:
1. Lake spaulding. You go google map that. It’s about 200 miles from the warm ocean, on the other side of the mountains.
2. Willows a bit closer to the ocean. It’s in the sacramento Valley.
3. Electra. About 200 miles from the ocean on the far side of a mountain.
4. Crater lake: closer to the ocean, but in a crater,
separated from the ocean by a ridge line.
5. Orleans. Much closer to the ocean. Behind a big ridge line.
I’m stumped. Why would these 5 sites show unusual cooling ? Hansen’s reply: they can’t. There must be something wrong with the station data.
Must. well, There are multiple explanations:
1. could be something wrong with the Ocean measurements.
2. Could be something amiss with the assumption that these site records always correlate well with the ocean data.
3. Could be something wrong with the warmer stations they are compared to. where is the list of the other stations.
How to decide? confirmation bias to the rescue.
Dump the cold data.
Continuing:
“If these adjustments were not made, the 100-year temperature change in the United States would be reduced by 0.01°C.”
A Small amount. does not even matter. Why drop it?
11 August 2007 at 10:34 AM
re: “Another week, another ado over nothing.”
If that is the last warming error in the GISS temp analysis then perhaps it is an embarrassing, but minor issue.
After an error is found an organization’s sales staff typically weigh in with “It is only one small error, the rest of the product is still solid”. Meanwhile the technical folks scurry around and make sure that that was the only mistake. Take a step back and look Gavin, your business card may need some updating.
re: “What is secret? The algorithms, issues and choices are outlined in excruciating detail in the relevant papers: http://data.giss.nasa.gov/gistemp/references.html”
Sorry, I don’t see the excruciating detail. I found no pseudo code, no code, no scripts. Had Hansen released code and scripts Steve McIntyre would have caught the problem long ago. That just made him work harder. Until code and scripts are fully released, other errors are going to be slower to surface. I am betting you know this.
11 August 2007 at 10:42 AM
One mistake and the “gotcha” squad comes out in full force and says the whole enterprise is wrong! Baloney!
The modelers admitted the error and promptly corrected it. This is what should be focussed on.
Jeremy Bernstein, the physicist and science writer visited Erwin Schrodinger,near the end of his life.Schrodinger developed the wave mechanic equations of quantum mechanics.Schrodinger told Bernstein “There is something that the ancient Greek scientists knew that we seem to have forgotton.” He paused and then said “and that is modesty.”
It seems that Schrodinger was wrong. The modelers had the humility to openly acknowledge the error and to correct it.
11 August 2007 at 10:56 AM
It’s already been this hot (1998) in the past. In other countries it has been hotter in other years. Icebears still live. There have always been droughts and floods. Now we just get informed better than 50 years ago and before.
In my opinion the rate of change doesn’t say anything about the following years. Temperature records show that after one year with an increase (maybe even a dramatic increase) - one or two years later it has been cooling again.
From what I gather there is no gain of momentum but a lot of jitter on the raw data, as well as the annual means.
If anyone could point me to (freely accessible?) scientific literature suggesting something like a momentum in global annual mean temperatures, please feel free to post: I am hungry to learn every day!
PS: I am currently working on some statistics concerning temperatures (min, avg, max). I will let you know when I’m ready to “publish”… As a preview you can look at this:
http://www.07000deuschl.de/co2/pubs/avgcharts_ffm/avgs.jpg
http://www.07000deuschl.de/co2/pubs/avgcharts_ffm/deriv.jpg
http://www.07000deuschl.de/co2/pubs/avgcharts_ffm/index.html
11 August 2007 at 11:05 AM
What do you mean its “estimated” to be? You don’t know what the error is? What’s the error on the error of the GST?
Stop obfuscating. It may not be better than Sweden–but I imagine it is. That’s not the point. What’s the network like in Botswana? Nigeria? Rural Mongolia? Siberia? Geez.
It is inappropriate and unprofessional to dismiss anyone and an “industry stooge” simply because they disagree with you. Unsubstantiated ad hominem attacks are never appropriate.
[Response: If we knew exactly what the error was, we’d fix it and then it would be zero. All errors are therefore estimated (based on spatial coverage, measurement accuracy, etc.). - gavin]
11 August 2007 at 11:14 AM
So, it is climatically significant that 1930-1934 is 3 hundredths of a degree cooler than 2002-2006, but it is not climatically significant that 1934 and 1998 have now changed places, by 3 hundredths of a degree.
Can you explain your criteria for assessing when 3 hundredths of a degree is climatically significant and when it is not? Because from where I sit it looks like the criteria is “significant if it supports the AGW hypothesis, insignificant if not”.
[Response: The longer the average the more relevant, and the more significant the difference. I was just trying to make the point that long term averages didn’t shift. If we consider 0.1 (as above) the ’significant’ level for single years, the five year mean significance is roughly 0.1/sqrt(5)= 0.045 and so the 1930-1934 and 2002-2006 difference isn’t significant, but the difference to 1998-2002 certainly is. - gavin]
11 August 2007 at 11:32 AM
2001-2006 is longer than 1930-1934 - what does it mean to compare averages of periods of different length?
And may we please get preview back?
[Response: I mistyped. It is 2002-2006, I’ll correct the post. Thanks. (Preview is coming but for some reason the compreval plugin doesn’t want to work). - gavin]
11 August 2007 at 11:33 AM
While the error may be minor it does make good copy.
11 August 2007 at 11:37 AM
Can someone tell me - what are the primary sources of scatter in the global mean temp from year to year?
[Response: El Nino/La Nina is important, the North Atlantic Oscillation during the winter can be seen, but mostly it is just ‘weather’ - unforced variability (which locally can be quite large) averaging out to give deviations of about 0.1 deg C from year to year. - gavin]
11 August 2007 at 11:50 AM
Just for global warming skeptics:
http://www.nowpublic.com/english_man_swims_1km_north_pole_raise_climate_change_awareness
Just for everyone else on the cause of global warming (non-scientific):
It’s true that it’s happening. It’s true that it is new to the Earth, CO2 levels are higher than they’ve been for … how many years? The question is the cause: It’s not from living plants, they reduce CO2; It’s being caused by a change of some kind… It sounds reasonable to assume that the change must be large to have so large an effect on the amount of CO2. It’s most likely a living organisim that emits CO2 whose population is consistently increasing indicating that has no natural predator to thin out their numbers… Could it be a fish? nah, we’d kill them and eat them. Could it be an animal? Could be, but we’d keep their numbers down through hunting/we’d see them. Could it be an insect, nah we’d kill them to keep their numbers down, they are pests after all. Well, what’s causing it: “I have met the enemy and them are us”.
11 August 2007 at 12:12 PM
Re #56: [All of this constitutes evidence for global warming. Some of it constitutes strong evidence for a particular theory of the mechanism…]
An interesting & informative read, if a bit long, but I think it’s still backwards. After all, most if not all of the points you cite as evidence of warming would be equally valid for warming from any cause, whether it be an increase in the sun’s output, cosmic rays, or the side-effects of UFO exhaust. That means having to refute each and every alternative explanation that the denialist community dreams up - and denialists being what they are, having to repeat each separate refutation ad nauseum.
Why not approach it from the other direction, from the theory of the mechanism? The amount of CO2 in the atmosphere has increased, a fact that only the completely irrational denialists try to dispute. That this increase is from fossil fuel burning is also beyond any reasonable question. The properties of CO2 have been measured quite accurately, and are likewise difficult for the denialists to dispute. So from that, and known radiation theory (which has been experimentally verified in many ways), we discover that there is this mechanism by which increasing atmospheric CO2 causes warming.
If the denialist community wants to try to refute this mechanism, they have to do some hard math and/or experimental science, rather than just randomly sniping at this or that data set. Instead of the foundation for the theory, your long list of warming examples becomes evidence supporting it. If one or more are shown to fail, due to instrument error or the effects of other climate drivers (e.g. sulfate aerosols), that doesn’t undermine the existence of the mechanism.
11 August 2007 at 12:36 PM
Gavin:
Steve McIntyre graciously accepted my suggestion and the two figures you present now are viewable side-by-side. Hopefully it will stimulate further explorations and careful assessments of the published temperature records.
11 August 2007 at 1:32 PM
RE 77.
Matthias, your data is from one city in Germany. Hardly data representative of the globe.
J
11 August 2007 at 1:50 PM
What I’d like to see is a least-squares analysis of these data sets, which would calculate a growth rate over the period of the data sets. Simple maxima and minima do not make a trend by themselves. The 1934 data for the US might be just an aberration in the trend. As I look at the graph of the data, just eyeballing it as it is, it certainly appears to me that there is a clear upward trend over the course of the 20th Century in the US data set.
Some of the comments also make it clear to me that there are a lot of folks who think of “pollution” as generic. That is, of course, total nonsense. The effects of particulates (e.g. fly ash), and carbon dioxide are quite different, as I’m sure those of you who are actually scientists know.
[Response: US trend (1880-2006): 0.043 +/- 0.021 deg C/decade (95% confidence levels), significant but less than the global trend. For the more recent period 1975-2006: 0.3 +/- 0.16 deg C/dec. - gavin]
11 August 2007 at 2:15 PM
It’s really no surprise how the antiwarming crowd jumps on any small error in the science (in this case a minor error in one data set) to discredit all the science. It is the same tactic the anti-evolutionists use. They do not understand that both theories, AGW and evolution, are based on multiple converging lines of evidence and to bring down either theory would require seriously attacking each of those independent lines of evidence successfully. This instance wasn’t even a small dent and still the fools cry hoax.
11 August 2007 at 2:39 PM
James (#84) wrote:
My apologies for the length, particularly so soon in the thread. Believe it or not, I was actually inclined to write something a bit longer to address some of your later concerns and to more properly treat some technical issues, but that would have been imposing a great deal upon everyone else, and I believe that given the length I had imposed more than enough as it is. Besides, what you don’t mention at one point you can always bring in later.
Granted, for something general purpose, one should focus more on the mechanism, particularly anthropogenic greenhouse gas emissions. We actually have that nailed. Likewise, there should be more focus on the foundations of climatology in physics. However, I was responding to Steve Mosher - and he is inclined towards skepticism with regard to the mere fact of global warming. Likewise, McIntyre is trying to cast doubt on the phenomena by focusing on a small fraction of a percent in temperature for one year in the United States - which is afterall what this thread is about. So focusing more on the warming and less on the mechanism seemed warranted, given the context.
11 August 2007 at 2:50 PM
“Stop obfuscating. It may not be better than Sweden–but I imagine it is. That’s not the point. What’s the network like in Botswana? Nigeria? Rural Mongolia? Siberia? Geez.”
How am I obfuscating here? Your original post was along the lines that since “the best” network hads given rise to an eror all the other were in doubt. I took Sweden as an example since I happen to be staying there for a few months now.
The reliability of data from different regions is of course important to know when deciding how reliable the averages are, but just assuming that our data is the best is no good. Your mention some countries you find unreliable, but what about Australia, New Zeeland, Japan, South Africa? Or Canada, which has more land area than the US and thereby a larger part of the world average.
11 August 2007 at 2:55 PM
#82 (including response)
Doesn’t the scatter in the surface temperature record really boil down to
the fact that most of the energy in the climate system is stored BELOW the
surface in the oceans? Thus, vertical transports of energy that would be
insignificant when compared to the total sub-surface energy can have a
measureable impact on SURFACE ocean temperatures.
11 August 2007 at 3:23 PM
88. “It’s really no surprise how the antiwarming crowd jumps on any small error in the science …to discredit all the science.”
This was a classic tactic of the tobacco industry. Any small error in a science article was used to describe the research as ‘flawed’ and unusable. Tobacco lawyers also sued researchers to get patient data: I know one scientist who was unable to visit Texas for a number of years because there was a warrant out for his arrest; he had refused an order obtained by the tobacco industry to turn over his patient records and data.
11 August 2007 at 3:24 PM
re 92
Don’t any of those foldk read their Grandmother’s journals for when the apples bloomed?
This year the apples bloomed earlier than in 1934.
It is hard to argue with an apple tree.
11 August 2007 at 3:26 PM
#6: Lee
11 August 2007 at 3:46 PM
All scientists are skeptics. To “believe” is associated with faith and religion, not science. To “deny” is associated with “believing” regardless of contrary evidence, and regardless of the fact that doing so is unscientific.
Those who use this terminology easily expose themselves as naive, manipulated, and propagandized. This goes both ways.
11 August 2007 at 3:53 PM
Roman,
No. I was explicitly looking at the total change in temp over a stated time interval, with the beginning and end times stated. ie, the total change over the last century, between an implicit zero at the beginning, to the end, was somewhere between 0.8C and 1.1C. That change is not sensitive to the base reference. Changing the base would change the absolute values for beginning and end, but would leave the difference between them unchanged.
Yes, it is slightly awkward - but only slightly. I chose to state it that way because claims are being bandied all over the internet that the change in the temp/trend was several percent change.
11 August 2007 at 3:53 PM
Here is the response of an Inuit child about all this climate change:
http://www.putfile.com/pic.php?img=6219791
11 August 2007 at 3:54 PM
Steve Mosher (#74) wrote:
Hansen (2001):
A closer look at United States and global surface temperature change
J. Hansen, et al (2001)
http://pubs.giss.nasa.gov/docs/2001/2001_Hansen_etal.pdf
They state:
As such, they are responding to sources of contamination which may not have been regarded as that important - prior to 1929. Not only does this have little to do with current trends, but it demonstrates a concern for accuracy. And as a matter of fact, the elimination of such contamination tends to reduce the apparent rise in temperatures over the twentieth century rather than magnify it.
In an earlier thread you claimed to be interested in accuracy and were attacking urban sites for their presumed contamination by the urban heat island effect. We of course pointed that you get virtually the same warming trend whether you use all stations or just rural stations - which you didn’t even seem to acknowledge.
Now Hansen is correcting some fairly obvious contamination, and you find this particularly problematic because he is applying some common sense. You vaguely suggest it involves ulterior motives so on the basis of a kind philosophic skepticism which on close analysis would appear to be of the radical kind roughly on par with Descartes and Hume.
For some reason, I don’t think that identifying the actual trends and causes of world temperatures (or more generally, for that matter) is your highest priority.
11 August 2007 at 4:06 PM
96 Lee
Then you are not talking about anomalies. An anomoly is a departure from a norm. If you want to say taht you are looking at the change from some staring point and some end point, you are entitled to do so, but don’t call that an anomoly unless you want us to believe that somehow the temperature at the beginning of the 1900s was the norrmal state of affairs.
If this webiste wishes to edit my post, I would appreciate it if they would not make it look like I was quoting myself. Thanks.
11 August 2007 at 4:24 PM
Roman, a change in anomaly over that time period is equivalent to the change in temp over that period. If the anomaly went from -0.3C to +0.7, then the change in anomaly, is equal to the change in temp, is equal to a delta of 1.0C.
Make any change to the reference period, change the baseline, and all that happens is you create equivalent offsets to the beginning and ending anomaly values, and the delta is unchanged.
My percentage calculation references the delta temp (or delta anomaly, which is equivalent), not the absolute anomaly value.
I wasn’t even imprecise in my language. The adjustment causes a slight change in the anomaly values post 2000, which changed the delta temp, which is what I explicitly referenced in my percent calculation.
11 August 2007 at 4:27 PM
Re 98 - Tim, I am so grateful for your knowledge, skill and patience in deconstructing plausible sounding strawman arguments and revealing them for what they are. Keep up the good work - and I am sure that it involves a great deal of work on your part. Your posts are a great asset on this site.
11 August 2007 at 4:30 PM
It is troublesome that in this day and age we still seem to be using, or over using, an 18th century approach to data capture, a thermometer mounted on a post. Problem is, thermometers don’t really measure what we need to know about energy in the atmosphere, so their measurements are largely irrelevant for models. Thermometers report some unidentified hodge-podge of LTE, upwelling and down welling radiation fields fields and air currents and radiations from the surround, whatever that happens to be.
What we need — it is 2007 after all — are lateral and vertical radiation field measurements concurrent with pressures, separate pressure fluctuations, wind velocity, and humidity, all sampled at least every five minutes. Stations should be located inside and outside UHI’s. I say let’s stop wasting time with postoffice or whatever thermometers. Let the private sector manage and sell this data. If NASA is going to be in the business of modeling, it needs clean, complete data and trash the rest of it.
11 August 2007 at 4:45 PM
Hansen’s correction shows that in the United States temperature oscillates with a frequency of about 65-70 years. Thjis oscillation was identified by Schlesinger and Ramankutty in 1994. The last peak was 1940 and we are about to experience the next decline. There is no evidence for “global warming” from Hansen’s record.
The same oscillation, without overall warming, is to be found in the published regional records for China, and for many well-maintained local records.
The supposed “global warming” shown in the surface record is an artifact caused by the many inaccuracies and biases in the recording af local temperatures (where average temperatures are never measured) and their influence by local changing circumstances.
[edit]
11 August 2007 at 4:52 PM
Re #7. Was there an answer to this reasonable request? It would be quite illustrative to see the US and global anomalies on a similar scale, don’t you think? Concerning the global anomaly graph, somebody should tell the graphics person not to plot values off the chart, but to choose the y-axis so that everything fits inside the picture.
11 August 2007 at 5:03 PM
RE: 92/Richard: Monbiot says that tobacco company documents disclose a concerted effort to fog the secondary smoke issue by discrediting science in general–using AGW research as the focal point of the disinformation attack. From that flowed junkscience.com, its progeny, and most if not all of AGW denial. As such, the denial collective toils, consciously or unconsciously, in the service of tobacco company (and by derivation, extractive company) profits. How proud they must be.
88. “It’s really no surprise how the antiwarming crowd jumps on any small error in the science …to discredit all the science.”
This was a classic tactic of the tobacco industry. Any small error in a science article was used to describe the research as ‘flawed’ and unusable.
11 August 2007 at 5:11 PM
#103
The probability that the cumulative effect of local “inaccuracies and biases” would be a statistically sigificant global warming signal is miniscule. (Unless there’s a conspiracy… calling Michael Crichton!)
11 August 2007 at 5:22 PM
Three Questions.
Had McIntyre discovered the error change the data in the other direction and made it much warmer would he have brought the error to light?
Would 1934 still be the warmest year had calculations for the year started in any other date than January 1 through December 31? Had the years been calculated from any other date, it seems very likely that time period we call a year would not stand out at all. The years before and after 1934 were considerably much colder unlike the years in recent decades. Recent years are all much closer together in warmth, hence the mean average being higher for a longer period.
1934, Dust Bowl right? If in 1998 we scraped the plains of plants so that dust could blow around would it have significantly beat the pants off 1934 for hottest year on record ever?
[Response: Actually, it wasn’t clear which way the error would go once it was spotted. I doubt that it would have got as much publicity if it had gone the other way though. - gavin]
11 August 2007 at 5:26 PM
As I said at DeLong’s typepad, where he has a good piece on this now:
You can draw a straight line right across the 1934 peak on the revised graph [as I could see it there], and see that it is still lower than the 200x wiggles.
Yeah, Rush was blabbering about fraudulent NASA scientists, and “Seventy years ago, it was way warmer than it is now” etc. BTW, why can’t groups of people like NASA scientists sue for libel if someone defames them?
[Response: It’s not worth the bother…, though sometimes one is tempted! - gavin]
11 August 2007 at 5:38 PM
In 103, Vincent Gray writes: ‘Hansen’s correction shows that in the United States temperature oscillates with a frequency of about 65-70 years. This oscillation was identified by Schlesinger and Ramankutty in 1994. The last peak was 1940 and we are about to experience the next decline. There is no evidence for “global warming” from Hansen’s record.’
How interesting. I realize you are only saying that Hansen’s data does not demonstrate global warming, but you imply that global warming does not exist, and is merely a phase of a natural oscillation.
Since, as you say, this idea has been around since 1994, and since the conclusion contradicts the conclusions of hundreds of peer reviewed studies, surely dozens of peer reviewed articles have been published in support of the idea. Could you refer us to, say, three or four published in the past five years?
11 August 2007 at 6:01 PM
#58
Gavin, I think that explains the difference, thank you.
11 August 2007 at 7:21 PM
The Y2K comment was McIntyre being flip. It has caused unneeded confusion. One of the dangers of blog science (tendancy for catchy headlines and the like, versus boring titles).
The station picture thing was related in an indirect way. Watts et al. had been looking at individual station pictures and also in some cases correlating them to the station temp graphs. Looking at individual temp graphs showed some large adjustments tending to happen at JAN2000. That’s what popped out to the eye and led to the insight. The adjustments are actually bimodal vice normal around the mean, so that it was easier to see an effect than one might expect. (None of this is to defend the station photo effort, which I’ve ridiculed. But to explain the form of the connection as it is. Really much more one of these things of one idea leading to another that happens in science. Heck some of my most interesting results from lit searches did not come from a search or references to references, but from being physically in the library books or bound journals and what you end up finding next to each other…)
11 August 2007 at 7:29 PM
49: (science being theory or experiment led). Both can happen. Fractional quantum hall effect, Michelson Morley experiment, Mossbuaer effect, Pennicilin discovery, etc. are examples of unexpected results. Of course, it is also often the case that instruments are specifically designed based off of an experiment.
11 August 2007 at 7:40 PM
Vincent Gray (#103) wrote:
The article is a bit dusty after thirteen years, I would presume. As such I was unable to find it, but I did find the abstract. It turned out to be a letter to Nature. Not peer-reviewed as I understand it, but nevertheless of some quality, I believe.
I have bolded the sentence which I found most significant, given the context:
Despite the title of Chapter 11 “Regional Climate Projections,” you seem to have thought that its purpose was of detailing observations made it the past rather than making projections regarding the future. Now it would appear that the letter which you are citing in support of your highly idiosyncratic views was written by authors who clearly do not share them. You might wish to consider reading things more carefully before commenting on them.
11 August 2007 at 7:42 PM
As I posted in the Arctic sea ice extent thread, the fact that Arctic sea ice is at an historic low this year is more the result of data changes made this January.
Prior to these changes (which may also contain the same kind of errors as this thread is about) 1995 was the lowest Arctic sea ice extent.
Here is a BEFORE and AFTER of the changes made in the data this January (hopefully the adjustments do not contain errors.)
http://img401.imageshack.us/img401/2918/anomalykm3.gif
11 August 2007 at 8:21 PM
#114 nice graphic, but wait until you see the present day sallatlite pictures vs the (modeled) satallite pictures of 100 years ago.
11 August 2007 at 8:29 PM
John Wegner (#114) wrote:
John, the following chart from 2005 would suggest that you are once again being creative…
28 September 2005
Sea Ice Decline Intensifies
Figure 1: September extent trend, 1978-2005
http://nsidc.org/news/press/20050928_trends_fig1.html
I must admit, though, that this is better than your decimal place trick.
11 August 2007 at 9:20 PM
papertiger said:
“I am wondering if the ocean is rising and flooding out Indonesia, why isn’t it flooding Cape Cod? are they not all at sea level?”
It ain’t quite so straightforward. Simply put, all sea level curves are local. This is for a few reasons. For one, the massive ice loads present at the last glacial max depressed the earth’s crust beneath them (intuitive) made it bulge upward along their edges (maybe not so intuitive, but similar to the beam problems I’m sure you enjoyed in diff eq). The removal of these loads caused a global isostatic readjustment, still ongoing, as the mantle flows in toward the depressions. This means that some places (previously beneath loads) are still rising quickly, others (those on the bulges) are falling slowly, and still others (in the far field) have had complicated post-glacial rises then falls.
Another important factor is that the global sea surface is nowhere near ‘flat’ in the way you might be picturing. Gravity varies spatially, as does the mass of seawater, resulting in hills and valleys on the sea surface. These move in time, sometimes very dramatically (e.g. El Nino, ENSO, etc.).
Finally, many places are tectonically active, and interseismic strain accumulation (and subsequent release during fault rupture) can complicate measurements mightily. This is the case in much of Indonesia, for example.
All said, measurements of sea level rise try hard to take these complications into account.
I’m not sure but I think Cape Cod might be in the forebulge region of the Laurentide ice sheet, so today the coastline there may be sinking in part due to post-glacial isostatic adjustment. Thus the question to ask is if recent sea level rise at Cape Cod has increased in rate (because the rate of isostatic change should be nearly imperceptible at the century scale).
It’s Saturday night - off to a party!
11 August 2007 at 10:01 PM
#59
Yes, I do understand the irrelevance of ‘fitting’ a single frequency sinusoid, - that was my entire point (I work in science/engineering so understand Fourier analysis) - Sorry, my remark was perhaps obtuse. I was simply criticising the hyper-analysis of linear trend segments in this data set (at arbitrarily defined endpoints, with subjective methods of pre-filtering the data)
#55
Yes, I work in research and publish myself. Have you involved yourself in an examination of the GISS data sets? Things are not as transparent as you readily state. The fact that this error has been present for so many years (referenced so often), and required great effort to reverse-engineer, indicates this is a serious problem.
It is not as if people did not care, and this was an error ‘missed’. A stream of scientists/engineers have been calling for a more transparent accounting, and we have been ignored under the absurd cackling of ‘Denialists!’
I think it is time for less social/political polarity on these issues, before things get entirely out of hand (if they have not already). The climate science community would do themselves no harm by applauding Stephen McIntyre’s, shall we say? - tenacity. Perhaps working with him in addressing other concerns raised within the CA sphere (that has the participation of many scientists/engineers). This is a different world from the traditional ‘publishing’ science - with serious ramifications for humanity and earth - therefore you will have to put up with us ‘outsiders’ having our say, auditing your data, questioning your results - even if it is outside the realm of our immediate expertise. This is the burden of having chosen such an important realm of scientific investigation - so be it.
regards, Steve
11 August 2007 at 10:40 PM
RE 98. Ataraxia. Pyrrhonism. Wiki it.
Hansen had several choices. See if you can name them all, grasshopper.
12 August 2007 at 12:26 AM
Steven (#118) wrote:
Steve,
There is a great deal of evidence besides US land-based surface temperatures which demonstrates that global warming is quite real and that it is accelerating.
I rattled off a list of twenty points (just before #57), where one of the most dramatic of which in my mind is global glacier mass balance - and by our best estimates, it threatens severe water-shortages for over a billion in Asia alone by the end of the century. Twenty largely independent lines of evidence that the world is undergoing a process of global warming, I will add, can and will take on a life of its own as various positive feedbacks come into play.
What is happening right now up in the arctic should be a pretty good indication of this. You guys might not take it seriously or might like to think it is all made up, but a number of national governments would appear to take it quite seriously - judging from how the US, Russia and Canada are trying to stake claims on the oil below - which until now have been out of reach.
However, your guys see the reading on one US temperature rise 0.02 degrees Celsius and the other fall 0.01 degrees Celsius, and are ready to say that global warming never happened. The mineralogist who crowned himself “climate auditor” is doing everything he can to encourage this. Somehow I think he is one of the last people in the world that one should ever call upon to keep a profession honest.
Your guys like to complain that you need more “transparency.” You can download the climate models if you want to, and if you are willing to look a little, you can generally find the data. Heck, you can get the raw data on all US land stations and stations throughout the world along with charts if you are willing to visit:
GISTEMP - EdGCM Dev - Trac
http://dev.edgcm.columbia.edu/wiki/GISTEMP
The guy in charge of the project is even trying to get pictures of all the stations for you and help you identify how close they are to urban centers. But somehow a bunch of guys snapping pictures who refuse to learn statistics, mathematics, physics or the simple fundamentals of the discipline they are attacking or even what information is actually available are going to tell climatologists how to conduct their science.
Give me a break!
I am no more of an expert than yourself, but at least I am willing to learn.
12 August 2007 at 1:07 AM
PS to #120
Steve,
As the charts in Gavin’s essay above suggest, it is quite true that the United States has had it easier than much of the rest of the world so far. Don’t count on our luck continuing as this progresses, though. If the models are as on the money as Jim Hansen’s twenty-year predictions (which were based on a single run with model much more primitive than today, mind you), we are going to make up for it.
By 2080, there will be a permanent dust bowl forming in the southwest and another in the southeast. We won’t be able to grow wheat south of the Canadian boarder. Wheat. In a world that will be facing severe water shortages, greatly reduced agricultural harvests and the depletion of fish populations.
This stuff is serious - and you guys are doing everything you can to stand in the way of the science because its telling you what you don’t want to hear. You want to continue with business as usual - even though we know where that will lead.
Oh well.
Maybe tomorrow I can quit trying to figure out the the twists and turns of human psychology and focus on learning more of the science. Genuine human achievement - in a world which may soon be seeing a great deal less of it. I guess we will see.
Time for bed.
12 August 2007 at 4:33 AM
Tim:”It turned out to be a letter to Nature. Not peer-reviewed as I understand it, but nevertheless of some quality, I believe.”
Hmm? Letters to Nature are certainly peer-reviewed, and held to the highest standards as they are the most well-read of all scientific publications.
12 August 2007 at 6:24 AM
I have been musing on why the “we need to audit the scientists” meme offends me so much. An exemplar of the meme from SCP in #23:
I’m not confident in the US data, let alone those from the rest of the world. Peer review is for science. The IPCC can do all the peer review they want. That doesn’t cut it for economies. Economies use accounting and audits. Failure to do so leads to situations like Enron and WorldComm. If we just want to do science, stick with peer review. If someone wants to influence economies, it’s long past time for auditors to start poking around.
If NASA doesn’t want to publish source code and all data, at a minimum maybe they should hire an accounting firm to audit climate related data, methods and processes and to issue a public report on the quality they find (and that firm should hire Steve McIntyre ; -).
Leaving aside the irony that both Enron and WorldComm were audited every year by an outside accounting firm, and the implied comparison of climate scientist with convicted felons, I have to wonder who the audit memers think they could hire to perform a valid audit? I’ve done enough data analysis to know that you have to know the field to know which data manipulations are valid and which aren’t. How does an accountant, say, decide which adjustments to the temperature records are acceptable? Accountants have the GAAP, FASB, and IRS code and regulations to guide them when they are checking a company’s books. Climatology doesn’t have any equivalent. Except, perhaps, for getting a PhD in Physics, Chemistry, Biology, Oceanography, or Geology and then doing post-doc time with climatologists. In the same way that a non-accountant is going to get hopelessly lost trying to audit exxon-mobile’s books, so will non-climatologists trying to audit climatology.
There is another dimension of difficulty with the audit meme. Unless the auditors are at least as skillful in climatology as those being audited, there is a powerful draw for the auditer to start seeing what is being audited as a tutorial: “this is how to solve this kind of problem and that technique works here” sort of attitude. And an auditor who falls into this trap will never find a problem.
Finally, you are not likely to entice a skillful climatologist into auditing someone else’s work.
Since we all make mistakes, how should we verify the work of climate scientists? Not suprisingly, science already has a mechanism: replicate results. The most effective way to check a scientist’s work is to try to reproduce what he did. And guess what, the climate science community is already doing that. Is there only one historical temperature data set? No. Is there only one GCM? No. Are there lots of studies that come up with a hockey stick? Yes.
So the answer to the audit meme is that old-time science: reproduce results. The climate audit folks are free to build their own temperature history and publish the results for scrutiny. They are free to build their own models and run them out to 2100 to see what they predict. That’s science.
(anybody want to start a pool on how long it is until someone starts blaming black helicopters on the IPCC?)
12 August 2007 at 7:43 AM
The Before and After sea ice extent images I linked to in #114 are from the Cryosphere Today.
The Before image is the saved version from the WayBackMachine in December 2006 (before all the data was changed), while the after is today’s graph (so it contains six months more data.)
12 August 2007 at 7:50 AM
Here are two Visible satellite images from a few hours ago of the NorthWest Passage (the satellites don’t centre right over it so you need a few different pictures.)
You could probably get through with a ship right now.
http://rapidfire.sci.gsfc.nasa.gov/realtime/single.php?2007223/crefl2_143.A2007223202001-2007223202500.4km.jpg
http://rapidfire.sci.gsfc.nasa.gov/realtime/single.php?2007223/crefl2_143.A2007223184001-2007223184500.4km.jpg
12 August 2007 at 9:00 AM
Gavin,
I still have not seen an explanation for the adjustment to years prior to 2000. The error McIntyre found should only have affected data for years 2000-2007. Why the adjustment to 1998 and 1934?
[Response: The corrections were of different signs in different stations. The urban adjustment uses the rural trends (which may have changed slightly) to correct urban stations and therefore the correction may differ slightly from what was used before. That made quite a few 0.01 or 0.02 differences before 2000. Compare this (from May) with this (from July). Note as well that occasionally there are corrections and additions to the GHCN data for earlier years that get put in at the same time as the monthly update. Thus this magnitude of difference - fun thought it might be to watch - is not climatologically significant. - gavin]
12 August 2007 at 9:41 AM
If 1934 had been the warmest year globally, we may have expected that 5000 year old Alps ice man to have melted back then, instead of in the early 90s. Where’s their thinking cap? Though, of course, local weather can be different from the global average.
12 August 2007 at 10:22 AM
I read in climateaudit.org that GISS does not share its code/algorithm for “fixing” the data. Gavin can you inform us why so? Just curious.
[Response: The algorithms are fully described in the papers - basically you use the rural stations trend to set the urban stations trend. It’s a two-piece linear correction, not rocket science. - gavin]
12 August 2007 at 10:35 AM
European Temperature records are also wrong
12 August 2007 at 11:56 AM
John Wegner (#125) wrote:
John,
Regarding the sea ice anomalies…
Ok. The images which you are using are the correct images.
However, I would look at them very closely. For example, the low that you are picking out for 1996 (actually the end of 1995) is especially narrow. Given the size of the graphic, it isn’t that great of a surprise that it isn’t showing up from one year to the next. I picked out the low for the end of 2005, put my finger on it and allowed your graphic to switch from one year to the next.
Blammo! Same point.
Principally what you are seeing is compression as the result of including one additional year in the same size graphic. It gets rather dramatic in terms of the difference in the images as you get towards the end, but this has to do with how quickly things are progressing in the arctic. Incredible.
Now I believe you can find all of the data files on the web. However, what I am seeing is principally gzs in ftps. And there is a great deal of data in those files. Additionally, the methods used to process those files into the graphics that you see at various sites is described in some detail, although perhaps not as much detail as I might like.
You’ve got to remember: the people who process the files and produce the graphics have jobs. The graphics are essentially labors of love. And to describe them in the detail that you or I might like would be time taken away from something else. If you want them to produce the level of detail, step by step instructions for the reproduction of their graphics, this will take resources.
But no doubt there is more that would be required to make this genuinely available to everyone with a net connection. For example, you would want to have the software available for free. But it would be pretty big. So preferably the software should reside on an internet server, perhaps as perl. This way anyone would have access to it. But then since a fair number of people might be using it at the same time, you will want the servers to be able to handle the traffic. More resources. The you should have the actual code being downloadable. More resources.
Are you beginning to get the picture? What all the “auditors” out there want is to have all of the resources dedicated to making it possible for them to duplicate what the experts do point-for-point and pixel-by-pixel. But they want these resources without any expansion of the budgets for the relevant agencies and organizations. And this means time and money taken away from a great many other things.
Anyway, someone who is more familiar with the rendering of data or of the statistical methods which are used might be able to say more. I myself have only a basic understanding. But I do know how to put my finger on the screen. And I believe that little more than this is required to understand the differences between the two images that you see.
My apologies for suggesting that you might have manipulated the images. Clearly you haven’t. I was just remembering the traveling decimal point from a few weeks back.
12 August 2007 at 12:28 PM
Count Iblis (#128) wrote:
Yep.
Or to be more precise, they were “wrong.”
Following your link I see that we had previously underestimated how bad things currently are relative to the past - because we had overestimated how bad things were in the nineteenth century. This means that our current trajectory is worse than we thought and that things are likely to get worse than we thought sooner than we thought.
Nice spin though, Count Ilbis.
12 August 2007 at 12:54 PM
Gavin,
In #126 above, I asked about the changes prior to the year 2000 when the error McIntyre found related to years 2000-2007. I was, of course, referring to changes to U.S. temp anomalies as that was the dataset that NASA changed. Your answer did not directly address why years prior to 2000 should be affected. Are you saying that in the corrected dataset NASA lumped in some other adjustments as well as the adjustments required to fix the error McIntyre found? Also, you linked to global temp anomalies which does not address the question. For US temp anomalies, the old version is found at http://web.archive.org/web/20060110100426/http://data.giss.nasa.gov/gistemp/graphs/Fig.D.txt and the new version is at http://data.giss.nasa.gov/gistemp/graphs/Fig.D.txt As you can tell by comparing the two, a large number of adjustments were made in years prior to 2000, but (as far as I know) no explanation for these changes has been made. Has NASA explained these changes somewhere so I can read more about them?
[Response:Actually, that old version is from 2006. I can’t say from this whether the changes were from the latest correction, or simply additions/corrections to the database from GHCN. If it is from the latest correction, it’s because the GISS urban adjustments come after the step we are discussing now. Therefore those changes could propagate down the line. Since the adjustments are linear in nature, the further back in time you get the more they’ll have an impact I suppose. - gavin]
12 August 2007 at 1:30 PM
Gavin, thanks for the links. I looked at Hansen’s paper on the urban correction/surface temperature (my first cursory look at a climate science paper… makes Ed Witten’s papers look terse). Seems pretty simple I agree (even simpler than Classical Mechanics, i.e., rocket science). Hope this is not too simplistic and that your models are predicting future trends accurately and verified repeatedly as physicists usually expect. Anyways, good luck and thanks for responding to curious strangers.
12 August 2007 at 1:47 PM
Gavin - response in #128 and earlier - I can’t tell whether you are being deliberately difficult or genuinely don’t understand some people’s concerns.
Yes, the papers describe what the algorithms SHOULD do but does the ACTUAL CODE do exactly what you think it does? Clearly in one instance (at least), it did not.
You seem to imply that the code is so simple that it does not need to be released, it only does A+B=C but if the actual code is released, anyone can verify that it correctly does A+B=C. Until the actual code and the basic station data that it processes are released, you are going to get some people wondering if there are more mistakes and they want to see exactly how corrections to individual data sets are applied. Science should be completely transparent and reproducible - so show people exactly how you arrive at the actual numbers. If the coding is fine then you are not vulnerable on that accusation - sure, people will then argue whether the UHI corrections (for example) are correctly applied or of the right magnitude but that is how science advances. If there are further coding problems then swallow your pride and thank whoever finds them for helping everyone understand the science better.
Judith Curry makes excellent points in #69 - even if you dismiss others as skeptics or deniers, please listen to her!
12 August 2007 at 2:46 PM
RE 101.
Timothy is not practicing “deconstruction” A deconstruction of my text would involve a variety of techiques to show how distinctions I used or binary oppositions I used were undermined by themselves in the text. Or perhaps how the implications of various tropes ( figures of speach like metaphor ) served to undermine the “stability” of the text, leaving one in a state of not knowing or not being able to fix or determine the meaning of the text. In short, deconstruction, is a method of commentary that multiplies the meanings of texts and shows how they cannot be controlled by method.
That’s the way Derrida explained it to me. Although I find it ironic that his anti method has come to mean “any kind of critique”
12 August 2007 at 3:14 PM
“Yes, the papers describe what the algorithms SHOULD do but does the ACTUAL CODE do exactly what you think it does?”
that would be the purpose of repeating the experiment yourself. the code would not need to be released to allow another person the ability to verify it. really not a hard concept.
also, the above corrections were not code errors as far as I’ve read.
12 August 2007 at 3:58 PM
The happiness of the denialists in this case is the same as the happiness from the Creationists/ID crowd when a part of evolution theory turns out to be incorrect or incomplete. They can use it to say that the scientists were wrong… This is actually a fantastic example of science at work. Somebody found the error, it was corrected. good.
12 August 2007 at 4:31 PM
#134: IL
I’m all for as much openness as we can get, but:
IL: how much software have *you* written, QA’d, documented, released, and made available *for free* (or close)? Have you done the work to make code portable among machines? Do you ship extensive test suites?
Do you understand that all this costs money and time?
Please make a case that you actually understand this well enough to have a non-naive discussion. For instance, I think a fine topic for RC would be:
+ start with the general issue of current science publishing that involves computer generated results. What’s the tradeoff between:
- (one extreme) just publishing the results, like in the old days and
- (other extreme) provide complete, portable, well-software-engineered source code, documentation, makefiles, extensive test suites, with code tested on a variety of platforms ranging from individual PCs (Windows, Macs, Linux, at least), through supercomputers.
[This takes a lot of work, and not all science researchers have professional-grade software-engineering resources available. Should that be required by journals?]
+ apply this in the climate research world. How much effort does this take? What do people do?
[There are serious arguments in other disciplines about similar issues, i.e., there are real issues about which reasonable, informed people can disagree.]
My personal opinion is that GISS does a very good job on software engineering, release, and accessibility … but then I only have 40 years experience with such things, so maybe I’m still naive about it, and could certainly be convinced otherwise by informed arguments.
IL: Assuming you are American, how much more taxes are you willing to pay to increase the resources for government agencies to put more of their code professionally online? There are lots of other government numbers generated by computers, and it is by no mean obvious that spending $X more for GISS to do even better would be more generally useful than spending $X on agencies whose results are also important, but whose code is far less accessible.
If you’re not American, you need to convince those of us who are that we should spend more money on this.
For instance, total areas, or world’s 510,065,600 km^2:
17,075,200 km^2 Russia (3.3%)
9,984,670 km^2 Canada (1.96%)
9,826,630 km^2 USA (1.93%)
2,166,086 km^2 Greenland (0.42%)
14,056,000 km^2 Arctic Ocean (2.76%)
If I *really* wanted to improve the accuracy of *global* numbers, I’m not sure I’d spend most of my effort chasing USA numbers around.
In particular, if I were Canadian, and I really wanted better numbers, I’d be chasing:
http://climate.weatheroffice.ec.gc.ca/Welcome_e.html
http://www.msc-smc.ec.gc.ca/ccrm/bulletin/national_e.cfm
The latter says “spring temperatures” +1.5C over 60 years, i.e., ~.25C/decade. Is that accurate? Has the code been checked? Canada is slightly bigger than the USA and therefore carries slightly more weight in global calculations, although of course each are only ~2%. Canada has plenty of seriously-rural weather stations. Has any auditor visited them, lately? Russia is 3.3%, but Canada is probably easier to check, especially for Canadians.
If one *really* wants to get more accurate calculations, Canada is slightly more important (by area) than the USA, although, of course, the numbers might not be to everyone’s taste, since climate science expects Canada to warm faster than the US.
Note: none of this is intended to ding Canada: I actually care about Canadian temperatures and precipitation because we own ski condos in B.C. and pay taxes up there.
12 August 2007 at 4:33 PM
#134
In my own field it is actually considered best to NOT release code, but give algorithms and data. The simple reason being that code is hard to read and misstakes in someone else’s code are easy to miss. Instead it is much better if the person wishing to verify a result writes his/her own code directly from the described algorithm. If the new program does not give the same result then either the algorithm is incorrect or one of the two programs.
Normally it is easier to write your own new code than to completely understand the other persons code too.
12 August 2007 at 5:08 PM
steven mosher (#135) wrote:
That is hard deconstructionism - well-explained in synoptic form.
From what I understand, soft deconstructionism may simply distinguish between meaning and significance, where the meaning of the text is independent of either the author or the reader, but is dependent upon a shared language, but where the significance will be dependent upon the reader’s context, which will include their historical context, the issues of the day, their values and standards. There will be ambiguities, but these can generally be resolved by reference to the larger context within the original text. But of course the significance, even for the author, may best be identified and explored by bringing in other elements.
However, the methods of hard deconstructionism typically consist of tearing one element or another out of its original context so that the reader will no longer refer to the original text, but will instead focus on the secondary text’s elaborate schemes of overinterpretation through ambiguous, archane and often highly idiosyncretic language where the interpreter seeks to impose his or her own often political agenda upon the original text, or alternatively, seeks to show that the written or even spoken word offers only the illusion of communicating the intent of the writer or speaker.
Of course, if the latter were true, the hard deconstructionist would be entirely incapable of communicating it. What we are speaking of at this point is simply an elaborate attempt to “demonstrate” the self-referentially incoherent position of radical skepticism with respect to all intended meaning.
12 August 2007 at 5:24 PM
Timothy (56), sorry but positively as simulating whole bunches of possible indications, most of which have measurement errors greater than indicated trend, and most of which come with very mushy cause and effect, into an undeniable global truth is every bit as bad, scientifically, as skeptics (but not me) justifying our position because the AGW theory is not 100% proven in every aspect and detail and tied up in nice little packages with pretty bows. You forgot the “cherry trees blossomed earlier last year” as another undeniable “proof”.
12 August 2007 at 5:26 PM
Re #139: DavidU — I agree that writing my own code is often easier than understanding someone else’s.
However, when the two programs do not give the same answer, it is oft the case that both are wrong!
12 August 2007 at 6:58 PM
To keep this in proper perspective, the area of the U.S. is 3.6 million square miles(9.3 million Sq.km) and the surface of the globe is 197 billion(with a B) sq. Mi.or 509.6 billion sq. Km. The ratio is 1.8×10^-5. Too small for any global consequences. Global warming isn’t affected.
We’ve learned once again that to err is human.Which is why peer review, and constructive criticism are so important to the sciences, and well as other in other areas of human endeavor.
12 August 2007 at 7:32 PM
Re #138:
First, for my credentials — over the years I’ve personally written somewhere on the order of 500,000 lines of code. I’ve also designed another several hundred thousand, reviewed several million, and QA’d some unknown amount. There are a lot of people here who use code I’ve written over the years. Some open source, some proprietary.
ANYWAY, it doesn’t cost anything at all to open source software, except some FTP bandwidth, and it greatly improves both the quality and functionality of the code in the long run. And frankly, if the code is so hard to comprehend that no one else with programming skills can read it (I have 28 years in the industry), the programmers need to be fired.
If someone were asking me my opinion, I’d say that the people who are spending the money don’t want to lose their budgets for either software developers or giant computers. That’s one of the biggest reasons software is kept closed.
12 August 2007 at 7:38 PM
I want to Contrast the stand up attitude of Ruedy, the good sense of Dr. Curry and Gavin’s openness with something quite different.
Here is the start of an email, written by a principal in this matter. I will not name him.
“Recently it was realized that the monthly more-or-less-automatic updates of our global
temperature analysis (http://pubs.giss.nasa.gov/abstracts/2001/Hansen_etal.html) had a flaw in
the U.S. data. In that (2001) update of the analysis method (originally published in our 1981
Science paper – http://pubs.giss.nasa.gov/abstracts/1981/Hansen_etal.html) we included
improvements that NOAA had made in station records in the U.S., their corrections being based
mainly on station-by-station information about station movement, change of time-of-day at
which max-min are recorded, etc.”
It was realized. Mistakes were made.
Some examples from other areas and how they responded:
http://www.the-scientist.com/news/display/39805/
http://pineda-krch.blogspot.com/2007/06/show-me-code.html
Quoting from the latter to show the guy at columbia how it us suppose to be done.
“Dear Colleagues,This to inform you that we must retract Hall, B.G. and S. Salipante. 2007. Measures of clade confidence do not correlate with accuracy of phylogenetic Trees. PLoS Comp. Biol 3: (3) e51.As a result of a bug in the Perl script used to compare estimated trees with true trees, the clade confidence measures were sometimes associated with the incorrect clades. The error was detected by the sharp eye of Professor Sarah P. Otto of the University of British Columbia. She noticed a discrepancy between the example tree in Figure 1B and the results reported for the gene nuoK in Table 1. At her request I sent her all ten nuoK Bayesian trees. She painstakingly did a manual comparison of those trees with the true trees and concluded that for that data set there was a strong correlation between clade confidence and the probability of a clade being true. She suggested to me the possibility of a bug in the Perl script. Dr. Otto put in considerable effort, and I want to acknowledge the generosity of that effort.”
12 August 2007 at 7:40 PM
For those interested, I’ve posted a comparison of GISS data before vs after applying the recent correction on my blog.
12 August 2007 at 8:09 PM
Land area Canada 9.09 million square km
Land area US 9.16 million square km
12 August 2007 at 8:16 PM
#120
Thanks Timothy for your response.
My goal is not to debate every point on your list, but rather to assert that one can be an AGW skeptic and still of sound mind. You presume that someone with a different view on the merit of this science must have some sinister motive - this is a polarising and simplistic view.
In the scientific community I am part of, members would not mind me proposing the following refutations to your first two points, as it would provide them an opportunity to correct my misunderstanding of the underlying ideas.
1. We have surface measurements in the United States which show an accelerating trend towards higher temperatures.
If I low pass filter the US surface temperature time series and examine the derivative of the signal, I see no evidence for this accelerating trend. The rate of recent increase (and length of its duration) is comparable to earlier periods in the record.
2. These are temperature measurements being taken by planes and satellites, and they show that the troposphere is warming - just as we would expect.
I believe the latest data does not support this point (for the tropics) and certainly does not match climate modelling predictions.
Christy J. R., W. B. Norris, R. W. Spencer, J. J. Hnilo (2007), Tropospheric temperature change since 1979 from tropical radiosonde and satellite measurements, J. Geophys. Res., 112, D06102, doi:10.1029/2005JD006881″
“Several comparisons are consistent with a 26-year trend and error estimate for the UAH LT product for the full tropics of +0.05 ± 0.07, which is very likely less than the tropical surface trend of +0.13 K decade−1.””
regards, Steve
12 August 2007 at 8:44 PM
Re 140: Thank you, Timothy Well said.
12 August 2007 at 8:45 PM
As The former Mayor of New York once said “When I make a mistake it’s a beaut”. The Surface Area of Earth is
197 million square miles ( with an M). What are three orders of magnitude among friends? It’s still holds that global climate figures aren’t affected.
12 August 2007 at 9:07 PM
Re: #143 (Lawrence Brown)
Sorry, but the area of the globe is 196 million (with an “m”) square miles, about 510 million (with an “m”) square km.
To err is indeed human.
12 August 2007 at 10:59 PM
re: #144 F.C.H.
Well, I said “there are real issues about which reasonable, informed people can disagree.” [FCH at least has relevant expertise, which is not clear for some people expressing strong opinions.]
After all, “open source” is not new in computing, having started no later than the early 1950s with John von Neumann’s distribution of the IAS plans, and continuing exchange of code via user groups (like SHARE & DECUS) in the 1950s/1960s onward, the UNIX dispersion of the 1970s, the related Software Tools User Group of the 1980s, and lately Linux, Apache, etc, etc.
Technical users (especially) have long shared code, but thankfully it has indeed gotten a lot easier to share, given the Internet and WWW. Making tapes gets old [at Penn State, in the early 1970s, we made hundreds of tapes of code I’d written, and that actually cost money] … but it cost more to design professional-grade distributions, and if there hadn’t been some grant money to help, we wouldn’t have done it. This was for code we knew people actually wanted to *use* daily, not just maybe look at to see if they could find bugs.]
The original UNIX distributions from Bell Labs originally said ~ “No warranty, no support, and don’t call us with bugs.” … but people knew perfectly well that we would in fact spend some time informally supporting it. Of course, we had monopoly money to play with…
Anyway, the issue is that in organizations in which researchers are supposed to generate research, it is a legitimate argument to figure out
- what level of source/data/test/documentation availability is appropriate,
- how much time researchers should spend responding to questions,
- what level of availability is required to publish in various journals
- and the tradeoff between getting results out in a timely fashion, with normal peer review (and consequently, unfound), versus doing excruciatingly-long testing, outside code reviews, etc, etc.
I don’t think there is any one right answer. I do think it is important for science to figure out answers, and if we raise the bar, we’d better figure out how to raise the funding, because it usually costs money - Sometimes it’s worth it, sometimes it’s not.
When I was at SGI, we went through a major exercise to figure out what we had to do to *usefully, professionally* make certain pieces of internal code open source,[not just stick them up on an FTP site], i.e., including the support found to be necessary.
Many of us wanted to do this, but we found that it simply wasn’t free, so we did it (spending a fair amount of money) for some things (like XFS -> Linux), and we didn’t for others, like Showcase. We also spent money on things like supporting Samba [i.e., hiring Jeremy Allison for a while to do that].
I would be delighted if every scientific paper that used computers offered a professional-grade distribution … but I can’t make myself believe that would be cost-effective. I have many times been in the position of doing budgets for projects where there was a choice between just building/using software and doing the work to make it usefully accessible elsewhere, and I’ve opted for the latter as often as possible, but it *was not* free.
Others may legitimately have other opinions, although I’d hope people will say why they have relevant experience (as FCH did).
12 August 2007 at 11:21 PM
Roger Pielke Sr and 14 others published an article in the latest BAMS noting problems with USHCN adjustment procedures beyond TOBS. Is it really possible to correct for station moves…especially in areas of complex terrain? Is it really possible to correct for UHI when land use changes around the station have not been documented very well, if at all? One flaw in the GISS adjustment code has been discovered. How do we know if there are more flaws? Gavin says that there methodology has been well documented, however, how do we know if that methodology was implemented correctly in the code? I understand that the U.S. has one of the best climate networks in the world (and better history information). What about Africa and Asia? It is clear to me that we don’t know what is UHI and what is a CO2 signal in our temperature records. That said, I know that the climate over the Northern Rockies (where I live) is changing. Our winters are milder and our snowpack hasn’t been doing well. It could be all the junk that China is putting in the air is affecting the weather patterns over the Pacific. Eventually that affects patterns over the Northern Rockies. Others say that this pattern change is CO2 induced. Neither side has yet to prove their case. I know the GISS people are high on CO2 causation, but they lost credibility when they keep their code secret (mistakes and all) from those who want to check their work.
13 August 2007 at 1:06 AM
This is how Stephen Mosher says “it should be done…”
So Stephen apparently thinks that it’s OK that the researchers didn’t open-source the Perl script, nor that the researcher uncovering the error had to do a lot of work.
On the other hand, NASA is Evil! Evil! Evil! for not having open sourced their code, and Evil! Evil! Evil! for writing an impersonal and slightly ungrammatical e-mail that correctly described what happened.
13 August 2007 at 2:45 AM
#147
Land area Canada 9,984,670 square kilometers
Land area US 9,629,091 square kilometers
#144
One thing you need to keep in mkind here is that the “programmers” in many scientific projects are not professional programmers, but rather just one of the climatologists/biologists/chemists… in the project who knows how to program, and sometimes learnt it 20 years and are still using the same progamming language as back then. People still write code in line numbered Fortran77!
These programs usually work just fine, they know how to write workign code, but to someone like the two of us who knows programming as a subject of it own it often looks hairrasing.
Hiring a professional programmer to do the work instead is normally not even close to realistic within a typical project budget, so there really isn’t anyone around to fire for their hard to read code.
#142
I agree with your point too. I actually often write two separate codes for doing the same job, in different languages, and don’t trust them until they both give the same results.
13 August 2007 at 5:58 AM
Timothy. RE 98.
Apparently the blog at my reply ( same thing happened over at Watts place, so I blame my Wifi )
I’ll give a condensed version:
You wrote:
“As such, they are responding to sources of contamination which may not have been regarded as that important - prior to 1929. Not only does this have little to do with current trends, but it demonstrates a concern for accuracy. And as a matter of fact, the elimination of such contamination tends to reduce the apparent rise in temperatures over the twentieth century rather than magnify it.”
Actually the don’t show contaimination they Hypthesize it to explain the record. They took cooling out
My main issue with that paragraph in Hansen 2001 is the lack of supporting data and analysis in the text or figures. If you like I will detail all the missing parts. But look for yourself. Find the ANALYSIS in the text. Not a description of the analysis. Simple example:
which sites in the “region” were the 5 sites in question compared to?
Next:
“In an earlier thread you claimed to be interested in accuracy and were attacking urban sites for their presumed contamination by the urban heat island effect. We of course pointed that you get virtually the same warming trend whether you use all stations or just rural stations - which you didn’t even seem to acknowledge.”
Let me schematize the UHI arguement for you using Peterson 2003, which Hansen cites ( as submitted) and which Parker quotes.
1. UHI exists: ( I’ll link studies if you like),
but see Petersons FIRST SENTENCE.
2. We expect to see differences between Rural and Urban stations.
3. These differences are NOT observed ( peterson, Parker, Hansen)in the climate network
4. THEREFORE, Urban stations must be well sited in COOL PARKS.
From Parker:
Furthermore, Peterson (2003) found no statistically
significant impact of urbanization in an analysis
of 289 stations in 40 clusters in the contiguous United
States, after the influences of elevation, latitude, time of
observation, and instrumentation had been accounted
for. One possible reason for this finding was that many
“urban” observations are likely to be made in cool
parks, to conform to standards for siting of stations.
From Peterson’s conclusion:
“Therefore, if a station is located within
a park, it would be expected to report cooler temperatures
than the industrial sections experience. But do
the urban meteorological observing stations tend to be
located in parks or gardens? The official National
Weather Service guidelines for nonairport stations state
that an observing shelter should be ‘‘no closer that four times the height of any obstruction (tree, fence, building, etc.)’’ and ‘‘it should be at least 100 feet from any paved or concrete surface’’ (Observing Systems Branch 1989). If a station meets these guidelines or even if any attempt to come close to these guidelines was made, it is clear that a station would be far more likely to be located in a park cool island than an industrial hot spot. ”
SO, you get the argument: we expect a difference, we find no difference, THEREFORE urban sites are in cool parks.
Simple Question: How do you test this last part?
You look. Go to Surfacestations. Look at Tuscon ( parking lot) Eureka ( on a roof) Santa Rosa ( on a roof)
Paso Robles ( on cncrete next to a freeway) And newport beach ( on a roof)
So now the argument looks like this.
1. UHI exists:
2. We expect to see differences between Rural and Urban stations.
3. These differences are NOT observed
4. Perhaps, Urban stations are well sited in COOL PARKS.
5. We haven’t found many if any Urban sites in a cool park.
6. Perhaps the Rural sites are corrupted at the MICROsite level. Things not visible on nightlights,
things like nearby asphalt, buildings, wind shelter,
Non compliant things.
So, how to test #6. Look. Take at look at Tahoe City and Happy Camp.
Essentially its the same logic as Hansen. He saw weird cooling and assumed the sites must have smething wrng with them and Hypthesized contaimination. We see similair weirdness ( No difference between Urban rural )
and hypothesize that rural sites have microsite issues.
Then we take the extra step of LOOKING. Peterson never checked his Supposition of COOL PARKS. We did. After checking 289 sites we havent found an urban site in a cool park. In parking lots? Yes, on Rooftps Yes, on Utility Poles, Yes.
Now, I would not call the matter closed. It’s never closed. If you like pick up a camera and go find a cool park site.
13 August 2007 at 6:44 AM
RE 56: Long Post timothy.
I’ll hit a few key points:
“Sounds like Quine to me.
Actually he’s not all bad - at least the bits he gets from Pierre Duhem.”
Really? Quine’s Two Dogmas was published in 1951 and Duhems work was not allowed to be published until 1954.
ZING!
Next:
“Now what about the claim that the world is at least five seconds old.
I certainly think it is, but you may regard this as nothing more than a “theory”
which is “underdetermined” by the “data,” where the data consists of the experience
of “memories” and anything else which I might claim as evidence for an older world.
But at this point we aren’t even talking about observation per se - we are talking about memory.”
Quine would say it’s a theory. A well supported theory. One that would be very complicated to
give up.
NEXT:
“So does this mean that when I look at an iceberg floating off,
it might actually not be floating away? ”
No. There is a choice. A theory of human perception ( that explains perceptual illusions) Or a theory
of human delusion. The first is more useful than the second. Both are underdetermined.
“Does this mean that if I am looking at the temperature displayed by a thermometer
I am holding is rising, that the temperature might not be rising? ”
No. There is a choice this time three options. The last being “instruments accurately record physical quantities”
Again, Quine would say they have the same epistemic status. Some are more useful at things like predictin
Next:
“If so, I would begin to wonder whether you are engaging in the philosophy of science
rather than in some freshman philosophy bullsession.
I would also have to wonder just how desperate the “global warming skeptics”
have gotten that they find it necessary to appeal to this kind of reasoning.”
I think someone who didnt even know that quine published before Duhem needs to actually read Quine.
And you will find that Confirmational holism makes one more amenable to AGW acceptance.
NEXT:
“Falsification can always be avoided by appealing to other data ( sea ice, SST, species migration, etc etc etc).
This isn’t the way that I normally hear it.
From what I understand, falsification can always be avoided by appealing to another hypothesis.”
Both ways actually.
NEXT:
“But lets focus on the phrase “appealing to other data.” …..
A hypothesis or theory which is justified by multiple lines of investigation
is generally justified to a far greater degree than it would be if it were simply
justified by any one line of investigation considered in isolation.”
Yes, just as quine says.
NEXT:
“Now the vast majority of the scientific community has accepted the view that:
1. The earth is getting warmer;
2. greenhouse gases are largely responsible for this; and,
3. That what has been raising the level of greenhouse gases are human activities.
You on the other hand are still stuck on (1). Not dogmatically denying it,
I understand, but simply doubting it with your healthy, “scientific” skepticism.”
Stuck on #1. Presently I am looking at #1. Have to start somewhere. Now, force me to decide, and I will say
Yes, the earth is probably getting warmer. That “probably” needs to be quantified and independantly confirmedand it’s magnatude estimated.
NEXT:
So in the interest of science, lets look at the evidence:
1. We have surface measurements in the United States which show an accelerating trend towards higher temperatures.”
Hmm. Which ten year trend shows a higher rate: 1997-2006 ( last ten) or say (1927-1936)?
Lets start with that one. I’ll address the other 19 in due course, but first things first. The simple task
of measuring air temps.
13 August 2007 at 6:49 AM
[[If I low pass filter the US surface temperature time series and examine the derivative of the signal, I see no evidence for this accelerating trend.]]
Why didn’t you just do a linear regression like everybody else in the world? You’re essentially admitting that you had to distort the data to get the result you wanted.
13 August 2007 at 8:23 AM
Why is there an assumption by the deniers that the US temp. data is the best?
Is this true? I find that my reginal NWS daily temp figures are often several degrees under what my home temp. gauge is recording.
13 August 2007 at 8:47 AM
RE 154.
You missed two points and invented a third.
1. The point was not the grammer. The point was attribution. Ruedy wrote a fine mail thanking SteveMc
for his work. The example I sited showed proper attribution for finding an error.
Like so for the Slow : ” I would like to thank Person X for finding..”
2 Nothing was said about NASA. I did not name the person who wrote the mail. And as far as the
accuracy goes you need to do some more reading.
3.Who said evil? you. If I had to use a word it would be obsfucating or opaque or lacking grace.
Some people like Ruedy, Dr. Curry, and Gavin are gracious. Hell, Gavin puts up with me.
Other folks are less gracious, myself included. Let’s leave it at that.
13 August 2007 at 8:49 AM
Steven (#148) wrote:
I am not a member of the scientific community - simply a philosophy major turned programmer who is on the outside - and with little practice in statistics. As such, there are certainly people that I would defer to, particularly in this area. Tamino is one obvious case - as I am impressed with his objectivity and skill.
Going off memory at the moment, I remember that the later trend of perhaps the last fifteen years was higher than the trend that of the last thirty, but I believe it was not statistically significant, and I had originally mentioned as much. There may be a valid approach which can demonstrate statistical significance. For example, one might perform an analysis in terms of the monthly averages and filter out the annual cyclical behavior, but I do not know the results of such an exercise. At present I would have to agree with you on this point and I will omit this statement in the future until I know otherwise.
(One point: even if it is demonstrated at some point that there is such an “acceleration,” I would expect it to be temporary, with a new higher slope being established until such time as we are able to reduce the geometric increase in the rate at which we are emitting carbon dioxide into the atmosphere. We have actually done worse in the past seven years than the previous ten, from what I understand.)
Steven continues:
There are other obvious examples. What is happening in the arctic is progressing much more rapidly than any of the models would predict - and I suspect that this will lead to more rapid climate change at a global level than models would predict. Alternatively, they are having some difficulty modeling the Indian Monsoon and a small western region of the Tibetean Plateau where glaciers are currently increasing in size as the result of increased snowfall.
The ability to observe the effect does not automatically imply the ability to model it - insofar as models have to be grounded in the actual physics, not the mere ability to identify trends through statistical analysis. I am sure there are others.
But the B scenario of Hansen’s 1988 projections performed admirably well in terms of global temperature - and it was primitive by today’s standards, for example, in the fact that it was based upon a single run. But overall, they are doing quite well, for example, in terms of the modeling of ocean circulation as of roughly 2001. The new GISS model is performing far better at modeling clouds, although there continue to be surprises in this area. For example, the recently discovered twilight zone where an invisible zone of higher water vapor extends for several kilometers beyond the visible edges of clouds. Likewise the modeling of both aerosols and the carbon cycle are at fairly early stages of development.
But I strongly suspect that Hadley’s new approach involving the initialization with real world data including natural variability will become the norm - which should improve both the modeling of climates at a variety of levels. Improved resolution will likewise improve model performance. We were able to model hurricane formation only with the creation of the NEC Earth Simulator - but then in 2003 projected increased cyclone formation in the South Atlantic, and Catarina formed in late March of 2004.
*
In any case, I genuinely appreciate the correction regarding temperature trends in the United States and will keep this in mind from now on.
13 August 2007 at 9:17 AM
Steven Mosher,
I will have to get back to most recent posts of 156 and 157 a little later. Currently I am about to go off to work. I hope you will understand. But if someone else would like to respond in the interim, I most certainly wouldn’t have a problem with this, although I would likely still personally respond later.
13 August 2007 at 9:50 AM
Re: #13, #18, #21, #22, #27, #30, #33, #41, #74, #79, #135, #145, #156, #157, #160
Steven,
Having read each of your posts, in which you use some fancy, uncommon words and refer to theories which I have not seen expressed before (which of course means nothing more than this: you have some education that I don’t have), I am left with one question: How, applying the theories you espouse, can you ever come to believe anything?
That is a serious question, and I guess we can approach it from the other side as well: What would it take for you to accept that this long list of unusual climate observations are related, and that they describe a coherent theory which is a) highly plausible and b) much more plausible than any other theory which has been applied to the same set of observations?
As Gavin said, in so many words: how do you explain these observations without a human component?
Further: academic exercises are all well and good (and I mean exactly that: they are essential and useful); but at what point do we put down the textbooks and notebooks and start trying to enact policies based on our observations?
13 August 2007 at 10:00 AM
Gavin started this topic with the sentence: “Another week, another ado over nothing.” How fitting! Now, 162 comments and 20 blog entries later, one has to wonder at most commentators’ ability to get worked up about something that is supposed to be nothing. But I suppose their zeal will not be diminished until the last denialist, skeptic, heretic or dissident has been silenced. This “ado” seems to be part of a recurring pattern in human behaviour.
13 August 2007 at 10:25 AM
Freeman Dyson has written a nice article in “Edge” expressing a heresy that GW climate models may be wrong. I think he makes too much of climate models and ignores other evidence, but there is no denying he is still a very smart guy.
http://www.edge.org/3rd_culture/dysonf07/dysonf07_index.html
Any comments on his thoughts?
[Response: Michael Tobis has a good commentary. - gavin]
13 August 2007 at 10:46 AM
My understanding is that McIntyre pointed the finger at something that was wrong, but that the NASA group actually figured out what it was and did the corrections.
So the scenario’s a bit different than in the letter you cited, where the woman in question did a huge amount of work and figured out in detail what was wrong.
And attribution has nothing to do with correctness, anyway.
And my point about the source code and work involved stands:
Note that she computed the phylogenic trees by hand, not using the script, and deduced that there must be an error in the script.
Publishing algorithms, not code, is SOP in many fields, climate science is not exceptional in this regard.
13 August 2007 at 12:01 PM
Re: #158 (BPL)
I disagree. A low-pass filter is a standard, and rather reliable, method of removing fast fluctuations from data, revealing the slower “trend”-like signal. I certaintly wouldn’t characterize it as an attempt to “distort the data.”
Unfortunately I don’t have monthly data for the lower-48 U.S. states, just annual averages. But looking at that data, I too don’t see any statistically significant acceleration in recent years (since 1975) in lower-48 U.S. temperatures.
13 August 2007 at 12:50 PM
I don’t want to start a tempest in a teapot, but what is the deal with Hansen’s email?
13 August 2007 at 12:57 PM
If the lower 48 US states have not been significantly warming (is that what people here are saying), then there must be other places that ARE significantly warming, if the global warming idea is accurate. Isn’t that that whole idea behind “the global average temperature” increasing? Avergage means that some places might stay the same & some might even get colder, but on average the whole data set for the entire world shows a warming trend. (And, it might be that the places currently staying the same or getting colder might eventually show a warming trend, if the problem continues.)
And if there’s lots of quibbling about how correct the data is, can we also use some other measures of temperature change, such as ice melting. I think the net melting in the Arctic, and the melting in the Old and New Worlds that has allowed some archaeological finds from thousands of years ago makes a good case that the world is warming. Not to mention the warming of the oceans.
Then there is a good theory to go along with this — the greenhouse effect — which not only explains the natural warmer than expected temps on earth, but also the colder temps on Mars, and much warming temps on Venus.
It seems to me it doesn’t take a rocket scientist or fancy statistician to come to grips that the world “ON AVERAGE” is warming. I assume that smart people like Steve McIntyre aren’t really suggesting there is no global warming — a idea that would go against this other evidence and a well-established theory.
I think it’s wonderful that scientists are able to tell us what is happening, so that hopefully we can solve this problem before it gets really bad.
13 August 2007 at 12:59 PM
gavin: In the response to the second comment you refer to “only the result”. But this result is a result after adjustments due to a factor which in “climate science” cools the earth today and therefor must be compensated for, but exactly what is done isn’t published. It’s also a result after other not published steps taken.
E.g., Hansen showed 1999 that the warmest year on the US record was 1934, but two years later he had a very different temperature record with higher temperatures the last decades, and lower earlier in the 20th century. The plotted data in both charts here:
http://www.coyoteblog.com/coyote_blog/2007/08/a-temperature-a.html
The problem is we only get data after the climate scientists has secretly adjusted them. Also we don’t know which non-rural stations that are not used due to rules described on the NASA diss data page; rejection of stations not within the long term trend. We don’t have raw station data or algorithms for selection or adjustment.
[edit]
[Response: Presumably that is new definition for the word secret? As in ’secretly’ publishing all the adjustments and consequences in the open peer reviewed literature? Please read the references: Hansen et al 2001. - gavin
13 August 2007 at 1:18 PM
re: 9. “When “deniers” make a claim (like yours), it’s based on the lack of serious research or considerable effort, and if an error is pointed out, it’s excused away or triggers a scrambling attempt to change the subject.”
Indeed. Or running away entirely, unable to admit being in error intentionally or not. In fact, we see it here by various anti-science layman skeptics/deniers who throw out any possible, usually unpublished comment (e.g. the surfacestations.org dribble) about GW desparately trying to make it stick. And who keep repeating it as if repetition somehow magically makes it true and to heck with literally thousands of climate researchers who publish in peer-review journals. Or who make absurd ascertains without absolutely any data or science to support their statements yet when confronted still do not make an apparent effort to learn why they were wrong.
13 August 2007 at 1:20 PM
But they have been significantly warming, no one in his right mind disputes that. The discussion has been whether the warming has been accelerating, and to my mind, over the last 30 years or so it hasn’t — the warming has been steady rather than getting faster.
The rate of warming seen in surface temperature data (since the correction to USHCN data and update of NASA GISS) since 1979 (the beginning of satellite temperature data) matches almost exactly the rate seen in satellite data for the lower-48 states of the U.S. This is illustrated in here.
13 August 2007 at 1:26 PM
Re 121
I am so glad to hear that the DUSTBOWL will not form until 2080.
Http://www.drought.unl.edu/dm/monitor.html and the state of the Sierra Snow Pack had me worried.
On the other hand, just as the Arctic sea ice is melting faster than the models predicted; perhaps North America may be drying out faster than the models predicted?
I do not think that one dry year makes a drought or a trend, but any noticeable change in the heat distribution of the Northern Hemisphere should put us on alert for follow-on effects.
Prudent men should consider and prepare.
13 August 2007 at 1:29 PM
#169
As I mentioned in an earlier post here I am in Sweden right now and here they are noticing the warming in ways that they don’t need any instruments for.
E.g. the news here has recently reported that there are now wild oak spreading in an area about 200 miles north of were oak could be found 15 years ago. There has been oaks in that area before, a few thousand years ago.
This is near a town called Örnsköldsvik, you can find it in Google earth, about 63 degrees north, which is quite far north even when considering that they are warmed by the Gulf stream here.
13 August 2007 at 1:42 PM
RE 166.
You missed the points again. I make it simple for you.
As some have claimed the algrithms are documented in the text.
1. They are not. They are generically described.
2. IF THEY WERE documented in the text, the issue remains:
3. Does the code implement the algorithm as designed.
[edit]
[Response: ‘Algorithms’ are just descriptions in various flavours of formality. If there is a question of whether the code correctly implements the algorithm it will be apparent if someone undertakes to encode the same algorithm and yet comes up with a different answer. Absent that, there is no question mark over the code. So if I generically describe a step as ‘we than add A and B to get C’, there is only a point in checking the code if you independently add A and B and get D. That kind of replication is necessary and welcome. With that, all parties will benefit. Simple demands to see all and every piece of code involved in an analysis presumably in the hope that you’ll spot the line where it says ‘Fix data in line with our political preference’ are too expansive and unfocused to be useful science. - gavin]
13 August 2007 at 2:24 PM
Re #169: [I think it’s wonderful that scientists are able to tell us what is happening…]
What’s even more wonderful is not that climate science can tell us what is happening, but that it told us, years ago, that it would happen. Isn’t that what science is supposed to be about: not just explanation, but prediction?
I’m going to harp some more, but there’s a quote in #157 that’s illustrative of the backwards viewpoint some people have taken:
“Now the vast majority of the scientific community has accepted the view that:
1. The earth is getting warmer;
2. greenhouse gases are largely responsible for this; and,
3. That what has been raising the level of greenhouse gases are human activities.”
A more accurate statement would be something like
The vast majority of the scientific community has accepted the view that:
1) The amount of CO2 in the atmosphere has risen considerably during the industrial period, and many lines of evidence show that the increase is human-caused.
2) Theory, well supported by experiment, says that increased atmospheric CO2 will trap more IR, leading to warmer temperatures.
3) There are many different lines of evidence that show that the Earth has in fact warmed by an amount consistent with theory.
This makes all the fuss over a minor adjustment to a few points in one data set (of many) supporting one line of evidence (again, of many) seem rather out of proportion, doesn’t it?
13 August 2007 at 2:47 PM
I have this doubt:
If it is man-made CO2 that pushes global warming, naively I would exspect that the US should be among the world regions in which the warming is MORE severely felt.
But now, after NASA revision, it seems that US are among the places LESS affected by long term warming…
There is something to ponder upon or everything is perfectly clear?
13 August 2007 at 2:48 PM
Gavin, could you address your fallibility as a scientist? Could you be wrong? Have you ever been wrong? How much do you really know? I know you trust in your models and conclusions, but I have learned never to trust anyone claiming to have the final word (on anything). I personally think your position on gobal warming is more rational than the deniers claims, but I would be a fool to put any money on your predictions (if I were a gambling man). Also could you please address your fallibility without using “if we wait any longer it will be too late” or without using the word ‘consensus’? Thanks much.
[Response: Moi, infallible? Hardly. I’ve made my fair share of coding errors and the like. But the whole point is that you shouldn’t pay any particular attention to me or any individual scientist - we all might have prejudices and biases that might colour what we say. However, assessment bodies like the IPCC and National Academies are much more impartial and since they go through very rigorous peer review , the chances that once person’s biases end up in the final product is very small. So, don’t listen to me. Read the IPCC report instead. - gavin]
13 August 2007 at 2:48 PM
Re: #177 (Mario)
I think your primary misconception is that CO2 emissions tend to have a stronger impact on the region in which they’re emitted. The truth is that CO2 is a “well-mixed” gas which stays in the atmosphere a long time, so it quickly (on a timescale of about 7 or 8 months) distributes itself around the globe. Hence CO2 emissions tend to affect the entire globe, rather than preferentially affecting the region in which they’re emitted.
Something that *does* tend to preferentially impact the region in which they’re emitted, because they have such a short atmospheric lifetime, are sulfate aerosols. These have a cooling effect, and historically have been very strong in the U.S. This may partially explain why the U.S. has shown less global warming over the century than the globe as a whole.
Finally, during the “modern global warming era” — 1975 to the present — as far as I can tell the U.S. has warmed just as fast as the rest of the world.
13 August 2007 at 2:59 PM
tamino (#167) wrote:
I wouldn’t be at all surprised if statistics can this in the case of global temperatures. Probably already has. But I actually doubt that an analysis in terms of the monthly data would turn up anything. My main point was simply that using the appropriate methods one may be able to uncover signals that superficially look like they would be drown out by the noise.
Anyway, my apologies.
I really should have found a better way of making the same point. I will be more careful in the future.
13 August 2007 at 3:26 PM
Aaron Lewis (#173) wrote:
Actually spelling it as two words seems to be more common. But one word spelled that way would seem to be a definite improvement over how I have spelled it in the past.
The more information and the more resources the better.
My central point in bringing it up in the first place is that most Americans still seem to be under the impression that climate change will have relatively little impact upon the United States - but this isn’t what the models say come the 2080s. Personally, I think that what may happen in Asia is a great deal more important, at least in terms of the numbers and global effect.
However, the United States has a disproportionate effect upon global policy, and this is largely a function of conservative attitudes. Despite the recent urbanization of conservativism, I suspect that pointing out the consequences of climate change in relation to US agriculture may be one of the more effective arguments, at least in the states.
I agree. And things may move more quickly as a result of the meltdown in the Arctic - and this is something that I may mention in passing. However, despite my earlier mistake regarding temperature trends, I think it is important to claim no more than what the weight of the evidence (either in terms of modeling or obvious trends) can bear. Otherwise it is all too easy for those on the other side to label us alarmists.
There are things which I worry about more than the drought in either the United States or Asia. I suspect that is obvious. But I won’t make it the centerpiece in any attempt to convince others that we need to do something about climate change.
But that is my own personal approach. I can certainly see others reasonably taking a different view.
13 August 2007 at 3:51 PM
Re #144 I have to say I agree with FCH that source code should be made available in science generally. Although I take the point that doing any more than just putting it online somewhere is going to take resources, I don’t think that is a sound argument against doing at least that much. Certainly in my own field (agent-based modelling), I’m among those trying to establish a norm that code, along with parameter settings and metadata concerning the hardware and software environment in which model runs were undertaken, should be put in the public domain.
13 August 2007 at 3:51 PM
re 169. Lynn V. wrote”If the lower 48 US states have not been significantly warming (is that what people here are saying), then there must be other places that ARE significantly warming, if the global warming idea is accurate. Isn’t that that whole idea behind “the global average temperature” increasing?”
Interesting developments in Crete.
http://www.npr.org/templates/story/story.php?storyId=12707473&ft=1&f=1004
Not that GW is necessarily the only culprit … but poor sustainability choices combined with apparent warming seems to have their drawbacks…
13 August 2007 at 3:52 PM
#170: Yes, gavin. These documents we have, but I’m probably a bit disappointed if that’s all you have. I’ve just started to go through them as good as possible, but of course here is no raw data, program code, or most important I don’t think there are detailed algorithms. It seems a lot to be a defence of methodology. Before I ask more specific questions I will however search for all answers on the site and elsewhere.
Now, the question why the US Temperature Anomaly 1880-1998 data 1999 are so different from that 2001 wasn’t answered. I shall not have to find it out myself, which adjustments that differ and the impact from them respectively. I stronlgy doubt it can solve this from these (more heuristically rethorical than mathematical?) documents that you gave me a link to.
Again: Why is the US Temperature Anomaly 1880-1998 data on page figure 6, page 37 in “GISS analysis of surface temperature change” (1999) so very different from US Temperature Anomaly 1880-1999 data an page 22 in “A closer look at United States and global surface temperature change” (2001).
These documents:
http://pubs.giss.nasa.gov/abstracts/1999/Hansen_etal.html
http://pubs.giss.nasa.gov/abstracts/2001/Hansen_etal.html
Just a general, not a detailed, answer why. Thanks!
[Response: The answers are in the papers! Basically, it is because of the corrections due to Time of Observation biases and calibrations for known station moves and the like. - gavin]
13 August 2007 at 3:54 PM
# 178 Michael
“but I would be a fool to put any money on your predictions (if I were a gambling man).”
So, would you put money against GISS predictions? If you think you’d be a fool to put $$ on their predictions, surely it would be very unfoolish to put money against them? Do you think a global 5-year average around 2020 will be the same? colder? than one around 2005? I’m still hunting someone to propose that side of a bet over on www.longbets.org…
13 August 2007 at 4:13 PM
Gavin, you make peer review sound flawless (re: response in #178).
http://www.eurekalert.org/pub_releases/2004-08/cu-bse081204.php
[Response: Interesting study. But still, I have never described peer review as flawless. When done properly and thoroughly (as it was for the IPCC reports) it is very useful at improving the quality of a document. When done badly….. well, let’s just say that it isn’t always done well. It’s just one step at improving the credibility of a statement. Necessary, but not sufficient! - gavin]
13 August 2007 at 4:22 PM
Re: #179 Tamino
Thanks for the explanation,
but your answer induces another “naive” doubt:
if CO2 is “world global” in its warming effects
while “short lifetime” sulfate aerosols are much more local
then for the 1940-1975 cooling trend I would also expect a neat divergence
between, say, the northern and southern emisphere of the globe
Now for example in
http://iabp.apl.washington.edu/Papers/JonesEtal99-SAT150.pdf
(see figure 4, p. 10)
I can’t see that much of a difference ,,,
13 August 2007 at 4:28 PM
Then just host a tarball–or give public read access to the SVN/CVS repo–and go about your business, doing the “useful science”. Is there any reasonable argument for withholding any of the relevant source code? How is that “expansive”?
[Response: Because working codes are generally a mixture of scripts, programs, utilities, dead ends, previous methodologies and unused options and not ‘nice’ web applets that anyone can run. Pulling together everything needed to the analysis so that it can be run elsewhere ‘out of the box’ is a significant undertaking. We’ve done that for climate models because that flexibility is required due to the number of users. For programs that just one person runs, it’s much more difficult and time consuming in general. Before going to all that bother, demonstrate that there indeed is some ambiguity in the published descriptions. Without that, this conversation is just posturing. - gavin]
13 August 2007 at 4:30 PM
No, but if the paper says C=f(g(h(A + g(h(B))))), it would be “nice” to have f(), g(), and h() fully specified and reproducible, with full access to the datasets containing A and B.
[Response: Read the paper and you will see that the GISS methodology for the urban adjustments are not that complex and all the raw data is already available. - gavin]
13 August 2007 at 5:38 PM
I saw the site on temperature anomilies. I wonder is there one on rainfall anomolies?
13 August 2007 at 6:00 PM
Re 181
Drought in California will have major impact on 30 million people, from lettuce farmers that flood their fields with Sierra snow melt to computer programmers that want a morning cup of coffee. A drought like the one that started in 1280 would be worse for California than any foreseeable earthquake, volcano, or plague.
I would rather risk being labeled an alarmist by speaking up, then to have people perish as a result of my not raising the issue soon enough, or not making my warning sufficiently urgent. So at what point do you raise the alarm? 2 % probability? 50 %? Do you wait until you are certain? By then, the danger may be so close that it is impossible to avoid. When does the philosopher speak? I think that a lot of people are going to come to grief because respectable scientists do not raise the alarm in time.
I think that the changes in the Arctic are evidence of changes in Northern Hemisphere surface sea temperatures and atmospheric circulation patterns. I think the surface sea temperatures in the Eastern Pacific are similar to those that I saw last year at this time, and I feel that those sea surface conditions contributed to California’s lack of precipitation last winter. Last year, I thought it was gong to give us a wet year, and instead the storm track moved north and we were dry. I think the same thing is going to happen this year. What is my confidence level? Maybe 33% - not high, but give the stakes, it is a heck of a bet. I would not give it a second thought if the Arctic ice was not melting, and my fruit trees were not blooming a day earlier ever year.
What happens if I shout DROUGHT too loudly? It is not like shouting fire in a crowded theater.
The worst that could happen is a few farmers put in drip irrigation systems and a few home owners convert their lawns to sedums. It is not like everybody is going to run for Oregon. Heck, half of the households in our little cul-de-sac do not believe in global warming anyway.
13 August 2007 at 6:03 PM
Here is Hansen et al paper on the urban adjustment: “The urban adjustment, based on the long-term trends at neighboring stations, introduces a regional smoothing of the analyzed temperature field. To limit the degree of this smoothing, the present GISS analysis first attempts to define the adjustment based on rural stations located within 500 km of the station. Only if these stations are insufficient to define a long-term trend are stations at greater distances employed.”
I have some questions:
– How does the analysis “attempt to define the adjustment based on rural stations”? What is the methodology?
– How are the local rural stations chosen?
– What is the test for insufficiency?
– In the event that local stations are deemed insufficient, how do you choose the longer range ones?
– Why 500km? Was that arbitrarily chosen?
– What is the methodology for making the adjustment for long range ones? How does it differ?
Also from the paper, regarding station history adjustments: “One of the best opportunities to make useful station history adjustments is in the United States. The USHCN stations were selected as locations with nearly complete temperature records for the 20th century, but also because they have reasonably good station history records that permit adjustments for discontinuities.”
– Um… this doesn’t describe the adjustment… at all.
Now, maybe I’m missing something, but I don’t see any papers referenced here which could answer any of these questions. Perhaps that is why some people are characterizing it as “secret”.
I’ll be the first to admit that I have NO IDEA whether there are or are not published papers out there that answer these, but they aren’t cited in Hansen et al 2001. With no mention of methodology on such important alteration of the raw data, how can this possibly be called “peer-reviewed”? I really don’t mean to sound snarky…. I’m sure I’m just missing something.
[Response: Umm… try reading past the abstract - all the full papers are freely available online. - gavin]
13 August 2007 at 6:21 PM
I quoted from sections 4.1.2 (Station History Adjustment) and 4.2.2 (Urban Adjustment), the relevant sections on the adjustments. I honestly quoted the most descriptive excerpts from the two sections… a full reading does not convey any more information toward answering my questions that do those excerpts. (You HAD to know that, so why such a dismissive response?)
All of my questions still stand.
[Response: Apologies if I jumped to conclusions, but I’m reading the same paper as you and the answer to your first question is clear: the rural stations are determined by the night light index (Section 3 and Plate 1). For the others, ‘insufficient’ refers to whether there are any rural stations within the radius (with enough values to define a clear trend). Of course, 500km was somewhat arbitrarily chosen - you want to minimise the size of the circle of influence while still maintaining enough stations for the method to work. The station adjustments are simply taken from USHCN itself - there is nothing extra added for that (and so no description required). If you think these choices matter, do it yourself with different numbers and see what happens. I guarantee that trying to do it for yourself will give you a much greater insight into the problems and potential solutions than simply looking at badly written fortran…. - gavin]
13 August 2007 at 6:55 PM
#161
No worries.
Now, moving on to global temperature anomalies, does anyone think there is an accelerating trend in the global data? Does anyone have some analysis of the time series to support this?
regards, Steve
13 August 2007 at 7:11 PM
Re Gavin’s response in 175:
“If there is a question of whether the code correctly implements the algorithm it will be apparent if someone undertakes to encode the same algorithm and yet comes up with a different answer.”
Gavin,
Geez. Just support the release of the code and scripts. The only thing worse than if there were more errors would be if there were more errors and no one caught them. Through support of keeping it difficult to find errors you are diminishing your value as a scientist.
[Response: You completely misread my point. I am certainly all for as much openness as possible - witness the open access for the climate models I run. But the calls for the ’secret data’ to be released in this case are simple attempts to reframe a scientific non-issue as a freedom of information one (which, as is obvious, has much more resonance). My points here have been to demonstrate that a) there are no ’secret’ data or adjustments, and b) that there is no reason to think there are problems in the GISTEMP analysis. The fact that no-one has attempted to replicate the analysis from the published descriptions is strong testimony that the calls for greater access are simply rhetorical tricks that are pulled out whenever the initial point has been shown to be spurious. - gavin]
13 August 2007 at 7:17 PM
First of all, Gavin, I wanted to say that its cool of you to come in here and actually interact with us and answer these types of questions. Regardless of whether we agree with you on a particular point (or are even qualified to agree or disagreee), you deserve credit for that. It’s rare.
Onto my response…
Well, the paper says that the “analysis” is “based on” the unlit stations. The use of such otherwise superfluous language suggests that there is something more going on than a simply-described, routine adjustment.
And regarding the insufficiency, should it not be made clear what constitutes a “clear trend”? The results could NEVER be reproduced without that information, which leads me back to my last question: how was this “peer-reviewed”? Are there publicly-available peer review documents?
Also from the paper: “The hinge date [for the urban adjustment] is now also chosen to minimize the difference between the adjusted urban record and the mean of its neighbors.”
What makes anyone think that this is correct? Is there some research that lends credibility to this approach? There is nothing cited for that either. It sounds like this method would arbitrarily minimize UHI when adjusting the data… rather, it sounds like the hinge point is chosen in a way that guarantees the highest possible trend. Maybe I’m reading that wrong though.
[Response: I think you have read it wrong. They don’t want the urban trends to have an influence at all, and so they look at the nearby rural trends. The urban long term change is then adjusted (using a two piece linear fit to the mean rural station trend) so that it matches the rural trend. They use a a two-piece linear fit (with an arbitrary hinge) because you don’t want to arbitrarily fix a particular date at which the urban trends become important (they were doing that prior to the 2001 paper). You could do better I think, for instance fitting a low order spline to the rural numbers, but I’m not sure that the improvement in fit would justify the work. The main point is not that the Hansen et al method is ‘right’, because there is no ‘right’ answer. There are just different approaches. Eli Rabett had a good example of how this works - and his ‘reverse engineering’ revealed that a) the GISS adjustment is a simple two-piece linear trend and b) the urban trends after adjustment are the same as the rural trends. That’s prima facie evidence that the code does what it claims to do. - gavin]
13 August 2007 at 7:24 PM
Re: #187 (Mario)
I agree that from the graphs in the paper you reference “I can’t see much of a difference.” So, I took the NASA GISS data for the northern and southern hemispheres, and computed the difference (northern - southern) from 1880 to the present. You can view the graph here.
As you can see, the northern hemisphere did indeed cool relative to the southern hemisphere from about 1945 to about 1975.
13 August 2007 at 7:41 PM
Re: #196 (tamino)
One more note: for most of the century it’s evident from the graph that the northern hemisphere warms relative to the southern. That’s because land warms faster than ocean (due to the immense thermal inertia of the oceans) and most of the land is in the northern hemisphere. The period 1945-1975 diverges from this pattern.
13 August 2007 at 7:52 PM
What happened to my comment on #196? Was there something inappropriate? Please let me know.
13 August 2007 at 8:23 PM
I urge everyone to review Eli’s rework, but make sure you read Steve McIntyre’s commentary.
13 August 2007 at 8:32 PM
So Gavin, besides posting replies here, how was your weekend? You have done yeoman’s work answering the queries. Thanks. I think a sticking point is that most of us would have a hard time replicating the GISS temps based on Hansen, et al. 2001 simply because we are not trained in the field. But I am sure many outside my field would have trouble replicating some of the vaccine work I published based on their reading of the Materials and Methods sections I painstakingly wrote. That’s the nature of the beast. Training in a scientific field does count for something.
I would just like to point out that with ClimateAudit.org down, Steve McIntyre has posted a couple of items on Anthony Watts’ blog (”Does Hansen’s Error “Matter”? - guest post by Steve McIntyre” and “Lights Out - Guest post by Steve McIntyre”). And I would recommend tamino’s data analysis to get an idea of the effects of the correction. It looks to me like the correction of the data for the lower 48 leads to a better slope match with the satellite data.
13 August 2007 at 9:17 PM
RE #191, & Drought in California will have major impact on 30 million people, from lettuce farmers that flood their fields with Sierra snow melt to computer programmers that want a morning cup of coffee. A drought like the one that started in 1280 would be worse for California than any foreseeable earthquake, volcano, or plague.
I just started SIX DEGREES by Mark Lynas (had to order it thru www.amazon.co.uk) & am starting on the first chapter. He writes about what happened in 1280 as happening with a 1 to 2C increase. And I understand (from other sources) that’s already in the pipes from previous GHG emissions. Yes, such an increase & concomitant harms will be very very bad for much of the U.S. (& the world).
13 August 2007 at 9:29 PM
“We of course pointed that you get virtually the same warming trend whether you use all stations or just rural stations - which you didn’t even seem to acknowledge.”
This is being claimed wrt US temps.
Is this true wrt global temperature studies? That is, only rural stations have the same trend? If so, what is the published study on it?
13 August 2007 at 9:58 PM
Oh, PUH-leeze, Gavin. You’re not talking to the computer illiterate. I’d tinker with the models if I had the right FORTRAN compiler, but all my best efforts — and I was working in FORTRAN 28 years ago — have been for naught. One thing that publishing the code would do is get it ported to something that doesn’t require a proprietary compiler. The next thing is I suspect someone would find a way to parallelize the code that would let it be run on distributed networks, rather than being limited to tightly coupled, shared memory systems — or at least, I’ve been told it currently only runs on tightly coupled systems.
[Response: Are we talking about climate models now? The available ModelE code was only for OpenMP systems, but does compile on a number of different platforms and compilers. Our current in-house version has MPI as well, and will compile with g95 (runs pretty slow though). At some point in the near future, I’ll update the available code to the version with the same physics but more flexible coding. - gavin]
13 August 2007 at 10:34 PM
Gavin thanks for your reply to 195. Obviously we are keeping you busy. I appreciate your dedication to the science and the discussion. Onto your comments:
“You completely misread my point. …”
Not sure how, I completely read your words. Your words said that the code should not have to be publicly inspected; the public should create a new program based on the original algorithm. If the public gets different results, then the public can raise an issue with the Author.
Reviewing code cuts to the chase. Making a new program makes the process more complex. Why on earth would you want to make it more complex?
” … I am certainly all for as much openness as possible - witness the open access for the climate models I run. …”
I am for as much openness as possible too, please support release of Hansen’s source code and scripts.
“…But the calls for the ’secret data’ to be released in this case are simple attempts to reframe a scientific non-issue as a freedom of information one (which, as is obvious, has much more resonance). …”
I don’t think so. I think people are curious how it works and if it really works correctly. Sounds like science to me. If you are correct then get it released, and the issue as you described goes away. If I am correct people will take the information, run it, look at, size it up, and comment. How can that not be good for Climate Science?
” … My points here have been to demonstrate that a) there are no ’secret’ data or adjustments, and b) that there is no reason to think there are problems in the GISTEMP analysis. …”
a) The code and scripts are apparently kept secret from me - thus the implementation of the data and adjustments is indeed secret. I don’t think your point a is valid.
b) Before Steve McIntyre found the subject GISTEMP problem you describe above, I don’t think you had reason to believe that that GISTEMP problem was there. I don’t think your point b is valid either.
” … The fact that no-one has attempted to replicate the analysis from the published descriptions is strong testimony that the calls for greater access are simply rhetorical tricks that are pulled out whenever the initial point has been shown to be spurious. - gavin”
Funny, I think it is strong testimony when someone releases important results publicly but withholds important information on generating those results. Hansen’s code and script details are obviously very important towards generating those GISTEMP results.
It is perfectly reasonable to subject every piece of code for completely transparent review if the code establishes one of very few standard data sets for GW measurement, as Hansen’s does. It may be poorly configured code and scripts that are embarrassing for Hansen to release, but the world will get past that if it otherwise passes scrutiny.
[Response: The methodology is important to the results, and the methodology is explained in detail (with the effect of each individual step documented) in the papers. The fact that the results are highly correlated to the results from two independent analyses of the mostly the same data (CRU and NCDC) is a strong testimony to the robustness of those results. However, you are I think wrong on one point, the rhetoric for more access and more data is actually insatiable. As one set of code is put out, then the call goes up for the last set of code, and the code and results from the previous paper, the residuals of the fits, and for the sensitivity tests and so on. Given that all this takes time to do properly and coherently (and it does), there will never be enough ‘openness’ to squash all calls for more openness. Whatever the result from releasing the current code (which very few of the people calling for it will ever even look at), the ‘free the code’ meme is too tempting for the political advocates to abandon. People who are actually genuinely interested in all of these questions, will, I assure you, be much happier in the end if they code it themselves. Think of it as tough love.
- gavin]
13 August 2007 at 11:06 PM
Hansen et al 2001 says: “Only 214 of the USHCN and 256 of the GHCN stations within the United States are in “unlit” areas.”
The data set http://data.giss.nasa.gov/gistemp/station_data/station_list.txt
lists 294 USHCN and 362 GHCN with lights=0 and 308 USHCN and 371 US GHCN as dark (A). Can you please obtain a list of the 214 USHCN and 256 GHCN sites that were actually used in Hansen et al 2001. What accounts for the difference between the numbers in the data set and the article.
[Response: I have nothing to do with that analysis and so you need to ask the authors. Bear in mind that GISTEMP is being updated in real time, as are the source datasets. What was available in 2001 is not necessarily the same as what is available now. But like I said, ask them. - gavin]
13 August 2007 at 11:14 PM
#205. For applied economics articles in major economics journals e.g. American Economic Review, it is mandatory to archive code and data as used, at the time of submission of the article. There’s no reason why this sort of “best practices” should not also be adopted in climate science, where there are important policy considerations.
And by the way, if you’re worried that no one’s going to look at the code, I promise that I will look carefully at the code.
[Response: As implied above, you will be much better off doing it yourself. It’s not a complicated procedure, and you could try all sorts of different methodologies on a consistent platform. If you come up with something substantially different, I’ll be surprised, but that would be constructive science. Graduate students sometimes put a lot of effort in deciding which choice to make in an analysis. My advice is invariably to simply do it one way and then go back and see if doing it the way other matters. If it doesn’t matter, it isn’t worth worrying about, and if it does, then you have an interesting result. The point is that idle theorising about potential issues is a waste of time when it is only a matter of days to actually find out. Work it out and see. If you are worried about microsite issues, do the analysis with only the ‘good’ sites. If you worry about urban issues, throw out the urban stations. etc. Each persons interest’s are independent, and the direction their tests will take them is unique. Waiting on someone else to do your analysis for you is foolish. -gavin]
13 August 2007 at 11:19 PM
I think that’s nonsense. Noone should have to “attempt” to replicate the results. They should be able to pull the actual, complete and unaltered dataset that was used and replicate it. They should see each and every station that was used to do the urbanization adjustment for any particular station. Every bit of that should be available, and there is absolutely no defensible reason why it isn’t.
I’m sorry, Gavin, but you’re wrong on this. People shouldn’t have to jump through hoops and do tons of trial-and-error hoping to “replicate” the results of a paper that somewhat ambiguously (despite your differing opinion on that) describes methodology, nor should their failure to do be considered evidence that they are asking for the data in bad faith.
And, in the end, it shouldn’t matter if they are asking in bad faith or not. Every last scrap of data should be available to everyone, even people who disagree with your conclusions, and ESPECIALLY to people who are trying to tear apart your conclusions.
So, what’s wrong with that? I was shocked over the last few days to learn the extent to which those things AREN’T available, to be honest.
[Response: Underlying all of this is, I think, a big misconception about what replication means in an observational field like climatology. Everyone is working with ambiguous and imperfect data - whether that is from weather stations, paleo-records, models or satellites. Every single source needs to be processed in ways that are sometimes ad hoc (how to interpret isotopes, how to model convection, how to tie two satellite records and, yes, how to adjust for urban heating effects). Robustness of a result is determined by how sensitive the results are to those different ad hoc assumptions (do different climate models have the similar sensitivity, do results from south Greenland match those of North Greenland, does the UAH MSU record match the RSS MSU record). The important replication step is not the examination of somebody’s mass spec, or the analysis of lines of code, but in the broader results - do they conform with other independently derived estimates? Climate science is not pure mathematics where there is (sometimes) a ‘right’ answer - instead there are only approximations. These are outlined in innumerable papers which for decades have been the method of recording procedures, assumptions and sensitivities. To be sure, there are sometimes errors in coding (in the MSU record, or in climate models), but these come to light because the end results seem anomalous, not because there are armies of people combing through code. The independent replication of results is by far the more important stage in evaluating something.
Take the Greenland ice cores. The US and EU wanted to drill an ice core in central Greenland. They could have drilled one core , split the samples and done indpendent analysis of each sample. That would have given a good test of how good analytical techniques are. Fine. But what they elected to do was to drill two separate cores, 30 miles apart and do everything independently. This was much, much more useful. Firstly they found that for most of the cores the results were practically identical (which demonstrated the robustness of the analytics as well as sharing the samples would have), but more importantly, they found that the cores diverged strongly near the bottom - a sure sign that there was something wrong at one or (as it ended up) both sites. Without the independent replication that non-robustness would have not been uncovered. Thus while there are already two independent replications of the GISS temperature analysis (CRU and NCDC) - both of which show very similar features, there is always room for more. That would demonstrate something. Looking through code that does a bi-linear fit of data will not. - gavin]
14 August 2007 at 12:13 AM
“Patrick: can you explain to us your expertise in physics, algorithms, statistics, analysis of imperfect data, software engineering, simulations? You have done most of this stuff professionally, right? [Of course, as an anonymous poster, it will be hard for us to check.]”
I already replied to this besides-the-point challenge but it was not posted.
Not sure why. Let me briefly say: I have a PhD in Computer Science and know many of the matters/areas related to this subject sufficient to do some work here, but my skill/background is besides the point. I joined the call others made for the scientists in the area to do good, professional science by being as reproducible as possible with data and data analysis algorithm transparency. Open the details and code up for review. If I am being asked to do what those working in the field won’t do, I’d have to decline, as I have a job. But Steve McIntyre seems eager to review things, so work with him. Willingly. (See Judith Curry, #69, she ‘gets it’.)
14 August 2007 at 12:16 AM
A problem with this thread is that we have two extremes, as seen before in other disciplines:
- Respected domain researchers (like Prof. Curry in this case), occasionally joined by others, like statisticians or software engineers who might have something constructive to add, who clearly want to improve the science, and are happy to make normal-science critiques. Reasonable people can and do disagree about the mechanisms, cost tradeofss, etc.
On the other hand:
- People who have little interest in improving the science, or actually getting better answers, and mainly want to slow down research whose answers they don’t like, raise the publication bar very high, and in general, cost-effectively waste researchers’ time.
If you don’t believe this goes on:
Consider “Good Epidemiology Practices (GEP)”, which describe the rules for doing good studies. This is good, and versions are created/debated by various researchers. Who could argue with that?
In Allan Brandt’s “The Cigarette Century”, p306-307: it turns out that in late 1992:
“Following years of fighting epidemiologists, Philip Morris now initiated a campaign for “Good Epidemiological Practices,” organized to “fix” epidemiology to serve the industry’s interests by changing standards of proof. One of the objective of the program, an internal memo explained, was to “impede adverse legislation.”
A really detailed analysis can be found (including the role of Steve Milloy) in:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1446868
The tobacco case is unusual, in that large amounts of *internal* documentation is publicly available.
Google: philip morris gep tobaccodocuments.org
gets plenty of pointers, including strategy:
‘generate letters to editors (scientific/general media) promoting study and highlighting EPA/other weak ETS methodology inc… encourage libertarian policy groups to promote study and criticise the weakness of epidemiology, especially on ETS * independent scientists to push feature articles promoting confounders arguments.’
Effective strategies proliferate.
For a view of the more widespread use of these techniques [for instance, Milloy has moved on from secondhand smoke to fighting AGW), see:
Chris Mooney, “The Republican War on Science”, especially:
Ch6 Junking “Sound Science”
p 71: ‘”sound science” means requiring a higher burden of proof before action can be taken to protect public health and the environment.’
The Data Quality Act is supposed to assure the Federal Agencies “use and disseminate accurate information”. Who could argue with that?
See:
http://library.findlaw.com/2003/Jan/14/132464.html
http://www.washingtonpost.com/wp-dyn/articles/A3733-2004Aug15.html
http://en.wikipedia.org/wiki/Data_Quality_Act
It has 12 separate references in Mooney’s book. On the surface, it’s supposed to assure good data. In practice, it has been used to inhibit publication of awkward data.
======
Anyway, I continue to be amazed at Gavin’s patient willingness to reply to posters who seem more like anonymous drive-by sockpuppets to me and often seem clueless about serious software engineering and what it costs. Having spent some time looking at climate websites, I think GISS does a terrific job cost-effectively making data and (lots of) code available.
I appreciate the different level of effort involved in getting the code done for a research study, and producing a systems program product that other people are expected to use. Again, if somebody is really interested in improving worldwide data, spending most of their effort chasing 2% USA Lower-48 is very weird. Reasonable, well-informed people can and do disagree about good procedures, but much of this thread seems right out of the Philip Morris GEP or Data Quality Act playbooks, and it does not help good science…
14 August 2007 at 12:45 AM
Gavin wrote in #205: However, you are I think wrong on one point, the rhetoric for more access and more data is actually insatiable. As one set of code is put out, then the call goes up for the last set of code, and the code and results from the previous paper, the residuals of the fits, and for the sensitivity tests and so on.
When you work on something unimportant, you are right, folks don’t care much and nobody asks many questions. But if you are working on something important, then I’m sorry but people will ask for more and more information. That’s the way it goes.
But don’t these types of questions come up anyway when someone is peer reviewing the work? If I am reviewing another engineer’s work, it’s common for me to ask that certain inputs change so that I can be satisfied of the impact on outputs. It’s a quick sanity check that catches more issues than you might imagine.
If you are only providing descriptions, then my guess is that most peer reviewers aren’t at all scrutinizing to the level that most think things are being scrutinized.
If I tell you I have a million lines of code that prove something you believe to be a stretch, and my code is secret, then investment needed by you to demonstrate an error in my thinking is astronomical if you have to recreate everything from scratch.
This is what absolutely bugs me about all this climate research. As I’ve noted before, 1MLOC typically has 5,000 to 10,000 bugs lurking about unless you have an army of folks on QA.
[Response: As I said above, complex codes can’t be derived directly from the papers and so should be available (for instance ModelE at around 100,000 lines of code - and yes, a few bugs that we’ve found subsequently). The GISTEMP analysis is orders of magnitude less complex and could be emulated satisfactorily in a couple of pages of MatLab. - gavin]
14 August 2007 at 1:43 AM
Gavin,
Here is some speculation that might be worth looking into by someone with sharper physics than mine.
I grew up in the desert of Eastern Washington State. I go back from time to time, and the weather has cooled. It used to be that every couple of years, late June or early July would have temperatures in the 110s. That doesn’t seem to happen now; I’ve been there in June when the temp didn’t get over 95 for a week. The common attribution of this change is irrigation. There are new irrigated vineyards, orchards, and even corn where there used to be only dry-land wheat.
My understanding of such things tells me that increasing humidity should depress high temperatures (and put a floor under lows). If this is true, then we may have the opposite of urban heat islands. At least in places where irrigation is increasing.
In poking around the web, I could find no records of relative humidity. I did find that about 2% of the total area of the US is under irrigation (to the tune of about 150 billion acre-feet a year); that the area under irrigation fell by about 10% from 1972 to 1984.
There are some interesting possibilities in looking at humidity as a moderator of temperature. Should high temperature in areas that humans have dessicated (LA has captured entire rivers to feed their water needs) count in temperature trends? Should areas that irrigation has cooled? Was 1934 so hot because so much of the country was very dry? Did the growth of irrigation suppress temperature in mid 20th century?
14 August 2007 at 1:47 AM
#146
Thanks for that Tamino. That’s a link I’ve already used on two other forums. Grand stuff.
14 August 2007 at 1:47 AM
A common definition of whether a software application is implemented correctly is that it “conforms to the specification” (a similar definition is sometimes used for the quality of products produced by a manufacturing process).
If I follow Gavin’s argument (and I do respect and appreciate Gavin’s efforts even if I disagree on this point), we should test Microsoft Word by developing a duplicate of Microsoft Word based on a hypothetical (since its from MS) detailed specification and if the duplicate performs differently than Microsoft Word, then we may have found an error in Microsoft Word (or the duplicate)? I suppose this approach might work but it does not seem to be a particularly realistic way to test software for conformance with the specification nor is it used widely to test the accuracy of software implementations. For all but relatively simple algorithms, it is a potentially labor intensive and time consuming approach to checking the accuracy of an implementation. This is not an issue of “political advocates” but straight forward traditional software engineering and also a major tenet of the open source movement. Thanks for listening.
14 August 2007 at 3:01 AM
John Norris wrote: Reviewing code cuts to the chase. Making a new program makes the process more complex. Why on earth would you want to make it more complex?
Reading to code won’t cut to the chase. In the first place, the code is the least informative artifact in the software development process. It has had too much of the what and why removed. What you really need to understand a program is a statement of the requirements for the program, a layout of the architecture of the program, and then a description of the high and low level design of the components of the program. Once you have all that, then you can start to make sense of the code. I doubt that much documentation exists.
I once worked for the Navy modeling *mumble*mumble* The point of the exercise was to gain understanding of certain phenomena and how certain weapons responded. Our product was understanding, not software. Our work cycle was, approximately, to find an area that didn’t seem to be right to us. We might do some research, we might run our simulation against new conditions. Then we would decide to try something, code it, and run it. Then if it was a better match for our reference data, we would keep it and start over. We didn’t have to be perfect; we couldn’t be perfect. We just had to keep getting better.
If we had to have operated in the kind of fishbowl you think would be good for climate science, our productivity would have gone in the toilet. Everything would have taken two or three times longer, because we would write documents we didn’t need for our work, but would be required by our auditors. And we would have been interrupted constantly by people wanting to know why we didn’t use longer variable names (F77, of course), don’t we know that common blocks are the root of all evil, and good gawd there’s a GOTO.
Exxon Mobile made, after taxes, nine thousand dollars every second of 2006. To think that they would not have folks poring over the code looking for things that just look funny (so that they could feed it to a blog to raise a stink) is, well. Not everybody just wants to understand.
14 August 2007 at 5:18 AM
Re #206
Tim,
It is already known that humidity sets a ceiling to high temperature. For instance in a jungle the temperature seldom goes above 80 F but in deserts over 100 F is not uncommon.
Water has several effects on climate with the first being that it prevents the surface from warming, because it absorbs the solar radiation as latent heat. This leads to more water vapour which is a greenhouse gas, and this heats the air near the surface. But water vapour is a lighter gas than air with a molecular weight of 18 compared with air at around 30. Thus the ‘wet’ air convects because it is warmer and lighter. The convection leads to cooling and the water vapour condenses and forms clouds. This cuts of the supply of solar radiation to the surface during the day and causes cooling. At night the blackbody radiation from the clouds keeps the surface warmer than on clear nights.
So it is not really the humidity that is causing the cooling. It is just a rather unpleasant side effect from the surface water.
14 August 2007 at 6:15 AM
[[Others say that this pattern change is CO2 induced. Neither side has yet to prove their case. I know the GISS people are high on CO2 causation, but they lost credibility when they keep their code secret (mistakes and all) from those who want to check their work.]]
The code from the GISS GCM is available on the web:
http://www.giss.nasa.gov/research/modeling/
As for CO2 accumulation raising surface temperatures, that has been clear since John Tyndall demonstrated that CO2 is a greenhouse gas back in 1859. It doesn’t depend on modern GCM results.
14 August 2007 at 6:22 AM
Re #172 — Tamino, you were right and I was wrong. For some reason I had gotten it into my head that you (or the original poster) was saying the lower 48 were not warming, whereas apparently you were saying the warming trend was not accelerating. My bad.
14 August 2007 at 6:25 AM
[[I have this doubt:
If it is man-made CO2 that pushes global warming, naively I would exspect that the US should be among the world regions in which the warming is MORE severely felt.
But now, after NASA revision, it seems that US are among the places LESS affected by long term warming…
There is something to ponder upon or everything is perfectly clear?]]
Mario —
CO2 is well mixed in the troposphere due to convection and turbulence. On a regional scale the proportion of carbon dioxide in the air is pretty much the same everywhere.
14 August 2007 at 6:29 AM
So if the GISS code implements a relatively simple algorithm, you don’t mind if only the algorithm(s), not the code’s been published?
That would seem to be the implication of your post.
Gavin:
Hmmm, hey, it *is* relatively simple.
14 August 2007 at 6:55 AM
[[Now, moving on to global temperature anomalies, does anyone think there is an accelerating trend in the global data? Does anyone have some analysis of the time series to support this?]]
I took the GISS global annual temperature anomalies from 1881 to 2000 and divided them into 12 decades, each with a time variable T = 1 to 10, and regressed the figures on T for each decade. I then took the coefficient of the T term for each decade and regressed that on the decade number 1-12. The trend was up but not significant, which is perhaps attributable to the small sample sizes involved.
14 August 2007 at 7:02 AM
BlogReader #72 said: Odd you didn’t mention that maybe some plates might be sliding lower into the ocean. Exactly how much of a rise is there? And how does it vary across the globe?
Well, plates do slide against each other, but sea level changes due to tectonics tends to be in the order of 1cm per thousand years, and this would generally result from doming events. Sliding plates can cause more dramatic changes when there is elastic springback after tension has been released, but there haven’t been any such events in human recorded times as far as I have found.
Any change in sea levels due to tectonic events would have both local and globals results: local changes will be due to the changes in plate tensions and buoyancy - global levels will result if the size of the oceanic basin changes.
14 August 2007 at 8:03 AM
Realclimate said:
“Another week, another ado over nothing.
… In the global or hemispheric mean, the differences were imperceptible (since the US is
only a small fraction of the global area)….”
From the link in 184: Hansen et al., 2001 says-
“… Although the contiguous U.S. represents only about 2% of the world area, it is important that the analyzed temperature change there be quantitatively accurate for several reasons.
Analyses of climate change with global climate models are beginning to try to simulate the patterns of climate change, including the cooling in the southeastern U.S. [Hansen et al., 2000]. Also, perceptions of the reality and significance of greenhouse warming by the public and public officials are influenced by reports of climate change within the United States…”
If Hansen et al. said accurate analysis of temperature changes are “important”, can we conclude now that the call for transparency is not actually “another ado about nothing”?
If I were inclined to read between the lines, that paragraph from Hansen et al, might raise my eyebrows for different reasons too, but that’s a topic for another day.
14 August 2007 at 8:07 AM
On the Toronto Star’s website this morning, Stephen McIntyre is quoted as saying that he caused “a micro-change. But it was kind of fun.”
14 August 2007 at 8:33 AM
#207. Gavin, I think that the problem that you’re failing to come to grips with is that when results are used for policy purposes, people expect different forms of due diligence than little academic papers, which have received only the minimal due diligence of a typical journal review. People are entitled to engineering-level and accounting-level due diligence.
The reason that current replication practices in econometrics require archiving of code is that this makes post-publication due diligence much more efficient. There are a number of interesting links and discussion of replication policy at Gary King’s website http://gking.harvard.edu/replication.shtml . The McCullough and Vinod articles are relevant and were relied on in the American Economic Review adopting its replication policy. (The editor at the time is presently the Chairman of the Federal Reserve.)
Your statement that no one had apparently tried to emulate the Hansen methods is itself evidence of the burden in trying to run the gauntlet of assembling the data and decoding the methods and precisely illustrates the obstacles to replication discussed in McCullough and Vinod (and its references) and why the American Economic Review changed its policy to require such archiving.
In addition, the GISS temperature series is essentially an accounting exercise, rather than a theoretical exercise. In an accounting audit, you don’t just hand a bunch of invoices to company auditors and say - well, do your own financial statements. Yes, maybe they’d learn something from the exercise, but that’s not the way the world works. Their job is not to re-invent the company’s accounting system, but to verify the company’s accounting system. Sure there’s a role for re-doing Hansen’s accounts from scratch, but there’s also a role for examining what he did. If he’d archived his code, then it would have been easy to see where he switched from using USHCN raw data to adjusted data. You’d ask - why did you change data sets? You might also consider the possibility that if they’d gone to the trouble of properly documenting and archiving their code, maybe they’d have noticed the error themselves.
The other problem that arises is that, if I do what you said - emulated GISS methods and arrived at different numbers, it’s not hard to imagine a situation where GISS said that my implementation of their method was wrong [edit] and right away everyone has a headache trying to sort out what’s going on.
The purpose of inspecting source code is precisely to avoid these sorts of games. I asked for code to avoid pointless controversies [edit]. Contrary to an impression that you’ve given, I wouldn’t try to run their code as is. My own practice is to re-write the code in R (as you would do in Matlab), recognizing that it is fairly trivial, and then try to test variations.
However even in “trivial” code, little things creep in. If you can read their Fortran code, this can elucidate steps and decisions that may not be described in the written text. [edit]
If one wants to test the impact of (say) using only rural stations on Hansen’s numbers or of using “good” stations on US temperature data, something that is on my mind, then you need to benchmark the implementation of Hansen’s methods against actual data as used and actual results, step by step, to ensure that you can replicate their results exactly and then see what the effects of changing assumptions or methods are. Only by such proper benchmarking can one ensure that you are analyzing the effect of rural stations and not unknown differences in methodology. This seems so self-evident that I don’t understand why you are contesting it.
[Response: Because, frankly, I find the ‘audit’ meme a distraction at best. I am much more interested in constructive science. Scientifically, independent replication - with a different set of ‘trivial’ assumptions is far more powerful (vis the Greenland ice core example) than any amount of micro-auditing. If there is a difference, then go to the code differences to see why (ie. UAH and RSS), but if you can show that the main result is robust to all sorts of minor procedural changes, then you’ve really said something. You have all the data sets from USHCN, GHCN, and GISS and you have demonstrated in a number of plots that all the GISS adjustment does is make a bi-linear adjustment to the stations based on close neighbour rural stations. How difficult is that to code? If the net result is significantly different than the GISS analysis then look into it further. If it isn’t, then why bother? In this field, methodology is not written in stone - it’s frequently ad hoc and contains arbitrary choices. Pointing out that there are arbitrary choice is simply telling us what we already know - showing that they matter is the issue. That kind of constructive analysis is how the rest of the field works - if you think you can do better and make better choices that are more robust to problems in the data, then that makes a great paper. Simply saying something is wrong without offering a better solution is just posturing. It’s worth pointing out that the GISTEMP analysis started out exactly because they were unhappy with how the station data were being processed elsewhere. - gavin (PS. edits to keep discussion focussed)]
14 August 2007 at 9:02 AM
For those interested, I’ve made a few supplemental charts from Gavin’s datasets, in particular the US and Global temperatures on the same chart:
http://graphoilogy.blogspot.com/2007/08/us-temperature-revision.html
14 August 2007 at 10:07 AM
Gavin:
You don’t seem to have addressed the point that the AEA has and presumably enforces a policy of archiving all data and code. Are you suggesting that such a practice has no value?
Khebab:Thank you, these are useful charts especially the one overlaying US and Global temperatures. (The one I twice asked Gavin to produce!) It sure focuses attention on the 30s and the question of whether the US pattern was really regional or global. I quickly checked Canadian data for the same time period and it seems to conform to the US pattern, which of course means that its weighting is significantly higher than 2% - probably approaching 13% of land based temperature measures.
14 August 2007 at 10:24 AM
I’ve never bothered to reproduce the entire NASA GISS analysis, mainly because I trust their results and reproducing it would be too much work. Besides, HadCRU and NCDC have done essentially the same work (albeit with different algorithm choices) and got essentially the same results — that’s pretty damn robust.
Most of the work would be in acquiring and formatting all the data and metadata. The fact is that making public all the code for all the programs and all the scripts wouldn’t much lighten the workload. Besides, I’m not at all interested in finding out whether running the same program on the same data will produce the same results — that’s obvious! Nor am I interested in going through all the programs and all the scripts line-by-line looking for potential errors. Program/script code can sometimes reveal simple errors, but it doesn’t really help very much in determining whether the answer is right according to the stated algorithm. Debugging my own code is a royal pain; debugging someone else’s is an exercise in self-flagellation.
I’d much rather acquire the data and metadata, then write my own programs to process it according to the stated algorithm. If my results agree, then I’d know that they got it right — and I did too. If not, I’d start debugging my own code until I was satisfied it was correct. If the results still disagree, I’d say so.
I’d never bother to debug someone else’s code. There are too many ways to make the code look wrong when in fact it’s right! And too many ways to make the code look right when it’s wrong. I’m not interested in disentangling someone else’s twisted logic, which very well might look right/wrong when in fact it’s wrong/right.
If NASA wants to release the code, and people want to pore over it line-by-line, more power to ‘em. But I’d rather my tax dollars were spent funding real NASA research than preparing programs and scripts for public release, to feed a call for “openness” which is really more motivated by a desire to discredit than a desire to discover.
As for actually reproducing the analysis from what is published, it’s a lot of grunt-work but really is not that complicated. Why haven’t the denialists done so? Here’s my theory: it involves a lot of actual work. You guys want to know whether NASA got it right? Get busy — put up or shut up.
14 August 2007 at 10:29 AM
Economics would have much less need to verify code correctness if its theory was ever allowed to meet observation.
14 August 2007 at 10:33 AM
#226. Khebab, Could you show the global data before and after? That’s what’s important, after all.
14 August 2007 at 10:40 AM
Didn’t “1998 is the warmest year” claim originate with the NOAA/WMO figures? That is, wasnt it NOAA the media was citing all this time, not NASA/GISS. Have NOAA’s rankings of largest anomalies been affected here, or do they remain the same?
[Response: Globally, all the indices showed that 1998 was a record breaker. In the GISS analysis, 2005 just pipped it to the post (as it does in NCDC product). For CRU, 1998 is still the warmest. The differences between the years are small, and the different products have slightly different rankings. Nothing from NOAA or CRU is affected by this correction to the GISTEMP analysis (and even there the global mean changes are too small to see). - gavin]
14 August 2007 at 10:43 AM
Dr. Curry writes “Climateaudit has attracted a dedicated community of climateauditors, a few of whom are knowledgeable about statistics (the site also attracts ‘denialists’)” and “in the long run, the credibility of climate research will suffer if climate researchers don’t ‘take the high ground’ in engaging skeptics….”
Mr. McIntyre and the “few … knowledgeable about statistics” are facing a choice Saul Alinsky has written about in “Rules for Radicals” — when you have a little success as a critic, you will need to decide whether you’re trying to improve the institution, or tear it down. You then choose either to stay with the people who got you to the gates, outside, or leave them to go inside.
Dr. Curry’s good advice is the same advice she gave over at CA to the people about the kid whose high school paper chart error got pointed out last month, deflating her attack on Hansen — when someone points out an error, check it, fix it, and move on.
A huge one-sided error like that high school kid’s graph blows the whole paper.
Little errors, when those are caught, that improves the product.
The successful ‘auditors’ who understand statistics are, in fact, improving the product. This can’t please the ‘denial’ crowd.
14 August 2007 at 10:47 AM
Re: #230 (Rob Negrini)
You can see that comparison here.
14 August 2007 at 11:00 AM
Re: #227
I’ve just added the chart.
14 August 2007 at 11:09 AM
If anyone wants to go through the source program of the GISS climate model, God bless them! Below is a small sample of FORTRAN statements from ModelE taken from “Field Notes From A Catastrophe” by Elizabeth Colbert(p 101-102)
C***COMPUTE THE AUTOCONVERSATION RATE OF CLOUD WATER TO PRECIPITATION
RHO=1.E5*PL(L)/(RGAS*TL(L))
TEM=RHO*WMX(L)/(WCONST*FCLD+1.E-20)
IF(LHX.EQ.LHS) TEM=RHO*WMX(L)/(WMUI*FCLD+1 .E=20)
TEM=TEM*TEMC
IFITEM.GT.10.) TEM=10.
CM1=CMO
IF(BANDF) CM1=CMO*CBF
IF(LHX.EQ.LHS) CM1=CMO
CM=CM1*(1-1/EXP(TEM*TEM)+1*100*)*(PREBARIL+1)+
*PRECNVL(L+1)*BYDTsrc)
IF(CM.GT.BYDTsrc) CM=BYDTsrc
PREP(L)=WMX(L)*CM
END IF
C**** FORM CLOUDS ONLY IF RH GT RHOO
219 IF(RH1(L).LT.RHOO(L» GO TO 220
C**** COMPUTE THE CONVERGENCE OF AVAILABLE LATENT HEAT
SQ(L)=LHX*QSATL(L)*DQSATDT(TL(L),LHX)*BYSHACM=CM1*(1.-1./EXP(TEM*TEM))+1.*100.*(PREBAR(L + 1) +
TEM=-LHX*DPDT(L)/PL(L)
QCONV=LHX*AQ(L)-RH(L)*SQ(L)*SHA*PLK(L)*ATH(L)
*-TEM*QSATL(L)*RH(L)
IF(QCONV.LE.O.O.AND.WMX(L).LE.O) GO TO 220 C**** COMPUTE EVAPORATION OF RAIN WATER, ER
RHN=RHF(L)
IF(RHF(L).GT.RH(U) RHN=RH(L)
There must be untold thousands of lines of code. It takes the largest computers today, a month for a single run to simulate 100 years of climate .It would be an enormous job for any individual, or even group of individuals.
Another difficulty would be that even though the components of the model that use the basic laws of physics,are the same for different models, the ones that are parameterized are different for different models.
14 August 2007 at 11:12 AM
Link to “the 2001 paper describing the GISTEMP methodology” is broken. Climate Audit has been sabotaged. An act of Gaia?
[Response: link fixed… thanks. - gavin]
14 August 2007 at 11:18 AM
Your criticism of climate science would be more persuasive if weren’t wrong, yourself.
5 of the top 10 hottest years GLOBALLY are still in the last decade. Which has been the claim all along. As has been pointed out, the “G” in “AGW” doesn’t stand for “The Lower 48″.
Sheesh.
14 August 2007 at 11:54 AM
Skeptic/deniers often claim, (like at Watts Up With That?) that climate scientists (like Hanson) won’t release their algorithms/SW (should see a joke in there…) I doubt that, and if not so, where can I find evidence of the algorithms/SW to show around?
14 August 2007 at 12:30 PM
In response to Steve Mosher’s #156…
I wrote:
steven mosher (#156) responded:
Looking at the raw data for Lake Spaulding, you can most definitely see what would appear suspect: it goes from roughly 11.75 to 7.25 within the span of approximately ten years - very early on in the records. That is suspicious. It is particularly suspicious when you compare it with the neighboring rural stations.
Which stations?
This will give you a list:
http://data.giss.nasa.gov/cgi-bin/gistemp/findstation.py?lat=39.32&lon=-120.63&datatype=gistemp&data_set=0
There are four rural neighboring stations within 71 miles.
Here is a chart of the raw data:
http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=425745010040&data_set=0&num_neighbors=1
You can compare it with the neighboring rural stations:
http://data.giss.nasa.gov/cgi-bin/gistemp/gistemp_station.py?id=425745010040&data_set=0&num_neighbors=3
One station doesn’t go back that far. Another shows almost comparable variability - but would not result in such a distortion, but the nearest neighbor shows nothing comparable.
Additionally, he does not simply posit contamination, but points out that we know in fact that such contamination took place in those years - and identifies the known potential sources. Can we say that these stations were affected in this way? Of course not. If we knew the exact way in which they had been “contaminated,” we could correct for this. But since we do not have this information, the uncertainty of the effect combined with the evidence that such an effect may have played a part in the temperature records is enough to throw out those years of data.
More importantly, the reason for dismissing the earliest data (at least in the case of the station that I investigated) is that the extreme variability at the earliest point was inconsistent with that of other stations and unrepresentative of any long term trend. I believe you will find that the same is true of the other stations if you are inclined to investigate as I have done. And as he said, the distortion would have been transmitted to the urban stations as the result of their methodology.
*
In any case, this is very early on in the temperature record. What we are concerned with when it comes to global warming is principally from 1978 forward. An adjustment of this sort would presumably require more justification during this period - particularly since we have better kept records and more rigorous procedures in place.
I wrote:
Steven Mosher wrote:
No need to link to the studies - I found them myself. Finding them takes only a few minutes, but the reading a bit longer.
Assessment of Urban Versus Rural In Situ Surface Temperatures in the Contiguous
United States: No Difference Found
THOMAS C. PETERSON
VOL. 16, NO. 18 Journal of Climate 15 SEPTEMBER 2003
http://www.ncdc.noaa.gov/oa/wmo/ccl/rural-urban.pdf
Steven Mosher wrote:
That is a very brief synopsis - enough to give the reader some idea of what is in a well-known paper - which they should probably read anyway, since in my estimation it is an excellent example of what research should be like.
Peterson goes through quite some history in terms of the studies which were performed prior to his study and lists the various factors the authors failed to take into account. There are uncertainties, and he acknowledges them. As far as the various factors which would influence station temperatures are concerned, he listed what the major ones were and the empirically determined adjustments that these would require - as identified by past literature - and then filtered out those effects.
I quote:
A detailed analysis of each of these factors as well as explanation of the manner in which such factors may be expected to affect the readings is given. Additional studies are cited for each of these effects. Progress, building upon earlier, more well-established work. The residual would be what is left over after all other factors are accounted for. This is what is attributed to the urban heat island effect: 0.04 C. It is not statistically significant. Once results were obtained, additional tests were performed for robustness.
You wrote:
Peterson suggests that there are a variety of well-known reasons why the urban heat island effect is relatively insignificant. Park cool island effects are certainly one of these. Another potential mitigating factor will be bodies of water - since urban sites are more likely to be near bodies of water. Clouds are more likely to form over urban areas as the result of urban heat, but in the process they are more likely to shade those areas where sites are located, reducing the effects of solar radiation.
How do you test for a strawman or cherry-picking? Park cool islands are just one of the well-known effects which may account for the unimportance of urban heat islands. Besides, a photo won’t pick up the thermal, but this is precisely what we are interested in as heat cells are what determine the boundaries which isolate a park cool island from its surrounding environment.
Peterson states:
Likewise, as has been pointed out to you on a rather large number of occasions:
http://www.realclimate.org/index.php/archives/2007/07/no-man-is-an-urban-heat-island/
… a site which is in a poor location will not produce a trend. A site where an air conditioner is installed beside it at some point will not produce a trend - but a jump. These are things which would be picked up by statistical analysis - but they would not produce a trend. As has also been pointed out to you on numerous occasions, we get virtually the same trend whether we are using all stations or simply rural stations. We get quite similar trends whether we are using surface stations or lower troposophere. Lower troposphere has higher variability, but shows essentially the same trend.
Steven Mosher wrote:
So I have noticed.
14 August 2007 at 12:48 PM
This is a response to Steven Mosher’s #157. My apologies at outset, but this will include some philosophy, although principally in defense of the contextual, falliblistic self-correcting approach of empirical science - as Steven Mosher’s position is fundamentally opposed not to the approach of climatology but of science. However, I doubt that I will ever have to deal with anything of this nature again, at least in this forum.
Please feel free to pass over it if this is not something which interests you. However, if you are inclined to read it, I have tried to make this easier by dividing it into titled sections.
*
Duhem and Quine
I had written:
Steven Mosher responded:
You are speaking of “The Aim and Structure of Physical Theory,” but this is an English translation of the book “La Theorie physique: son objet et sa structure,” which was published in 1906, although the original article which expressed the central insight was published as an essay in 1892 as “Quelques reflexions au sujet des theories physiques.”
In all honesty, I couldn’t remember after all these years whether it 1892 or 1893, so I had to look it up. But as part of an essay which critiqued early twentieth century empiricism that included logical positivism in all major forms, first individually and then as whole by means of self-referential argumentation, operationism, operationalism, etc and finally offering a critique of the analytic/synthetic-dichotomy by means of self-referential argumentation, I refered to it in a critique of Karl Popper’s principle of falsifiability and expounded roughly the same argument.
As I have said, I was a philosophy major.
*
The Fatal Flaw in Relativism and Radical Skepticism
I had written:
You responded:
Since he is a strong coherentialist, he would argue that even such distinctions as those between identification and evaluation, perception and emotion, subject and object are not ultimately based in observation, but are simply part of a web of belief in which even the law of identity is a subjective preference of sorts. His argument is essentially that in dealing with quantum mechanics, one may choose to employ an alternate logic. This much is true - as such an alternate could very well offer one greater theoretical economy.
But from this he concludes that it may be appropriate to abandon even at root the law of identity itself. As such, his “pluralistic realism” is fundamentally a form of extreme relativism. This however runs into a problem in that for an alternate logic to be regarded as an alternative to conventional logic, it must be internally consistent, but one must presuppose conventional logic and the law of identity in some form simply in order to test for internal consistency.
Any physical theory which is internally consistent would be logically untenable. Even in the case of quantum mechanics, one would have to abandon the theory if it proved internally inconsistent. However, assuming quantum mechanics is tenable, which is something which I believe we could both agree upon for our present purposes, it must be internally consistent.
Quantum mechanics may be counterintuitive - and it is.
However, for it to make specific predictions, even if they are expressed only in probablistic form, it must be internally consistent - otherwise, as a matter of logic it would be possible to derive any contradictory set of predictions in which case it would be untestable. Given the need for internal consistency, the law of identity is at root unavoidable, although one may in logic choose a formalism which expresses itself in some alternate formulation of it.
I will also point out that extreme relativism presupposes a form of radical skepticism in which the world that exists independently of our awareness of it is fundamentally unknowable. It should be clear given your statements regarding Quine’s theory that this applies to Quine as well. However, extreme skepticism is self-referentially incoherent.
This means that it has a fatal flaw which is closely related to the internal inconsistency which results from stating “I know that there is no knowledge.” This problem lies in the fact that when one asserts that radical skepticism is true, implicit in one’s assertion is the assertion that it is something which one knows. As such, there is a fundamental, fatal internal inconsistency implicit in its affirmation which renders it entirely untenable.
*
The Purpose of this Forum
Now I should quickly point out that the above is merely a brief summary. To properly deal with the issues which I have mentioned above at the level of technical philosophy would take a great deal longer. But this is not a graduate level philosophy course. This is a forum devoted to climatology. For this reason, I will not go into any more depth on these issues than I have treated them here. Duhem’s thesis could very well be a different issue as it deals with the interdependence of scientific knowledge, but since it is already something that we both agree upon I believe that is unnecessary.
*
The Nature of Science
Now with regard to climatology, no doubt you will point out that in my previous post it became quite clear that in arriving at any of the conclusions of the authors, one had to presuppose a fairly extensive background of assumptions - in very large part the conclusions of earlier papers. However, it should should also be clear that this is how science works. It is cummulative.
Even when an earlier theory is superceded by a more advanced theory, however much the form in which the more advanced theory is expressed differs from the earlier theory, it must be consistent with the evidence which formed the basis for the acceptance of the earlier theory. The form changes, but much of the substance is preserved.
This forms the basis for the correspondence principle which you are no doubt aware of. As such, there is nothing invalid in the cummulative nature of climatology. Additionally, it is clearly falliblistic. We will make mistakes. But given the systemic nature of human knowledge and empirical science, we can expect to uncover our mistakes in time.
There are degrees of justification, and where a given conclusion is justified by multiple independent lines of investigation, the justification that it receives is often far greater than the justification that it would receive from any one line of investigation considered in isolation. This applies to all empirical science. As such, the fact that its methods are fallible and its conclusions fail to achieve cartesian certainty cannot be held against it any more than it could to any empirical proposition.
*
Given the preceding sections, I believe we can skip much of the rest which follows, at least those sections dealing with issues in philosophy and the philosophy of science.
*
You state:
We will return to some of this in a moment.
But how much is to be determined by means of scientific investigation, and the results of such an investigation are to be regarded as our best estimate until further scientific investigation determines otherwise. It is not something to be decided by means of philosophy, word games or ideology.
Statistically, since 1978 they have been increasing worldwide according to according to virtually every study which has investigated it. Likewise, the global temperature was considerably higher for in 1998. This has been sufficient for every major scientific body which has taken a position on climate change to acknowledge that it exists and for mainstream science to acknowledge that by any rational human standard it is quite dangerous. Word games at this point will according to our endanger the a great many people, the world economy and quite possibly even more.
We have dealt with that at length - in the thread:
http://www.realclimate.org/index.php/archives/2007/07/no-man-is-an-urban-heat-island/
Now returning to your earlier statement, you begin by quoting me and then respond:
But this is not what has been suggested ever since you began participating in the debate here.
From your previous post:
This has clearly been your attitude since you first arrived - which is in line with the radical skepticism that I have dealt with above.
*
Your Motivation
Previously I had stated that the active skeptics with regard to anthropogenic global warming were generally either motivated by a misguided concern for the economy, financial concerns or ideology. The last of these was exemplified by you and your interaction with Ray Ladbury. At the time I did not know what your ideology was, but it was obvious that you are fairly intelligent and that your motivation has nothing to do with a concern for the genuine facts of the matter.
As I like to know who I am dealing with, I decided to do some digging.
Within five minutes, I found that you are the president of Population Research Institute, a spinoff of Human Life International, a pro-life organization. You advocate population growth as you view any attempt at zero population growth as being contrary to your pro-life stand.
Here is the evidence for you position as president of PRI:
An Interview with Steven W. Mosher, President of the Population Research Institute
By John Mallon
http://www.pop.org/main.cfm?id=151&r1=10.00&r2=1.00&r3=0&r4=0&level=2&eid=678
Here is the logic of your ideological position against the acknowledge the acknowledgement of what your organization views as the environmentalist nature of climatology - as expressed by your vice president:
300 Million and the Environment
Friday, October 20, 2006
By Joseph A. D’Agostino
http://theologyofthebody.blogspot.com/2006/10/300-million-and-environment.html
Now I do not care to debate ideology with you. However, your ideology is irrelevant to climatology and your approach is fundamentally anti-science. You will not be swayed by any evidence or argumentation.
We have no further reason to debate you.
14 August 2007 at 12:56 PM
“If anyone wants to go through the source program of the GISS climate model, God bless them!”
Sure, I’ll go through it, and I suspect many others in the ‘open source’ SW community would jump on the chance to help others utilize this.
Where is it available?
[Response: http://www.giss.nasa.gov/tools/modelE - this is the code that was run for the AR4 simulations, and so is now a little out of date (Feb 2004) - thus any issues are likely to have been fixed in our current version. But it gives you a feel for what the models are like and how they run. I plan on putting up the latest version at some point in the near future. - gavin]
14 August 2007 at 1:11 PM
> #20 Niller writes:
2) The North Pole is melting so that there will soon >be a North-West Passage to which Canada is laying >claims.
Your information is wrong. The Northwest passage is already open for commercial purposes.
The open, peer-reviewed journal Science reported that in 1999 the Northwest passage was already open:
“In 1999, Russian companies sent two huge dry docks to the Bahamas through the usually unnavigable Northwest Passage, which winds through the labyrinthine…”
This is old news in the open, world-wide, peer-reviewed scientific community.
http://www.sciencemag.org/cgi/content/summary/291/5503/424
http://www.google.com/search?hl=en&q=1999+russians+northwest+passage+bahamas&btnG=Google+Search
14 August 2007 at 1:13 PM
This general reader found Gavin’s comments dead on. The only point of vulnerability seems to be the danger that non scientists, like Mcintyre, can animate the boob circus masters to work through the political system and fasten a choke hold on a field of science. You can see what happens then if you review the grim story of Bush administration interference and outright sabotage of science for political ends in the blog by Rick Piltz, Climate Science Watch.
I suspect it will not be enough to gather round the water hole from time to time and agree on how unfair all this is, and perhaps the scientists in this field, climate science, who will always be working with close approximations and arbitrary bits here and there will need to set up a self defence organization. What will they need to defend? Their right to work the science as they please, not as meddling strangers want to control them to do, which would strangle the science. Perhaps it is turning out that Mann had exactly the right way to handle the non scientific opposition, stiff them, knock them down, give them a few more kicks. Like Terry O’Reilly played Hockey–always finish a check.
The union of concerned scientists seems to have impact, and you can see they use some of the weapons of the system itself.
14 August 2007 at 1:34 PM
Re: #225, and Gavin’s response,
As a software engineer myself, I make sense of where Steve is coming from. Software is the analytical tool by which these scientific conclusions are reached. As I recall in reading my history of the development of computers in climate science, it was not possible to crunch the numbers faster than the time span being measured, until the advent of fast computers. This means that these computers, and the software which runs on them, are the tools.
In fact, to go Steve one better: it matters on what hardware the simulation was run, and it matters what the OS was, and it matters what the release level was, and it matters what else was on that computer. Any of these things could cause corruption.
As Steve points out, this is about policy-level decision making, and I can assure you that this member of the public seeks the highest possible standards from science which has such potential to define economic paradigms far into the future. Probably the future of the nation state depends on how well nations organize to confront AGW.
So, Gavin, there is just a little import to these issues. I’m sure you’d agree. And while your point is valid that good science means doing it yourself and seeing what happens, Steve is also right that controversy (”my results are different than yours”) only leads us back to where we are. In other words, audit-as-you-go is a very sensible approach.
I disagree with Steve on one point: the auditing can, and probably should be
in-house. It will take considerable understanding of the processes involved, to fairly audit them. However, the auditor needs to have broader knowledge than the climate scientists and engineers, in a few areas: 1) he/she must be well versed in the issues I cited above regarding hardware and OS; 2) he/she must have a strong background in statistics and other forms of analysis. It would be mighty helpful if this person or persons had previous software auditing experience.
The sad truth is that there is no alternative. Gavin, your approach leads us here: there could be an error in both your process and somebody else’s; verifying them against each other will not reveal the error. Or: your results are different than somebody else’s, but he is poorly funded and has little support, and his concerns are dismissed. Or: Nobody has the time or money to replicate your gigantic simulations, so there is nothing with which to compare your results.
Unless you have some other, heretofore unknown empirical method for validating the results of your simulations, then I side with Steve: your processes must be thoroughly and competently audited, on an ongoing basis.
The work you do is just too important.
14 August 2007 at 1:52 PM
RE #225 & when results are used for policy purposes, people expect different forms of due diligence than little academic papers, which have received only the minimal due diligence of a typical journal review. People are entitled to engineering-level and accounting-level due diligence.
I don’t know much about this discussion, the codes & data sets, but I think there is a big difference between balancing the account books and addressing climate change so that we avert harms.
I remember how as a bookkeeper many years ago (before they used computers for it) I sweat bullets trying to find that 1 cent mistake I had made. I couldn’t just give them a penny, I had to find the mistake and correct it.
However, there may be some parallels with engineering a bridge so that it is strong enough to withstand maximum traffic load under worse-case conditions, except that with global warming we’re not talking about a bridge load of cars crashing into a river, but the possibility of huge problems around the world that may harm millions or even billions of people over the long time the CO2 is in the atmosphere (maybe up to 100,000 years even — see http://www.realclimate.org/index.php/archives/2005/03/how-long-will-global-warming-last/).
In that situation, we need to err on the side avoiding global warming and its harms, than on the side of helping fossil fuel companies that refuse to diversify.
As I’ve mentioned on this site many times, mitigating GW can be done to a large extent cost-effectively, and the potential harms are so great, that we really don’t need much evidence global warming is happening and is or will be harmful. Scientific standards of .05 significance are way too stringent. Even if there is less than a 50/50 chance the climate scientists are right that global warming is happening and is or will be harmful, we need to mitigate the problem. That should be the policy of a moral, even self-interested society. Anything else is tantamount to dereliction of the policy-maker’s duty. And it means we shouldn’t even be having debates about policy at all. It should have been our policy to mitigate GW with all our effort since at least 1990 — 5 years before the 1st scientific studies reached .05 on GW.
But here’s an idea. Rather than spend a lot of time and expense trying to find the mistakes of climatologists (I think they find each other’s mistakes — and get feathers in their caps when they find some gross ones that really matter), those who question the climate scientists’ accuracy and conclusions could really mess them up by getting the whole world to substantially mitigate GW. That would mess up their data sets. They wouldn’t have any increasing CO2, and if they are right that the temps rise with carbon, then they wouldn’t have any rising temps either (allowing for some lag time to pass). And then what could their codes do about it, without such great data?
14 August 2007 at 1:55 PM
re: #241 Patrick
Great! [although, it’s a little strange that anyone really interested in this, didn’t know where the code was: SOFTWARE is one of the top-level menu items on the GISS website], but when people make a big effort to provide well-documented, relatively-portable code, it’s nice if people read it.
This is actually an interesting test case:
a) How about describing your background, i.e., familiarity with F90, OpenMP, and the relevant physics? What target machine(s) and software will you be using? (likewise for other people you get interested in this).
b) How about a status report, say once a week? Tell us how far you’ve gotten, and how much effort it took?
[These are not new questions, I’ve often asked these of people using software I’d written, to get feedback about match with intended audience.]
14 August 2007 at 2:58 PM
What effect did the drought and lack of good irrigation have in 1934 average temperature as compared to more recent years where even in during a drought millions of acres are irrigated and potentially reduce the extreme high temps due to evaporation.
14 August 2007 at 3:33 PM
In 241 Patrick is willing to go through the program,and even sounds eager to do so. As Gavin points out in one of his responses above, Model E contains about 100,000 lines of code. Back in the stone age of computers, I carried boxes of punch cards for both programs and data to a CDC 6600 located at a local university. If there was a bug in the program,it was a painstaking process to locate and correct for it. And these were programs with a lot less statements(by a factor of 100).
If folks are willing to go through it, to search for discrepancies, my hat’s off to them.
14 August 2007 at 4:39 PM
Re: #247
Interesting point.
One may wish to consider the effect of irrigation on the Ogallala Aquifer:
http://www.waterencyclopedia.com/Oc-Po/Ogallala-Aquifer.html
http://en.wikipedia.org/wiki/Ogallala_Aquifer
14 August 2007 at 5:24 PM
Rod asks about the effect of irrigation. Google Scholar has plenty of info, try a search or two. For example:
http://www.agu.org/cgi-bin/wais?hh=A41E-0074
14 August 2007 at 5:34 PM
Tamino, your claim in #9 that it didn’t “cool” between 1940 and 1975 is incorrect. It’s not much, but if you chart global mean at NOAA’s GCAG, you’ll see the anomaly went from slightly under 0 to about -.1 C
It’s not much “cooling”, sure. The trend is only -.01 a decade and the significance is only 82%. But it did go down.
(GISS analysis using either 51-80 61-90 or 71-00 as the base shows -.14 (slightly confusing me as to why changing the base doesn’t affect the anomaly.))
But I did notice you saw the cooling trend for 1945-75 later in #197 I see. And you make a good point about the software in #228. Which makes all this discussion before and later on about “the code” rather perplexing.
———————-
Gavin, I understand your point in #188 perfectly, that’s a great way to explain it. I wish you’d put it that way in #43
So just let’s give all that junk to those asking for it and we don’t have to spend any more time on it. If that’s how it’s all used, just give it that same way and be done with it. I do disagree a bit with #189, at least if Dr. McIntyre has a reason to say he doesn’t fully understand it, which I have no reason to doubt. So if somebody like he with as much insane detail as he goes into statistically on his site can’t figure it all out, who can? For example, your response to #211, if it’s just “a couple of pages of MatLab” why is all this debate going on? Or is he just complaining for no reason? I don’t see that. Plus as he later points out, it’s not the original stuff anyway if he “reproduces it” Why reinvent the wheel? Although you do make good points about balancing against the other methods that other adjustment schemes make showing that it’s probably stable.
I commend you for the link to the references but many of those papers and the papers they reference are not linked or available. Which ones have the algorithms in them is rather difficult to tell, also. And as John N. pointed out in #75, it’s not all that detailed, I don’t see it. I think the point of all this is that rather than those that want to independently validate everything needing to demonstrate anything, but rather it’s up to those being audited to assist. I suppose that’s what the argument is about. That and “read all the papers” isn’t a very satisfying answer to a lot of this.
And your response to #195… Perhaps others have tried to “replicate the analysis from the published descriptions, and it’s more difficult or less complete than you’re making it out to be? (I’m just asking if you’ve considered that as well as you could have….) And in #196, the link to Eli’s blog and the discussion there shows there is an issue that “both sides” are sure they have valid points. I don’t think the views are mutually exclusive.
But again, I believe the disagreement is if “start Reading The Fine RFCs, all the informtion you need is there” is a valid answer or not. I’m sure some specifics might clear this up, so now we’re back to “just give all of it and let’s stop pointlessly debating matters of opinion”. Your discussion with John N. and your reply in #205 is prime evidence of the conflicting viewpoint, and your response to Steve M. in #206… Um, I believe he has and hasn’t gotten an answer. And #208’s response seems rather not an answer to the point that was being made. etc further on down the line with other back and forth comments.
Thanks.
———————-
Dylan in #50, if you actually read what Dr. McIntyre says on his blog and the subjects he’s interesting in talking about (mainstream accepted scientific literature) that’s been the point all along; it’s not what the data shows, it’s making sure it’s correct by independently validating it. That’s what confuses many of us; the audits could show more warming than previously thought, so why do so many attribute other motives to what’s just a bunch of data collection and validation?
———————-
Patrick, #51, I certainly agree with you.
Good point in #144 FCH.
———————-
But John M., in #53, those are the GCMs. (Later discussion in #240 on starts talking about the GCMs again…). But the adjustment software is not there. But goood point!!!
Instead of arguing about .1C or .003C or whatever, or complaining about doing station surveys, why don’t we all work on doubling or tripling their budget? (Perhaps more support would occur if the results could easily be replicated and easily independently verified?) And in #138, the point is this is a publicially funded organization, the citizens own that code. If somebody has experience coding or not is immaterial. If it’s copywritten or purchased software, why not just say that? If it’s a mess and not one package, why not just give it up? What’s the big deal about giving a researcher code that exists only to adjust the data?
Certainly it has a limited use to anyone else. Or how about a published paper with instructions and/or algorithms detailed “all in one place” instead, then? All the pushback or ignoring of the subject confuses some of us. On the other hand, your comments in #210 strike me as somewhat like Gavin’s in #208. As I said above, I think the issue of “serious software engineering and what it costs” is beside the point — if all the materials are little tidbits and expensive to put together, give them out as is! “Prove there’s a need” is not really a good answer.
shrug
———————-
That is a good point in #155 David. I knew a person with a doctorate in physics that did a lot of complicated analytical coding in C, but wasn’t a programmer. I understood the basic flow but the code was a mess. Aside from the fact I didn’t understand the math itself which was the bulk of everything.
———————-
All other discussion of other issues, or quibbling about details, detracts from what should be the conversation and goals. One such distraction is trying to compare periods less than 15-30 years. (like 2002-2006) We should be talking like Gavin did in the note in #87 about US temp trends (” For the more recent period 1975-2006: 0.3 +/- 0.16 deg C/dec”) My take on it all is “So 1934 and 1998 are the same anomaly for the US. So what.”
My question is the base period still 1961-1990 instead of 1971-2000? And does it matter and why or why not?
14 August 2007 at 6:31 PM
Re: #251 (Hal P. Jones)
We’ve been through this before, on another thread. But it’s an important distinction, so here goes …
I don’t dispute that 1975 was slightly cooler than 1940. I disputed the claim that “1940-1975 the temperature was falling.” I often hear it said that mid-century we experience 30 or more years of cooling, and frankly, it just ain’t so.
Fit a trend line 1940 to 1975; the slope is slightly negative (cooling). Now fit a trend line 1950 to 1975. The slope is slightly positive (warming). And that is *not* 30+ years of cooling. It’s more correct to say that it cooled from about 1944 to 1951 (7 years), then levelled off for 24 years.
Then there’s the fact that neither the cooling 1940-1975 nor the warming 1950-1975 is statistically significant. It’s not cooling or warming; it’s fluctuating.
The oft-repeated claim of 30+ years of global cooling mid-century, is excellent fodder for denialist propoganda. But it’s simply not true.
14 August 2007 at 6:41 PM
# 251 Hal
Somehow the message isn’t getting through.
How much computer-based scientific research & software engineering do you do? I’m happy to debate with people with relevant experience [as always, reasonable people can differ], but when people essentially keep insisting things are free, it goes rather contrary to a *lot* of experience.
My concern is to *cost-effectively* help good science happen, and I want my tax dollars to be used well. I’d be delighted to see GISS budget doubled or tripled … and I’d bet we’d see more things on websites, but I’d trust GISS to figure that out, given that they do better than many, from what I can see.
Yes, 2X-3X more budget … but no strings. I wouldn’t insist on procedures designed to slow down research, like those I mentioned in #210. PLEASE go read some of the things I pointed to there.
14 August 2007 at 6:54 PM
John Mashey, thanks for #210. This is really cautionary:
http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=1446868
14 August 2007 at 7:09 PM
Thanks Tamino. I think the issue here is one of phrasing. The proper way to say that is “The trend from 1940-1975 went in a negative direction.” And if we talk about “The trend from 1880-2006 went in a positive direction” that’s fine too.
Sure, I don’t disagree that 7 years of falling coupled with 24 years of being steady can be described a great many ways. My point is that choosing your measurement period can allow anything to be shown.
My outlook is that if we talk about what it’s doing at some arbitrary point (pick one: 1893-1900, 1900-1909, 1929-1944, 1944-1976, 1992-1998, 1992-2006, or make up your own) as being the greatest rise/fall of x years, that’s a dis-service. That there’s no doubt that from 1950-2006 it’s gone up a lot (.1C/decade, globally) — that is the key, and the subject I think. The meaning of that is a different subject and I would say so is the accuracy and a whole lot of other subjects rolled into one. It’s difficult if not impossible to try and talk about all of them at once!!!!
All I’m saying is that it’s globally trended up .1C/decade in how we measure it over the last 56 years (100% significance). We shouldn’t be discussing short term trends, that’s my only real point.
14 August 2007 at 7:10 PM
There is an unfortunate interpretation that those wishing to see source code wish only to find fault. But verification (or what some call “auditing”) cuts both ways - while it may find faults, the process also results in verification that a method is implemented properly and would thereby strengthen the interpretation of the output of the computer model. I suspect there is a strong organizational culture difference from those of us in engineering versus those in academic settings. We do make mistakes in engineering - and go through significant quality assurance steps to prevent errors from occurring, or finding errors that might have occurred. Perhaps a description of what quality assurance steps are used in the design, implementation and verification of various GISS data analysis and models would be helpful?
14 August 2007 at 7:17 PM
Re: Gavin’s response to McIntyre’s #225. I’m an academic lightweight, but I feel I have a good reason to disagree. Having the code and making specific changes to it is like doing controlled experiments. I think that is a very good way to do sensitivity analyses. Changing multiple factors at once seems a less effective method of learning, even if it could be a faster way to find a more robust model. Perhaps I’ve misunderstood the point?
14 August 2007 at 7:56 PM
I understand your message John, I just don’t think the same way about it that you do. I hope you understand that key point. I spent quite a bit of time explaining what this debate is about. One group doesn’t see the need and the other group does. Fine.
GISS does a wonderful job, and I don’t question anyone’s motives. It just seems to me that this non-issue is taking up a lot of effort better spent elsewhere. I have no control over their funding, but if I did I’d increase it. Moot.
Nope, not a programmer. Immaterial. Zip up everything in the code directory and sub directories and make it available for download. Case closed, questions finished. Issue done. Costs nothing. The code is produced by the government and is not secret, it should be available. Regulations don’t prohibit the release of it. All this other stuff is just cobwebs in the way. Who cares why anyone wants it or if it needs to be checked? Just put it out there.
But I’ll tell what my qualifications are. I worked for the government for many years, and know exactly how everything functions (probably more than I want to, in fact!!!) Owned businesses, been in charge of large teams of people, ran the IT (with the help of multiple team leaders) in an organization of hundreds. I have been involved in this industry since the ’70s and my degree is in computer science. I have almost 10 years experience teaching computers and networking down to the digital theory level and up to the business and policy aspects. Also have training and some various level of experience in Ada, tcl/tk, dBase, BASIC, shell scripts, batch files, HTML, XML, Pascal, various IDE stuff, CVS, and so on. I certainly consider myself qualified to discuss these issues, and on multiple levels. Political, economic, social, scientific and otherwise.
14 August 2007 at 8:27 PM
Here’s the correct link to “the 2001 paper describing the GISTEMP methodology”.
http://pubs.giss.nasa.gov/docs/2001/2001_Hansen_etal.pdf
14 August 2007 at 8:45 PM
Have any of the people who want to audit any of the existing models done their own model? I’d think that would be a convincing exhibit of bona fides, to show that they know how this stuff works and make their own code public.
I realize that some infighting about what data to include would be an issue —some people may want to rule in or out particular data sets or methods.
But if a group set up a public-source model, they’d get a lot of attention and be able to prove their competence in the field.
14 August 2007 at 9:35 PM
Re #251 (Hal Jones): Just to note that McIntyre is two degree levels short of being “Dr.”.
14 August 2007 at 9:48 PM
Re: #259
Hank,
I really think the best way is for NASA-GISS (and other similar organizations) to have robust DP audit departments - I’d be shocked if they didn’t have something along those lines established already.
The issue would be: what is their expertise in this sort of validation? In other words, as I mentioned earlier, there are issues which go beyond climate science and even beyond the programming language used. These areas of expertise are not critical for the software engineers and scientists who do the modeling, but they are essential for the auditors.
I’d love to hear from Gavin with regard to what NASA-GISS already has in place.
Of course, Steve would say: not enough, or else why did he and not they find the data-switch error?
14 August 2007 at 10:25 PM
Re: #197-198 (Tamino)
Many thanks again for the effort in producing a well documented answer to my naive doubts
your graph at
http://tamino.files.wordpress.com/2007/08/nh-sh.jpg
on the northern-southern hemisphere temperature anomaly is really interesting and thought provoking
For example…
1. If the main driver of global warming is man-provoked CO2 increase, then it would seem that already in the ’80s of 19th century (!)
Northern Europe-US “industrial revolution” was somehow able to make itself felt thru a (quite fast) Northern hemisphere differential warming
True: railways and carbon burning were rising very fast then,
but can this be enough?
one could give a look look - say - at figure 2 of
http://www.epa.gov/climatechange/emissions/globalghg.html
2. Thermal inertia of the southern oceans must really be “immense”, as you say, because even in recent years, when northerners sulfate aerosols are far in the past, and CO2, with its “7 or 8 months” diffusion time over the globe, the North-South differential is still rising!
but this - I suppose - could for example be the effect of a possible strong acceleration in recent northern CO2 emissions…
3. But then another notable kind of “thermal inertia” must be operating in the US, otherwise, after the demise of sulfate aerosols, one could expect a quick alignment to - say - a kind of northern hemisphere mode,
that is a QUICKER warming “to recover lost time”, so to say.
On the contrary, now US among the northern regions seem to be the warming laggard,
as the sulfate aerosols era had left a lasting heritage…
Now if we assume some kind of big “thermal inertia” all can be neatly explained, and this is perhaps the correct thing to do,
but then
unless we also find a robust way of independently verifying and measuring “thermal inertia”,
admission in the discourse of this additional “free entity” reduces greatly the forcefulness of our theoretical construction,
because other competing explanations of global warming would become workable too:
because it is enough that these “competitor theories” adjust the non-directly-measured-but-conveniently-assumed “free entity” at the level that best fits their needs.
One would then be forced then to admit a higher level of ignorance on climate mechanisms than it’s pleasant to do.
14 August 2007 at 10:35 PM
“so why do so many attribute other motives to what’s just a bunch of data collection and validation?”
I guess I am more cynical, but after seeing what Bush did to the gulf Coast after the storms, and after seeing him put the scientist guy that companies use as paid witness to defend lead poisoning cases into the EPA with a high ranking position , it is pretty obvious to me that there are other motives and agendas at play here. And one side has proven itself to lie and exaggerate exponentially more than the other.
14 August 2007 at 10:59 PM
Re #260 - So what is your point?
14 August 2007 at 11:15 PM
Steve L: Having the code and making specific changes to it is like doing controlled experiments. I think that is a very good way to do sensitivity analyses. Changing multiple factors at once seems a less effective method of learning, even if it could be a faster way to find a more robust model. Perhaps I’ve misunderstood the point?
Not a software type, are you? The nasty fact is that software is formally chaotic. There are several ways to get to that conclusion, but the one I like best is that a running program in a Von Nuemann computer is an iterated map, and iterated maps are known to be chaotic. Which adds up to the fact that making random changes to code usually blows things up.
That is beside the point, however. The game that Gavin, or any modeler of natural processes, plays with his models is to try to learn how things works. Coding a new feature, or trading one way of calculating with another are, essentially, experiments. The code is not important. The equations expressed in the code are important. And running the model is how you evaluate a particular set of equations. The rules of economics or engineering don’t really apply here. In a profound way, a sensitivity analysis is meaningless in this domain.
14 August 2007 at 11:28 PM
Gerald, 260 corrects an error in 251; Dr. McIntyre is someone else.
15 August 2007 at 12:26 AM
The side discussion of tectonics in the context of sea level rise is getting a bit tangled (”BlogReader said: Odd you didn’t mention that maybe some plates might be sliding lower into the ocean . . .and john mann replied, “Well, plates do slide against each other, but sea level changes due to tectonics tends to be in the order of 1cm per thousand years . . .”). And this epic set of comments is WAY too good to get sidetracked, so I won’t wade in with much more.
Barton summed it up nicely: Sea level is rising on average, as determined by several independent lines of evidence (satellite ranging; satellite gravity measurements; tide gauge analyses, among others) - but the effects on individual coastlines are decidedly local. A previous Realclimate post (http://www.realclimate.org/index.php?p=314) touched on these issues without explicitly dealing with coastal changes (unless I missed a more relevant post; sorry if so). In most places eustatic sea level rise due to AGW is purely bad news. All this is probably best taken up in another thread sometime.
15 August 2007 at 12:36 AM
Re. #258: amazing. That paper contains not one formula that I can see. If the algorithm is as trivial as claimed, surely it might fit into the space of one of the many graphs supplied in the index?
Has anyone here gone to the trouble of turning those ten pages of dense text into something more directly comprehensible (i.e., a formula)? If so, would they mind posting it here?
15 August 2007 at 12:43 AM
Gavin, thanks for the link, I’ve downloaded source and am looking through it now. While you may argue such code-sharing isn’t important like primary work, it certainly helps in diffusion of knowledge.
JohnM: My platform is Linux. In 25 years of SW experience, I have studiously avoided Fortran as best I could, but unfortunately have to run into it occasionally; I can certainly compile, read and understand as well as most other languages I work with. As I mentioned before, I have a day job and other responsibilities besides, so my intention in doing this myself is for curiosity/education, not to find any bug. Should it parlay beyond that I cant commit to now; expect no weeklys, they are painful enough even when paid.
The one thought I did have that might be a contribution was the thought that, like the SETI project, there is perhaps a way to harness many people’s machines to run these GCM simulations. Have others had this idea as well? I think so. Anyway, an available open-source GCM code base would be an enabler of such an effort, hence my ‘what the heck, I’ll run it’ bid to take it.
[Response: Watch this space… - gavin]
15 August 2007 at 1:49 AM
I feel persuaded by Gavin on this one RE: response to McIntyre.
Both don’t seem to disagree in the value of the check and balance system of replicability. Where they disagree is in the required level of precision in the record. However, in this case, the accuracy of the record remains the same even after the corrections, for all intents and purposes (and that accuracy was coupled with a good level of precision nonetheless). It seems as if McIntyre is not so much concerned with accuracy (and I think that Gavin was right to refer to his exercise as “micro auditing”) as he is with precision; I don’t think he has made the case for why greater precision must be had (at least in this instance).
What Gavin is arguing is that when a question of accuracy arises, it is much more fruitful to form a competing replication. Since there are different codes used for the record constructions anyway (we can, I think, rightly assume this), it’s clear that the code changes aren’t going to affect whether or not the record is accurate. Analysing a replication through creating a competing (and perhaps, null) replication will tell you a lot more.
Justin
15 August 2007 at 2:02 AM
When a “skeptic” scientist offers up a view of why they may disagree with your theory you are very quick to dismiss them (I have never seen so much as a “well you have a point..”). So people do build things from scratch and it ends up in a “he said-she said” argument.
This issue gives everyone a huge opportunity to move closer together on climate change analysis as it cuts to the heart of many skeptics’ problems. Computer models. Silly dim old skeptics can’t quite believe how computer models are able to model something as complex as the weather. Out to 100 years in the future. But they do, and many people say that such forecasts should dictate public policy. So when you say it’s about the science, build your own model, you are being disingenuous: it has become both about the science and the models. Climate science without the models would be meaningless.
I am a drive-by skeptic with no scientific training. What do they call such people? Let me see..oh yes, voters. There are billions of us. Threads like this, seeing the many ways you have tried to deny there is a need to open any aspect of your scientific investigation to anyone who wants it, makes it less likely we will vote for any “anti-global warming” measures which today we see as futile and economically destructive.
15 August 2007 at 3:24 AM
re 259 H Roberts …if a group set up a public source model…
It is a surprise that this hasnt been proposed already.
The Linux project shows that massive public codes can be developed.
With modern cluster and sharing management large amounts
of computer power is freely available.
Maybe the Free Software Foundation could be the vehicle.
Or, Anthony Watts seems to be able to inspire public
participation. All we need is one Torvalds!
15 August 2007 at 3:31 AM
Re #259: I don’t know the details, but IIRC NCAR’s CCSM is set up more or less for wide access, which raises the question of why the “auditors” are interested in the GISS model in particular.
Re #262: It was an appeal to lack of authority, Gerald.
15 August 2007 at 6:48 AM
Well, apologies to Jim Hansen, but that has to be the worst coding style I’ve ever seen. No indentations or blank lines, barely any comments, gotos all over the place… if he’d wanted to make it unreadable he couldn’t have done a better job. And I assume that’s Fortran-77? It sure doesn’t look like ‘95. God forbid that I knock ‘77, because it kept me employed as a programmer for eight years, but please tell me they’re not still coding in it.
15 August 2007 at 7:02 AM
Tamino states in #252 that:
“I often hear it said that mid-century we experience 30 or more years of cooling, and frankly, it just ain’t so. . . . Then there’s the fact that neither the cooling 1940-1975 nor the warming 1950-1975 is statistically significant. It’s not cooling or warming; it’s fluctuating. . . . The oft-repeated claim of 30+ years of global cooling mid-century, is excellent fodder for denialist propoganda. But it’s simply not true.”
According to the NOAA global temperature anomaly data, prior to 1981, the highest average global temperature anomaly in a single year was .2143 degrees Celsius, in 1944 (The average global temperature anomaly in 1940 was .1187 degrees Celsius, nearly .1 degrees Celsius below the peak level attained in 1944). From January 1945 to December 1976 (a total of 384 months) the global temperature anomaly was lower in 369 months, or 96% of the total. The standard deviation of the first difference of the global temperature anomaly is about .108 degrees Celsius. In the same 1945 to 1976 period, in 317 months, or 83% of the total, the global temperature anomaly was greater than one standard deviation below that average in 1944. Furthermore, in 206 months in that same period, the global temperature anomaly was greater than two standard deviations below the 1944 average.
One would have expected that, were the global temperature “not cooling or warming; [but] fluctuating” between 1945 and 1976, we would have expected that roughly 50% of the data points in that period would have been above and below that 1944 average, that 33% would have been one standard deviation below and that 5% would have been two standard deviations below.
For those who claim that this writer is cherry picking, if we take the average global temperature anomaly for the 1940 to 1944 period of .1434 degrees Celsius, the corresponding percentages are 90%, 60% and 31%. These percentages are significantly different from expected percentages of 50%, 33% and 5%.
Furthermore, Tamino claims that the relatively flat temperatures between 1950 and 1975 dispel the notion that one could consider that there was a general cooling trend. The average global temperature anomaly during this period was -.0036 degrees Celsius (the corresponding figure for the 1952 to 1975 period is .0063 degrees Celsius). In 277 months out of 312 (89%) during this period the global temperature anomaly was below the 1940 to 1944 average; 179 months (57%) were 1 standard deviation below the 1940 to 1944 average; 93 months (30%) were two standard deviations below the 1940 to 1944 average.
Slimply put, the global temperature anomaly was, very simply, cooler in the 1945 to 1970 period and these cooler temperatures were statistically significant compared with the five-year period between 1940 and 1944 when global temperatures reached their highest level attained before 1980. Regardless of your position on AGW, the above analysis is not “denialist propaganda”, but a careful examination of reality based on the data.
15 August 2007 at 7:36 AM
Re #226. Thanks for the graphs. Looks much better now that global warming doesn’t fly off the chart anymore. Let’s hope the GISS graphichs person takes note.
15 August 2007 at 8:09 AM
#272 Alan,
This incident proves that climate scientist take skeptics seriously when they bring something new and worthwhile to the table.
But all I see from skeptics is that that recycle the same already refuted arguments over and over again, millions of times, I would estimate. Nobody has time to even take note of this stuff.
For instance, your just trucked out the old “models don’t work” chestnut again. Been there, done that, nobody has time to keep refuting it over and over again for every skeptic with a keyboard [edit]. There are just too many of you.
15 August 2007 at 8:10 AM
re: 272. Do you apply your model skepticism broadly? Or selectively to science? For example, the next time you step on an airplane for a flight, do you realize that models were used to develop and test the plane? Or do you demand to see the open models that were used first before allowing the plane to take off? BTW, those models were also tested and peer-reviewed.
“…which today we see as futile and economically destructive.” Ah, now the unobjective statement behind your question becomes obvious. “WE”, the voters do not see that at all. Check the various polls re: the need for action on GHGs.
15 August 2007 at 8:20 AM
Re: #276 (John Tofflemire)
You have misunderstood me. I’ll say it again…
I have never disputed that the period 1950 to 1975 was cooler than the period 1940 to 1945. What I dispute (correctly) is that is was cooling *from* 1945 *to* 1970. If you want to say that mid-century we saw 25-30 years of “cooler,” that’s one thing. But if you say 25-30 years of “cooling,” then you are quite simply mistaken.
The “denialist propoganda” to which I refer is the claim that the globe was cooling for 30+ years mid-century. The impression which is intended is that the planet cooled, and kept cooling, for three decades. It just ain’t so.
We beat this to death already on an earlier thread.
15 August 2007 at 8:52 AM
# 278 Tom, why should people not continue to express doubts? Just because you say so? [edit]
# 279 I do apply my model scepticism broadly. I have done enough (albeit financial) modelling to know that you can make an output what you want to make an output. A plane will have been proved to fly - ie in real life. Would you believe a model that told you what type of flight will exist in 100yrs?
15 August 2007 at 9:03 AM
RE #277, that’s science for you. We lay people look at something and see not much happening, but a scientist looks at it and sees all sorts of things. It’s a matter of many years of education and training. And I’ll trust what the scientists say over what the untrained laypeople say any day.
BTW, I just read on ClimateArk.org that British scientists have predicted the temps will level off for 2 years, then after 2009 rise sharply, and all hell is about to break loose. See:
http://www.climateark.org/shared/reader/welcome.aspx?linkid=81736 and
http://www.climateark.org/shared/reader/welcome.aspx?linkid=81891
Of course, if you’re 99 and expect not to be here then, you’ll probably miss the worst.
15 August 2007 at 9:27 AM
279. “re: 272. Do you apply your model skepticism broadly? Or selectively to science? For example, the next time you step on an airplane for a flight, do you realize that models were used to develop and test the plane?”
Dan, an inapt analogy. We have had tens of thousands of actual airplane flights of experience to validate that the models are matching observation. Airplanes were not built on models alone, but multiple test flights. We have yet to live through the 21st century to validate climate models longterm. Its an ongoing validation and calibration, year by year.
For climate modelers, the challenge is that its such a complex system with complex cause-and-effect, that one can match past experience yet have a model with not much skill in predicting the future (viz 281).
For example, I could give you a computer model that perfectly matches how the stock market performed up to today. Would you put your entire life savings into using it to make a bet on the stock market in the future? I suspect you would give it multiple ‘trial runs’ before making such a bet, right?
[Response: You are erecting and knocking down strawman arguments. Climate models have nothing to do predictions of the stockmarket. In climate modelling, there are indeed many ‘trial runs’ that give people confidence in their projections. See previous threads on this exact same point: here and here. Further discussions of climate models is OT in this thread. - gavin]
15 August 2007 at 10:42 AM
We haven’t touched on this before that I have seen, but something occurred to me last night as I read the latest climate story from Elizabeth Kolbert in The New Yorker. Her story is about declining bee populations. I also saw in last week’s Newsweek that certain Central American frogs were wiped out in 2 years in the late 1980s. Also, certain butterflies no longer exist in their previous habitat, having moved north and up to seek the cooler temperatures they prefer.
In other words, nature is changing out from under us, long before we ‘feel’ the impacts of climate change in our daily lives. And long before ice sheets break up, or storm patterns change for the worse, or previously wet regions turn to desert.
Long before that, animals and plants which are much more sensitive to environmental changes will feel the effects and be forced to adapt. This will have unknown consequences for man, but one thing is for sure: there will be consequences.
15 August 2007 at 11:05 AM
Another issue with having the code is that there is a very strong tradition in the open source community of improving code. Some of the fastest compilers, as I think John Mashey will testify to, are products of open source efforts.
On the subject of “support”, I was an early UNIX user — I’ve seen 7th Edition and System III source code, as well as some variants that never saw the light of day. We managed to do just fine without support from Bell Labs. The same was true in the early days of Linux, back when we installed it using a stack of floppy disks. There was no support for Linux then, and look at it today. I wouldn’t be surprised to learn that a lot of the platforms running these models are Linux systems
15 August 2007 at 11:11 AM
Steve McIntyre observed that the GISS corrections does attempt to eliminate UHI for urbanizing sites. However, Steve McIntyre also observed that the GISS corrections added a warming trend to sites that is rural in nature and hasn’t moved for quite some time. I’d like Gavin to explain the physical reason why a warming trend was added to rural sites and how it was added. This is no small matter. Subtracting UHI from urbanizing sites, but adding UHI from non-urbanizing sites, still keeps UHI effects in the GISS analysis. The validation of climate models can’t be done with surface data that is corrupted with effects unrelated to climate change. This makes any error found a BIG deal in my mind, because it opens the possibility of more errors.
[Response: The virtue of reading the references:
Why might that be?
Therefore any one station, even if rural, has its trend set by the average of raw rural station data in the area. - gavin]
15 August 2007 at 11:38 AM
Re: #286
Agreed.
My point was, climate change will affect these populations in a direct way long before they affect humans in a direct way.
And yet, these indirect effects will have significant consequences for humans. For example, without bees we cannot grow as much of certain fruits, vegetables and even nuts as we demand.
That’s just one of what will certainly be many examples.
15 August 2007 at 11:52 AM
#283 & We have yet to live through the 21st century to validate climate models longterm.
Well, see the scientists want to do as much as they can now with whatever evidence they have, before what actually happens during the 21st century has a chance to validate or invalidate their models. That’s because by 2100 there may not be any scientists left. They may have to do like the rest of the remnants of humanity will be doing — scrounge around for food, fight off mauraders, escape mega floods, hurricanes, storms, and forest fires, and such.
15 August 2007 at 12:13 PM
Gavin, So what if the www.surfacestations.org effort determines that some of the rural stations used in the regional average is corrupted with non-climate change effects? This may not subract enough trend from urbanized sites and add too much trend to the rural sites.
Of course, the nearest USHCN site to me is Huntley Exp Station, MT. They irrigate all around the station, so while it can be considered rural, they have land use effects corrupting the climate change signal (posible cooling effect during May-Aug?). Is it possible to know how the levels of irrigation has changed in the area during the last 100 years? And even if we knew the irrigation levels, do we know how to correct the data?
I think we need to fund a climate station network that sites stations free of land use effects and changes.
[Response: The regional trend is the average trend of the regional rural stations. In the raw data, some will be more than the mean, some less. There is a funded climate station network - CRN - which is exactly what you want. - gavin]
15 August 2007 at 1:13 PM
Ahem, looks like the numbers got moved. Just to clarify, #260 is Hank Roberts discussing people making their own ways of performing adjustments. #261 is Steve Bloom correcting me on Steve McIntyre not having a doctorate (The title I used in #251) Hank (#267) responds to Gerald (#265) asking what the point of Steve Bloom’s comment about not having the title of doctor. (So Hank, same person, wrong title).
Which is correct, I was wrong, I went and looked it up. It’s a bach sci from U Toronto in 1969 and graduating Oxford 1971 after studying philosophy, politics, and economics. I’m unsure if Oxford included a degree or what kind if so, it just says graduated. I just always thought he had a PhD, sorry.
Also, Ralph (269) comments on “#258″ (barry’s link to Hansen 2001 in #259 ) hillrj (#273) is commenting on Hank in #260 not #259, Steve B. (#274) has #259 and #262 but that’s #? and #265, etc.
FP, in #264 you are talking about the US govt and matters of politics vis ulterior motives. I was talking about the people surveying or auditing.
Justin, in #271 you talk about precision. I don’t think that’s it. It’s more an issue of validating a method by analyzing the method itself versus validating it by creating a different method that does the same thing. I suppose the discussion is all about which way is the “better way” to see if things are doing what they’re supposed to? Gavin thinks the other methods around already do that. Steve wants the checking the original method directly. So what I see is that one doesn’t see the need and the other doesn’t understand why the need has to be seen. I don’t think there’s a solution to that.
Steve B, your #274 I believe you’re saying that my mistakenly calling him Dr. as a title is trying to say what I wrote was an appeal to (lack of) authority. Reading the bio, it seems to me that regardless, he’s qualified to statistically analyze these sorts of things. YMMV
#276. John T. #280 tamino I suppose it depends if you’re talking about constantly cooling every year versus the general trend.
15 August 2007 at 1:38 PM
Regarding the adjustments, at one time (August 1999)1934, 1921, 1931, 1953 and 1998 was the order of US temperature records.
See:
http://www.giss.nasa.gov/research/briefs/hansen_07/fig1x.gif
From:
http://www.giss.nasa.gov/research/briefs/hansen_07/
[Response: Interestingly, that was prior to the adoption of the USHCN corrections for time of observation biases etc. - which is what the 2001 paper was all about. - gavin]
15 August 2007 at 1:44 PM
This is definitely an interesting to discussion to read as non-climatologist scientist. All of the attention and scrutiny is, in the long run, very good for the science.
I would agree w/ many of the comments above…the skeptics, denialists, auditors, whatever…they are making the case for AGW stronger with their probing. I’m interested to hear about the results of the due diligence.
And I would love to contribute computing power in the SETI-type way mentioned in #270.
15 August 2007 at 2:10 PM
#292
Are you looking for something like this?
Climateprediction@home
15 August 2007 at 2:17 PM
Hal,
when you say “validating the method”, do you mean validating the code or validating the method without its coded form?
Justin
15 August 2007 at 2:23 PM
Re#293: DavidU…thanks!…I will give that site a look
15 August 2007 at 2:50 PM
I trust those who are clamoring for access to the computer code used by climatologists in order to scrutize those codes for possible errors are similarly motivated to do quality control checks on the models underlying gloom and doom economic forecasts of dealing with AGW:
Emissions Dilemma High Cost Of Reductions Estimated; Others Say Doing Nothing Would Cost More
By ALAN ZIBEL
Associated Press
August 14, 2007
WASHINGTON
Making big cuts in emissions linked to global warming could trim U.S. economic growth by $400 billion to $1.8 trillion over the next four decades, a new study says.
The study published Monday by a nonprofit research group partially funded by the power industry concludes that halving emissions of carbon dioxide - the main greenhouse gas linked to global warming - will require “fundamental” changes in energy production and consumption.
The Electric Power Research Institute said the most cost-effective way to reduce the level of carbon dioxide in the atmosphere is to make many changes at once, including expanding nuclear power, developing renewable technologies and building systems to capture and store carbon dioxide emitted from coal plants. Reducing demand for fossil-fuel power is also key, the institute said.
The EPRI cost estimate is based on a 50 percent cut in total U.S. carbon emissions from 2010 levels by 2050. Without such a cut and the shifts in technology it would bring, the Energy Department projects that U.S. carbon emissions will rise from about 6 billion metric tons a year in 2005 to 8 billion metric tons by 2030.
The report calls for more modest cuts in emissions than some proposals being considered in Congress. Bigger cuts could well be more expensive…
Disclaimer: Many years ago, I was involved in research funded by EPRI.
15 August 2007 at 3:15 PM
Hal P. Jones (#290) wrote:
Quick note: even when on rare occasion a post gets removed (and this has happened to me before when I implied a rather ill-chosen comparison), if you have hyperlinks to the posts that you are refering to, people will still be able to follow. Anyway, thank you for the effort that you are putting into keeping the context.
It helps.
Personally, I have a variety of problems with Steve McIntyre. First, he is not a climatologist, and he has deliberately made use of statistical methods which he should know are invalid to try and discredit the hockey stick. Second, he tries to create the impression that taking photos of the stations will be sufficient to determine whether they are capable of providing accurate and useful information, that is, that somehow the photos will show whether or not the stations are in park cool islands, whether other staticians are using the appropriate statistical methods for separating the signal from the noise, etc.. Third, he cherry-picks the stations and misrepresents the data which is being recieved from them. Fourth, he pretends as if somehow a problem with a particular station will result in a continuing upward trend when all it would produce is a jump, not a trend. Fifth, he pretends as if this is the only source of information we have for reliably determining trends.
So I do not question his qualifications as a statician. [edit]
That said, he is attracting more talent at present. Who knows ?- maybe something of real value will come out of his group despite his involvement. Stranger things have happened.
But I won’t be holding my breath.
15 August 2007 at 4:21 PM
Re #296, makes me wonder why anyone thinks that fossil fuels starting with Oil are going to be around in suficient quantities to fuel economic growth for decades to come. From all of the available evidence I would suggest not. And to think that something else is available to replace Oil, then gas and coal immediately with no economic pain is at the present seemingly presumptous and slightly foolish.
Still Peak Oil people are seem as doomongers much like environmentalists and as yet not quite mainstream enough thinking.
15 August 2007 at 4:48 PM
Chuck Booth (#296) wrote:
Electric Power Research Institute
http://www.sourcewatch.org/index.php?title=Electric_Power_Research_Institute
Hmmm… Exxon funding, Chauncey Starr in previous years froom George C. Marshall Institute…
Starr, Chauncey (George C. Marshall Institute)
http://www.mediatransparency.org/recipientprofileprinterfriendly.php?recipientID=137
Involved in…
HARVARD CENTER FOR RISK ANALYSIS
According to its website, the HCRA “was launched in 1989 with the mission to promote public health by taking a broader view. By applying decision science to a wide range of risk issues, and by comparing various risk management strategies, HCRA hopes to empower informed public responses to health, safety and environmental challenges by identifying policies that will achieve the greatest benefits with the most efficient use of limited resources.” (http://www.hcra.harvard.edu/about.html; accessed 03/29/06)
according to Center for Science in the Public Interest / Integrity in Science Database
http://www.cspinet.org/integrity/nonprofits/harvard_university.html
… tactics out of tobacco industry playbook.
Not good.
PS
Chuck - Great to see you made it to this front now that the other has cooled down.
Same war, though.
15 August 2007 at 5:04 PM
Resources for researching the disinformation industry:
Integrity in Science Database
Center for Science in the Public Interest
http://www.cspinet.org/integrity
Source Watch
http://www.sourcewatch.org
Center for Media and Democracy: PR Watch.org
http://www.prwatch.org
Media Transparency
http://www.mediatransparency.org
Climate Science Watch
http://www.climatesciencewatch.org
DeSmogBlog
http://www.desmogblog.com
Society of Environmental Journalists
http://www.sej.org
See inks for more: http://www.sej.org/resource/index18.htm
15 August 2007 at 5:17 PM
Still Peak Oil people are seem as doomongers much like environmentalists and as yet not quite mainstream enough thinking.
Favorite Far Side cartoon:
Two fish are outside their small fishbowl watching a fire consume their fishbowl castle. One says to the other, “Thank heavens we made it out in time. Of course, now we’re equally screwed.”
15 August 2007 at 6:36 PM
re: #301
Peak Oil
ASPO-USA will be held in Houston in October. You can check who’s speaking and decide whether they actually know anything, or whether they are random doommongers.
(ASPO = Association for the Study of Peak Oil & Gas),
http://www.aspo-usa.com/
Alternatively, look at
http://www.lastoilshock.com/
and get Strahan’s book (Amazon Canada or UK, not USA).
Peak = 2015 +/- 5 years is the consistent estimate. Personally, I’m planning for $10/gal gas here within 10 years.
Peak Oil & Global Warming are rather tightly coupled. If we do our best to burn up oil fast, we not only increase CO2, but when the gas gets really expensive, we get to have a World Depression that makes it hard to invest very quickly in replacements, and will almost certainly mean we’ll be burning a lot of coal before we figure out how to sequester it.
Maybe we can avoid this; younger folks will get to see it firsthand!
15 August 2007 at 6:55 PM
Tamino says (#280):
“The “denialist propoganda” to which I refer is the claim that the globe was cooling for 30+ years mid-century. The impression which is intended is that the planet cooled, and kept cooling, for three decades. It just ain’t so.”
In #256 Tamino said:
“It’s more correct to say that it cooled from about 1944 to 1951 (7 years), then levelled off for 24 years.”
In fact, one could claim, with a reasonable degree of validity, that it is “denialist propoganda” the earth warmed, and kept warming, for the 30+ year period from 1976 to 2006. For example, the average global temperature anomaly in 1976 was -.1182 degrees Celsius (using the NOAA data). In 1981 the average global temperature anomaly was .2392 degrees Celsius, an increase of .3574 degrees Celsius in just 5 years. In 1996, the average global temperature anomaly was .2564 degrees just .0172 degrees higher than 15 years earlier. In 1998, just two years later, the average global temperature anomaly was .5772 degrees Celsius. In comparison, the average global temperature anomaly in 2006 was .5391 degrees Celsius or .0381 degrees Celsius cooler than in 1998!
Thus, one could claim that the earth’s temperature rose rapidly between 1976 and 1981, then essentially “fluctuated” between 1981 and 2006, except for the rapid increase between 1996 and 1998. Under such logic one could further claim that the earth has not “warmed, and kept warming for three decades” to pharaphrase Tamino’s statement noted above.
Of course, such a claim is sheer nonsense. The earth has been in a warming period since 1976 and to deny that is to deny reality. That is, temperatures have, on average, risen and have, on average, tended to remain at elevated levels. Similarly, between 1944 and 1976, temperatures, on average, fell and, on average tended to remain at supressed levels. The difference is that the total temperature increase in the warming period since 1976 has been much greater than the total temperature decrease that took place in the 1944 to 1976 cooling period. That is also reality.
Now, Hal Jones notes (#290) that:
“#276. John T. #280 tamino I suppose it depends if you’re talking about constantly cooling every year versus the general trend.”
In fact, between 1976 and 2006, in 17 years the year-on-year change in the average global temperature anomaly was positive and in 13 years the year-on-year change in the average g.t.a. was negative. In other words, average temperatures fell nearly half the time year-on-year during a warming period! The earth’s temperature fluctuates up and down whether the period is warming or cooling. It NEVER constantly warms or cools!
Why is all of this important and why is this thread not “much ado about nothing”? The U.K.-based Climate Research Unit last week forecast that the average global temperature anomaly in 2014 will be .3 degrees Celsius higher than in 2004. Since the NOAA average g.t.a . in 2004 was .5344 degrees Celsius, the CRU is forecasting that the NOAA g.t.a. in 2014 will be .8344 degrees Celsius (+/- some confidence interval). There is a lot riding on this forecast for, if it is correct, then it would be impossible for anyone but the delusional to argue that AGW is real. On the other hand, if the actual average g.t.a. is significantly lower than this figure (crossing our fingers that we don’t have an accursed major volcanic eruption to screw things up), the AGW theory as currently held will very much be in doubt. Thus, it is crucial that the temperature measurements taken over the next seven years be as accurate as possible. Steve McIntyre has everyone a favor (most of all AGW proponents) here in this regard.
15 August 2007 at 6:58 PM
Sorry! I meant to say: “it would be impossible to argue for anyone but the delusional to argue that AGW is false”
15 August 2007 at 7:10 PM
RE #3
Nick, you are referring to Ramanathan, V. et al (2007). That is not the conclusion and the study has no bearing on the 1940-70 period.
15 August 2007 at 7:13 PM
As Gavin has continued to point out, this study indeed means nothing. See the graph on global temperatures which will not change.
http://www.epa.gov/climatechange/images/triad-pg.gif
The U.S. map gives you some feel for how variable climate change can be regionally and locally over long periods of time
http://www.epa.gov/climatechange/images/tempanom.gif
15 August 2007 at 8:26 PM
#306
And what assurance do you have the global temperature will not change?
15 August 2007 at 9:33 PM
Re 288, Lynn says:”Well, see the scientists want to do as much as they can now with whatever evidence they have, before what actually happens during the 21st century has a chance to validate or invalidate their models. That’s because by 2100 there may not be any scientists left.”
This will be more than compensated for by the fact that there won’t be any economic forecasters left either. Theres a lot of angst about the future of civilization,as we approach the point of no return. Noboby wants to see our progeny as nomads roaming the the arctic on a subsistence level living. So in that light when our backs are to the wall,which they soon will be,the skeptics and their “uncertainty” arguments will be drowned out by reality.
BTW the source code for ModelE is readily available on line.See Gavin’s response in #211. He may have been playing cat and mouse for awhile and then concluded that some of us we’re too challenged to find it(like yours truly).
15 August 2007 at 9:43 PM
In this high polarized debate, I feel that everyone should remember that the vast majority do not deny that man has an impact on this planet. The question is how much, how soon. That is not irrational.
The label denier is a bit much. Skeptical is better. While the correction of the US historical data does not indicate cooling it does indicate warming without acceleration. That would be a good thing. While that may apply to only 2% of the earth’s surface, it is an indication that implementing new technology to reduce emissions and improve efficiency may be made before the tipping point with less financial impact.
If you are absolutely certain that all the data is 95% plus correct, unwilling to fine tune the data with other statistical methods, unwilling to bare peer review from less than cliquish peers, then the science is settled. Some are not that certain.
To the request for code, that should not be required unless someone has built their own model based on the provided algorithms and data and found significant differences. If the results cannot be replicated from the provided information by a competent source, then comparison of code may be in order. Is that unreasonable?
Since the majority of the science now boils down to statistical analysis, should not a variety of statistical methodology be used to validate the models?
I am not a climatologist, only studied engineering, the only thing I can offer is the KISS rule. If it is a statistical problem, get a statistician.
15 August 2007 at 9:52 PM
Bill, I think Chris means that you won’t be able to see the difference in the chart — before and after this particular correction being discussed here is made. The picture — at that scale — won’t change visibly. Chris, izzat what you meant?
15 August 2007 at 10:53 PM
Quick question.
1998 was originally calculated to be warmer than 1934, and was so until at least 2001. By this year, it was calculated to be warmer, which the recent correction reversed. Does anyone know when 1998 surpassed 1934 in the NASA calculations? I just read a story where it was said that 1998 had “long been believed” to be warmer than ‘34 but that’s a bit of an exaggeration.
15 August 2007 at 11:33 PM
Sorry that should be “1998 WASN’T originally calculated to be warmer than 1934 and WASN’T so until at least 2001″
16 August 2007 at 12:35 AM
#311. You are incorrect to say that “1998 was originally calculated to be warmer than 1934″ - I presume we’re talking U.S. here. 1934 was originally almost 0.6 deg C warmer than 1998 (Hansen et al 1999) and NASA 1999 news release. In the next two years, 1998 gained 0.6 deg C on 1934.
Contrary to one of Gavin’s posts, the time-of-observation bias adjustment was included in Hansen et al 1999.
[Response: Not so. Read the abstract of Hansen et al (2001): “Changes in the GISS analysis subsequent to the documentation by Hansen et al. [1999] are as follows: (1) incorporation of corrections for time-of-observation bias and station history adjustments in the United States based on Easterling et al. [1996a]”. - gavin]
The dramatic increase in 1998 relative to 1934 appears to originate in Karl’s “station history adjustment”, which was added to NASA calculations between 1999 and 2001, with dramatic results. [edit]
[Response: Also not true. The Plate 2 in Hansen et al (2001) clearly shows that the effect of the TOBS adjustment between the 1930s and 1990s is larger than that of the station history adjustment (both of which are significant however). - gavin]
16 August 2007 at 1:24 AM
re: #270
Well, if not weekly, let us know what happens sometime; most of the people who’ve argued about this have just seemed to disappear.
====
re: SETI@home, etc
1) Here’s a good list of such projects:
http://en.wikipedia.org/wiki/List_of_distributed_computing_projects, of which one is:
2) http://en.wikipedia.org/wiki/Climateprediction.net
3) Distributed PCs can work well for certain kinds of work, of which SETI@home, finding primes, etc are good examples. Note that 2) is an ensemble project: that is, each PC runs a completely separate simulation that will fit there, and people look at the ensemble results. [Of course, if someone really doubts a simulation code, running a bunch of instances on different machines won’t remove their doubt … in fact, it would increase the need to check each run. :-)]
Algorithms that work well this way usually have following characteristics:
A) There are huge number of independent tasks.
B) Ideally, as a machine becomes idle, it gets the next task (hopefully small), spends a lot of time computing, and then hands back a short answer [ideally, YES or NO], i.e., the work has a HIGH compute:communication ratio.
(A lot of particle physics people have used workstation/PC farms to look for interesting events (which hardly every happen). Each system takes an event, spends a lot of time looking for interesting patterns, and then returns YES or NO. SETI@home is similar to this, as are various other kinds of problems. Somewhat similar have been renderfarms at special-effects shops; I don’t know current times, but in olden days a system might run 2 hours to generate 1 frame of a movie.)
C) If you get a result back from a machine that you don’t own, whose software environment might be unknown, you can VERIFY interesting results easily. Hence, if some PC doing SETI@home says “YES”, you can easily check that by rerunning the algorithm. [If a PC somehow uses bad code and misses the little green men, you may not notice.] Of course, if you are dealing with PCs you don’t trust, you can at least send the same inputs to multiple machines and compare.
D) Unfortunately, none of this helps much to parallelize a single run of physics-based time-stepped gridded algorithms across multiple distributed PCs, especially with dense 3D grids.
I don’t think any of the projects in the above are of this type. If you know of a project that is (seriously) doing this, please post.
People can and do make serious CFD or FE codes work on Linux clusters, for example, but it certainly takes work, and people usually use fast Ethernet, Myrinet, IB, etc, and dedicated machines to minimize latency.
(more later, relatives visiting for a couple days).
16 August 2007 at 1:35 AM
#279 oh and “Ah, now the unobjective statement behind your question becomes obvious. “WE”, the voters do not see that at all. Check the various polls re: the need for action on GHGs.”
I’m sure every poll says there is a need for action on GHG, the same polls probably say people will vote for higher taxes to help the poor and would be happy to donate 10% of their income to charity. The reality, meanwhile, is that people actually vote with their feet.
eg. the first one off the top of google
http://www.oag.com/oag/website/com/OAG+Data/News/Press+Room/Press+Releases+2006/Global+Warming+fears+fail+to+dampen+demand+for+air+travel+0910064
16 August 2007 at 2:16 AM
I would imagine that the uncertainties on these numbers would mean that any reordering of the rankings would be statistically meaningless as you point out. What are the uncertainties on these numbers, because without them I’m not even sure how to interpret what I’m looking at? This also means that there shouldn’t have been any hoo-ha when 1998 and 2006 turned out to be the warmest years.
16 August 2007 at 8:29 AM
Gavin,
As you say, “The algorithms, issues and choices are outlined in excruciating detail in the relevant papers” and “The error on the global mean anomaly is estimated to be around 0.1 deg C now.”
However, to me the fact that no one caught a 0.15 deg C error in the US data for several years shows that the level of transparency has been inadequate to the goal of achieving accuracy. It would certainly be easier for organizations providing “corrected” data to include all the corrections in their database than for outside researchers to hunt through various articles and from them make guesses about exactly when and how various corrections have been made to each piece of data. Until this level of transparency has been achieved, I think it is reasonable for people to be skeptical of claims that any area’s temperature record is accurate to 0.1 deg C.
[Response: The error in the global mean anomaly is around 0.1 deg as I said. The fact that a revision 0.15 deg C in the US made no appreciable difference to the global mean implies that local area errors need to be much larger to have a significant impact. Note that no claim is made that individual stations data is correct to 0.1 deg C - the low error for the global mean comes from the power of large numbers and the large scale averaging that goes on. - gavin]
16 August 2007 at 8:49 AM
Yes Gavin, I understand that that the US is only 2% of the world’s area and so the 0.15 C in the US is trivial globally. However, no one noticed a 0.15 C error in the US data for several years, and I am skeptical that data in most of the rest of the world is under much better scrutiny.
16 August 2007 at 9:23 AM
Given the amount of noise in the data, it is not surprising that the 0.15 C error wasn’t detected immediately. Year to year variation in the USA’s mean temperature is far greater than 0.15 deg. So the error would likely not have been spotted by *anyone* until several subsequent years of data were collected and analyzed.
And given that skeptics have had full access to both the corrected and raw data for years, and given that there has been plenty of funding available for skeptics’ activities, the fact that this 0.15 boo-boo is all the skeptics have to show for their efforts is an indication that the data overall are pretty robust. If there *were* serious problems with the data, a couple of dedicated analysts with a few 10’s of K of funding from Exxon-Mobil would have been able to uncover them. Exxon and others would have been throwing money at research and analysis instead of paying to have puff-pieces published in the National Enq^H^H^HReview, WSJ, and other partisan publications.
16 August 2007 at 10:09 AM
Dallas Tisdale said: Since the majority of the science now boils down to statistical analysis, should not a variety of statistical methodology be used to validate the models?
The adjustments to historic temperature is not “the models.” Climate models are generally constructed from first principles of physics and chemistry. They are independent from historical data except that the skill of the model is judged, in part, by its ability to reproduce historical trends. Since there are several independent temperature series that all agree to the level of accuracy needed to evaluate models, there seem to be better places to spend the science budget.
16 August 2007 at 10:45 AM
re 309 -
Some thoughts from a non-scientist type who has been watching this unfold and come to some conclusions accordingly regarding labels such as denialism, skepticism and the so-called “debate”. I don’t comment on this site to often – I don’t have the technical background to hold my own, for one thing. I, like many people in my situation, have to rely upon what I can read in reports and science popularizations. Often, it comes down to observing little things like tactics, asking questions about motives, and understanding the basics of how science works across all fields. In that regard I am qualified to comment.
Simply put, this is a debate, yes, but only in the rhetorical sense, IMHO. As a scientific debate, I believe the facts supporting GW so far outweigh the arguments against GW that to call it a debate would be, at best, silly. There is just too much data, from far too many sources, to suggest that there is any legitimacy to the so-called AGW “Skeptics” position.
In fact, from all appearances, what is really occurring is what David Michaels, professor at the George Washington University School of Public Health, correctly characterized as manufactured uncertainty:
http://www.ajph.org/cgi/content/abstract/95/S1/S39
See also:
http://www.ucsusa.org/news/press_release/ExxonMobil-GlobalWarming-tobacco.html
http://thepumphandle.wordpress.com/2007/01/11/exxonmobil-says-it-will-stop-manufacturing-uncertainty-%E2%80%93-who-is-next/
http://www.motherjones.com/news/feature/2005/05/some_like_it_hot.html
Peer review is another indicator. If there is a real scientific debate, it would be there where I expect to see it in play. But this does not seem to be the case:
http://www.sciencemag.org/cgi/content/full/306/5702/1686
It also helps to understand the expertise of the people involved. Paralleling the so-called Evolution-Creationism debate, the majority of these folks criticizing GW science have no background in the science they seek to criticize, do no research of substance into the phenomena but instead engage in often spurious critiques of the science, much like Intelligent Design’s Discovery Institute, which does no research, just publishes books and op-ed pieces while funding efforts to undermine school policy in places like Kansas and Pennsylvania. Even from a casual perspective, this has to raise questions as to the legitimacy of the critiques they offer of the people who actually studied and work within the field of climatology.
A great dissection at how far apart the methodology of the so-called “skeptics” is from that of the climatologists can be found here:
http://www.realclimate.org/index.php?p=74
While on the surface this link it is a discussion of sci-fi author Michael Crichton’s cherry-picking of data to support his work of fiction, it is also an excellent snapshot of the types of arguments and tactics employed by the denialists over the past few years. Far too often, we see denialists take bits and pieces of data out of context and attack the resulting straw man they create (like the attempt to discredit Mann et al’s Hockey Stick), creating an illusion of a debate, of doubt far in excess of the actual uncertainty, that is at best disingenuous and, if I may be so bold, possibly criminal, particularly if the long-term effects of AGW end up being as dire as the middle-of-the road projections suggest.
In light of this, to suggest that the GW “Skeptic” crowd is truly Skeptical in the sense of how skepticism is employed in science is, at best, quite a stretch:
http://www.skeptic.com/about_us/discover_skepticism.html
Or are you seriously willing to suggest that people like Crichton or the makers of “The Great Global Warming Swindle” …
http://www.realclimate.org/index.php/archives/2007/03/swindled/
…represent a skeptical approach to the issue? To suggest that this current issue which, more and more, appears to be the tempest-in-a-teapot the lead article makes it out to be, somehow refutes global warming (as many of the folks Denialist camp appear to want people to believe)?
I could go on, but I think you get where I’m coming from. It’s not any one thing that causes problems for the AGW “Skeptic” crowd, but instead an overwhelmingly obvious pattern of tactics and less-than-forthright behaviors that cause problems for them with anyone who takes the time to watch and research their methodology in action over time. They are not doing any science of significance; they are not offering anything new to give the science something to chew on. Instead, they are sewing uncertainty, often playing fast and loose with the facts. People like Steve McIntyre can keep playing this game, doing their part in enabling behavior in the general population classically referred to as ‘Fiddling While Rome Burns’, and they will likely succeed in the continual seeding of doubt, at least for a time.
But whatever it is you think they are doing, it is apparently not about engaging in legitimate skepticism.
16 August 2007 at 11:06 AM
DWPittelli (#318) wrote:
If you just think in terms of the normal distribution, assuming thermometers could only measure with an accuracy of a degree, the larger the number of thermometers, the more accurate the average, and a large enough number of thermometers could make the uncertainty regarding the average temperature arbitrarily small where the uncertainty of the average is proportional to the inverse of the square root of the number of thermometers.
Statistics 101.
I assume that those who have actively participated in this debate for a while will be aware of this.
Others might want to check out the following for a look at this from a refreshing perspective…
The Power of Large Numbers
July 5th, 2007
http://tamino.wordpress.com/2007/07/05/the-power-of-large-numbers
PS
I might like to point out that this is a special case of the principle that a conclusion which recieves justification from multiple independent lines of investigation is often capable of a degree of justification far greater than that which it would receive from any one line of investigation considered in isolation. The evidence from surface stations, satellite measurements of troposphere, sea surface temperatures, etc all add up, and much more quickly than someone just off the street might think.
16 August 2007 at 11:26 AM
Incidentally, while I most certainly don’t mean to suggest that it is common, no doubt there are some who believe or at least suspect that climatologists are deliberately manipulating the numbers in order to make it appear as if temperatures are going up or going up more rapidly than they actually are.
Time to retire that belief. As the result of the recalculation, 1998 went down, not up.
Additionally, for those who believe that climatologists simply aren’t concerned with accuracy, it is worth keeping in mind that the recalculation was part of a deliberate and systematic attempt to improve accuracy. It succeeded.
And looking at the chart for the global average temperature trend, there really isn’t room for much doubt with regard to the direction of the trend, or for that matter, the near magnitude of the trend.
16 August 2007 at 11:35 AM
[[ Silly dim old skeptics can’t quite believe how computer models are able to model something as complex as the weather. Out to 100 years in the future.]]
You have weather confused with climate. Weather is chaotic and can’t be predicted beyond about five days. Climate is a long-term regional average (formally, 30 years or more), and is deterministic. An example to distinguish the two: I don’t know what the temperature will be tomorrow in Cairo, Egypt. But it’s a safe bet that it will be higher than in Stockholm, Sweden.
16 August 2007 at 11:38 AM
re: #281 FCH
I must run off, and I’ll come back to this, but FCH: think about why your analogy doesn’t fit very well.
- Open source is useful and cost-effective for some kinds of software.
- It isn’t for others, i.e., the return on the effort to to make it widely open, document it appropriately, respond to bug reports, etc isn’t worth it.
In particular, UNIX/Linux source:
- was/is used by programmers who typically want to use the resulting programs often, for real work, and as needed, make modifications to make them do additional work, or write related systems software
- provided a way to get a large mass of widely-useful software onto some hardware platform or other
- was/is of direct use to software engineers with the relevant expertise and motivation to make it work, without or without any help
- is structured as large collections of usually-small, relatively-independent modules, of which many have purposes and code easily accessible to a high school student with no special background. Anybody who can read and compile C can add a new flag to a lot of commands.
- UNIX: Ken & Dennis & Brian & co were happy to make source available for *lots* of things … but had they been told they had to make everything they ever did available, and that their code would be assessed for its quality, and that various random people would be suggesting changes they needed to consider … that would haver been the end of that. As it happens, over time, some people got to be considered as people who might actually have useful suggestions, but only because we proved over some years that we understood what was going on and were actually useful. Without naming names, some people were appalled at some of the “improvements” that were done elsewhere…
It is a labor of love for Linux and his lieutenants to do what they do.
Let us also reflect that GCC (a fine piece of software), in the last few years, has finally started to acquire global optimizations of the sort found in {MIPS, HP, IBM, etc} compilers 20 years ago. Likewise, things like XFS for Linux didn’t happen because a few people were looking at filesystem code and decided they’d like to play at doing better
Now, if there were a UNIX/Linux systems programmer-sized community of people whose daily work is climate-simulation&analysis, who have both the relevant scientific backgrounds and programming skills and motivation to knowledgably examine source code and help improve it, that would be nice.
[One of these years, I’ll have to do an essay that backtracks the 60-year history of open-source, and look at why it works well where it does, and not where it doesn’t … but not here :-)]
16 August 2007 at 11:49 AM
#324, the problem is that you can’t just average climates. There are just to many variables. The climate on a Caribean island is much too different than in the Canadian Rocky Mountains. Regional and microclimates are important as is variance, wind, humidity, clouds, etc etc.
16 August 2007 at 12:02 PM
Ref: 320
I understand. The historical data and proxy data may benefit from further statistical analysis. The data used to determine the skill of the models, sorry if validate was a poor word.
As far as the expense, given the urgency of the situation?
It is obvious that the Earth is warming and obvious that some portion of that warming is anthropogenic. I am confident that the business as usual mentality is changing, a change for the better. I just don’t share 95% confidence in the estimates of the rate of global warming.
16 August 2007 at 12:14 PM
Over at Number Watch they have a chart showing the ever increasing “Difference Between Raw and Final USHN Data Sets” as well as photos of temperature sensors near outdoor air-conditioning fans, etc. They claim the first graph shows the adjusting factors are partly responsible for the warming trends, and their second complaint is that temperature stations aren’t monitored and corrected well enough (or is the first the answer to the second?) Comments please, ty.
[Response: The adjustments to the US data are not responsible for global trends. Continental trends on every continent (except Antarctica) show similar patterns. The adjustments certainly do matter, and the largest one is related to changes in the time of day people in the US took measurements, other ones deal with station moves and instrument biases. Are they claiming that known problems shouldn’t be corrected for? - gavin]
16 August 2007 at 12:14 PM
Dave Blair (#326) wrote:
Actually you can - if you perform multiple runs with slightly different initial conditions. It gives you the spread - and it gets more accurate the higher the resolution.
With the NEC Earth Simulator from a few years back we were performing 32 trillion floating point calculations per second. Things have improved since then. Hadley is now using the different initial conditions from empirical observations over consecutive days and running their models with measurements from past years they have found surprising accuracy in terms of their projections over the near-term scale of a decade.
And that is with models grounded in physics, not some attempt to fit the model to the data. Climate models do the former, not the latter.
16 August 2007 at 12:20 PM
#316 & This also means that there shouldn’t have been any hoo-ha when 1998 and 2006 turned out to be the warmest years [in the U.S.].
I don’t think there’s been any hoo-ha over which year in the U.S. recorded the highest temps. Even in years when the entire world records the highest temp (which is more pertinent to the concept of “global” warming), there’s no hoo-ha on any channel I watch. (Until last week I didn’t have cable, so maybe the science channels mentioned it.)
The sad truth is that the well-oiled media just don’t mention global warming much at all (and they talk about solutions even less), with only a slight pick up after Katrina (or A.K.), followed by a sharp deceleration. The public brainwaves have been flatlined by the media on global warming.
The only thing that might grab media and public attention is very very severe global warming disasters, one upon another (like the equivalent impact of a Katrina happening every month … and assuming we are not at war at the time) … which means we will have already passed the runaway tipping point of no return by the time people take global warming seriously enough to start doing something about it.
I sincerely hope I’m wrong.
Which brings me to the best GW policy IMHO: Hope for the best, and expect (& try to avert) the worst.
16 August 2007 at 12:29 PM
Re: 313 “You are incorrect to say that ‘1998 was originally calculated to be warmer than 1934′”
That’s why I corrected it in the post immediately following.
Just to reiterate,
1998 (US) was originally calculated to be cooler than 1934.
This remained so until at least 2001.
By 2006, it was recalculated to be slightly warmer (that is, it had a “bigger number”)
Now, it is slightly cooler again.
Does anyone know when 1998 “surpassed” 1934, since skeptical sites seem to be making a big deal about how “long” 1998 was on top.
“In the next two years, 1998 gained 0.6 deg C on 1934.”
Certainly you mean 0.06 degrees.
[Response: Actually it’s larger than that - about 0.3 deg C. The difference that the TOBS adjustment and station history moves made in the US ranking was quite large. For the 1930’s to 1990’s there is about 0.2 deg C for TOBS bias correction, 0.1 deg C for station history adjustment, 0.02 for instrument changes, -0.03 for urban effects as applied to the USHCN raw data - all of this is described in the 2001 paper (see plate 6). McI’s claim of 0.6 is probably from a misreading of figure 6 in the 1999 paper which used the convention of Dec-Nov ‘years’ rather than the more normal Jan-Dec average. The appropriate comparison is Figure A2 (d) (1999) and Plate 6(c) (2001). - gavin]
16 August 2007 at 1:21 PM
You are reported in the Daily Telegraph as saying that global warming is “a global phenomena”. I hope the hack got it wrong and you in fact called it “a global phenomenon”, which it is BTW.
[Response: Possibly I can blame the phone connection…. - gavin]
16 August 2007 at 1:22 PM
#327 & I just don’t share 95% confidence in the estimates of the rate of global warming.
It does seem that the rate is fairly slow in lay (though perhaps not geological) terms. And we seem to be only at the beginning of seeing the GW trend come out of the noise. I think the first studies to reach .05 on AGW was in 1995. So it may be too early to tell how fast the warming might speed up, or stay steady, or decelerate. That’s why they have a big range in projected scenarios (involving variations in sensitivity and GHG emissions).
My thinking has been that scientists tend to underestimate the problem, since they can only work with quantifiables. There are many factors that are thought to have impact, but are not (easily) quantifiable. Like the melting of the ice sheets & mechanics of their disintegration.
Now using my kitchen physics, when I defrost my Sunfrost frig (www.sunfrost.com) everything looks very stable, then after a long time lag (kitchen, not geological, time), the top ice sheet just breaks off KA-BOOM. Something like catastrophe theory in mathematics (which I know very little about) might be more applicable to cryosphere dynamics than linear algebra.
Then, of course, when the world’s ice and snow start vanishing en mass, that leaves dark land and sea to absorb more heat.
Other factors include a warming world releasing nature’s stores of GHGs — as in ocean hydrates and permafrost. Again, I guess it’s hard to quantify what these rates will be. But I imagine there will be threshhold points for this — the point at which ice melts, say, at various levels of the underground or sea (tho sea currents impact this too) — and recent studies find some ocean methane hydrates at shallower levels than previously thought, and other studies indicate stored permafrost carbon going a lot deeper than previously thought. Undersea landslides are also a factor.
And then at a certain point of land/vegetation desication (warming causes more WV to be held in the atmosphere, taking it out of the land & plants), and fiercer wind storms, we can expect greater forest and brush fires. Those not only reduce the CO2 sequestering plant-life, but also release CO2 into the atmosphere. And again, I imagine that such events would be hard to quantify, though I’m sure scientists are busy working on that.
There’s just a whole lot left out of the codes and equations and calculations that might not only indicate increases in the warming, but also accelerations of it — like some wild domino effect (I remember the nuclear fission demonstration as a kid with mousetraps and ping-pong balls).
I recenlty read on ClimateArk.org that British scientists have predicted the temps will level off for 2 years, then after 2009 rise sharply, and all hell is about to break loose. See:
http://www.climateark.org/shared/reader/welcome.aspx?linkid=81736 and
http://www.climateark.org/shared/reader/welcome.aspx?linkid=81891
But, again, does their “code” contain all these hard-to-quantify factors? Are they being too scientifically cautious with “fudge factors” if at all they included them. Maybe it’ll even be worse than they suggest, but I sure HOPE (Help Our Planet Earth) that they are wrong and we luck out.
16 August 2007 at 2:25 PM
Gavin said:
Even if there is no indication of anomalous data in the rural station?
[Response: The GISS analysis is done to get the best regional trend, not the most exact local trend. Once again, read the reference: “the smoothing introduced in homogeneity-adjusted data may make the result less appropriate than the unadjusted data for local studies”. If you want to know exactly what happened at one station, look at that one station. If you want to know what happened regionally, you are better off averaging over different stations in the same region. The GISS adjusted data is *not* the best estimate of what happens at a locality, but the raw material for the regional maps. Think of an example of a region with two rural stations - one has a trend of 0.15 deg/dec, the other has a trend of 0.25 deg/dec - the regional average is 0.2 deg/dec, and in the GISS processing, both stations will have the trend set to 0.2 deg/dec prior to the gridding for the maps. Different adjustments for different purposes. - gavin]
16 August 2007 at 4:00 PM
Gavin,
In reading Hansen 2001 I Came across this:
“The strong cooling that exists in the unlit station data in the northern California region is not found in either
the periurban or urban stations either with or without any of the adjustments. Ocean temperature data for the same
period, illustrated below, has strong warming along the entire West Coast of the United States. This suggests the
possibility of a flaw in the unlit station data for that small region. After examination of all of the stations in this
region, five of the USHCN station records were altered in the GISS analysis because of inhomogeneities with
neighboring stations (data prior to 1927 for Lake Spaulding, data prior to 1929 for Orleans, data prior to 1911 for
Electra Ph, data prior of 1906 for Willows 6W, and all data for Crater Lake NPS HQ were omitted)”
Now, the elimination of this data on the presumption of Contamination, doesn’t lead to much of a change. That is not my issue. My issue is the documentation of the process. Since the “other sites” in the region are not listed I can’t really duplicate the analysis.
But, I tried. So I looked at Lake spaulding and sites within 70KM or so. I had no idea what kind was used around each site to do the checks. Anyway If you compare Lake Spaulding with Tahoe City, Colfax, Nevada city you can see that Lake spaulding has a cooling trend from 1914 -1931 ( not 1927) that differs from these other stations. Then Lake spaulding starts correlating with nearby stations in a more regular fashion. The problem is I’m curious about what objective statistical method was used to judge Homogeniety? Text doesnt say. Related question.
When you ingest data from USHCN, do you just ingest
Monthly Means? or do pull in Daily detail data? ( tmax, tmin)
Second I looked at Crater Lake NPS HQ( which is a very cold place relative to its surrounding sites ) Which sites did Hansen 2001 compare Crater Lake to? The two I glanced at showed no trend differences with Crater Lake, They were warmer than crater Lake station.. In fact the difference in bias ( there was no trend difference)
between Crater Lake and the other Station was fully accounted for by altitude differences ( using Hansens 6C per KM). Crater Lake just happens to be an isolated snowy cold place, But since warming trend is what matters ( hey the artic is cold) I was wondering if I could get a couple of pointers before I plunge into Crater Lake in any depth.
Can I get a pointer to the EXACT test used for homogeneity and a pointer to the exact list of stations that Crater Lake NPS HQ was compared to ( or a radius from its location)
16 August 2007 at 4:17 PM
#329, Timothy, weather prediction is also based on physics and have some large computing power running their models too. The prediction for weather this afternoon will probably be pretty accurate, but 2 - 3 weeks from now? Same for climates predictions, you say they are accurate for the next decade, maybe so and time will tell, but when we hear the predictions for 50 or 100 years from now that when you wonder why all the movie stars and politicians are getting involved.
16 August 2007 at 4:21 PM
Two questions. Beginner´s questions, actually:
- Is there any chance of mankind quit using fossil fuels before the planet runs out of it? (Unlikely in my view, but I would like to hear more educated guesses)
- What levels of CO2 would we reach if all that buried carbon is released to the atmosphere? (and what would that mean in terms of GW?)
16 August 2007 at 4:40 PM
#335
Well the obvious question here is: Have you sent an email to the authors of the paper you discuss?
Unless they are retired or dead the authors of a paper is normally best persons to ask things. Unless you have asked them you will rapily become suspected of prefering to make a lot of noise rather than actually getting answers.
16 August 2007 at 4:55 PM
In another argument (sci.physics.foundations), comment was supplied on http://www.numberwatch.co.uk/manmade.htm
What I found interesting here was the website was showing substantially different graph from global satellite temperature than the RSS, UAH curves. Can anyone enlighten me on where their graph may have come from?
Also in that context, http://www.dailytech.com/Blogger+Finds+Y2K+Bug+in+NASA+Climate+Data/article8383.htm
claims “Hansen refused to provide McKintyre with the algorithm used to generate graph data, so McKintyre reverse-engineered it. The result appeared to be a Y2K bug in the handling of the raw data.”
Now given the algorithms seems to be published, I find the claim weird. Anyone know the back story here??
16 August 2007 at 4:56 PM
Dave Blair, your belief that studies of weather and climate are the same is one that’s frequently asserted; it’s been answered elsewhere repeatedly; response would be off-topic here.
You could find that clarified in the basic info at
http://www.realclimate.org/index.php/archives/2007/05/start-here/
16 August 2007 at 5:05 PM
[[#324, the problem is that you can’t just average climates. There are just to many variables. The climate on a Caribean island is much too different than in the Canadian Rocky Mountains. Regional and microclimates are important as is variance, wind, humidity, clouds, etc etc.]]
Hit the wrong button. Apologies if this message shows up twice.
The things you cite — wind, humidity, clouds — certainly affect the local temperature. But the temperature itself always measures the same thing — the heat content of the surface or the low atmosphere. It’s perfectly valid to average the temperatures of different regions, since it’s the same sort of thing being measured.
16 August 2007 at 5:12 PM
[[ What levels of CO2 would we reach if all that buried carbon is released to the atmosphere? (and what would that mean in terms of GW?)]]
We have at least enough coal and oil to quadruple the CO2 in the atmosphere, which would most likely raise global temperature about 5 K. We’d probably lose the polar ice caps; summer at each pole would always be enough to melt all the ice. A lot of the present coastal cities would be under water, including New York, Miami, Houston. The entire country of Bangladesh would be submerged.
16 August 2007 at 5:26 PM
RE Steven Mosher (#335)
As we have both noticed, Gavin tries to keep the door open for everyone. But I would like it if we could keep this polite.
I believe it would be in everyone’s interest.
Of course I realize that you are making an effort in this way at present. But in light of whats happened before, I think you can understand and perhaps even share my concern.
16 August 2007 at 5:27 PM
If denialists, instead of giving climate scientists nonsense arguments, would go outside and plant some trees, perhaps we will survive a few more centuries.
http://environment.newscientist.com/article/dn12496-forget-biofuels–burn-oil-and-plant-forests-instead.html
16 August 2007 at 6:07 PM
Phil, look up the ‘Global Warming Swindle’ stuff, I think the first graph may be from that program; it’s being posted about on a lot of skeptic discussions but with no attribution that I can find for it.
16 August 2007 at 6:09 PM
Because a) the raw data are publicly available and b) papers are supposed to contain enough detail to allow others to repeat the analysis. If a paper says ‘we then add A and B’, you don’t need code that has “C=A+B”. - gavin
And this is why so many serious scientists have trouble dealing with the public. This may be true and it’s probably even ideal in the academic and scientific world. But a refusal to release the code in the political arena is suicide. You went through this with Mann’s code so why repeat the same mess? Are those lessons so easily forgotten? Heck the refusal to release the code was discussed in the halls of Congress almost ten years after the fact!!!! Is it really so easy to forget? The vast majority of people are going to assume you are hiding something when you refuse to release the code. A large portion of the legitimate skeptics will view the paper as an attempt to confuse and stall those who are trying to replicate your work. If you have a legitimate reason to release the code then say what it is. Squeezing extra papers from software is a legitimate reason. Release part of the code if you have to. Refusing to release the code simply because you feel like keeping it secret is political suicide.
16 August 2007 at 6:46 PM
sparrow’s points explain why so much money has been spent to encourage denialists: more pressure on climate scientists, lessened probability that real steps might be taken to actually do something about global warming, steps that will in most likelihood, cost somebody money. We have denialists because their existence helps save somebody money.
16 August 2007 at 6:49 PM
Phil, that “28 years” graph is found in Powerpoint file written by a David Archibald for the Lavoisier Group - slide #1, but he gives no source for it or much else. Without a source, it’s just argument.
16 August 2007 at 7:09 PM
The skeptics are troublesome for GISS and climate scientists everywhere.
Solution: Transparency.
It is as simple as that.
[Response: We publish hundreds of papers a year from GISS alone. We have more data, code and model output online than any comparable institution, we have a number of public scientists who comment on the science and the problems to most people and institutions who care to ask. And yet, the demand is always for more transparency. This is not a demand that will ever be satisfied since there will always be more done by the scientists than ever makes it into papers or products. My comments above stand - independent replication from published descriptions - the algorithms in English, rather than code - are more valuable to everyone concerned than dumps of impenetrable and undocumented code. - gavin]
16 August 2007 at 7:32 PM
Justin, I think the two go together (#294)), the method and the calculations to implement the method’s goals. The code purports to do something. If you have the specific code used, you can analyze it and ensure that it accomplishes what it’s supposed to be doing. Um, if I write a program to generate random numbers, I can run that program over a period of time and validate they are random. I am trying not to go into anything OT here…
Thanks for the idea! I can see why some would have problems with those issues, Timothy (#297) But let me cover those issues from what I see happening. 1. I don’t know enough about it to know if anyone did or didn’t do anything to Dr. Mann’s graphs. But the point of that is you want an outsider, who is an expert at a discipline, to examine the scientific validity of something from they’re angle. How good he is at it or not, like I said not my area and a different issue anyway. 2. He’s not trying to do anything with the stations. Photographic documentation is part of the standards, and it lets you go relook and see how things are later without having to go back. Plus some aren’t sited well. 3. That may be. I think we have to wait and see once every one of them is looked and so everything can be analyzed. I am only interested in having the best data available. No matter which way it goes. 4,5 I don’t get that out of it.
Let’s see. It’s not his project, he doesn’t have control over who goes to what site where. Sure, perhaps some of it is a little over-enthusiastic, but maybe just because it’s interesting and exciting . Sure, instead of “good” it’s maybe better to use “meeting siting standards and minimizing adjustments” and not “bad” but “not meeting siting standards and needs a lot of adjustments” would be more neutral, but it gets tedious to read that. However, if 7 people get 7 sites and 6 are “bad” and 1 is “good” and he compares them to see what effect that may have, I don’t consider that “cherry picking” it’s just what’s there. I can’t fault anyone for focusing on the one issue of the sites and possible effects and not bringing in glaciers, sea ice, ocean temperatures, carbon dioxide and methane levels, albedo and the rest into every discussion and aspect of this.
But Pete (#298) some of this all assumes that solar, wind, hydrogen, hydro, nuclear, clean coal, and the like won’t become more viable and less expensive in the future, that no new oil discoveries of any import will ever happen, and that at some time the oil in shale won’t become at the same relative expense as that in the ground. Who knows, perhaps those with untapped liquid oil are buying it from others to save their own? I think there’s a reason it’s called “black gold”.
Now you’ve lost me Timothy (#299). Why wouldn’t an oil company (or any industry) fund endeavors to protect its interests? Governments and schools do it all the time. If I’m doing experiments that could turn out to be helpful to some entity, why wouldn’t they fund me? They’re going to fund somebody, even if it’s those trying to find new sources of oil, doing R&D into related fields (solar panels for example), and so on. I don’t know about you, but I like having gasoline, electricity, available food and the like. It’s just economics. Think about this: When the government of the United States making money off of every bit of gasoline sold in the entire country starts complaining about oil company profits (which they benefit from every single one of,) why are their motivations any less suspect than that of say Chevron/Texaco?
You are misunderstanding me I believe, from what you said in (#303) John. I’m not making any comment on the validity, scope, or importance of the trend from year x to year y or how somebody interprets it. Or the validity, scope, or importance of the measurements themselves or how somebody interprets it. I’m just saying if you go year to year you can say one thing, if you pick specific periods or specific lengths of time, you can say other things. I totally agree that we need to know as much as possible exactly what’s going on, and as accurately as possible. That the trend of the measurements shows we are warming over time is not really up for debate, anyone can chart it at NOAA.
Speak for yourself Lawrence (#308) I certainly want to see our progeny as nomads roaming the arctic on a subsistence level…. But that’s beside the point, it’s always been easy to get ModelE I believe.
Well cce it’s like this (#311) The difference is not great enough for it to matter between the two years, nor the effect upon the global numbers even if it was. But the point is that finding and correcting errors is the goal of many people, regardless of what “they’re trying to do” or others think “they’re trying to do”. On “either side”.
Jake, it’s all about the trend dude. (#316)
I don’t think this is the same conversation going on in (#317), since I agree with both DWPitelli’s comment and Gavin’s response.
You’ve lost me also caerbannog (#319). Your first paragraph is fine, the second is just, just, just… Not very helpful to supporting the arguement in the first.
J.S. McIntyre (#319). I see it all the time from everyone. Nobody has a lock on the rhetoric about this subject.
In an issue such as this, there’s going to be a very very wide range of people that believe a whole lot of things(#323). Matters of opinion are always such.
(#324), I don’t think that’s a very good analogy Barton. “Climate” has nothing to do with saying “It will be cold in the Arctic and Antarctic this December” or “Water boils at 100C”. (Comparing Egypt and Sweden on the same day.) Nah, I don’t have a better one. But your comment in (#341) bears a bit of explanation; “the temperature” itself always measures the same thing; the temperature of the area you are measuring in the material you are measuring it with the device you’re using to measure it.
I think that rather than how you phrased it Dallas (#327), it is obvious the anomaly is trending up, and if the measurements are stable as to the anomaly, then we are warmer “now” than we were “then”.
Alexandre in #337 most of those two questions are answered in my posts I think. The short answer suddenly is…. Nobody knows the answer to either. But we can make projections, some of which will be true and others not, to various levels of accuracy and margin of error. That’s what everyone is discussing all the time. All that you can get is the opinions of those who have looked into various aspects of this.
16 August 2007 at 7:40 PM
Re #335: [Anyway If you compare Lake Spaulding with Tahoe City, Colfax, Nevada city you can see that Lake spaulding has a cooling trend from 1914 -1931 ( not 1927) that differs from these other stations.]
I happen to live in the area, and thought I remembered something about this lake. Sure enough, the first hit from Google gives me this:
“Lake Spaulding rests at an elevation of 5,014 feet in a glacier carved bowl of granite. The lake has a surface area of 698 acres surrounded by giant rocks and a thick pine forest. The lake was originally built for hydraulic mining in 1912…”
OK, so people built a dam, and changed a square mile or so of rocky river valley into a lake. Water has a lot more thermal inertia than rock, and at those middle elevations it would probably get a good thickness of ice accumulating every winter. Wouldn’t you expect a cooling trend in nearby temperatures, quite independently of any larger trends?
16 August 2007 at 8:11 PM
Re 321: Amen.
Nobody ever erected a statue to a critic.
16 August 2007 at 8:39 PM
RE 349
Gavin. [edit]
Its two pages of code… At least tell us which method you used for In homogeniety testing?
Easterling? Kohler? Salinger? Zhang? SNHT? Berry?
Vincent? There are bunches. Which was used?
[Response: RTFR. GISS only adds the urbanisation adjustment and does not do any homogeneity testing, the station history adjustments are taken from USHCN as is clearly described. The only further culls were for obvious weirdness such in the early Cal. data as was, again, clearly described. What is so hard about reading the papers? - gavin]
16 August 2007 at 8:41 PM
Re: Steven Mosher
Steve,
One thing.
I don’t ever really harbor any hard feelings towards anyone. And assuming you feel the same way and can make it out to Seattle some time, I would be willing to spring for a couple of drinks.
Strictly non-alcoholic. Hopefully tea or coffee would be alright.
Here’s my email address:
timothychase at g mail.com (No spaces, well you can figure that out.)
Honestly - feel free to take me up on this.
16 August 2007 at 8:44 PM
RE 322 and others: I have repeatedly seen the assertation that if you have large enough numbers errors in data will be statisically insignificant. However, lets say I am conducting an experiment in the biology lab measuring how the production of an enzyme by bacteria in petri dishes is effected by different nutrients. And some of those dishes were contaminated by fungus tha competed for those nutrients. If only a few were effected that might not change the results. But if 50% or 30% or even 20% were contaminated no matter how many dishes I had I would always get incorrect results. One of the ways to protect against this is to visually examine the dishes (including with a microscope) to look for contaminants. At some point it is important to come out from behind the computer screen and check where the data is coming from, how it is collected and look for possible errors in tabulation/computation.
A very recent example of finding that the real world does not always fit the computer model is: http://www.nasa-news.org/documents/pdf/Wentz_How_Much_More.pdf
16 August 2007 at 8:59 PM
Fix the bad data.
From NOAA:
In summary, climatic normals are intended to serve as a baseline for both spatial and temporal comparison. They are values derived from the frequencies of identically distributed observations over a 30-year period of record. At most locations, however, non-climatic
influences such as station relocations, siting changes, instrument changes and recalibrations, etc. preclude obtaining a climatically homogeneous record of daily observations for 30 years. The statistical problem of detecting the full range of these inhomogeneities from the observational record is currently intractable.
16 August 2007 at 9:56 PM
You don’t “fix” data — nobody’s going to go back through history and change what’s recorded.
You replicate — with better instruments. That’s what’s being done, rolling out new stations with better gear and more consistent criteria.
You run in parallel for a while. That gives you a parallel record of the old instrument sites and the new ones.
Then you can evaluate the old data because you’ve been able to check each of the old instrument setups running in parallel with the new instruments.
This is, perhaps, exactly what the know-nothings want to avoid by insisting the old data be fiddled or discarded.
Because a consistently biased observer is still a reliable observer —- and the long record made by a consistently biased observer becomes more valuable once you’ve run the old observer and the new observer in parallel.
Get a grip, folks, the new instruments going out set up with the new criteria are going to make all the old data _more_ useful.
Just as it is.
Without throwing it out. Without fiddling with it.
16 August 2007 at 11:14 PM
Timothy Chase Says:
11 August 2007 at 4:03 AM
I had written (#19):
Earth will feel the heat from 2009: climate boffins
By Lucy Sherriff
10th August 2007 15:31 GMT
http://www.theregister.co.uk/2007/08/10/climate_model/
It appears that climatologists are in the process of improving their short-range forecasting.
papertiger (#29) responded:
Interesting. It seems to be inproved just enough to cover the time period right after the next election.
You sure this is a non political website?
Different country - Hadley out of England - notice the UK in the website address. But I suppose they could be part of a global conspiracy. You have to watch those conspiratorial types pretty darn closely - particularly the scientists when they start getting uppety….
So you think that climate models can predict weather? You endorced their site as if it were authoritative. That same office predicted an unremarkable 2007 summer for the UK. As we have seen the UK had an exceptionally wet summer, well below average temperatures, and widespread floods.
Why do you endorce this model, if not due to politics?
16 August 2007 at 11:43 PM
Come on, that’s a link to The Register.
Good grief, if you don’t read the actual science paper, at least read the comments of someone who has:
“… So I read the actual paper containing the new predictions. It turns out that the press reports are considerably overblown (surprise!); they give the unmistakeable impression that the HadCRU team has made definitive predictions of the future progress of global warming over the next decade or so, as though we now know with confidence how global average temperature will evolve up to 2014. If you read the actual paper you’ll find that is simply not so…. ”
http://tamino.wordpress.com/2007/08/14/impure-speculation/
16 August 2007 at 11:47 PM
Gavin said:
Why not just say the regional average is 0.2 deg/dec and leave the individual station data alone?
Anyway, I thought the purpose of the GISS adjustments is to correct any urban heating, not just to homogenize all stations.
[Response: The GISS-adjusted record is simply the last step in the process before you grid the data, it is not a claim that this is what was the most accurate history for that station. The purpose is to provide a regional and global analysis that is based on rural trends. - gavin]
17 August 2007 at 12:04 AM
Paper indeed sir! Humans are pattern-seeking creatures. Scientists deal in substantiated trends. See the difference between this and your fallacy? If not, review and report back.
17 August 2007 at 1:23 AM
The graph of global mean temperatures has disappeared fron Gavin’s text. Let’s hope it will soon re-appear with the y-axis similar to that in the US graph. This is basic stuff in statistical graphics: if you compare two time series, you visualize them in similar coordinates. And no values off chart, please.
It’s not that GISS does not know how to visualize their global data. There is for example one graph, showing that global warming more or less stopped about six years ago:
http://data.giss.nasa.gov/gistemp/graphs/Fig.C_lrg.gif
17 August 2007 at 4:03 AM
Re in-line comment to #349 [My comments above stand - independent replication from published descriptions - the algorithms in English, rather than code - are more valuable to everyone concerned than dumps of impenetrable and undocumented code. - gavin]
But Gavin (and you know I’m no denialist, nor do I think this matter undermines the immense weight of evidence from AGW from numerous independent lines of research), code should not be impenetrable and undocumented. If it is, that’s a flaw in the work. Whether it’s a big flaw or a little flaw depends on context, but it’s a flaw.
17 August 2007 at 4:35 AM
#359 - Errr… it looks more like the anomaly had a large peak in 1998 and has then returned to a rising trend. Also helpful if you show data before 1997.
17 August 2007 at 8:01 AM
The demands for transparency and the cackling over a teensy adjustment in temps don’t reflect well on a group that allowed an order of magnitude change in a y-axis scale to back up one of their talking points. And that’s simply as a question of form and rhetoric. Counter science with science, folks, not with Death by Quibble.
17 August 2007 at 8:40 AM
“So you think that climate models can predict weather? You endorced their site as if it were authoritative. That same office predicted an unremarkable 2007 summer for the UK. As we have seen the UK had an exceptionally wet summer, well below average temperatures, and widespread floods.”
Actually, despite the fact that the summer isn’t yet over, they predicted the temperature to be about average - and it has been so far. They also predicted the northern part of the UK to tend towards wetter than average. The southern part was predicted to tend towards drier and that’s where the main error on the part of the forecast has been. The UKMO state that the seasonal forecast is experimental and has a success rate of roughly 60%.
However, that is using a different forecasting model (it’s based on NAO predictions and SSTs partly using statistical techniques), and by a completely different group (the seasonal forecasts are carried out by the UKMO whereas the climate research is done by the Hadley Centre which is a semi-autonomous division - that might seem a trivial distinction but it’s not). The DePreSys model uses the HADCM3 model but with improved data initialisation. It is more akin to running the current UKMO GM for ten years than their seasonal forecast.
That said, the project has been running for over four years and is based in the UK, so any link with US elections is rather fanciful and a bit silly.
Finally, their paper is really a marker for a work in progress. It’s possibly the first time that anyone has done a climate prediction of this type. They acknowledge the limitations of the DePreSys forecast and I’d expect the forecast to change with newer runs.
17 August 2007 at 8:47 AM
[ gavin : My comments above stand - independent replication from published descriptions - the algorithms in English, rather than code - are more valuable to everyone concerned than dumps of impenetrable and undocumented code. ]
Huh? I’m a computer programmer and the first thing I skip when tackling a problem is the docs and go straight to the code as that’s where the skeletons lie. What you’re advocating is like Ford telling a jury that they shouldn’t look at the blueprints of their pinto or at executive memos but rather at press releases.
This is akin to security by obscurity. It is the last thing that I would expect from scientists.
[Response: Again, you miss the point. It is better for someone to come up with the same answer by doing it independently - that validates both approaches. - gavin]
17 August 2007 at 8:48 AM
Skeptics leaping on the need for NASA to revise their figures have reached Sweden, where I work. Letters to the editor in regional newspapers (hd.se), no less. Under the heading “En Obekväm Nyhet” which means “An Uncomfortable Piece of News”, the letter writer neglected to mention that the new figures were for the US, and simply said that temperatures in 1934 were as warm as 1998, that this news (studiously avoided by the press because it’s so uncomfortable) would “disappoint” people with so much invested in the climate threat, and even that “the climate catastrophe has been cancelled”!
Needless to say, there’s a critical reply/correction on the way to the editor!
Keep up the good work RealClimate team.
P.
17 August 2007 at 9:20 AM
RE351.
The cooling TREND at Lake spaulding ( compared to nearby sites ) is nearly linear. That would indicate
smething like a sensor going bad. Post 1931 it matches the other sites in the area nicely. Statin history records might show replacement of the sensor.. Havent checked that yet.
17 August 2007 at 9:42 AM
RE 175. Gavin inlined:
”
Response: ‘Algorithms’ are just descriptions in various flavours of formality. If there is a question of whether the code correctly implements the algorithm it will be apparent if someone undertakes to encode the same algorithm and yet comes up with a different answer. ”
Well, If we both made the same error they would match.
If we used different math libraries and one had a flaw
they would mismatch for a different reasn.
If they didnt match, how would we resolve the mismatch?
By sharing each others code. Plus Nasa software policy
encourages you to share code.
Next:
“Absent that, there is no question mark over the code. So if I generically describe a step as ‘we than add A and B to get C’, there is only a point in checking the code if you independently add A and B and get D. That kind of replication is necessary and welcome. With that, all parties will benefit. ”
I would have to more than merely claim that the results
didn’t match, right? I suspect that you would want to
see my code. Can you imagine if 10 people tried to
match what you did and then all sent you their code to
check. that would be rather a time waste for you.
Next:
“Simple demands to see all and every piece of code involved in an analysis presumably in the hope that you’ll spot the line where it says ‘Fix data in line with our political preference’ are too expansive and unfocused to be useful science. - gavin”
I don’t expect to find comments like that. Fundamentally I believe in transparency. The default should be release the code, unless there is an IP issue.
[Response: Why talk about hypotheticals? Do your emulation and see. - gavin]
17 August 2007 at 10:09 AM
J.S. McIntyre (#319). I see it all the time from everyone. Nobody has a lock on the rhetoric about this subject.
==============
Actually, for the sake of reference, it was post 321.
No one said otherwise, and I find it interesting you would infer that was my sole point.
As I outlined in my remarks, there is a very large difference between what we see emerging from the people promoting the science of Global Warming and the so-called “AGW Skeptics”, far beyond just “rhetoric”.
17 August 2007 at 10:18 AM
re #367,
Do Climatologists do the programming themselves or do they give the algorithms to programmers or comp. sci.students(in the case of a University) to program?
[Response: Depends on the size of the group. The Hadley Centre and NCAR have specific programmers, GISS is a smaller institution and the scientists do most of the work themselves though they do get some help from GSFC programmers. - gavin]
17 August 2007 at 10:32 AM
Re 358 papertiger “the UK had an exceptionally wet summer, well below average temperatures” in fact temps for June 1.1 degrees above 1961 - 1990 average; temps for July 0.3 degrees below 1961 - 1990 average.
is this the same papertiger in ref 24:
Just to expand on my last point at 12, (somehow) many see this as indicating that we can’t trust anyone (especially Hansen) to handle the data properly.
I think the point is that we shouldn’t have to trust someone as in a single person or entity such as Nasa GISS to develope what is in effect policy for our country.
It’s un democratic and un scientific.
A person could make a mistake. Ahem.
So we can’t trust anyone handling data? I know who I would trust with attempting to be accurate and honest. Sadly typical that those who cast aspersions on the validity of this scientific work don’t seem to be able (or willing?) to treat facts with such respect as those they criticise.
17 August 2007 at 10:42 AM
> I would have to more than merely claim that the results didn’t match, right?
You’d have passed a peer review and gotten a publication in a science journal.
You’d have coauthors, whose track record and reputation would support your conclusion.
You’d be taken seriously and people would assume you were doing honest science, and look for flaws in your approach.
Why not try it, as Gavin and many others suggest? So far every group that _has_ done a climate model finds much the same result.
Once you’ve done it — you’ll understand how it’s done.
As Willy Ley supposedly said, analysis is all very well, but you can’t understand a locomotive by melting it down and analyzing the mess. You have to build one to understand how it works.
17 August 2007 at 10:59 AM
Gavin: While the magnitude of what happened is small, the dimensionality — errors by a respected agency — is bad for the agency. My town once had a school superintendent whose approach to criticism was to 1)ask the selectmen to form a study committee 2)appoint the critic as the chair 3)make the “full resources of the department” available for the study. The dynamics of this response are interesting. First, the critic cannot decline without losing credibility. Second, the school department still maintained most of the control of the study process. Third, it got to know what was being found and correct additional problems before they got out of hand. Further, the committees sometimes found solutions that would have required paid consultants, so they more than paid for themselves. Everyone was happy.
It is not possible for anyone to review all or even most of a GCM, so there will always be questions. You might find it useful to have a mechanism in place for semi-formal review of data, design, or algorithms by outsiders via the