Isn’t it amazing how climate science must reach out to other disciplines. You describe some fundamentals of communications research. Professional pollsters really can be helpful here. And you will quickly note that even poll organizations can be biased in obvious ways. We don’t need to be reminded that polling scientists will only let us know their opinions, it does not validate or dismiss the science.
Hearing that gubernator Arnold Schwarzenegger is planning a Climate change Summit in November. It might be nice to give him some results prior to that event.
One would think the creators of this survey would be aware of the dangers of ambiguous questions in a survey.
I suppose part of the difficulty comes from trying to address professionals in very different specialties.
A survey targeting scientists by specialty asking them to weight their agreement with interpretations of data with respect to AGW hypotheses could identify areas of uncertainty and flesh out areas of consensus. A well constructed suryvey could be very revealing otherwise it may well end up being a tool for deniers.
It is interesting to note that there is a recall effort to get rid of Schwarzenegger by the right-wing in California. This is a nation-wide phenomenon. The attempt to shut down debate and action appears to be agenda-driven, sadly. Worse, the flimsiness and vagueness of the survey, combined with its flawed set of recipients, makes it a poor instrument by which to make any conclusions about scientists’ opinions. It thus plays into the hands of those who consistently cherry-pick data and surveys like this to bolster their own otherwise poorly supported case. Notwithstanding that people are certainly free to interpret these things as they wish, it is incumbent upon active climate scientists and their peers in related disciplines to rebut bad reasoning and it is also incumbent that surveys of scientists be constructed in a much more robust and specific format. The longer we delay action by dithering back and forth on these matters, the worse the outcome for all of us, advocates and denialists alike.
Comment by Stephen Missal — 29 Sep 2008 @ 11:12 AM
If you don’t like their survey, why not try running your own? Design one for the recognized authorities, and a smaller one for regular commenters here.
Comment by Richard Sycamore — 29 Sep 2008 @ 11:24 AM
I would argue that such surveys are pointless because no single scientist is qualified to comment on all topics. For example, a biologist might be certain that the climate is warming because of evidence in the eco-systems studied by that biologist but that same biologist is not qualified to comment on why the climate is warming.
Similarily, an aerospace engineer with experience in numerical modelling would be qualified on comment on whether the climate models are likely to provide useful information but could not reasonably comment about the accuracy of paleoclimate studies.
Of course, this is a moot point because these surveys are simply propoganda tools and whoever sets the criteria for who is “qualified” to respond will choose the criteria that will produced the outcome desired.
Some of the questions seem quite pertinent to my recent postings, particularly 9 thru 19. It would be very, very interesting to know the responses to these questions, especially from those who work in a climate science discipline as their occupation.
I hope, when the results are published, the answers are categorized in such a way that we can weigh the responses against the background of the respondents.
Some of these answers may prove quite useful, if done correctly.
So have the respondent identify their areas of primary, secondary, tertiary expertise and weight their responses accordingly. Or better perhaps: beside each question have them rate their degree of confidence in their response.
Comment by Richard Sycamore — 29 Sep 2008 @ 12:22 PM
I wonder how usefull a survey of the public would be? Would the out come of either the survey of scientist or the public be useful in shaping the policy related to taking action on climate change? Would it make government more inclined to take action as they seem very reluctant just now?
I agree with Richard Sycamore and with the two separate surveys idea. Who better than the RealClimate people to construct these? And here are some suggestions from someone that does this kind of work regularly: The survey should be shortened, to increase the response rate. In my opinion, that would include eliminating many of the questions (45ff) concerning communication (some of which are leading, as you note). It would be interesting to see if there is a range of opinions among climate scientists on many of these questions, but beyond saying that climate scientists agree or disagree about this issue, what else do you do with those answers? I believe the survey would better reflect the actual state of knowledge of climate scientists (question 9-44) if separate “Unsure” and perhaps other similar responses (like “Cannot be known currently”) were added to the response rating categories. Otherwise, people who do not know or who feel that something cannot be known tend to fall in the middle (4) on the 7-point scale items. Or they leave the questions blank and you have no idea why. I would volunteer my time to help construct such a survey if wanted.
”Contrary to recent assessments based on theoretical models [IPCC, 2007] the anthropogenic warming estimated directly from the historical observations is more pronounced between 45°S and 50°N than at higher latitudes…”
“Climate models may therefore lack – or incorrectly parameterize – fundamental processes by which surface temperatures respond to radiative forcings.”
If these are the world’s leading climate experts, the least they can do is give answers in the short paragraph format.
Just to go over it again, there are three general areas of climate expertise: paleoclimate, climate modeling and real-time data collection.
These areas span the academic separation of Earth sciences: land, air, biosphere and water, or as divided up traditionally into geology, oceanography, ecology and meteorology.
Each of these areas is influenced by the even more traditional disciplines of mathematics, physics, chemistry and biology, to greater or lesser extents.
More recently, it has also become clear that the study of the “natural system” must incorporate the effects of human activities, i.e. inputs and outputs and land mass use – so human economic-industrial activity now must be included as part of the “natural system.”
Specific problems in climate involve all of these issues. One way to organize things more neatly is to look at things in terms of elemental and molecular cycles: the atmospheric circulation, the ocean circulation, the carbon cycle, the nitrogen cycle, the water cycle, the sediment cycle, the ocean-crust-recycling cycle and so on.
This kind of scientific organization requires a good deal of interdisciplinary knowledge, but also allows for division of labor and specialization.
Take a specific problem like ocean anoxia. A warming ocean could be a more stagnant ocean with less circulation, due to physical and chemical issues. Add a lot of nitrogen (via human industry and agriculture) and the oxygen is depleted even more, thanks to biological activity. There you have physical forcing, aqueous chemistry, agricultural economics, and the biological nitrogen cycle all interacting, and the result is increasingly anoxic ocean conditions, which are sure to have other negative repercussions.
Given these levels of complexity, it’s better to conduct literature surveys and see what scientists have actually been saying in print. That’s what the IPCC was intended to do – conduct a comprehensive survey of climate scientists and get a written report from them.
The most succinct summation of that synthesis survey, from the last two pages of the report: “6: Robust Findings and Key Uncertainties”
A comparison between two recent national surveys among nanoscientists and the general public in the US shows that, in general, nanoscientists are more optimistic than the public about the potential benefits of nanotechnology. However, for some issues related to the environmental and long-term health impacts of nanotechnology, nanoscientists were significantly more concerned than the public.
Public Understanding of Science 2008, doi:10.1177/0963662507079373
In recent decades, corporate and special interests have developed a wide repertoire of methods to manufacture doubt about science that threatens their interests. In the case presented here, a trade association issued a rich assortment of rhetorical claims intended to sow public confusion about university studies that threatened to undermine its industry’s activities. Journalists’ use of these claims appeared to vary largely as a function of their perceptions of their journalistic roles and of their audiences, though their knowledge of science also appeared to play a role. Our findings offer insight into how and why reporters respond to rhetorical claims about scientific ignorance and uncertainty that actors use to discredit threatening science. In so doing, they contribute to growing scholarship on journalists’ contributions to the social construction of ignorance in scientific controversies.
Polls are routinely used to understand research needs, e.g.
Minimum detectable difference.
—The goal of the
video transecting methodology is to be able to
detect the smallest biologically meaningful
changes in percent coral cover with high pow-
er at 0.05. We consider a change of 5–10%
coral cover to be biologically meaningful,
which is a stricter criterion than the 10–20%
figure obtained by M. J. Risk and A. C. Risk
(1997) in a poll of reef scientists …
A PROFILE OF FOREST CANOPY SCIENCE AND SCIENTISTS-WHO WE ARE, WHAT WE WANT TO KNOW, AND OBSTACLES …
academic.evergreen.edu By means of a survey to canopy scientists. ..
Climate science affects policy more than any other science currently. It is important for policy makers and politicians to know what Americans think. It is important for Americans to know what scientists think. Most Americans believe in global warming http://www.foxnews.com/story/0,2933,250571,00.html
However, most Americans are pragmatic enough to put medium to low priority on a distant risk and treat the here and now as a priority http://www.redorbit.com/news/science/843724/americans_believe_global_warming_is_real_but_not_a_top/index.html – it’s the economy stupid.
To decide who should have a say or who is enlightened enough to have a say is shrill and elitist. I’m interested what all scientists think when it comes to global warming, not just the right scientists. Polling experts should be devising the polls. When enough polls are available, we can review the results and statistically quantify what a majority of American scientist actually think about global warming. I think that information would be helpful in policy discussions before politicians raise my taxes.
When you have +69 questions in a survey, the survey is automatically flawed. Too many questions. A survey should be concise otherwise biases due to the sheer magnitude of answering dozens of questions come into play. Perhaps Bray and van Storch should have consulted survey professionals before implementing their current survey.
As a social scientist, having read the pdf, I have a few comments to make. Your article states: “There are too many questions in this survey to go over each one in detail…The series of questions Q15 through Q17, typify a key issue – precision”. In my view, there are too many questions full stop! 76 is far too big a
number of questions to throw at busy people. You are right about the lack of precision. I’d single out questions 21 (really two questions in one)and 22. I also find some of the questions to be irrelevant or
liable to produce meaningless results, eg 68-70. Question 70 asks what the opposite of science is, not what you think it is nor what in your opinion it is. One option is ‘other’: within other I might include “lies” or “the opposite of science is ideology” or “religion”. What does it mean, then, if 10% of respondents or 90% tick the all-embracing other? Finally, I’d have problems with the sample -how it is chosen? -and with the language bias- heavily towards those who speak English as a first language. Hope this helps.
My main response to this: where do these people find their time, and what do they hope to achieve? This looks like a monumental waste of effort all round.
How about this for a variant on Q52?
“Some scientists having become aware of potentially catastrophic impacts related to climate change suppress their findings so as not to alarm the public. How much do you agree with this practice?”
Of course we don’t know if anyone has done this (by definition) but an answer to this question would be a good counter to any bias inherent in the wording of Q25.
Q38 to me smacks of ignorance: “The IPCC reports are of great use to the advancement of climate science.” Anyone who understands the overall process should know that the IPCC is not advancing climate science but summarising and evaluating it. I couldn’t answer this question with a strong positive without stopping to think what spin the authors could put on responses.
Richard (#19) is absolutely right. The sheer size of such a survey will discourage many people from completing it. A scientist with more important things to do will not set aside an hour to complete a survey.
My other problem with Bray and Von Storch’s survey is who qualifies as a “climate scientist”. Climate science is multi-disciplinary. Concentrating just on computer modellers and other people who identify themselves as climate scientists is not good enough and the results will be biased. The area of climate science is contributed to by biologists, oceanographers, glaciologists, physicists, atmospheric chemists, geologists, geophysicists and many other professions. Really the survey should be open to all scientists. Our training is sufficiently robust to allow us to make informed assessments of the research regardless of whether we are classified as a “climate scientist” or not.
Walt #14, once we’re quote mining, here are a few more from Lean and Rind 2008:
“None of the natural processes can account for the overall warming trend in global surface temperatures. In the 100 years from 1905 to 2005, the temperature trends produce by all three natural influences are at least an order of magnitude smaller than the observed surface temperature trend reported by IPCC . According to this analysis, solar forcing contributed negligible long-term warming in the past 25 years and 10% of the warming in the past 100 years, not 69% as claimed by Scafetta and West ”
and, apropos of climatologists not being motivated to try and break things or ignoring evidence conflicting with their cherished models:
“With the goal of using the geographical patterns to quantitatively constrain simulated climate change, detailed comparisons with the GISS middle atmosphere GCM are underway.”
Well, in a health survey, you’d probably rather they ask doctors in the relevant specialties. For example, if I were to survey docs on the dangers of smoking, a pulmonologist might be a little more use than a dermatologist. Or you or me. Is that elitist and shrill? Deciding what kind of climate scientists to ask is mostly about figuring out whose expertise is more useful in gauging this kind of stuff.
Distant risk — that is, the risk that something will happen in the future — doesn’t mean no risk. See smoking — the risk of you getting lung cancer is elevated if you do it, but you’re saying “smoke away! it won’t be a problem for 20 years, likely!” And you’d be right, smoking doesn’t cause problems in one day, or a week, or even a year or two.
Also, there is precedent for dealing with such risk and even raising taxes to do it. The Netherlands is not underwater precisely because the government there took steps after disastrous floods decades ago. Yes it raised taxes. Yes the risks were far in the future. But the Dutch are probably happier that those steps were taken.
But people aren’t good at assessing risks that take place over decades. Our lives are only 70-90 years so for most of us such risks may not matter, at least as individuals. But the problem now is that there are risks to much of our civilization.
When talking about mitigating risk of global warming, there are several ways to do it. It’s better to take a relatively small cost now — prevention — than it is to try the really expensive bit later. It’s a heck of a lot less expensive to move from coal power, repair the rail tracks, and encourage a replacement of the auto fleet with hybrids and more efficient models than it is to try and build massive walls around Florida. I might add that moving to more efficient cars was hugely successful, even in the US. (It’s worth noting that gas and oil are both cheaper now, adjusted for inflation, than they were in 1979-80). No, you may not be able to drive an SUV anymore. But that shouldn’t be a problem unless you have manhood issues. (Minivans are much more efficient and can do most of the same things).
In #11 I’ve suggested methods that get around both #23 and #24. Send the survey out widely. With 69 wide-ranging questions you want 690+ respondents in order to make sure your sample is not pure noise. The effect of varying domain expertise can be controlled for by factoring in the certainty coefficients provided on each statement. Stop making up excuses why this can’t be done.
Comment by Richard Sycamore — 30 Sep 2008 @ 8:12 AM
Interesting post, it raises good questions. Sixty-nine questions is a long survey. I’m assuming that it will bias the results by over-representation of scientists who are motivated enough to complete it. This then brings in the question of motives which is very messy.
Questions 61 and 62 could be more specific, like should scientists participate in public blogs and if so what type of blog, i.e. moderated science blogs like Real Climate, unmoderated larger issue blog like Gristmill or contributing to contrarian sites like Climate Audit. Perhaps a separate survey for climate scientist on blogging would be useful.
Comment by Joseph O'Sullivan — 30 Sep 2008 @ 8:49 AM
How about the folks in the various science organizations which contribute to RC, answer 9 thru 19 and let us know how it breaks down?
Better to be looking for bias and not finding it, than to not be looking for it.
They can do it over lunch. No need to disrupt their important work.
I doubt any good will come of such a poll. Too long, too convoluted, and so many questions that are really just statements. Even if it were better, the answers folks listen to are those given by the ones whom they perceive to be right. The answers will \prove\ to those who agree with the results that they were right all along, and will \prove\ to those who disagree that the poll is fatally flawed because of A, B, and C. And as everyone knows, the brilliant contrarian who happens to agree with \ME\ beats all the cattle who voted against the Truth. Naw, we’ll be kicking and screaming and fighting pointless battles with polls and politicians until the experiment we started a hundred plus years ago runs to completion. There are too many jurisdictions and owners. Even if 90% of countries or people agree (an impossible figure), then the other 10% will take advantage of the void and convert the rest of the dead dinosaurs to cash. Saudi et al stop pumping oil? China/US stop digging coal? Not gonna happen until the hard data – as in results, not science – is in. And then it’s too late, so why not continue partying like it’s 1999?
From the abstract, it looks like the paper discusses (among other things) the impact of removing aerosols from burning coal. It attempts to expand upon AR4 by recalibrating the medium-turn warming to which we are committed to take into account the aerosol reduction that would occur with immediate abatement of coal-burning…
If this continues and increases, I don’t believe that IPCC 100 year projections have even incorporated this into scenarios yet. Large Arctic Ocean bed methane clathrate releases weren’t expected for another 100 years at least.
I have heard conjecture that this is possibly due to warming river waters feeding the Arctic which were not previously incorporated into temporal calculations.
Comment by Richard Ordway — 30 Sep 2008 @ 11:06 AM
In response to Ian McLeod #18:
In computer science there is a category of decision making processes labeled “greedy algorithms”. When defining all possible outcomes or solution paths becomes too intractable, one of these is often brought into play. They can be thought of as simpler processes that take the best option they can foresee, knowing and accepting the fact that they can’t see to the end and may well choose a path to solution that is less than optimal. Like in a computer chess game, you can sometimes set how many moves ahead the program should look. Players that realize they are up against one of this type of chess program can detect when, say the computer chooses a move that steers the game down a path that looks good (to the computer) for the next three moves, but leads to it being checkmated in five. I’m an American, but if I had to categorize Americans, I’d have to say that the majority operate using a greedy algorithm.
Next, regarding your elitist concerns, it is not elitist to ask a plumber what he thinks about a plumbing problem, and his answer means a lot more than what an electrician has to say about about it.
Last, climate change is a global concern; so, I would not restrict my assessment to American scientists. I would gather the reports from thousands of scientists who are “plumbers” and publish a briefing that represented sort of a middle ground of all their conclusions. Sound reasonable so far? And, I would call it, I don’t know, an IPCC Report? Just because your plumber tells you something is going to cost you a lot does not mean that you are better off gambling that he is wrong.
Climate change will affect food production. If you assume that farmers have a pretty good idea how to optimize the production on their land under current climatic conditions, then any change in climate can be predicted to have a negative impact. Couple this with the fact that oil is our cheapest, broad-use energy source and is becoming harder to get, and you get a double-whammy looming just over most people’s horizons. It takes a lot of energy to produce food at the rates we need. It’s not just the gas for the tractor; it’s also the fertilizers, insecticides, and herbicides required to maintain a high production. Rising energy costs are slowing the economy; climate change costs will add stress to the situation.
#32 Richard Ordway: I don’t see this news release as being evidence that there are “large” Arctic ocean methane releases. In fact, in the link they state that it is likely that these releases have been probably been going on for 15,000 years. They _speculate_ that global warming may increase the rate… but I see no firm evidence that this will be a larger source than the various positive feedbacks from permafrost methane release that are incorporated in several existing models. (yes, this can go on our list of “known unknowns” but I think we are a long way from having this be in a standard model)
Also check out the Morgan and Keith (1995) work on estimating climate sensitivity: you can pick out Dick Lindzen’s “the climate sensitivity is low with no uncertainty” anonymous entry easily.
Having said that, I think that studies like the Morgan and Keith work are useful, but the point is that they are also difficult to do well. And they are easier to critique than create: I would wager that rather than having Gavin try and run such a survey himself, it would be more useful to have Gavin work in collaboration with Granger Morgan or some other survey expert instead.
#30 — there’s a great example in the news yesterday of the power of democracy to check/balance capitalism, even US capitalism. Granted, not everyone everywhere lives under the constraints of a constitution today, but all rulers ultimately must respect and embrace the wills of their peoples. So the greatest tool of the ruler is to manage/manipulate the will of the people to coincide with the ruler’s objectives.
But if 70% of people across the world ever agree that weening ourselves off of carbon is good for everyone, the changes will be made, whether the other %30 fall in line or not.
And of the people I’ve talked to, even the people who deny AGW, almost all recognize that the seasons are weird getting weirder. Just based on the changes I’ve seen in the attitudes of people I know, I think within five years there will be a 51-plus percent popular consensus (in Kansas!) that local climate is changing for the worse. (I think there will be a 70% majority of Kansans in favor of doing something about climate change within 10 years…mostly because oldest generation is going to kick the bucket during that time, and because most of the boomers I know are reasonable people….) These people may or may not care why it’s changing–they may never accept it as anthropogenic, partly because the word doesn’t mean anything to most of them–but they will CERTAINLY ask what can be done. So the real battle, as I see it, is not about convincing people that AGW is real and happening, but rather, about convincing people that adopting non-CO2 lifestyles will be a more effective solution, especially in the long term, than throwing money at other types of geoengineering.
If I were going to do a survey, I’d ask regular people where they stand on climate change and what they think should be done about it. Like a reprise of this or something.
There are umpteen different kinds of “plumbers”, not to mention the various “electricians”, etc. that make up the “consensus” on what to do about this house we live in. McLeod #18 is suggesting polling widely rather than narrowly. Assuming you can factor out the various skillsets that introduce various biases into the analysis, what’s wrong with a democratic approach? To suggest that this is what IPCC has already done is to argue that Bray and Von Storch are wasting their time duplicating IPCC’s work. Don’t you think they’ve read the IPCC reports? Don’t you think they are capable of recognizing a data gap? The question is not “what’s the consensus”, but “what’s the relationship between individual and group opinion”. These are obviously very different questions.
Comment by Richard Sycamore — 30 Sep 2008 @ 3:22 PM
Warm river water floats. It might explain the Russian observation. In fact, there have been hints of methane over the Sea of Leptov since 2003. However,the British observation seems to have deeper sources. It is interesting enough that we should wait for the peer reviewed publication before speculating. In short, the huge range of depths that seem to be producing the methane plumes seen by the British have shocked me into silence.
37 AC, you’re talking 5-10 years out for democracy to crank up to agree, and I’m saying the Arctic ice and methane will conk us over the head in a few years, and so democracy will no longer be needed to “vote” on the science, so I agree, take a poll or a vote in 10 years, and it will be a landslide. As you fear, geoengineering will probably win the next vote. Here’s a bit on a rather benign (hopefully) type of geoengineering. David Keith at the U of Calgary says he’s got a filter that grabs 20 tons of CO2 from regular air for less than 100KWH of electricity. Doesn’t give a capital cost or materials used or how to sequester the CO2 captured, but an interesting development. http://planetsave.com/blog/2008/09/30/air-capture-system-can-filter-carbon-dioxide-from-any-air-anywhere/
On your 70% of countries, how do you propose stopping (pick any 2 of the following) the US, Russia, Canada, Saudi Arabia, China, or India? Oil and other fossil fuel supply is incredibly inelastic. Nearly all costs are capital, so even if oil plummets to $20, it will still be pumped and a buyer found. Imagine the financial meltdown if Alberta’s tar sand mining was stopped. Imagine the turmoil in Iraq if we said, “You can’t sell your oil.” I suppose that’s one reason folks are going after coal. Oil, including tar sands, is just too volatile to touch. Even coal, imagine closing coal power plants. Imagine crushing perfectly good SUVs. It’s all fine and dandy saying that if 70% say so, (that 70% will certainly be those with zero investment in the technology to be taxed/banned), the 30% whose very livelihood/transportation depends on the tech in question will meekly agree. Remember, democracy and polls are just phase 1. In phase 2, it gets messy in lawsuit land and rogue nation world.
This is an excellent post. The results to this “survey” will be biased to the deniers of climate science. I hope that this post will be shared widely so that we can limit the damage of this faulty poll before it becomes used by the deniers. Without Real Climate the public would be fooled by nonsense that looks sensible. Thank you for advancing the truth.
RichardC — Thanks for the link. I’m not sure how I misunderstood your post in #30, but based on #41, our views seem more similar than I first thought. For the record, I’m opposed to geoengineering only when it means manipulating some piece of atmospheric chemistry other than CO2 in an effort to solve what (for the moment) is a CO2 problem.
I doubt I’ve understood the true meaning of the link tho…so someone please (please!) help me out. Since I don’t want to waste everybody’s time with what is probably gibberish, I’ll throw that on my own page and hope somebody from ’round these parts will stop by to straighten me out.
I don’t understand why they claim that the methane being liberated happens because of permafrost melting. How can there be permafrost in the sea bed? On top of the sea bed you have liquid water. Liquid water is over 0ºC. And as you go deeper into the sea bed, the temperature only rises. Is that compatible with the existence of permafrost in the sea bed? I thought that permafrost was a land-only phenomenom. I never heard of permafrost under the sea.
Of course the skeptics might-miss use the survey results. That is their modus operandi. However, I can understand what the aim of the survey is, and it may help to convince some people who are undecided.
I note with interest the survey of American public opinion (#18). It would be nice to see the Presidential candidates taking a stronger stand on mitigation. Perhaps this is the time to put the pressure on them to commit to strong action, when they are trying to get elected.
They need to be pinned down to a target for 2020 so that the USA can go to Copenhagen and lead the world.
Comment by Ricki (Australia) — 2 Oct 2008 @ 5:01 AM
I believe oceanic methane is held in clathrates in sea floor sediments, not in permafrost.
“The discovery of this system is important as its presence provides evidence that methane, which is a greenhouse gas, has been released in this climactically sensitive region since the last ice age,” Professor Westbrook said.
Permafrost is a land only phenomenon, but there are huge methane ice reserves in deep sea sediments in addition to methane ice stored in permafrost on land.
There is liquid water at the bottom of the ocean but it is very cold, close to zero degrees in most places and below zero in some. The reason it is not frozen is partly the huge pressure of the overlying water column and partly deep sea currents moving the water around.
However at those low temperatures and high pressures, methane is in hydrate form and is stable as ice in the sediments.
Nylo, it is not so simple. Here is a quote from Wiki that may help: “While it is stable at a temperature of up to around 0°C, at higher pressures methane clathrates (ie., the methane-containing permafrost) remain stable up to 18 °C.”
Questions 11-17 have too many problems to be useful. RC has already discussed 15-17. In 11-14, what does “adequate” mean? Adequate for what? Detecting an anthropogenic influence? Determining that swift action is needed? Quantifying the future precipitation changes throughout the globe?
Consider Q12 in particular. Which models? Some atmospheric models only parameterize convection, while others have sufficient resolution to model it explicitly. Should I consider both? How do I weight the two types in my answer? Does the word “can” mean that I should consider only the best-performing models or the theoretical potential of models, or should I ignore the word entirely and just consider current models?
Well-done! Thanks for letting us know what’s perking the climate science world, and also for debunking this ‘survey’. The best surveys are short, and come up with the answers to the questions you want. This 76 question monstrosity seems designed to tire the recipient out, with its ill-thought and leading questions. Why would any respectable climate scientist devote her or his time to such an endeavour? I’m with you – nonresponse rate will dominate the results.
An ideal poll is short, to the point, and asks an interesting research question. Something like:
1. In your opinion, is the observed warming of the Earth in the past 100 years due mostly to human emissions, or not?
2. In your opinion, can state-of-the-art climate models predict long-term climate response to doubling of CO2 to within 1.5 degrees Celsius?
3. In your opinion, what would be a good threshold below which to limit CO2 emissions to avoid very serious damage?
That’s what the public wants to know about… why not ask that?
Obviously, there is some confusion on the topic (including on my part–mea culpa), however the definition of permafrost does not include a chemical component–it comprises, for instance, both water ice and rock, as well as other components of soils, either terrestrial or subsea.
The survey of 34,000 `scientists’ keeps cropping up. Yesterday on Chicago tonight, an old Chicago area TV weatherman, John Coleman, was interviewed on Chicago Tonight on the local public TV station. He denied there was any warming at all or that it would be a significant problem, referring to those so-called scientists who don’t believe in global warming. He also made the usual claim that the IPCC was motivated by ideology.
Unfortunately, the interviewer, although he tried to challenge Coleman on a couple of matters was entirely inept. I posted something on the WTTW website in response, mentioning for example, the fact that the National Academy of Sciences and other such organizations agreed with the IPCC. There is a good chance the station will read my comment and other similar comments on the air. But I haven’t been able to locate any information about those 34,000, whatever they are. If I remember correctly, they were mostly not even scientists, and, except for a small minority, didn’t have any qualifications in the relevant disciplines.
It would help if ReadlClimate has an easy to find reference to which people could refer when they need to make a quick response.
P.S. In my comment to the station, I suggested they interview Ray in response, so I hope he will forgive me for bringing up his name.
Figen, is this a useful search to get information on these processes? I know that pingos used to be considered as ice heaves but I’ve seen comments that pingos now above the ocean may have been formed when the ocean was higher, and new ones are forming below sea level. I’m not sure how new this idea is.
Pingos at Nioghalvfjerdsfjorden, eastern North Greenland by Bennike puts me in mind that “Pingo” may not be a geologically precise word. The shape may be more a function of local weather than the formative mechanism
Just to add on the: Arctic Ocean seabed methane clathrate release
“Since 1994, he (Dr Semiletov) has led about 10 expeditions in the Laptev Sea but during the 1990s he did not detect any elevated levels of methane.
However, since 2003 he reported a rising number of methane “hotspots”, which have now been confirmed using more sensitive instruments on board the Jacob Smirnitskyi.”
Marcus, I think you are correct to state that any conjecture is premature until peer reviewed, juried, refereed journal articles appear, and I appologise if I seemed to overstate its possible significance.
What I am trying to suggest is that its possible intensity change is perhaps a “new event” and bears more scientific investigation and attention just in case.
I’m suggesting that scientific knowledge is not increased by taking a poll of existing knowledge. Further, I’m suggesting that a poll where the respondents volunteer to respond is a notoriously poor way of getting an accurate assessment of the population’s real attitudes and beliefs. I can tell you I spent a little time in research (Psychology) where it was generally accepted as a given that you could resort to this method if you had to, but the results would be treated with skepticism. (Take a look at recent political polls, conducted by expert pollsters; who is winning seems to vary depending on who is taking the poll.) So, given that no new knowledge is gained by a poll and the result of any poll should be treated with skepticism, you are absolutely correct, I am suggesting that they are wasting their time and ours.
When you ask people if they would like to contribute their opinion (Bray and von Storch), you get different results from when you sample opinions already stated (IPCC). If you have the opportunity to examine how the person formed that opinion (IPCC) versus when you don’t (Bray and von Storch), it’s even better. The risk of bias is reduced when you take larger sample sizes, and here I am talking about the sample size of the authors. Bray and von Storch are a sample size of two authors; the IPCC report has 37 authors.
Are you suggesting that these 37 authors did not sample widely enough? I’ve looked over the references.
They look pretty wide too me. If you made a tree of the references cited in those references, it would grow very large, very quickly. It starts with around 900 direct citations. The number of research hours that this report represents is astronomical.
When I look over how the polls have been used, I do not get the impression that they are used to examine the variance between opinions. It appears more like they are used to argue that the consensus among climate scientists is not what some (most) climate scientists are telling us, when they say that it is real, anthropogenic, and dangerous. I am not saying that is the intent of Bray and von Storch; they are just as subject to misinterpretation as everyone else.
I assume that Bray and von Storch have read the IPCC report, but I can offer no opinion concerning their thoughts on it. I can only give you my reasons for trusting it over the results of any questionnaire survey.
I’m sorry, but science is not served by a democratic vote of the general population, and in this case the population is best served by accurate science. If you ask the average person really simple questions about Newtonian physics, you will get a lot of answers that are just plain wrong. As the questions get more difficult, the answers will get less and less connected to reality, and the correct answer in this subject depends on an understanding that stretches our best minds. You might as well have asked the general population in the early 40s about whether an atomic bomb were possible or whether setting one off would ignite the atmospheric nitrogen. The climate is like a vast multi-body problem were the movement of one body affects the movement of others, which is hard enough, but unlike gravity, we are are struggling to understand how much movement of one affects movement of the others. Most people have no idea how little they know about it. That’s not a condition that leads to good decisions about what, if anything, to do about any problem.
Most engineer types can appreciate the idea that a change in the thermodynamic balance will result in a new equilibrium state. This can easily be put in terms the general population can understand as well. The fact that change is and will happen is not the question, the question is how quickly and how much will it change. Those answers depend on how everything interacts. The fact that the general population has a deeply vested interest in the answers does mean they are best served by letting them make an uninformed decision. Conducting surveys of general beliefs will not add to the accuracy of those beliefs.
I know that sounds elitist; I will admit to being concerned about letting the average person make this decision for me and my children. I would much rather do without some luxuries than risk some of the extreme scenarios that could play out, but I’m skeptical that the average person is as willing to make sacrifices as I am and I really don’t think they understand the risk.
At the risk of using a really bad example, I’ve run a lot of simulations (computer games) involving competing societies and limited, common resources. Most often the society that dominates the others is the one that figures out how to collect and consume the resources faster than the others. It seems to me that humans are doing this in real life. Americans have been particularly good at it.
If you couple peoples’ competitive nature with the desire for fairness observable in primates (We want at least as much as the other guy is getting.), you get a situation where Americans will not give up the advantages we have and the more populous, but poorer, societies will not give up their pursuit of getting the same kinds of things that we have. The implications of where this will take the climate, without a major shift in where the global economy gets energy, are not good. My belief is that the world is heading toward a train wreck; the best we can do is to try to slow it down so that when it hits, it won’t be as bad.
I’d like to suggest an alternative way of going about a survey of opinions that would get around all of the problems listed.
1. Set up a survey website that allows scientists to join as members.
2. The members would be able to enter details into the site database about their experience and their fields of expertise.
3. The members would be able to enter a list of their papers, along with details of each, including optionally the abstract.
4. All members would be able to rate each of the papers of other members on a scale of importance/soundness.
5. To assist busy scientists who have lots of papers, they would be able to enable other members, including lay members, to enter papers for them. (With an approval flag feature implemented to show when the member has approved a paper that has been entered for them.) This would be a task that students for example might be interested in helping with.
6. To prevent abuse, site administrators would need to be able intervene to bar bogus members, or to confirm their credentials. Both members and lay visitors would be able to flag potentially bogus members and papers (but not be able to do so for confirmed members).
7. The site administrators would post survey questions periodically, where-upon members would be notified and asked to participate. Answering questions would not be compulsory and there would be no time limit on answering them. Confirmed members would be able to submit questions to the administrators.
8. When answering questions a certainty rating would need to be given with the answer.
9. Members would be able to rate each question on various quality criteria (eg. ‘ambiguous’, ‘confusing’ etc).
10. Members would be able to return to a question and re-answer it. Each answer would be stored with a time stamp, so that it would be possible to track changing sentiments within the community.
11. Survey summaries would be able to be queried in a manner that presented the results in relation to member creditability as demonstrated by their publication record and the rating of those papers by other members. Answers from members who have been flagged as bogus, or who have been disbarred, would be filtered from the result sets.
Does anyone out there have the expertise to do this? I’ve been contemplating it since I suggested it here about six months ago, and have been busily learning how to create such applications using PHP and ExpressEngine, but I reckon I’m still at least six months away from being able to do it.
I apologize for spamming this. I’ve been working on a “Layman’s Guide” to global warming intended to summarize the science and controversy. It covers the broad topics (including mitigation) with a focus on answering skeptical claims along the way. It’s presented as a narrative, where each section builds on the last, and not as a point by point rebuttal like Skeptical Science or Coby Beck’s guide. Most significantly, it is presented both as html and as a flash-based slideshow/presentation with narration.
The HTML version is basically done, aside from a few minor revisions that I intend to make as I create the slideshow. I learned the hard way that it is extremely difficult to record a halfway decent narration, so I’m looking to correct any errors or lapses in judgment first. (At this point I’m not looking to make stylistic changes, or re-organize it). I have no actual expertise in any of this so it is a true “layman’s” guide, but there are some things I understand better than others. Fixing problems with the narration after the fact is problematic.
It is 14 sections, plus introduction. Each section will be about 18 minutes long.
The sections are:
0. Introduction (my reasons for doing this)
1. Primer and History
2. The Scientific Consensus
3. A New Ice Age (case study of the claim that scientists were predicting a new ice age)
4. The Temperature Record
5. Temperature Reconstructions (Mostly Hockey Stick stuff. Andrew Dessler did a post about my summary on Grist some months back)
6. Solar & Cosmic Rays
7. Attributing Mankind
8. Climate Models
9. Hansen’s ’88 Scenarios (case study)
10. A Changing World (summarizes environmental changes to present)
11. “Who Cares?!” (why this is a bad thing)
12. “Why Now?” (why we’ve run out of time)
13. Facing the problem (high level mitigation issues)
14. Technologies and Strategies (specific solutions)
I’ve posted the slideshow version of the introduction to give an idea of what the final format will look like. I also plan to upload versions for Google Video.
We need more good efforts like this, but there is an inherent psychological problem which severely limits their usefulness.
Adults who have not made up their minds on this issue by now, are not likely to have the interest required to make the effort to sort out all the technical points in the controversy.
People who have made up their minds that AGW is a scam, are unlikely to accept any arguments to the contrary, because most of these people firmly believe AGW is some kind of political plot, or based on economic motives. Reading the web sites, I find the deniers will believe any quasi scientific narrative that supports their idea no matter how stupid and incorrect it may seem to someone with any real scientific background. The lack of qualifications of the people who make these bogus scientific statements does not matter. It has been shown that sometimes the more debunking you do, the more the incorrect idea becomes set in the mind.
You can see this demonstrated on more blogs and discussion groups than you would have time to visit.
I haven’t had a chance to read it yet, but this survey looks like it’s full of good examples on how not to write survey questions I can use in research methods class. Examples of all the various mistakes.
Another flash — if a tree falls in a forest & there’s no one to hear it, does it make a sound?
If scientists are unaware of climate change (or some of its awful effects), does that mean it doesn’t exit?
Here’s something we should be made aware of: PEAK FOOD. Due to using finite resources to TEMPORARILY boost food production, and climate change effects, etc. — see: http://peakfood.co.uk
67 Eric, you’re right. Nothing will change until terror sets in. Unfortunately, the pretend panacea of geo-engineering will prevent that. Yep, temps can be stabilized no matter the CO2 level, but the oceans will still die and the poles will still melt. (melt is determined by equator/polar temperature difference, NOT average temps) Modern folks have been conditioned to doing whatever is fun and expecting a Magic Pill, then another Magic Pill that will counteract the harm caused by the first Magic Pill. Well, ultimately, we run out of Magic and the patient dies a miserable death. Permanent Red Tide, anyone?
The article focuses on the questions and not who was asked to participate. Why were there questions about activism and advocacy? The issue is about what the scientists think. I could get any answer I want if I could pick the respondents, e.g. a gun control survey sent to Wyoming republicans.
A scientist with real credentials and peer reviewed publications.
68 lynn: I read the peak food articles. They do state that part of the problem is that Asian, and other countries, who for centuries have eaten plant food diets, are now eating more like us in the West. i.e., a lot of meat, poultry, fish, eggs, and dairy. That takes a lot of land to feed one person, compared to that person eating a vegan diet. About as much as 15 to 1. So we can buy some time by encouraging as many people as possible to eat as near vegan as possible. A very important side benefit would be a much healthier population, with lower health care costs.
Sorry, this is really OT, but I’ve gotta rant to someone about an interesting lingo thing I caught in Gov. Palin’s stock line about AGW. You guys (RealClimate) seem interested in how scientists’ cautious, precise use of language puts them (you, us) at a disadvanage in the “marketplace of ideas”. Gov Palin has provided a great counter-example, as someone not constrained by a preference for clarity.
First, in the Couric interview:
Couric: Is it [Global warming] man-made, though in your view?
Palin: “You know there are – there are man’s activities that can be contributed to the issues that we’re dealing with now, these impacts. I’m not going to solely blame all of man’s activities on changes in climate. ”
2nd, during the VP “debate”:
“I’m not one to attribute every … activity of man to the changes in the climate. ”
Three times, in her most public moments, she has garbled the causal directionality of the same phrase in almost exactly the same way. Three times, she – sorta – declines to blame climate for man’s actions, when asked whether man’s actions are to blame for climate change.
What are the chances of this occuring randomly? We know she’s no scientist, but is her concept of causality really that fuzzy? Or has she learned – or been taught – to answer that question like that, to avoid answering the question at all? This wouldn’t be the only question she hasn’t answered, of course, but has she avoided answering any other question in such a consistent way?
Color me paranoid, but I think it’s intentional obfuscation. My pet theory is that Exxon (indirectly) paid some clever unscrupulous gnome to write her answer.
The kicker is her answer when Gibson asked her if she didn’t believe in AGW. It was something like (paraphrasing, I don’t have the transcript) “… I never said that. You can’t find where I ever said that…”. Which is true, because I can’t find where she ever said anything coherent about this at all!
Its also a great way to keep the issue out of the limelight. Most text-based media wouldn’t use a quote which didn’t make sense, because it can’t be woven into the context.
And who is most interested in keeping the issue hidden and muddled? Hmmm, let me think a minute here, with my brain…
Some brilliant wanted HSS to switch from cow meat to kangaroo meat because the latter’s digestive system barely produces methane. The fences around the pastures need to be a tad higher then :D
As for Sarah Palin, I’m in awe with her. As a none native English speaker she almost seems to say that man’s actions are a result of climate changes. Yes, man are planting palm trees in gardens way more northerly than before ;)
but that number needs to be undetstood, and comes with significant cautions
“… sector emissions in particular are extremely difficult to
quantify and the values reported to the (United Nations Framework Convention on Climate Change) UNFCCC for this sector are known to be of low reliability. This sector is therefore often omitted in emissions reporting, although its share is thought to be important.”
RE Question #9, “How much do you think the direction of research in climate change sciences has been influenced by external politics in the past 10 years?”
I’m thinking that so much energy has been tied up in refuting contrarians. It’s one thing for science to advance from legitimate, logical, (some) evidence based skepticism — that spurs scientists to strive for greater heights of knowledge. But contrarian attacks that are based more on ideology and less on logic or evidence — or a general sense & knowledge of how the physical world works — I think, is counterproductive. I’m thinking part of the purpose of their critiques is to derail the science — get scientists involved in somewhat side issues.
Another thing that has happened is that the debate, which should be between false-positive-avoiding scientists and false-negative-avoiding laypersons and policy-makers (who would strive to avert harms, requiring much lower confidence than scientists that there’s a dangerous problem, before acting to mitigate it), has been shifted so it is between the false-positive-avoiding scientists and the false-positive-avoiding contrarians (who have much higher standards than scientists, and would not accept AGW is a problem unless they were much more than 95% confident, some seemingly to demand 101% confidence — requiring some bridge to never-neverland).
The contrarians have managed to convince a lot of the public and politicians that the science just isn’t in yet. That’s got to be demoralizing to hard-working scientists.
RE #72 and Elkern’s comments, here’s an insight into the bogus science behind Palin’s claims:
Do the names ExxonMobil, the American Petroleum Institute, the George C Marshall Institute, the Heartland Institute, Willie Soon, Sallie Baliunas, and Timothy Ball ring any bells? These and other like-minded persons are the orgs and “scientists” who inform Palin’s knowledge about global warming, according to the article.
Too bad this type of news is only available outside America, and the American media are not picking up on it. We live in the Dark Ages here.
The 18% came from the FAO’s report: “Livestock’s Long Shadow”. Note that that is for worldwide emissions. I calculated the percent for the USA. It works out to about 8%, due to our using much more fossil fuel than most countries. So for some countries, their percentage is higher than 18%.
The main offenders are ruminants, because of their methane emissions. So if you want to eat flesh, you’d be best off with poultry. I’d include fish, but much energy is expended bringing them to your plate.
While I’m at it, you might want to read “The China Study”, by T. Colin Campbell, on the health effects of eating animal foods. There are many other books on the subject, but I think that one is the best.
I really like your idea above of discussing questions publicly as part of the survey process. This would help reduce leading question problems as well as help design the survey instrument to provide the community with the most helpful kinds of feedback.
At the risk of using a really bad example, I’ve run a lot of simulations (computer games) involving competing societies and limited, common resources. Most often the society that dominates the others is the one that figures out how to collect and consume the resources faster than the others.
If you mean actual computer games, such as civilization, age of empires, etc, almost all of those are deliberately designed to favor the rapid acquisition of resources. So you’ve observed these games work as designed. They don’t indicate anything about the real world.
I don’t disagree with your claim that societies that gather the most resources tend to dominate others – that is, after all, largely what the historical record appears to show (with some important exceptions). But experience with computer games does not support your claim.
I can say that, whatever the other problems with this survey it has not yet been FREEPED by any of the climate sceptic groups I am aware of, including the folks on the climate sceptics mailing list (who got hold of it last time).
Jack, the document I linked to above (the FTP link) is, I think, using the same numbers you found. Here’s a cite to it. FAO is documenting their longterm serious attempt to work out answers to these questions that are useful for comparing different food sources including the costs to the climate. It’s an amazing amount of work.
Nice to be reminded someone, somewhere, makes this kind of effort.
FAO/WFT Expert Workshop
24–28 April 2006
Comparative assessment of the environmental costs of aquaculture and other food production sectors
Methods for meaningful comparisons
RE #8 (Raven)
“Similarily, an aerospace engineer with experience in numerical modelling would be qualified on comment on whether the climate models are likely to provide useful information”
(Slightly off topic because the article was about the questions not the questioned)
I have strong doubts. An aeroplane is different from a climate system. The opinions of such people might be useful especially if they asked for plenty of time , first, in which to learn about climate models. The trouble is that people who over-estimate their own knowledge, are just the ones whose opinions are of the least value. I keep on reading comments from people who claim expertise in a general subject they call computer modelling. They probably belong to the latter category.
Returning to the topic of questions; how about another kind? i.e. questions designed to elucidate whether the person concerned has any knowledge of the subject? These should be taken under examination conditions. Why should we respect the opinions of any old scientist if (s)he is incapable of demonstrating that (s)he has read any of the leading papers?
A suggestion. Choose one well defined question such as the range of estimates and confidence for climate sensitivity i.e roughly the same as done by Morgan and Keith (mentioned above) but with a larger sample of experts. The survey being discussed is much too vague and dodgy.
Oddly enough, Bray and von Storch have responded to this post here but curiously did not see fit to let us know.
One minor typo in their post is worth pointing out. In quoting the post above they replace the word ‘corrupted’ (in reference to previous sampling problems, paragraph 4) with the word ‘corrupt’. This might seem minor, but it is the difference between an accidental problem that makes data untrustworthy and implying the researchers themselves are untrustworthy. This is certainly neither expressed nor was intended.
As I said in the post, I think that there could well be a great deal of community input that would be useful in improving these kinds of surveys, but avoiding talking to us – even to notify us of a response – does not bode well for that interaction. Shame.
Why not discuss the substance of their post rather than your indignation?
It is indeed ironic that you complain about not being notified when your original post was put up without notifying Bray or von Storch. You also posted their survey without asking.
Rather than the indignation, how about some substance from you?
[Response: Who’s indignant? I just clarified a possible misinterpretation. And as for complaining, I merely made an observation. The survey questionnaire was not described as confidential, and had B+vS asked me to take it down once I posted it, I would have. Instead, your post is the only communication they have made (albeit an indirect one and one I only found by accident), two weeks after the initial posting. I am really not that difficult to contact, even snail mail would have got to me faster. I may comment on their comment if I get some time later on – my comment above was merely a notification to the readers here that a response existed. I am not however blessed with instant judgment on posts I have yet to digest. – gavin]
[Further response: You should probably also correct the other misquotations. I did not say “These problems have meant that the results were less than useful.”, I said “These problems have meant that the results were less useful than they could have been…”. Neither did I say there were ‘too many questions’ (though others did in the comments), I said “There were too many questions … to go over each one in detail”. Both of these errors distort the meaning of what was said. And since the bulk of the my post was on the ambiguity of imprecise language, this is mildly ironic. – gavin]
Comment by Roger Pielke. Jr. — 13 Oct 2008 @ 6:40 PM
Was it their typo, or Pielke mistyping?
The typo immediately following must be his, he writes:
> Real Climate had posted their survey on their site
> without first asking.
RC had not posted their survey. Posted _about_ it.
The previous survey was corrupted when people not intended as recipients were able to answer it.
FYI, the post appeared at Prometheus but was authored by Bray and von Storch. Telling me to correct their work is probably not too useful ;-)
[Response: Well, you appointed yourself go-between. But they can read this in any case, and so I don’t really care who fixes it. – gavin]
I am sure that they would be responsive to your complaints should you wish to engage their post. I have copied your comments over at Prometheus and notified Bray and von Storch of them by email. So now everyone is aware of everyone’s comments and we can get past who did or didn’t email whom about what.
Too often blog discussions devolve into accusations of “misrepresentation” and who said what really meaning something else, and whose feelings were hurt because their words were “distorted” (Shame!).
How about just engaging them on their survey? All surveys are imperfect and can be improved, but does their survey tell us anything at all about the views of climate scientists? What might be done specifically to improve it for the next iteration? Presumably these are the sorts of questions that would make for an interesting conversation.
[Response: What do you think the original post was about? – gavin]
Comment by Roger Pielke. Jr. — 13 Oct 2008 @ 7:47 PM
The survey does appear on the website and as I read things Pielke did not write any of the post.
Clarify — the first survey had its response HTML link spread around and people filled it out who weren’t invited to as a result, corrupting the results. The second one — how are people’s responses to that being checked? Are they limiting responses to those they know are part of the population they intend to survey?
I think that’s the difference.
Thanks for asking . . . . I absolutely think that the first post was a preemptive hit job on bray and von Storch, designed to discredit their work in very public manner — and judging by the discussion that ensued on your site, it was very effective to that end.
Now, maybe I am very wrong in my interpretation of that first post and what you really wanted to do was start a productive, high-level conversation with Bray and von Storch about survey methods etc. If so, you are in luck as you now have their full attention as they have responded at some length to your criticisms with a post of their own.
So the question is, are you interested in spin, misdirection, and discrediting your peers? Or maybe instead you are interested in a substantive public discussion among experts on surveys of climate scientists? I have my views as to the answer, but feel free to prove me wrong.
[Response: Huh…… Well, I’m actually stunned that you wrote this. Until this comment, readers could have assumed that you were just helping improve the lines of communication, facilitating discussion and furthering goodwill amongst men and women. However, you appear to much more interested in stirring the pot, insulting people and generally being nasty and unpleasantly petty. You can therefore consider our interaction terminated. If Bray or von Storch want to communicate with me and readers here, they can do it directly, and frankly, it’s likely to go much more smoothly. – gavin]
Comment by Roger Pielke. Jr. — 13 Oct 2008 @ 8:24 PM
As is all too typical, whenever Roger comes by to play games, he leaves mis-communication and rancour behind. Since he is someone who is frequently misquoted, you might think that he’d be sensitive to people incorrectly cutting-and-pasting statements… but let’s put that aside.
While, I have little interest in continuing to engage with him on this or any other topic, I do have an interest in improving the level of communications of scientists with the public and that includes the communication of what scientists think. Unlike Roger’s paranoid fantasies about what this post was about, I really do think that these surveys could be done much better – and the idea proposed above of having an open wiki where the questions could be hashed out ahead of time is still I think a good one. Bray and von Storch did not respond to that, nor to the issues of precision in the questions. These are important issues to get right.
The only one of the original points they did address was with regard to what I described as a leading question. I have to say, I am surprised at what the responses were. While the result was clearly skewed towards disapproval of the use catastrophic scenarios to alert the public, more people than I expected saw this favorably. Why this is so and what it means unfortunately can’t be easily discerned. A more specific question involving a real case would have been a much better thing to ask about.
As I said originally, this critique is too late for this survey, but I sincerely hope that the issues can be taken on board for any future attempts. There is a great deal of interest in what the community of climate scientists think outside of what we write in assessment reports and papers, and it will only take a few more poorly designed surveys before they’ll cease to respond to anything – however well crafted. Thus I urge Dennis Bray (who I think wrote the response) to engage more directly, and maybe ask himself (and the readers here) why the responses were as they were? There may well be lessons to learn there too.
I once set up a polling group and engaged experts to write the questions, a bit like engaging professional ststisticians to help with climate science (Wegman).
Take a question from Bray and von Storch to examine methodology.
“24. With how much certainty can we attribute recent climate related
disasters to climate change?”
This question is logically meaningless because it requires the respondent to agree that there have been recent climate related disasters. Some of us think climate is BAU, so any answer we give is, by definition, wrong. Extremely bad poll methodology to lead respondents to an answer they know has to be wrong. I could give a dozen other method errors in this amateur survey.
Try Sir Humphrey Appleby on compulsory national military service:
Humphrey: You know what happens: nice young lady comes up to you. Obviously you want to create a good impression, you don’t want to look a fool, do you? So she starts asking you some questions: ” Mr. Woolley, are you worried about the number of young people without jobs?”
Humphrey: “Are you worried about the rise in crime among teenagers?”
Humphrey: “Do you think there is a lack of discipline in our Comprehensive schools?”
Humphrey: “Do you think young people welcome some authority and leadership in their lives?”
Humphrey: “Do you think they respond to a challenge?”
Humphrey: “Would you be in favour of reintroducing National Service?”
Bernard: Oh…well, I suppose I might be.
Humphrey: “Yes or no?”
Humphrey: Of course you would, Bernard. After all you told her you can’t say no to that. So they don’t mention the first five questions and they publish the last one.
Bernard: Is that really what they do?
Humphrey: Well, not the reputable ones no, but there aren’t many of those. So alternatively the young lady can get the opposite result.
Humphrey: “Mr. Woolley, are you worried about the danger of war?”
Humphrey: “Are you worried about the growth of armaments?”
Humphrey: “Do you think there is a danger in giving young people guns and teaching them how to kill?”
Humphrey: “Do you think it is wrong to force people to take up arms against their will?”
Humphrey: “Would you oppose the reintroduction of National Service?”
Humphrey: There you are, you see Bernard. The perfect balanced sample.
Comment by G H Sherrington — 14 Oct 2008 @ 1:03 AM
I left the following post over at Roger Pielke’s blog (sorry for length),
Dennis Bray and Hans von Storch,
Thank you for the time to post your critique of the RealClimate post. This seems to be turning into another episode of the “ha…I gotcha” game between blogs, with cheerleaders for either party, so I hope to provide a bit of focus. If you are new to the blogosphere, I’d recommend you ignore the cheerleading and simply respond to the primary post, since you’ll drive yourself mad going around various sources and reading each and every comment (and expecting them all to be rational). For instance the claim by Stef that this survey was “debunked” clearly makes no sense, as the RC post was not an attempt to “debunk” anything, nor did you make any positive scientific claim that could possibly be “right” or “wrong”…it was simply a survey, and the RC piece an opinion piece on the quality of questions asked.
For the most part I agree with your responses to the remarks made by RC commenters, but you said very little about what Dr. Schmidt actually wrote. Some of the objections raised by Gavin are probably too trivial, such as what you meant (Q16) concerning the ability of climate models to predict the next 10 years, etc. Gavin’s point is that a) there are many different meanings to “the models ability to reproduce temperature observations” and b) the ability cannot be determined, since it would require us to be around 10 years later to assess what actually happened to what was modeled…point b is kind of silly and nitpicking at words, since it doesn’t seem to be open to any other interpretation, however point a is very reasonable. He does make a good point however– models do projections, not predictions. This means an entirely different thing, since a projection assumes a given scenario (e.g., a rise of CO2 by 2% per year, no asteroid impacts, the sun doesn’t explode, etc) while a prediction is meant to forecast all possible socio-economic and physical conditions, so unlike a projection where you have “if x, then y” a prediction guarantees x, therefore y. We cannot predict volcanic eruptions or asteroid impacts, and scientists who answer this survey may be nitpickers and give an answer to that question which would have been different than had you said “projections.”
Clearly Gavin was incorrect in saying that no scientist would be in favor of “exaggerating” in a public setting, but he also makes good points on what “extreme events” are, etc. There were clearly very poorly phrased questions in the survey (e.g., what are things well enough for in Q15?). In my opinion, the effects of a doubling of CO2 are understood well enough for policy action on mitigation, but not good enough to stop research funding and going on to something else. So I’d have to assume what you mean. The essence of the RC post is this– The less assumptions your respondents need to make, the better. There is no way you can argue with that, or the fact that certain questions in this survey require assumptions. You need to see the forest a bit and not the trees in your response of their critique of your survey– Quibbling over who was not contacted or what he said, she said, etc is cute, but RC wanted to assure that questions are properly phrased, not open to question on meaning, and everyone here is hoping that they will not be misused by special interests (denialists or alarmists). I’d hope we all agree with this, so it’s not an attempt by them to discredit everyone else’s opinion as you say in your last paragraph.
By the way, Gavin is certainly right in that a few of his words were definitely misquoted, and those misquotes lead to a much different context. I hope that you can correct these mistakes (and why a [sic] next to “since?”). I do hope that you guys come over to RC and discuss this directly, so these types of backs and forth don’t get too out of hand. It seems Roger Pielke simply wants to insult Gavin which is non-productive, and certain RC commenters apparently have not understood the RC post, or have not bothered to read the survey. I doubt the aim was to “discredit your work in a public manner” as Roger Pielke thinks, but the public needs to be aware that polling can be a tricky business, and your piece had great intentions but a few of the questions they made examples out of. However, the overall procedure and intent deserves kudos, and I hope we can get reasonable assessments of what the mainstream community thinks. If anything, let’s hope respondents to this survey read the RC post to catch a few of the subtleties.
I would briefly comment on some of the cyber shenanigans of late concerning the survey of climate scientists.
First, to give Hank Roberts peace of mind: The first survey (1996) was a hard copy mailout in multiple languages. The second survey (2003) was on-line.
Gavin wrote: ‘I really do think that these surveys could be done much better – and the idea proposed above of having an open wiki where the questions could be hashed out ahead of time is still I think a good one. Bray and von Storch did not respond to that, nor to the issues of precision in the questions. These are important issues to get right.’
On the Wiki suggestion …
The idea of the Wiki proposal raises a couple of concerns which seem to predominate many of the comments on the survey. How would you control the respondents, i.e. would the questions be open for discussion by anyone so wishing to participate or would the participants have to be a practitioner of that particular specialty? Would there be community censorship on sensitive questions? (Just as an aside, haven’t some of the more damning criticisms – some of which might be valid – come from outside of the climate science community in the last couple of years?) What if questions were posed that could not be answered, would they simply be ignored? Who would be the intended audience of the responses? Would it be an in-house assessment with no room for critical comment? Might it turn out to be little more than asking the members of the Vatican to evaluate the significance of the Vatican?
[Response: (Thanks for stopping by). I don’t see this as a problem at all. You are free to ignore suggestions that aren’t constructive, but having pre-screening of questions would avoid many of the problems. Each question would have a discussion page where the purpose, clarity and appropriateness of the questions could be discussed prior to any edits for instance. Remember people aren’t discussing the answers, just the questions. – gavin]
So first I would suggest it is necessary to establish the PURPOSE of the survey. Ours was to get scientific OPINIONS concerning broad aspects of the climate sciences. We clearly stated the intention of the survey in the letter if invitation:
1 identifying areas in need of increased research and/or focus
2.suggesting funding priorities for research
3.providing the opportunity for the science community to express an opinion concerning the dissemination of scientific results
It will provide a collective perspective of the climate science community on matters of climate science and climate change from the perspective of climate scientists.
[Response: …but this a little of the problem. The questions were not directly addressed to these issues. No choices for funding priorities were suggested, and the spread of the questions dealt with what is a very limited set of climate science questions. Just compare the money that goes towards remote sensing vs. climate modelling and then the profile of each field in this survey. – gavin]
On precision …
The precision necessary to finely tune questions to each specialization in climate science would further result in a much lengthier questionnaire than any of the original surveys, with a much reduced sample size with the ability to respond to such specific questions, so what would be the representativeness of the survey?
[Response: That is a danger. But the alternative is to have people not really know what they are being asked about. What did you mean for people to think when you said ‘temperature’ for instance? It doesn’t take too much more text to say ‘global mean anomalies since 1900’ if that was indeed what you wanted. – gavin]
Still on the matter of precision …
Gavin also comments concerning the precision of a question asking as to the acceptance of presenting extreme scenarios. He says ‘A more specific question involving a real case would have been a much better thing to ask about.’
A ‘more specific real case’ might be fine for a more specific sample. Unfortunately, the global climate science community does not all read the same newspaper nor watch the same TV station. We simply wanted to know if climate scientists agreed with this practice and the question as posed captures this opinion, in my assessment, without any problem. Or, is it the case, that you feel some cases should be exaggerated and some not?
[Response: But without an example, how is one to judge whether it was exaggerated or not? One presumes that the author of any such piece didn’t think so when they wrote it. However, as I said above, I am surprised by the responses you received. That may well be worth following up in more detail. – gavin]
precision continued …
We had in mind to address a general broad spectrum of climate scientists and to do so the questions needed be be somewhat general.
question content and preemptive conclusions
Back to the general posting, Gavin stated ‘One set of questions (Q68+Q69) obviously come from a social rather than a climate scientist: Q68 asks whether science has as its main activity to falsify or verify existing hypothesis or something else [Gavin, you were very critical of my misquoting you in one of your comments. Please note that the question does NOT read ‘verify existing hypothesis or something else’, it reads ‘verify existing CONDITION and OTHER. A minor point to be sure and no less petty than your own comment]; and Q69 whether the role of science tends towards the deligitimization or the legitimization of existing ‘facts’ or something else. What is one to make of them? [I am a sociologist and oddly as it may seem, I have an interest in sociology] There are shades of Karl Popper and social constructivism in there, but we’d be very surprised if any working scientist answered anything other than ‘other’.
Well, once more your preemptive assumptions prove to be incorrect:
The frequencies of responses are as such:
68. Concerning what science is in general, what would you say is its main activity?
to falsify hypothesis – 88
to verify existing conditions – 100
other – 179
69. Concerning science in general, the role of science tends towards
deligitimization of existing ‘facts’ – 45
legitimization of existing ‘facts’ – 132
other – 179
Surprise (again), but that is what the data says.
[Response: Again, I am indeed surprised. But I’m still not any the wiser about either what the questions were trying to get at or what the scientists who answered them with other than ‘other’ were thinking. But given that ‘other’ was around 50% in both cases indicates that the question was not well framed. What did you hope to find out here? – gavin]
This part I do not understand:
Gavin concludes: ‘Thus I urge Dennis Bray (who I think wrote the response) to engage more directly, and maybe ask himself (and the readers here) why the responses were as they were? There may well be lessons to learn there too.’
Why the responses were as they were? Are the responses incorrect? Or is it perhaps they simply do not fit with preconceived notions and are therefore incorrect? Furthermore, little in the way of results have been made public, so how on earth do you know what the responses are?
[Response: I was referring the comments here, not the responses to the survey which of course I have no knowledge of. – gavin]
‘I once set up a polling group and engaged experts to write the questions, a bit like engaging professional ststisticians to help with climate science (Wegman).’
Just out of interest and for the sake of clarity: How did it turn out? Are there references to the survey? Setting up a polling group means research group or respondent sample selection?
‘Take a question from Bray and von Storch to examine methodology.
“24. With how much certainty can we attribute recent climate related
disasters to climate change?”’
For the reader, the response options were 1 – none, 7 – very much. G H Sherrington, if you are going to cite the questions please be complete.
‘This question is logically meaningless because it requires the respondent to agree that there have been recent climate related disasters. Some of us think climate is BAU, so any answer we give is, by definition, wrong.’
Well I guess the hurricanes that were so debated and the evacuation of coastal regions this year (and the failure to do so in the past) might warrant the title of a climate disaster, but then, this is just an assumption. If respondents are likely to deny these things happened then the question might well be ill posed. As for BAU a response declaring that the (non?)climate disaster is not related to climate change would be an indication of the BAU position, would it not? Following Katrina, this was a major debate. Climate related disasters are somewhat different from climate-change related disasters. What I have asked is much different from what you imply the question says. To adhere to your logic the question would have to read ‘With how much certainty can we attribute climate change related disasters to climate change.
[Response: This is another example where precision would have helped. If you wanted results related to US landfalling hurricanes (or specifically Katrina) you should have said so. But the answer might have been different if you’d said the ongoing drought in the American Southwest, or the 2003 European heat wave. By grouping together all climate-related disasters you end up losing the granularity that would allow you to say something about how specifically the hurricane/climate issue is understood in the wider field. – gavin]
>Gavin, you were very critical of my misquoting you in one of your comments
> A minor point to be sure and no less petty than your own comment
Just a note: the prometheus site has Gavin’s quotes clearly set aside in a format that suggests verbatim quoting. It is, of course, always possible to still be misleading by quoting out-of-context, but at the least it is expected that such quoted text be accurate (especially given the ease of cut-and-paste from one electronic document to another).
Gavin, in his post, probably should not have changed “condition” to “hypothesis”, but there were no quote marks about his restating of your question, and therefore his gaffe was not at the same level as yours. Nor did it really change the meaning or tone of your question nearly as much as your misquoting did of his, as far as I can see.
In a more substantial note: I do think the survey results will be of value, just not as much as it could have been with more outside input. It would be interesting to know how you pre-tested the survey questions, and whether any of the issues raised in this post were raised in the pre-testing? (in addition to Granger Morgan, Ansolabehere also does some interesting survey work in this area. They both had different foci than you, but there might be mutual learning in discussions with them)
And one more example of a specific question that is ambiguous: “Given our current state on knowledge, climate change is now mostly a…” (political issue or scientific issue): I’m not sure how I would answer that. I might say “well, I think the science is sufficiently resolved that we should be treating it as a political answer in order to work towards solutions”, or I might say “well, clearly studying climate change is a scientific issue! Despite all those deniers who say that the IPCC is political, it is scientific!”
A question that gets a 50% response of “other” is VERY poorly put. Either the authors have not a clue to what the range of responses will be or opinion about the issue is so widely spread that there is no point in asking the question.
Ah – Eli Rabett – like your friend over at Deltoid, you are a little late into this fray this time. I could say much more but I have neither the time nor the inclination.
My foray into blogworld has been what … amusing I guess. But I must bid farewell to the Hatfields and the McCoys. Alas, I must return to more responsible duties – I am milk monitor of the week!
Before I go I would like to offer a little prayer, penned by Robertson Davies long before Digital Daze. It goes ‘God give me oblivion from the small small voices of small small people’
Amen to that
[Response: Direct interaction on blogs is not for everyone, but there are legitimate issues that have been raised here, and good suggestions for making improvements in the future. I’d still like to have had a discussion of exactly what you were trying to find in the more imprecise questions, but I hope that you are able to take those constructive criticisms on board. – gavin]
How did it turn out? We spent the main part of the survey exercise to determine if the random respondents (a) knew what we were polling about and (b) cared what we were polling about; concluded a negative to both and pulled the plug.
Comment by G H Sherrington — 15 Oct 2008 @ 3:07 AM
The indigination at not being notified of the Promethus post is misplaced, if Bray and von Storch were not contacted before Gavin’s post on RC.
But for the rest, Gavin’s criticisms of the surveys and the misquotations are right on.
BTW, #96 looks like a typo:
“But given that ‘other’ was around 50% in both cases indicates that the question was well framed.” I assume Gavin meant “not well framed”. (Or does this refer to Gavin’s “question” about the questions?).
[Response: Yes. my bad. I’ve edited it above for clarity. – gavin]
While I agree that the von Storch/Bray survey has a lot of problems, here’s one that’s even worse. It was a public survey in Canada sponsored by the Frontier Centre for Public Policy.
The report’s title says it all:
“Immense Public Frustration with Politicians Over the Global Warming and Climate Change Debate ”
Sample report heading:
“One-Sided Media Reporting the Main Apparent Driver of Public Opinion on Global Warming and Climate Change”
Believe it or not, FCPP actually has charitable status in Canada (as does the Fraser Institute).
I just took a look at the Bray/von Storch survey and it’s even worse than I thought. Here’s a really poor question:
36. The best approach to the mitigation of anthropogenic climate change would be based on:
voluntary actions (1) enforced regulations(7)
In Canada, we have just been through an election campaign in which a proposed carbon tax was a major plank of the Liberals, one the major parties (losers to the Conservatives as it happens). That proposal had the support of most economists and climate scientists in Canada.
But it’s not clear where such a proposal fits on the continuum in the question. After all, a tax shift onto carbon is not really regulation as such, and it would presumably work through market forces (i.e. the sum of “voluntary” actions). A much better question might be ask for degree of agreement with the general proposal for “putting a price on carbon” (for example, through cap-and-trade system, carbon tax or some combination).
Revisit my 103 and forget about the survey if these two objective pre-tests fail.
Global warming has become so far detatched from science that public opinion surveys have a high probability of reporting “conditioning” “belief” and “propaganda”. None of this is of much use in understanding science or in enjoying the benefits of nice warm weather.
Comment by G H Sherrington — 16 Oct 2008 @ 4:22 AM
“In Canada, we have just been through an election campaign in which a proposed carbon tax was a major plank of the Liberals, one the major parties (losers to the Conservatives as it happens). That proposal had the support of most economists and climate scientists in Canada.”
Most? please…It had the support of Liberal climate scientists and economists..
Don’t forget the Primer Minister himself is an economist and he had no love for that wealth distribution plan. You can’t take money off people all year long, then hope they make past your cut off date to get some of it back, anyone who’s primary cost is fuel ends up lending the government money for 12 months, for farmers and small transportation companies, thats alot of money to be lending for free. Also, it would never be revenue nuetral, bureaucrats do not work for free.
Giving the benefit of the doubt we will assume that you are a well qualified milk monitor. As a designer of surveys it is quite clear that you are not very well qualified.
Eli’s point remains, one which was not made previous to this very small (but cute) Rabett posting. You did not even try to provide an answer but attempted to blow us off. If you had tried your little trick in a bar, there would have been very cross words and more, but, as you point out this is a very polite blog.
Thus, a reasonable person must conclude that you agree that a question that gets a 50% response of “other” is VERY poorly put and the authors have not a clue to what the range of responses will be or opinion about the issue is so widely spread that there is no point in asking the question. But Eli repeats himself.
No doubt about it, the IPCC needs to be more clear in its diction and description of certain aspects of its report. It would be a shame if people continued to misconstue, distort and minimize global warming due to flawed articulation of data, lack of presentation or minor errors overlooked through vague design of the reports themselves.