{"id":4699,"date":"2010-08-03T10:34:23","date_gmt":"2010-08-03T15:34:23","guid":{"rendered":"http:\/\/www.realclimate.org\/?p=4699"},"modified":"2010-08-05T10:31:18","modified_gmt":"2010-08-05T15:31:18","slug":"expert-credibility-in-climate-change-responses-to-comments","status":"publish","type":"post","link":"https:\/\/www.realclimate.org\/index.php\/archives\/2010\/08\/expert-credibility-in-climate-change-responses-to-comments\/","title":{"rendered":"Expert Credibility in Climate Change – Responses to Comments"},"content":{"rendered":"
\n

Guest commentary by William R. L. Anderegg, Jim Prall, Jacob Harold, Stephen H. Schneider<\/small><\/p>\n

Note: Before Stephen Schneider’s untimely passing, he and his co-authors were working on a response to the conversation sparked by their recent paper in the Proceedings of the National Academy of Sciences on climate change expertise. One of Dr. Schneider\u2019s final interviews<\/a> also addresses and discusses many of the issues covered here. <\/i><\/small><\/p>\n

We accept and rely upon the judgment and opinions of experts in many areas of our lives. We seek out lawyers with specific expertise relevant to the situation; we trust the pronouncement of well-trained airplane mechanics that the plane is fit to fly. Indeed, the more technical the subject area, the more we rely on experts. Very few of us have the technical ability or time to read all of the primary literature on each cancer treatment\u2019s biology, outcome probabilities, side-effects, interactions with other treatments, and thus we follow the advice of oncologists. We trust the aggregate knowledge of experts \u2013 what do 97% of oncologists think about this cancer treatment \u2013 more than that of any single expert. And we recognize the importance of relevant expertise \u2013 the opinion of vocal cardiologists matters much less in picking a cancer treatment than does that of oncologists. <\/p>\n

Our paper Expert Credibility in Climate Change<\/a> is predicated on this idea. It presents a broad picture of the landscape of expertise in climate science as a way to synthesize expert opinion for the broader discourse. It is, of course, only a first contribution and, as such, we hope motivates discussion and future research. We encourage follow-up peer-reviewed research, as this is the mark of scientific progress. Nonetheless, some researchers have offered thoughtful critiques about our study and others have grossly mischaracterized the work. Thus, here we provide responses to salient comments raised.
\n<\/p>\n

Definition of groups:<\/strong> The first of four broad comments about our study examines the relevancy of our two studied groups \u2013 those Convinced of the Evidence that much of the warming of the last half century is due in large part to human emissions of greenhouse gases, as assessed by the IPCC, which we term \u201cCE,\u201d and those who are Unconvinced of the Evidence (“UE”). Some have claimed<\/a> that such groups do not adequately capture the complexity of expert opinion and therefore lose meaning. To be sure, anthropogenic climate change (ACC) is an immensely multi-faceted and complex area and expert opinion mirrors this complexity. Nonetheless, society uses simplifications of complex opinion landscapes all the time (e.g. Democrat versus Republican for political views) that don\u2019t \u201close their meaning\u201d by ignoring the complexity of nuanced differences on specific topics within these broad groups. <\/p>\n

The central questions at hand are: are these groups (1) clearly defined, (2) different in views of ACC, (3) reasonably discrete, and (4) in the main mutually exclusive? Our definition of groups, based entirely in the case of the UE group on their self-selected, voluntarily signed statements and petitions expressing various versions of skepticism about ACC, is clearly defined in the published paper. The strongest evidence indicating that our CE and UE groups satisfy the second and third criteria is that only three of 1,372 researchers fell into both groups\u2014and in two of those cases, the researcher unwittingly added themselves to a statement they did not in fact support. Thus, if only one researcher of 1,372, or 0.07%, legitimately falls into both of our groups, this suggests that the two groups both differ starkly and are discrete. Any statistical analysis would be only trivially altered by having three redundant members of the cohort. Furthermore, the CE and UE groups are coherent, as around 35% of signers in each group also signed another statement in that set. <\/p>\n

Another researcher suggests<\/a> that his views have been \u201cmisclassified\u201d by our inclusion of older public statements, as he signed a 1992 statement. Using a sweeping set of public statements that cover a broad time period to define the UE group allows us to compile an extensive (e.g. make an effort to be as comprehensive as possible) dataset and to categorize a researcher\u2019s opinion objectively. However, were we to reclassify this researcher, it would only strengthen our results as then none of the top fifty researchers (rather than one researcher, or 2%) would fall in the UE camp. <\/p>\n

Others have contended<\/a> that the only experts we should have analyzed were those researchers involved specifically in detection and attribution of human-caused climate change. Importantly, much of the most convincing evidence for ACC comes from our understanding of the underlying physics of the greenhouse effect, illuminated long before the first detection\/attribution studies, and these studies provide only one statistical line of evidence. The study could have been done in this manner but let us follow that logic to its conclusion. Applying this stricter criterion to the CE list does cause it to dwindle substantially\u2026but applying it to the UE list causes it to approach close to zero researchers. To our knowledge, there are virtually no UE researchers by this logic who publish research on detection and attribution. Following this logic one would have to conclude that the UE group has functionally no credibility as experts on ACC. We would, however, argue that even this premise is suspect, as ecologists in IPCC have done detection and attribution studies using plants and animals (e.g. Root et al. 2005). Finally, applying a criterion such as this would require subjective judgments of a researcher\u2019s focus area. Our study quite purposefully avoids making such subjective determinations and uses only objective lists of researchers who are self-defined. They were not chosen by our assessment as to which groups they may or may not belong in. <\/p>\n

Some<\/a> have<\/a> taken issue with our inclusion of IPCC AR4 WGI authors in with the CE group, in that the IPCC Reports are explicitly policy-neutral while the four other CE policy statements\/petitions are policy prescriptive. However, we believe our definition of the CE group is scientifically sound. Do IPCC AR4 WGI authors subscribe to the basic tenets of ACC? We acknowledge that this is an assumption, but we believe it is very reasonable one, given the strength of the ultimate findings of the IPCC AR4 WGI report. We classify the AR4 WGI authors as CE because they authored a report in which they show that the evidence is convincing. Naturally, authors may not agree with everything in the report, but those who disagreed with the most fundamental conclusions of the report would likely have stepped down and not signed their names. The presence of only one of 619 WGI contributors on a UE statement or petition, compared to 117 that signed a CE statement, provides further evidence to support this assumption. Furthermore, repeating our analysis relying only on those who signed at least one of the four CE letters\/petitions and not on IPCC authorship yields similar results to those published. <\/p>\n

No grouping of scientists is perfect. We contend that ours is clear, meaningful, defensible, and scientifically sound. More importantly, it is based on the public behavior of the scientists involved, and not our subjective assignments based on our reading of individuals\u2019 works. We believe it is far more objective for us to use choices by scientists (over which we have no influence) for our data instead of our subjective assessment of their opinions.<\/p>\n

Scientists not counted:<\/strong> What about those scientists who have not been involved with the IPCC or signed a public statement? What is their opinion<\/a>? Would this influence our finding that 97% of the leading researchers we studied endorse the broad consensus regarding ACC expressed in IPCC\u2019s AR4? We openly acknowledge in the paper that this is a \u201ccredibility\u201d study and only captures those researchers who have expressed their opinions explicitly by signing letters\/petitions or by signing their names as authors of the IPCC AR4 WGI report. Some employers explicitly preclude their employees from signing public statements of this sort, and some individuals may self-limit in the same way on principle apart from employer rules. However, the undeclared are not necessarily undecided. Two recent studies tackle the same question with direct survey methods and arrive at the same conclusion as reached in our study. First, Doran and Kendall-Zimmerman (2009) surveyed 3,146 AGU members and found that 97% of actively publishing climate researchers believe that \u201chuman activity is a significant factor in changing mean global temperatures.\u201d A recently published study, Rosenberg et al (2010), finds similar levels of support when surveying authors who have published during 1995-2004 in peer-reviewed journals highlighting climate research. Yes, our study cannot answer for \u2013 and does not claim to \u2013 those who have not publically expressed their opinions or worked with the IPCC, but other studies have and their results indicate that our findings that an overwhelming percentage of publishing scientists agree with the consensus are robust. Perfection is not possible in such analyses, but we believe that the level of agreement across studies indicates a high degree of robustness. <\/p>\n

Publication bias:<\/strong> A frequent response to our paper\u2019s analysis consists of attributing the patterns we found to a systematic, potentially conspiratorial suppression of peer-reviewed research from the UE group. As of yet, this is a totally unsupported assertion backed by no data, and appears untenable given the vast range of journals which publish climate-related studies. Notably, our publication and citation figures were taken from Google Scholar, which is one of the broadest academic databases and includes in its indexing journals openly receptive to papers taking a different view from the mainstream on climate. Furthermore, recently published analysis (Anderegg 2010) examines the PhD and research focus of a subsample of the UE group, compared to data collected by Rosenberg et al. 2010 for a portion of the climate science community publishing in peer-reviewed journals. If the two groups had similar background credentials and expertise (PhD topic and research focus \u2013 both non-publishing metrics), it might indicate a suppression of the UE group\u2019s research. They don\u2019t. The background credentials of the UE group differ starkly from that of the \u201cmainstream\u201d community. Thirty percent of the UE group sample either do not have a documented PhD or do not have a PhD in the natural sciences, as compared to an estimated 5% of the sample from Rosenberg et al; and nearly half of the remaining sample have a research focus in geology (see the interview<\/a> by Schneider as well). <\/p>\n

“Blacklist”<\/strong>: The idea<\/a> that our grouping of researchers comprises some sort of \u201cblacklist\u201d is the most absurd and tragic misframing of our study. Our response is two-fold:<\/p>\n

    \n
  1. Our study did not create any list. We simply compiled lists that were publicly available and created by people who voluntarily self-identified with the pronouncements on the statements\/letters. We did not single out researchers, add researchers, drop researchers; we have only compiled individuals from a number of prominent and public lists and petitions that they themselves signed, and then used standard social science procedure to objectively test their relative credibility in the field of climate science. <\/li>\n
  2. No names were used in our study nor listed in any attachments. We were very aware of the pressure that would be on us to provide the raw data used in our study. In fact, many journalists we spoke with beforehand asked for the list of names and for specific names, which we did not provide. We decided to compromise by posting only the links to the source documents \u2013 the ‘raw data’ in effect (the broader website is not the paper data), where interested parties can examine the publically available statements and petitions themselves. It is ironic that many of those now complaining about the list of names are generally the same people that have claimed that scientists do not release their data. Implying that our list is comparable to that created by Mark Morano when he worked for Senator Inhofe is decidedly unconvincing and irresponsible, given that he selected individuals based on his subjective reading and misreading of their work. See here<\/a> for a full discussion of this problematic claim or read Schneider\u2019s interview above. <\/li>\n<\/ol>\n

    In sum, the various comments and mischaracterizations discussed above do not in any way undermine the robust findings of our study. Furthermore, the vast majority of comments pertain to how the study could have been done differently. To the authors of such comments, we offer two words \u2013 do so! That\u2019s the hallmark of science. We look forward to your scientific contributions \u2013 if and when they are peer-reviewed and published \u2013 and will be open to any such studies. In our study we were subjected to two rounds of reviews by three social scientists and in addition comments from the PNAS Board, causing us to prepare three drafts in response to those valuable peer comments that greatly improved the paper. We hope that this response further advances the conversation. <\/p>\n

    References<\/strong>
    \nAnderegg, W.R.L. (2010) Moving Beyond Scientific Agreement. Climatic Change, 101 (3) 331-337.
    \nDoran PT, Zimmerman MK (2009) Examining the Scientific Consensus on Climate Change. Eos Trans. AGU 90.
    \nRoot, T.L. et al. (2005) Human-modified temperatures induce species changes: Joint attribution. PNAS May 24, 2005 vol. 102 no. 21 7465-7469
    \nRosenberg, S. et al (2010) Climate Change: A Profile of U.S. Climate Scientists’ Perspectives. Climatic Change, 101 (3) 311-329.
    \n<\/small><\/p>\n\n<\/div> ","protected":false},"excerpt":{"rendered":"

    Expert credibility in climate science, consensus and the Anderegg PNAS study<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[1,34],"tags":[],"class_list":{"0":"post-4699","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-climate-science","7":"category-skeptics","8":"entry"},"aioseo_notices":[],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/4699","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/comments?post=4699"}],"version-history":[{"count":10,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/4699\/revisions"}],"predecessor-version":[{"id":4813,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/4699\/revisions\/4813"}],"wp:attachment":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/media?parent=4699"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/categories?post=4699"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/tags?post=4699"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}