{"id":109,"date":"2005-01-20T12:37:15","date_gmt":"2005-01-20T16:37:15","guid":{"rendered":"\/?p=109"},"modified":"2015-01-25T11:54:22","modified_gmt":"2015-01-25T16:54:22","slug":"peer-review-a-necessary-but-not-sufficient-condition","status":"publish","type":"post","link":"https:\/\/www.realclimate.org\/index.php\/archives\/2005\/01\/peer-review-a-necessary-but-not-sufficient-condition\/","title":{"rendered":"Peer Review: A Necessary But Not<\/em> Sufficient Condition Evaluation par des pairs : une condition n\u00e9cessaire mais pas<\/em> suffisante<\/lang_fr>"},"content":{"rendered":"
\n

by Michael Mann and Gavin Schmidt<\/small><\/p>\n

On this site we emphasize conclusions that are supported by “peer-reviewed” climate research. That is, research that has been published by one or more scientists in a scholarly scientific journal after review by one or more experts in the scientists’ same field (‘peers’) for accuracy and validity. What is so important about “Peer Review”? As Chris Mooney<\/a> has lucidly put it:<\/p>\n

\n

\n[Peer Review] is an undisputed cornerstone of modern science. Central to the competitive clash of ideas that moves knowledge forward, peer review enjoys so much renown in the scientific community that studies lacking its imprimatur meet with automatic skepticism. Academic reputations hinge on an ability to get work through peer review and into leading journals; university presses employ peer review to decide which books they’re willing to publish; and federal agencies like the National Institutes of Health use peer review to weigh the merits of applications for federal research grants.\n<\/p>\n<\/blockquote>\n

Par Michael Mann et Gavin Schmidt (Traduit par Alain Henry)<\/small><\/p>\n

Dans ce site, nous insistons sur les r\u00e9sultats de recherches sur le climat soumis \u00e0 des \u00ab \u00e9valuations par des pairs \u00bb [NdT: l’expression fran\u00e7aise \u00ab \u00e9valuation par des pairs \u00bb \u00e9tant lourde et peu satisfaisante, nous utiliserons dans la plupart des cas l’expression anglaise originale peer review<\/em> et nous d\u00e9signerons les reviewers comme des \u00e9valuateurs]. C\u2019est-\u00e0-dire, des recherches publi\u00e9es par un ou plusieurs chercheurs dans un journal scientifique, apr\u00e8s avoir \u00e9t\u00e9 \u00e9valu\u00e9e par un ou plusieurs experts dans le m\u00eame domaine (des \u00ab pairs \u00bb) pour en v\u00e9rifier la pr\u00e9cision et la validit\u00e9. Quelle est l’importance de ces peer review<\/em> ? Comme le dit tr\u00e8s lucidement Chris Mooney<\/a> : <\/p>\n

[Le peer review<\/em>] est incontestablement une pierre angulaire de la d\u00e9marche scientifique moderne. Concept central au choc comp\u00e9titif des id\u00e9es qui fait avancer la connaissance, le peer review<\/em> b\u00e9n\u00e9ficie d\u2019une telle renomm\u00e9e au sein de la communaut\u00e9 scientifique que les \u00e9tudes qui n\u2019ont pas son imprimatur sont consid\u00e9r\u00e9es avec scepticisme. Les r\u00e9putations acad\u00e9miques d\u00e9pendent de la capacit\u00e9 \u00e0 franchir le peer review<\/em> pour \u00eatre publi\u00e9 dans les principaux journaux ; les presses universitaires emploient le peer review<\/em> pour d\u00e9cider quels livres elles publieront ; et les agences f\u00e9d\u00e9rales comme l\u2019Institut National pour la Sant\u00e9 utilisent le peer review<\/em> pour \u00e9valuer les demandes de fonds f\u00e9d\u00e9raux pour la recherche. <\/p><\/blockquote>\n

(suite…)<\/a>
\n<\/lang_fr>
\n<\/p>\n

Put simply, peer review is supposed to weed out poor science. However, it is not foolproof — a deeply flawed paper can end up being published under a number of different potential circumstances: (i) the work is submitted to a journal outside the relevant field (e.g. a paper on paleoclimate submitted to a social science journal) where the reviewers are likely to be chosen from a pool of individuals lacking the expertise to properly review the paper, (ii) too few or too unqualified a set of reviewers are chosen by the editor, (iii) the reviewers or editor (or both) have agendas, and overlook flaws that invalidate the paper’s conclusions, and (iv) the journal may process and publish so many papers that individual manuscripts occasionally do not get the editorial attention they deserve. <\/p>\n

Thus, while un-peer-reviewed claims should not be given much credence, just because a particular paper has passed through peer review does not absolutely ensure that the conclusions are correct or scientifically valid. The “leaks” in the system outlined above unfortunately allow some less-than-ideal work to be published in peer-reviewed journals. This should therefore be a concern when the results of any one particular study are promoted over the conclusions of a larger body of past published work (especially if it is a new study that has not been fully absorbed or assessed by the community). Indeed, this is why scientific assessments such as the Arctic Climate Impact Assessment (ACIA)<\/a>, or the Intergovernmental Panel on Climate Change (IPCC)<\/a> reports, and the independent reports<\/a> by the National Academy of Sciences<\/a>, are so important in giving a balanced overview of the state of knowledge in the scientific research community.<\/p>\n

There have been several recent cases of putatively peer-reviewed studies in the scientific literature that produced unjustified or invalid conclusions. Curiously, many of these publications have been accompanied by heavy publicity campaigns, often declaring that this one paper completely refutes the scientific consensus<\/a>. An excellent account of some of these examples is provided here<\/a> by Dr. Stephen Schneider<\/a> (Stanford University).<\/p>\n

<\/a>Perhaps the most publicized recent example was the publication of a study<\/a> by astronomer Willie Soon<\/a> of the Harvard University-affiliated Harvard-Smithsonian Center for Astrophysics<\/a> and co-authors, claiming to demonstrate that 20th century global warmth was not unusual in comparison with conditions during Medieval times. Indeed, this study serves as a prime example of one of the “myths”<\/a> that we have debunked elsewhere on this site. The study was summarily discredited in articles by teams of climate scientists (including several of the scientists here at RealClimate<\/a>), in the American Geophysical Union (AGU) journal Eos<\/em><\/a> and in Science<\/em><\/a>. However, it took some time the rebuttals to work their way through the slow process of the scientific peer review. In the meantime the study was quickly seized upon by those seeking to sow doubt in the validity behind the scientific consensus concerning the evidence for human-induced climate change (see news articles in the New York Times<\/a>, and Wall Street Journal<\/a>). The publication of the study had wider reverberations throughout the academic and scientific institutions connected with it. The association of the study with the “Harvard” name caused some notable unease among members of the Harvard University community (see here<\/a> and here<\/a>) and the reputation of the journal publishing the study was seriously tarnished in the process. The editor at Climate Research<\/em> that handled the Soon et al paper, Dr. Chris de Frietas, has a controversial record of past editorial practices (see this ‘sidebar’<\/a> to an article in Scientific American<\/em> by science journalist David Appell). In an unprecedented (to our knowledge) act of protest, chief editor Hans von Storch<\/a> and 3 additional editors subsequently resigned from Climate Research<\/em> in response to the fundamental documented failures of the editorial process at the journal. A detailed account of these events are provided by Chris Mooney in the Skeptical Inquirer<\/em><\/a> and The American Prospect<\/em><\/a>, by David Appell in Scientific American<\/em><\/a>, and in a news brief<\/a> in Nature<\/em>. The journal’s publisher himself (Otto Kline) eventually stated<\/a> that “[the conclusions drawn] cannot be concluded convincingly from the evidence provided in the paper”. <\/p>\n

<\/a>Another journal which (quite oddly) also published the Soon et al study, “Energy and Environment”, is not actually a scientific journal at all but a social science journal. The editor, Sonja Boehmer-Christensen<\/a>, in defending the publication of the Soon et al study, was quoted<\/a> by science journalist Richard Monastersky in the Chronicle of Higher Education somewhat remarkably confessing “I’m following my political agenda — a bit, anyway. But isn’t that the right of the editor?”. <\/p>\n

<\/a> Shaviv and Veizer (2003) published a paper<\/a> in the journal GSA Today<\/em>, where the authors claimed to establish a correlation between cosmic ray flux (CRF) and temperature evolution over hundreds of millions of years, concluding that climate sensitivity to carbon dioxide was much smaller than currently accepted. The paper was accompanied by a press release entitled \u201cGlobal Warming not a Man-made Phenomenon”<\/a>, in which Shaviv was quoted as stating,\u201cThe operative significance of our research is that a significant reduction of the release of greenhouse gases will not significantly lower the global temperature, since only about a third of the warming over the past century should be attributed to man\u201d. However, in the paper the authors actually stated that “our conclusion about the dominance of the CRF over climate variability is valid only on multimillion-year time scales”. Unsurprisingly, there was a public relations offensive<\/a> using the seriously flawed conclusions expressed in the press release<\/em> to once again try to cast doubt on the scientific consensus that humans are influencing climate. These claims were subsequently disputed in an article in Eos<\/em> (Rahmstorf et al, 2004)<\/a> by an international team of scientists and geologists (including some of us here at RealClimate<\/a>), who suggested that Shaviv and Veizer’s analyses were based on unreliable and poorly replicated estimates, selective adjustments of the data (shifting the data, in one case by 40 million years) and drew untenable conclusions, particularly with regard to the influence of anthropogenic greenhouse gas concentrations on recent warming (see for example the exchange<\/a> between the two sets of authors). However, by the time this came out the misleading conclusions had already been publicized widely. <\/p>\n

<\/a>Next, we discuss the first of three so-called “bombshell” papers that supposedly “knock the stuffing out of”<\/a> the findings of the IPCC. Patrick Michaels<\/a> and associates billed his own paper (McKitrick and Michaels, 2004) (co-authored by Ross McKitrick<\/a> ), this way:<\/p>\n

\n After four years of one of the most rigorous peer reviews ever, Canadian Ross McKitrick and another of us (Michaels) published a paper searching for \u201ceconomic\u201d signals in the temperature record. \u2026The research showed that somewhere around one-half of the warming in the U.N. surface record was explained by economic factors, which can be changes in land use, quality of instrumentation, or upkeep of records.\n<\/p><\/blockquote>\n

It strikes us as odd, to say the least, that, after one of the “most rigorous peer reviews ever”, nobody involved (neither editor, nor reviewers, nor authors) seems to have caught the egregious basic error that the authors mistakenly used degrees rather than the required radians<\/a> in calculating the cosine functions used to spatially weight their estimates**. This mistake rendered every calculation in the paper incorrect, and the conclusions invalid — to our knowledge, however, the paper has not yet been retracted. Remarkably, there were still other independent and equally fundamental errors<\/a> in the paper that would have rendered it entirely invalid anyway. To the journals credit, they published a criticism of the paper by Benestad (2004)<\/a> to this effect. It may come as no surprise that McKitrick and Michaels (2004) was published in Climate Research<\/em> and was handled by none other than Chris de Frietas.<\/p>\n

<\/a>The other two “bombshell” papers were published in the AGU journal Geophysical Research Letters (GRL) which publishes over 1500 papers per year. It can be conservatively estimated that they publish no more than 70% of the papers received, and thus probably process over 2000 papers per year. That gives each of the typically 8 or so editors of the journal almost a paper per day to evaluate. While GRL publishes many excellent papers and provides an important forum to the research community for rapid publication of important results, occasionally, poor papers slip through the net. These two papers were authored by Douglass and collaborators (Douglass et al, 2004a;2004b) the first<\/a> with Fred Singer<\/a> as a co-author and the second<\/a> with both Singer and Michaels. Both papers*** argue that recent atmospheric temperatures have been cooling, rather than warming, based on the analysis of data over a selective (1979-1996) time interval that eliminates periods of significant warming both before and after, and using a controversial satellite-derived temperature record whose robustness<\/a> has been called into question by other teams<\/a> analysing the data. An excellent discussion of both papers is provided by Tim Lambert<\/a>.<\/p>\n

<\/a>Another relevant GRL paper was the article by Legates and Davis (1997) which criticized the use of “centered correlations” common to numerous “Detection and Attribution”<\/a> studies supporting the detection of human influence on recent climate change. They argued that correlations could increase while observed and simulated global means diverge. However, as pointed out<\/a> in the chapter on Detection and Attribution<\/a> in IPCC (2001)*, centered correlations were introduced for precisely this reason: to provide an indicator that was statistically independent of global mean temperature changes. As noted by the IPCC, “if both global mean changes and centered pattern correlations point towards the same explanation of observed temperature changes, it provides more compelling evidence than either of these indicators in isolation”. Again, a basic logical flaw in the authors’ criticism of past work was not caught in peer review.<\/p>\n

<\/a>Next, we consider the paper by Soon et al (2004) published in GRL which criticized the way temperature data series had been smoothed in the IPCC report and elsewhere. True to form, contrarians immediately sold the results as ‘invalidating’ the conclusions of the IPCC, with the lead author Willie Soon<\/a> himself writing an opinion piece<\/a> to this effect. Once again, a few short months later, a followup article<\/a> was published by one of us (Mann, 2004) that invalidated the Soon et al (2004) conclusions, demonstrating (with links to supporting Matlab source codes and data) how (a) the authors had, in an undisclosed manner, inappropriately compared trends calculated over differing time intervals and (b) had not used standard, objective statistical criteria to determine how data series should be treated near the beginning and end of the data. It is unfortunate that a followup paper even had to be published, as the flaws in the original study were so severe as to have rendered the study of essentially no scientific value.<\/p>\n

<\/a>There are other examples of studies that have even been published in high quality venues that were heavily publicized at the time, but in retrospect were flawed (though not as egregiously as the examples above). For instance, Fan et al (1998)<\/a>, on the size of the carbon sink in the continental US, rebutted by Schimel et al. (2000)<\/a>. Or the solar-cycle length\/climate correlation described by Friis-Christensen and Lassen (1991) whose seeemingly impressive correlation for the latter half of the 20th Century disappears if you don’t change the averaging scheme half way along (Laut, 2003; Damon and Laut, 2004<\/a>). <\/p>\n

The current thinking of scientists on climate change is based on thousands of studies (Google Scholar<\/a> gives 19,000 scientific articles<\/a> for the full search phrase “global climate change”). Any new study will be one small grain of evidence that adds to this big pile, and it will shift the thinking of scientists slightly. Science proceeds like this in a slow, incremental way. It is extremely unlikely that any new study will immediately overthrow all the past knowledge. So even if the conclusions of the Shaviv and Veizer (2003) study discussed earlier, for instance, had been correct, this would be one small piece of evidence pitted against hundreds of others which contradict it. Scientists would find the apparent contradiction interesting and worthy of further investigation, and would devote further study to isolating the source of the contradiction. They would not suddenly throw out all previous results. Yet, one often gets the impression that scientific progress consists of a series of revolutions where scientists discard all their past thinking each time a new result gets published. This is often because only a small handful of high-profile studies in a given field are known by the wider public and media, and thus unrealistic weight is attached to those studies. New results are often over-emphasised (sometimes by the authors, sometimes by lobby groups) to make them sound important enough to have news value. Thus “bombshells” usually end up being duds. <\/p>\n

However, as demonstrated above, even when it initially breaks down, the process of peer-review does usually work in the end. But sometimes it can take a while. Observers would thus be well advised to be extremely skeptical of any claims in the media or elsewhere of some new “bombshell” or “revolution” that has not yet been fully vetted by the scientific community.<\/p>\n

_______________
\n*Note added 1\/21\/05: It has come to our attention that Legates and Davis (1997) were similarly rebutted in a separate publication by Wigley et al (2000). <\/small>
\n**Note added 1\/21\/05: McKitrick and Michaels have published an errata correcting the degrees\/radians error in CR 27, 265-268 which now shows that latitude correlates much better with temperature trends than any economic statisitic.<\/small>
\n***Note added 1.25.05: Chip Knappenberger correctly points out that the the second Douglass et al paper doesn’t actually make the claim that the atmosphere is cooling. We therefore withdraw that specific comment, but note that the comment concerning the selective use of data series and time periods stands. <\/small><\/p>\n

References:<\/b><\/p>\n

Benestad, R.E., Are temperature trends affected by economic activity? Comment on McKitrick & Michaels.<\/a>, Climate Research<\/i>, 27, 171-173, 2004. <\/p>\n

Damon, P. E. and P. Laut, Pattern of Strange Errors Plagues Solar Activity and Terrestrial Climate Data<\/a>, Eos<\/i>, 85, p. 370. 2004<\/p>\n

Douglass, D. H., Pearson, B.D., and S.F.Singer, Altitude dependence of atmospheric temperature trends: Climate models versus observation, Geophys. Res. Lett., 31<\/i>, L13208, doi:10.1029\/2004GL020103, 2004.<\/p>\n

Douglass, D. H., Pearson, B.D., and S.F.Singer, Knappenberg, P.C., and P.J. Michaels, Disparity of tropospheric and surface temperature trends: New evidence, Geophys. Res. Lett., 31<\/i>, L13207, doi:10.1029\/2004GL020212, 2004, 2004.<\/p>\n

Fan, S., Gloor, M., Mahlman, J., Pacala, S., Sarmiento, J., Takahashi, T., Tans, P. A Large Terrestrial Carbon Sink in North America Implied by Atmospheric and Oceanic Carbon Dioxide Data and Models, Science<\/em> 282: 442-446, 1998.<\/p>\n

Friis-Christensen, E., and K. Lassen, Length of the Solar Cycle: An indicator of Solar Activity Closely Associated with Climate, Science 254, 698-700, (1991).<\/p>\n

Legates, D. R. and R. E. Davis, The continuing search for an anthropogenic climate change signal: limitations of correlation based approaches, Geophys. Res. Lett., 24<\/i>, 2319-2322, 1997.<\/p>\n

Laut, P., Solar activity and terrestrial climate: An analysis of some purported correlations, J.Atmos. Solar-Terr.Phys.,65, 801-812. 2003<\/p>\n

Mann, M.E., On Smoothing Potentially Non-Stationary Climate Time Series<\/a>, Geophys. Res. Lett., 31<\/i>, 2319-2322, L07214, doi: 10.1029\/2004GL019569, 2004.<\/p>\n

McKitrick, R., and Michaels, P.J., A test of corrections for extraneous signals in gridded surface temperature data., Climate Research<\/i>, 26, 159-173, 2004. <\/p>\n

Rahmstorf, S., D. Archer, D.S. Ebel, O. Eugster, J. Jouzel, D. Maraun, G.A. Schmidt, J. Severinghaus, A.J. Weaver, and J. Zachos, Cosmic rays, carbon dioxide, and climate<\/a>, Eos, 85, <\/i>, 38,41, 2004. <\/p>\n

Schimel, D., Melillo, J., Tian, H., McGuire, A. D., Kicklighter, D., Kittel, T., Rosenbloom, N., Running, S., Thornton, P., Ojima, D., Parton, W., Kelly, R., Sykes, M., Neilson, R. and Rizzo, B., Contribution of Increasing CO2 and Climate to Carbon Storage by Ecosystems in the United States, Science<\/em> 287: 2004-2006, 2000<\/p>\n

Shaviv, N, and J. Veizer, Celestial driver of Phanerozoic climate?, GSA Today, 13, <\/i>, 4-10, 2004. <\/p>\n

Soon, W., D. R. Legates, and S. L. Baliunas, Estimation and representation of long-term (>40 year) trends of Northern-Hemisphere gridded surface temperature: A note of caution, Geophys. Res. Lett., 31, <\/i>, L03209, doi:10.1029\/2003GL019141, 2004. <\/p>\n

Soon, W., and S. Baliunas, Proxy climatic and environmental changes over the past 1000 years, Climate Research<\/i>, 23<\/i>, 89-110, 2003.<\/p>\n

Soon, W., S. Baliunas, C, Idso, S. Idso and D.R. Legates, Reconstructing climatic and environmental changes of the past 1000 years, Energy and Environment, 14<\/i>, 233-296, 2003.<\/p>\n

Wigley, T.M.L, Santer, B.D and K.E. Taylor, K.E., Correlation approaches to detection, Geophys. Res. Lett.,<\/i>, 27, 2973-2976, 2000.
\n
\nTout simplement, le peer review<\/em> devrait \u00e9liminer la science de mauvaise qualit\u00e9. Ceci dit, il n\u2019est pas \u00e0 toute \u00e9preuve \u2013 il peut arriver qu\u2019un article tr\u00e8s imparfait soit publi\u00e9, et ceci pour toute une s\u00e9rie de raisons : (i) le travail est soumis \u00e0 un journal en dehors du domaine pertinent (par exemple un article de pal\u00e9oclimatologie soumis \u00e0 un journal de science sociale), o\u00f9 les \u00e9valuateurs sont probablement choisis dans une \u00e9quipe qui n\u2019a pas les comp\u00e9tences pour \u00e9valuer correctement l\u2019article, (ii) le r\u00e9dacteur choisit des \u00e9valuateurs trop peu qualifi\u00e9s ou en nombre insuffisant, (iii) les \u00e9valuateurs ou le r\u00e9dacteur (ou les deux) ont des emplois du temps trop charg\u00e9s et laissent \u00e9chapper les erreurs qui invalideraient les conclusions de l\u2019article, et (iv) le journal peut traiter et publier un si grand nombre d\u2019articles que, occasionnellement, un manuscrit ne re\u00e7oit pas toute l\u2019attention qu\u2019il devrait. <\/p>\n

Donc, s’il ne faut pas donner trop de cr\u00e9dit aux travaux qui n’ont pas \u00e9t\u00e9 soumis \u00e0 un peer review<\/em>, le fait qu’un article ait franchi ce cap ne garantit pas totalement que ses conclusions sont correctes. Les \u00ab fuites \u00bb du syst\u00e8me d\u00e9crites ci-dessus permettent malheureusement \u00e0 des travaux qui ne sont pas parfaits d’\u00eatre publi\u00e9s dans des journaux utilisant le peer review<\/em>. On devrait donc \u00eatre particuli\u00e8rement attentif lorsque les r\u00e9sultats d’une \u00e9tude particuli\u00e8re vont \u00e0 l’encontre des conclusions d’un plus grand nombre de travaux publi\u00e9s pr\u00e9c\u00e9demment (particuli\u00e8rement si c’est une nouvelle \u00e9tude qui n’a pas \u00e9t\u00e9 compl\u00e8tement absorb\u00e9e et \u00e9valu\u00e9e par la communaut\u00e9). C’est bien pourquoi des \u00e9valuations scientifiques tels que Arctic Climate Impact Assessment (ACIA \u2013 Evaluation de l’impact pour le climat dans l’Arctique<\/a>) ou le Groupe d’experts intergouvernemental sur l’\u00e9volution du climat (GIEC)<\/a>, et les rapports ind\u00e9pendants<\/a> de l’Acad\u00e9mie nationale des sciences<\/a> (NdT: aux USA), sont particuli\u00e8rement importants pour fournir un \u00e9tat des lieux \u00e9quilibr\u00e9 des connaissances dans la communaut\u00e9 de la recherche scientifique. <\/p>\n

On trouve dans la litt\u00e9rature scientifique r\u00e9cente plusieurs cas d’\u00e9tudes pr\u00e9tendument \u00e9valu\u00e9es par des pairs dont les conclusions sont injustifi\u00e9es ou invalides. Curieusement, beaucoup de ces publications ont \u00e9t\u00e9 accompagn\u00e9es de lourdes campagnes de publicit\u00e9, d\u00e9clarant souvent que ce seul article r\u00e9futait compl\u00e8tement le consensus<\/a> scientifique. Un excellent compte rendu de quelques-uns de ces exemples est fourni ici<\/a> par le Dr. Stephen Schneider<\/a> (Universit\u00e9 de Stanford).<\/p>\n

L’exemple r\u00e9cent dont on a le plus parl\u00e9 est peut-\u00eatre la publication d’une \u00e9tude<\/a> par l’astronome Willie Soon<\/a> du Centre Harvard-Smithsonian pour l’astrophysique<\/a>, affili\u00e9 \u00e0 l’Universit\u00e9 de Harvard, et ses co-auteurs, qui pr\u00e9tendent d\u00e9montrer que la temp\u00e9rature globale au 20\u00e8me si\u00e8cle n’est pas inhabituelle si on la compare aux conditions du Moyen-\u00e2ge. Cette \u00e9tude est bien un excellent exemple de ces \u00ab mythes<\/a> \u00bb que nous avons d\u00e9gonfl\u00e9s ailleurs sur ce site. Cette \u00e9tude a \u00e9t\u00e9 discr\u00e9dit\u00e9e par des articles \u00e9crits par des \u00e9quipes de climatologues (dont plusieurs de RealClimate<\/a>), dans le journal de l’Union am\u00e9ricaine de g\u00e9ophysique (AGU) Eos<\/a> et dans Science<\/a>. Il a toutefois fallu un certain temps pour que ces r\u00e9futations suivent le lent cheminement de l’\u00e9valuation scientifique par des pairs. Entre temps, l’\u00e9tude fut rapidement utilis\u00e9e par ceux qui cherchaient \u00e0 semer le doute sur la validit\u00e9 du consensus scientifique \u00e0 propos des preuves de l’origine humaine des changements climatiques (voir articles dans le New York Times<\/a> et le Wall Street Journal<\/a>). La publication de cette \u00e9tude eut un grand impact au sein m\u00eame des institutions acad\u00e9miques et scientifiques qui y \u00e9taient li\u00e9es. L’association de l’\u00e9tude au nom de \u00ab Harvard \u00bb a caus\u00e9 quelques malaises parmi les membres de la communaut\u00e9 de l’Universit\u00e9 de Harvard (voir ici<\/a> et ici<\/a>) et la r\u00e9putation du journal qui publi\u00e9 cette \u00e9tude a \u00e9t\u00e9 ternie par cet \u00e9pisode. Le r\u00e9dacteur de Climate Research<\/em> qui g\u00e9ra l’article de Soon et al, le Dr. Chris de Frietas, a eu par la pass\u00e9 des pratiques \u00e9ditoriales controvers\u00e9es (voir cet article annexe dans un article de Scientific American par le journaliste scientifique David Appell<\/a>). Dans un acte de protestation sans pr\u00e9c\u00e9dent (\u00e0 notre connaissance), l’\u00e9diteur principal Hans von Storch<\/a> et trois autres \u00e9diteurs ont par la suite d\u00e9missionn\u00e9 de Climate research, en r\u00e9ponse aux dysfonctionnements fondamentaux et document\u00e9s du processus \u00e9ditorial de ce journal. Un compte rendu d\u00e9taill\u00e9 de ces \u00e9v\u00e9nements est fourni par Chris Mooney dans Skeptical Inquirer<\/a> et The American prospect<\/a>, par David Appell dans Scientific American<\/a> et dans une br\u00e8ve d’information<\/a> de Nature. L’\u00e9diteur du journal lui-m\u00eame (Otto Kline), d\u00e9clara<\/a> finalement que \u00ab [les conclusions] ne peuvent \u00eatre tir\u00e9es de fa\u00e7on convaincante \u00e0 partir des preuves fournies dans l’article \u00bb.
\nUn autre journal qui (assez bizarrement) a aussi publi\u00e9 l’\u00e9tude de Soon et al, Energy and Environment, n’est en fait pas du tout un journal scientifique, mais un journal de science sociale. La r\u00e9dactrice, Sonja Boehmer-Christensen<\/a>, en d\u00e9fendant la publication de l’\u00e9tude de Soon et al, a \u00e9t\u00e9 cit\u00e9e par le journaliste scientifique Richard Monastersky dans Chronicle of Higher Education<\/a>. Elle confessait assez remarquablement : \u00ab Je suis mon programme politique \u2013 un peu en tous cas. Mais est-ce que ce n’est pas le droit du r\u00e9dacteur ? \u00bb<\/p>\n

Shaviv et Veizer (2003) ont publi\u00e9 un article dans GSA Today<\/a> o\u00f9 ils pr\u00e9tendent avoir \u00e9tabli une corr\u00e9lation entre le flux de rayons cosmiques et l’\u00e9volution de la temp\u00e9rature sur des centaines de millions d’ann\u00e9es. Ils concluent que la sensibilit\u00e9 du climat au CO2 est beaucoup plus faible qu’on ne le pense actuellement. L’article \u00e9tait accompagn\u00e9 d’un communiqu\u00e9 de presse intitul\u00e9 \u00ab le r\u00e9chauffement global n’est pas un ph\u00e9nom\u00e8ne du a l’action humaine <\/a>\u00bb dans lequel Shaviv dit : \u00ab la signification op\u00e9rationnelle de notre recherche est qu’une r\u00e9duction significative des \u00e9missions de GES n’abaissera pas sensiblement la temp\u00e9rature globale, puisque seulement un tiers du r\u00e9chauffement du si\u00e8cle pass\u00e9 devrait \u00eatre attribu\u00e9 \u00e0 l’homme \u00bb. Cependant, dans l’article, les auteurs disent en fait : \u00ab notre conclusion sur la dominance du flux des rayons cosmiques sur le climat n’est valide qu’\u00e0 l’\u00e9chelle de temps de nombreux millions d’ann\u00e9es. Sans surprise, une offensive de relation publique utilisa les conclusions s\u00e9rieusement erron\u00e9es du cit\u00e9es dans le communiqu\u00e9 de presse pour, une fois de plus, jeter le doute sur le consensus scientifique que les hommes influencent le climat. Ces pr\u00e9tentions furent contr\u00e9es dans un article dans Eos (Rahmstorf et al 2004) par une \u00e9quipe internationale de scientifiques et de g\u00e9ologues (dont quelques-uns uns de RealClimate), qui sugg\u00e9r\u00e8rent que les analyses de Shaviv et Veizer \u00e9taient bas\u00e9es sur des estimations non fiables et peu reproductibles, des ajustements s\u00e9lectifs des donn\u00e9es (d\u00e9calant les donn\u00e9es, dans un cas, de 40 millions d’ann\u00e9es) et qu’ils tiraient des conclusions ind\u00e9fendables, en particulier sur l’influence sur le climat des concentrations de GES d’origine humaine (voir par exemple cet \u00e9change entre deux groupes d’auteurs). Cependant, les conclusions trompeuses avaient d\u00e9j\u00e0 \u00e9t\u00e9 largement diffus\u00e9es lorsque cette information f\u00fbt publi\u00e9e. <\/p>\n

Ensuite, nous discutons le premier de trois articles \u00ab explosifs \u00bb qui devaient vider de leur substance les r\u00e9sultats du GIEC. Patrick Michaels et ses associ\u00e9s annonc\u00e8rent leur propre article (McKitrick et Michaels 2004, co-\u00e9crit par Ross McKitrick) de cette fa\u00e7on:
\n\u00ab Apr\u00e8s quatre ann\u00e9es du plus rigoureux des peer review jamais r\u00e9alis\u00e9s, le canadien Ross McKitrick et un autre d’entre nous (Michaels) publi\u00e8rent un article qui recherchait des signaux \u00e9conomiques dans les donn\u00e9es de temp\u00e9rature. La recherche montra qu’environ la moiti\u00e9 du r\u00e9chauffement dans les donn\u00e9es de surface des Nations unies \u00e9taient expliqu\u00e9es par des facteurs \u00e9conomiques, notamment les changements dans l’am\u00e9nagement du territoire, la qualit\u00e9 des instruments et la maintenance des enregistrements. \u00bb<\/p>\n

Il nous semble bizarre, pour dire le moins, qu’apr\u00e8s un des \u00ab plus rigoureux peer review<\/em> jamais r\u00e9alis\u00e9s \u00bb, aucune des personnes impliqu\u00e9es (ni le r\u00e9dacteur, ni les \u00e9valuateurs, ni les auteurs) ne semblent avoir saisi l’\u00e9norme erreur des auteurs, qui utilisent des degr\u00e9s plut\u00f4t que les radians qu’il aurait fallu utiliser pour calculer le cosinus utilis\u00e9 qui pond\u00e8re spatialement leurs estimations (**). Cette erreur rendait faux chaque calcul de l’article et invalidait les conclusions. A notre connaissance, toutefois, l’article n’a pas encore \u00e9t\u00e9 r\u00e9tract\u00e9. Remarquablement, il y avait dans cet article d’autres erreurs, ind\u00e9pendantes et \u00e9galement fondamentales, qui l’aurait rendu enti\u00e8rement invalide de toutes fa\u00e7ons. Au cr\u00e9dit du journal, ils publi\u00e8rent une critique de l’article par Benestad (2004). Cela ne surprendra personne d’apprendre que McKitrick et Michaels (2004) f\u00fbt publi\u00e9 dans Climate research et ne f\u00fbt g\u00e9r\u00e9 par personne d’autre que Chris de Frietas. <\/p>\n

Les deux autres articles \u00ab explosifs \u00bb furent publi\u00e9s dans le journal de l’AGU Geophysical Research Letters (GRL), qui publie 1500 articles par an. On peut estimer, de fa\u00e7on conservatrice, qu’ils publient 70% des articles re\u00e7us et qu’ils traitent donc probablement plus de 2000 articles par an. Ceci donne typiquement \u00e0 chacun des r\u00e9dacteurs du journal presque un article par jour \u00e0 \u00e9valuer. Si GRL publie beaucoup d’excellents articles et fournit \u00e0 la communaut\u00e9 scientifique un important forum pour la publication rapide de r\u00e9sultats importants, occasionnellement, des articles de mauvaise qualit\u00e9 passent \u00e0 travers les mailles du filet. Ces deux articles furent \u00e9crits par Douglass et ses collaborateurs (Douglass et al 2004a, 2004b), le premier avec Fred Singer comme co-auteur, le second avec Singer et Michaels. Les deux articles (***) soutiennent que les temp\u00e9ratures atmosph\u00e9riques r\u00e9centes ont baiss\u00e9, plut\u00f4t qu’augment\u00e9, sur base de l’analyse des donn\u00e9es d’une p\u00e9riode limit\u00e9e (1979-1996), qui \u00e9limine des p\u00e9riodes de r\u00e9chauffement significatif avant et apr\u00e8s et qui utilisent des donn\u00e9es de temp\u00e9rature controvers\u00e9es, obtenues par satellite, et dont la robustesse a \u00e9t\u00e9 mise en question par d’autres \u00e9quipes qui les ont analys\u00e9es. Une excellente discussion des deux articles est fournie par Tim Lambert. <\/p>\n

Un autre article pertinent de GRL fut l’article de Legates et Davis (1997) qui critiquait l’utilisation des \u00ab corr\u00e9lations centr\u00e9es \u00bb communes \u00e0 de nombreuses \u00e9tudes de \u00ab d\u00e9tection et attribution \u00bb qui soutiennent la d\u00e9tection de l’influence humaine sur les changements climatiques r\u00e9cents. Ils pr\u00e9tendaient que les corr\u00e9lations peuvent augmenter lorsque que les moyennes globales observ\u00e9es et simul\u00e9es divergent. Toutefois, comme indiqu\u00e9 dans le chapitre sur la \u00ab d\u00e9tection et attribution \u00bb des rapports du GIEC (2001)*, les corr\u00e9lations centr\u00e9es furent introduites pr\u00e9cis\u00e9ment pour cette raison: fournir un indicateur statistiquement ind\u00e9pendant des changements de temp\u00e9rature globale moyenne. Comme indiqu\u00e9 par le GIEC : \u00ab Si les changements de moyennes globales et les observations centr\u00e9es pointent vers la m\u00eame explication des changements observ\u00e9s, les preuves sont plus fortes que si chacun des indicateurs \u00e9tait corr\u00e9l\u00e9 s\u00e9par\u00e9ment \u00bb. De nouveau, une erreur de base dans les critiques faites par l’auteur des travaux pr\u00e9c\u00e9dents ne f\u00fbt pas identif\u00e9e lors du peer review. <\/p>\n

Ensuite, nous consid\u00e9rons l’article de Soon et al (2004), publi\u00e9 dans GRL, qui critique la fa\u00e7on dont les donn\u00e9es de temp\u00e9rature ont \u00e9t\u00e9 liss\u00e9es dans les rapports du GIEC et dans d’autres \u00e9tudes. Fid\u00e8les \u00e0 eux-m\u00eames, les \u00ab opposants \u00bb diffus\u00e8rent imm\u00e9diatement ces r\u00e9sultats comme invalidant les conclusions du GIEC. L’auteur principal Willie Soon \u00e9crivit lui-m\u00eame un article d’opinion \u00e0 ce sujet. A nouveau, quelques petits mois plus tard, l’un de nous (Mann 2004) publia un article de suivi qui invalidait les conclusions de Soon et al (2004), en d\u00e9montrant (avec des liens vers le code source en Matlab et les donn\u00e9es utilis\u00e9es) comment (a) les auteurs avaient, sans le mentionner, compar\u00e9 inappropri\u00e9ment des tendances calcul\u00e9es sur diff\u00e9rentes p\u00e9riodes et (b) n’avaient pas utilis\u00e9 les crit\u00e8res statistiques objectifs habituels pour d\u00e9terminer comment les s\u00e9ries de donn\u00e9es devaient \u00eatre trait\u00e9es pr\u00e8s du d\u00e9but et de la fin des donn\u00e9es. Il est malheureux qu’il ait fallu publier un article de r\u00e9action, car les erreurs de l’\u00e9tude originale \u00e9taient si graves qu’elle la rendait essentiellement sans valeur scientifique. <\/p>\n

Il y a d’autres exemples d’\u00e9tudes, parfois m\u00eame publi\u00e9es dans des revues de haute qualit\u00e9, dont on fit beaucoup de publicit\u00e9 \u00e0 l’\u00e9poque, mais qui r\u00e9trospectivement \u00e9taient erron\u00e9es (quoique pas aussi gravement que dans les exemples ci-dessus). Par exemple, Fan et al (1998), \u00e0 propos de la taille du puits de carbone du continent nord-am\u00e9ricain, r\u00e9fut\u00e9 par Schimel et al (2000). Ou la corr\u00e9lation entre la longueur du cycle solaire et le climat d\u00e9crite par Friis-Christensen et Lassen (1991), dont l’apparemment impressionnante corr\u00e9lation pour la deuxi\u00e8me moiti\u00e9 du 20\u00e8me si\u00e8cle dispara\u00eet si on ne change pas la m\u00e9thode de calcul de la moyenne \u00e0 mi-chemin (Laut 2003, Damon et Laut 2004). <\/p>\n

Le consensus scientifique actuel sur le changement climatique est bas\u00e9 sur des milliers d’\u00e9tudes (Google scholar donne 19.000 articles scientifiques pour la recherche sur la phrase \u00ab global climate change \u00bb). Toute nouvelle \u00e9tude sera un petit grain de preuve qui s’ajoute \u00e0 ce grand tas. Il fera l\u00e9g\u00e8rement \u00e9voluer la pens\u00e9e scientifique. La science proc\u00e8de ainsi, lentement et pas \u00e0 pas. Il est extr\u00eamement improbable qu’une nouvelle \u00e9tude r\u00e9volutionne toute la connaissance pass\u00e9e. M\u00eame si les conclusions de Shaviv et Veizer (20003), discut\u00e9es ci-dessus, avaient par exemple \u00e9t\u00e9 correctes, elles n’auraient constitu\u00e9 qu’une petite preuve, oppos\u00e9e \u00e0 des centaines d’autres la contredisant. Les scientifiques auraient trouv\u00e9 cette contradiction int\u00e9ressante et digne de plus amples investigations. Ils auraient r\u00e9alis\u00e9 d’autres \u00e9tudes pour identifier la source de cette contradiction. Ils ne jetteraient pas tout d’un coup tous les r\u00e9sultats pr\u00e9c\u00e9dents. Pourtant, on a souvent l’impression que le progr\u00e8s scientifique consiste en une s\u00e9rie de r\u00e9volutions o\u00f9 les scientifiques \u00e9cartent les vieux r\u00e9sultats chaque fois qu’un nouveau est publi\u00e9. C’est souvent parce que le grand public et les media ne connaissent, dans un domaine pr\u00e9cis, qu’une petite poign\u00e9e d’\u00e9tudes largement diffus\u00e9es, que celles-ci re\u00e7oivent une trop grande importance. Les nouveaux r\u00e9sultats sont souvent exag\u00e9r\u00e9ment mis en valeur (parfois par l’auteur, parfois par des groupes de pression) pour recevoir une couverture m\u00e9diatique. Les articles \u00ab explosifs \u00bb ont souvent des rat\u00e9s. <\/p>\n

Toutefois, m\u00eame si au d\u00e9but il peut \u00eatre gripp\u00e9, le processus du peer review<\/em> finit habituellement par fonctionner. Parfois, cela peut prendre du temps. Les observateurs seraient donc bien avis\u00e9s d’\u00eatre extr\u00eamement sceptiques face \u00e0 l’annonce, dans les media ou ailleurs, d’une nouvelle r\u00e9volution qui n’a pas encore \u00e9t\u00e9 minutieusement confirm\u00e9e par la communaut\u00e9 scientifique. <\/p>\n

* Note ajout\u00e9e le 21 janvier 2005 : Il est venu \u00e0 notre attention que Legates et Davis (1997)\u00e0 ont \u00e9t\u00e9 r\u00e9fut\u00e9 de fa\u00e7on similaire dans Wigley at al (2000).
\n** Note ajout\u00e9e le 21 janvier 2005 : McKitrick et Michaels ont publi\u00e9 un errata qui corrige l’erreur degr\u00e9s\/radians dans Climate Research 27, 265-268, qui montre d\u00e9sormais que la latitude est bien mieux corr\u00e9l\u00e9e avec la temp\u00e9rature que n’importe quelle donn\u00e9e \u00e9conomique.
\n*** Note ajout\u00e9e le 21 janvier 2005 : Chip Knappenberger remarque \u00e0 juste titre que le second article de Douglass et al ne pr\u00e9tend en fait pas que l’atmosph\u00e8re se refroidit. Nous retirons donc ce commentaire pr\u00e9cis, mais nous notons que le commentaire concernant l’utilisation s\u00e9lective des s\u00e9ries de donn\u00e9es et des p\u00e9riodes de temps reste valable. <\/p>\n

Bibliographie:<\/b><\/p>\n

Benestad, R.E., Are temperature trends affected by economic activity? Comment on McKitrick & Michaels.<\/a>, Climate Research<\/i>, 27, 171-173, 2004. <\/p>\n

Damon, P. E. and P. Laut, Pattern of Strange Errors Plagues Solar Activity and Terrestrial Climate Data<\/a>, Eos<\/i>, 85, p. 370. 2004<\/p>\n

Douglass, D. H., Pearson, B.D., and S.F.Singer, Altitude dependence of atmospheric temperature trends: Climate models versus observation, Geophys. Res. Lett., 31<\/i>, L13208, doi:10.1029\/2004GL020103, 2004.<\/p>\n

Douglass, D. H., Pearson, B.D., and S.F.Singer, Knappenberg, P.C., and P.J. Michaels, Disparity of tropospheric and surface temperature trends: New evidence, Geophys. Res. Lett., 31<\/i>, L13207, doi:10.1029\/2004GL020212, 2004, 2004.<\/p>\n

Fan, S., Gloor, M., Mahlman, J., Pacala, S., Sarmiento, J., Takahashi, T., Tans, P. A Large Terrestrial Carbon Sink in North America Implied by Atmospheric and Oceanic Carbon Dioxide Data and Models, Science<\/em> 282: 442-446, 1998.<\/p>\n

Friis-Christensen, E., and K. Lassen, Length of the Solar Cycle: An indicator of Solar Activity Closely Associated with Climate, Science 254, 698-700, (1991).<\/p>\n

Legates, D. R. and R. E. Davis, The continuing search for an anthropogenic climate change signal: limitations of correlation based approaches, Geophys. Res. Lett., 24<\/i>, 2319-2322, 1997.<\/p>\n

Laut, P., Solar activity and terrestrial climate: An analysis of some purported correlations, J.Atmos. Solar-Terr.Phys.,65, 801-812. 2003<\/p>\n

Mann, M.E., On Smoothing Potentially Non-Stationary Climate Time Series<\/a>, Geophys. Res. Lett., 31<\/i>, 2319-2322, L07214, doi: 10.1029\/2004GL019569, 2004.<\/p>\n

McKitrick, R., and Michaels, P.J., A test of corrections for extraneous signals in gridded surface temperature data., Climate Research<\/i>, 26, 159-173, 2004. <\/p>\n

Rahmstorf, S., D. Archer, D.S. Ebel, O. Eugster, J. Jouzel, D. Maraun, G.A. Schmidt, J. Severinghaus, A.J. Weaver, and J. Zachos, Cosmic rays, carbon dioxide, and climate<\/a>, Eos, 85, <\/i>, 38,41, 2004. <\/p>\n

Schimel, D., Melillo, J., Tian, H., McGuire, A. D., Kicklighter, D., Kittel, T., Rosenbloom, N., Running, S., Thornton, P., Ojima, D., Parton, W., Kelly, R., Sykes, M., Neilson, R. and Rizzo, B., Contribution of Increasing CO2 and Climate to Carbon Storage by Ecosystems in the United States, Science<\/em> 287: 2004-2006, 2000<\/p>\n

Shaviv, N, and J. Veizer, Celestial driver of Phanerozoic climate?, GSA Today, 13, <\/i>, 4-10, 2004. <\/p>\n

Soon, W., D. R. Legates, and S. L. Baliunas, Estimation and representation of long-term (>40 year) trends of Northern-Hemisphere gridded surface temperature: A note of caution, Geophys. Res. Lett., 31, <\/i>, L03209, doi:10.1029\/2003GL019141, 2004. <\/p>\n

Soon, W., and S. Baliunas, Proxy climatic and environmental changes over the past 1000 years, Climate Research<\/i>, 23<\/i>, 89-110, 2003.<\/p>\n

Soon, W., S. Baliunas, C, Idso, S. Idso and D.R. Legates, Reconstructing climatic and environmental changes of the past 1000 years, Energy and Environment, 14<\/i>, 233-296, 2003.<\/p>\n

Wigley, T.M.L, Santer, B.D and K.E. Taylor, K.E., Correlation approaches to detection, Geophys. Res. Lett.,<\/i>, 27, 2973-2976, 2000.<\/p>\n

<\/lang_fr><\/p>\n\n<\/div> ","protected":false},"excerpt":{"rendered":"

by Michael Mann and Gavin Schmidt On this site we emphasize conclusions that are supported by “peer-reviewed” climate research. That is, research that has been published by one or more scientists in a scholarly scientific journal after review by one or more experts in the scientists’ same field (‘peers’) for accuracy and validity. What is […]<\/p>\n","protected":false},"author":12,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[1,9,2,4],"tags":[],"class_list":{"0":"post-109","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-climate-science","7":"category-instrumental-record","8":"category-paleoclimate","9":"category-sun-earth-connections","10":"entry"},"aioseo_notices":[],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/109","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/comments?post=109"}],"version-history":[{"count":3,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/109\/revisions"}],"predecessor-version":[{"id":18078,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/109\/revisions\/18078"}],"wp:attachment":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/media?parent=109"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/categories?post=109"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/tags?post=109"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}