{"id":19100,"date":"2016-02-09T09:54:43","date_gmt":"2016-02-09T14:54:43","guid":{"rendered":"http:\/\/www.realclimate.org\/?p=19100"},"modified":"2016-02-09T10:05:38","modified_gmt":"2016-02-09T15:05:38","slug":"anti-scientists","status":"publish","type":"post","link":"https:\/\/www.realclimate.org\/index.php\/archives\/2016\/02\/anti-scientists\/","title":{"rendered":"Anti-scientists"},"content":{"rendered":"
\n

Ross McKitrick was so upset about a paper \u2018Learning from mistakes in climate research<\/em>\u2019(Benestad et al., 2015)<\/a><\/span> that he has written a letter of complaint<\/a> and asked for immediate retraction of the pages discussing his work. <\/p>\n

This is an unusual step in science, as most disagreements and debate involve a comment or a response to the original article. The exchange of views, then, provides perspectives from different angles and may enhance the understanding of the problem. This is part of a learning process.<\/p>\n

Responding to McKitrick’s letter, however, is a new opportunity to explain some basic statistics, and it’s excellent to have some real and clear-cut examples for this purpose.<\/p>\n

<\/p>\n

The ‘offensive’ paper (Benestad et al., 2015)<\/a><\/span> has supporting material that criticises some of McKitrick\u2019s publications (e.g. McKitrick and Michaels, 2004<\/a>), explaining why<\/em> they are flawed. However, this is only a very small part of the analysis presented therein.<\/p>\n

There is one crucial point that McKitrick seems to have missed, which is that nearby temperature trends are related because the trend varies smoothly over space. <\/p>\n

An important point made in (Benestad et al., 2015)<\/a><\/span> was that a large portion of the data in the analysis of McKitrick and Michaels (2004) came from within the same country and involved common information for the economic statistics (GDP, etc). In technical terms, we say that there were dependencies within the sample of data points. <\/p>\n

I repeated their work which was the basis for my original comment (Benestad, 2004<\/a>), and the point about dependencies and spatial correlation was made in this paper as well as in one of the first posts on RealClimate (Are temperatures affected by economic activities?<\/a>):<\/p>\n

In order to reduce the possible effect of inter-station dependency, the data were sorted according to latitude. Then, half of the data (latitudes from 75.5\u00b0 S to 35.2\u00b0 N) was used to calibrate the statistical models and the remaining data were used for evaluation (latitudes 35.3\u00b0 to 80.0\u00b0 N)… The negligence to account for inter-station dependencies in the analysis resulted in spurious results and inflated confidence levels in the analysis of McKitrick & Michaels.<\/p><\/blockquote>\n

McKitrick responded with a strange description of my approach as \u201can extreme version of the withholding test<\/em>\u201d. This view is strange because my choice was the only way to ensure that the split-sample would not give spurious results due to data dependency, as the quote above explains.<\/p>\n

McKitrick and Michaels had not carried out a proper split-sample test, as it involved dependencies between the calibration and evaluation samples, and they had used an approach which involved <\/p>\n

500 split sample withholding\/prediction tests in which 30% of the data were randomly withheld each time and predicted by a model fit to the remaining 70%.<\/p><\/blockquote>\n

This does not<\/strong> resolve the problem of interdependencies and is not proper a split sample test. Statistically, their approach favours splits where stations from the same region or the same country were prepresented in both the calibration and the evaluation samples.<\/p>\n

In his letter<\/a>, McKitrick also makes several false statements. One is that I predicted \u2018the Northern Hemisphere data from the (smaller) Southern Hemisphere<\/em>\u2019. <\/p>\n

The data set in question, however, contained more locations with temperature trends in the northern hemisphere, and an even split set the boundary for the two samples at 35.25\u00b0N. This was explained in the original comment (Benestad, 2004) and is apparent from the quote from it above.<\/p>\n

McKitrick also wrongly states that only the residuals matters in terms of autocorrelation, implying that the number of degrees of freedom<\/a> is not affected by the correlation of the values themselves. He claims to have proven so in a journal remote from natural sciences and climatology called Journal of Economic and Social Measurement<\/em>. However, his claim is unconvincing and this journal is not really relevant for climate research questions. <\/p>\n

He nevertheless insists that (here “SAC”=”spatial autocorrelation”)<\/p>\n

without providing any evidence, that SAC would reduce the effective degrees of freedom in MM04 sufficiently to undermine the significance of the conclusions.<\/p><\/blockquote>\n

His claim is obviously false, as the analysis in my original comment did indeed provide evidence that McKitrick was wrong about the autocorrelation. The figure below provides an illustration and is from my original comment. <\/p>\n

\"Results<\/a>
Results of regression analyses with different models using data from stations within the latitude range 75.5\u00b0 S to 35.2\u00b0N to calibrate the model and stations in the latitude range 35.3\u00b0 to 80.0\u00b0 N and corresponding depending variables for prediction and evaluation. The unit along the y-axis is \u00b0C\/decade (adapted from from Benestad (2004) with original figure caption).<\/figcaption><\/figure>\n

The grey points in the figure show the temperature trends (\u201cSTREND\u201d) and the coloured ones are the results of the regression model. The results show that the economic variables cannot explain the trends at the stations in the evaluation sample (station index greater than 110).<\/p>\n

The code of this demonstration is in the replicationDemos-software<\/a> available from Figshare.<\/p>\n

McKitrick’s view on autocorrelation is also wrong because in theory, you could get uncorrelated residuals from a model prediction for large number with high autocorrelation where the magnitude of the residuals are much smaller than the predicted values. <\/p>\n

His letter also reveals that he continues to fail to grasp other crucial points, and that the observation is still valid that he <\/p>\n

failed to address the important question of how many PCs were included in the calibration and how much of the variance they could describe<\/p><\/blockquote>\n

Although he cites his own paper in a journal with a questionable reputation, he fails to address the question in his response. Rather, he has a hang-upon the particular shape of the leading principal component. <\/p>\n

Again, the crucial point is the number of principal components that were included in the regression analysis and the proportion of the variance that they account for. This is just mathematics.<\/p>\n

Another way to interpret McKitrick’s letter is that it is an attempt to silence scientists. This is the second attempt by McKitrick to act as a gatekeeper (he himself falsely accused climate scientists of exactly this<\/a>). He managed to prevent the paper from being published on an earlier occasion when it was submitted to Climatic Change<\/em>.<\/p>\n

Attempts to censor articles with which one disagrees may have become more commonplace in terms of motivated rejection of well-established scientific propositions (Lewandowsky & Bishop, 2016)<\/a><\/span>.<\/p>\n

A Norwegian organisation that challenges the IPCC (\u201cklimarealistene\u201d) has twice tried to silence my public discourse on climate science and my comments on the Humlum paper<\/a> by writing to the director of my institute.<\/p>\n

Attempts to silence criticism runs against the principles of science.<\/p>\n

References<\/h2>\n
    \n
  1. <\/a>\nR.E. Benestad, D. Nuccitelli, S. Lewandowsky, K. Hayhoe, H.O. Hygen, R. van Dorland, and J. Cook, \"Learning from mistakes in climate research\", Theoretical and Applied Climatology<\/i>, vol. 126, pp. 699-703, 2015. http:\/\/dx.doi.org\/10.1007\/s00704-015-1597-5<\/a>\n\n\n<\/li>\n
  2. <\/a>\nS. Lewandowsky, and D. Bishop, \"Research integrity: Don't let transparency damage science\", Nature<\/i>, vol. 529, pp. 459-461, 2016. http:\/\/dx.doi.org\/10.1038\/529459a<\/a>\n\n\n<\/li>\n<\/ol>\n\n<\/div> ","protected":false},"excerpt":{"rendered":"

    Ross McKitrick was so upset about a paper \u2018Learning from mistakes in climate research\u2019 that he has written a letter of complaint and asked for immediate retraction of the pages discussing his work. This is an unusual step in science, as most disagreements and debate involve a comment or a response to the original article. […]<\/p>\n","protected":false},"author":11,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[41,1,38,34,45],"tags":[],"class_list":{"0":"post-19100","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-climate-impacts","7":"category-climate-science","8":"category-scientific-practice","9":"category-skeptics","10":"category-statistics","11":"entry"},"aioseo_notices":[],"post_mailing_queue_ids":[],"_links":{"self":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/19100","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/comments?post=19100"}],"version-history":[{"count":68,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/19100\/revisions"}],"predecessor-version":[{"id":19175,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/posts\/19100\/revisions\/19175"}],"wp:attachment":[{"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/media?parent=19100"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/categories?post=19100"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.realclimate.org\/index.php\/wp-json\/wp\/v2\/tags?post=19100"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}