RealClimate logo

Technical Note: Sorry for the recent unanticipated down-time, we had to perform some necessary updates. Please let us know if you have any problems.

Lindzen and Choi Unraveled

Filed under: — group @ 8 January 2010

Guest Commentary by John Fasullo, Kevin Trenberth and Chris O’Dell

A recent paper by Lindzen and Choi in GRL (2009) (LC09) purported to demonstrate that climate had a strong negative feedback and that climate models are quite wrong in their relationships between changes in surface temperature and corresponding changes in outgoing radiation escaping to space. This publication has been subject to a considerable amount of hype, for instance apparently “[LC09] has absolutely, convincingly, and irrefutably proven the theory of Anthropogenic Global Warming to be completely false.” and “we now know that the effect of CO2 on temperature is small, we know why it is small, and we know that it is having very little effect on the climate”. Not surprisingly, LC09 has also been highly publicized in various contrarian circles.

Our initial reading of their article had us independently asking, how we could have missed such explicit evidence of the cloud feedback as shown in LC09? Why would such a significant finding have gone undiscovered when these feedbacks are widely studied and recognised as central to the projections of climate change? We discovered these common concerns at a meeting last year and then teamed up to address these questions.

With the hype surrounding the manuscript, one would think that the article provides a sound, rock solid basis for a reduced climate sensitivity. However, our examination of the study’s methods demonstrates that this is not the case. In an article in press (Trenberth et al. 2010 (sub. requ.), hereafter TFOW), we show that LC09 is gravely flawed and its results are wrong on multiple fronts. These are the major issues we found:

  • The LC09 results are not robust.

    A goal of LC09 was to quantify the cloud feedback by examining variability in top-of-atmosphere (TOA) radiative fluxes in the tropics as it relates to variability in mean sea surface temperature (SST). To do this they examine only tropical data. In general, they find that during periods of higher-than-normal SST, the radiation emitted and reflected to space by the earth goes up as well, cooling the Earth and amounting to an overall negative climate feedback. To show this, they select intervals of warming and cooling (in a time series of monthly averaged values) and compare fluxes at their endpoints (see Figure). They didn’t provide an objective criterion for selecting these endpoints and in some instances (see their Fig. 1), the selection of these intervals actually appears to be quite odd.

    Fig. 1: Warming (red) and cooling (blue) intervals of tropical SST (20°N – 20°S) used by LC09 (solid circles) and an alternative selection proposed derived from an objective approach (open circles) (TFOW, 2010).

    The result one obtains in estimating the feedback by this method turns out to be heavily dependent on the endpoints chosen. [edit] In TFOW we show that the apparent relationship is reduced to zero if one chooses to displace the endpoints selected in LC09 by a month or less. So with this method the perceived feedback can be whatever one wishes it to be, and the result obtained by LC09 is actually very unlikely. This is not then really indicative of a robust cloud feedback.

  • LC09 misinterpret air-sea interactions in the tropics

    The main changes in tropical SST and radiative fluxes at TOA are associated with El Niño-Southern Oscillation (ENSO) and are not necessarily indicative of forced variability in a closed system. ENSO events cause strong and robust exchanges of energy between the ocean and atmosphere, and tropics and subtropics. Yet LC09 treat the tropical atmosphere as a closed and deterministic system in which variations in clouds are driven solely by SST. In fact, the system is known to be considerably more complex and changes in the flow of energy arise from ocean heat exchange through evaporation, latent heat release in precipitation, and redistribution of that heat through atmospheric winds. These changes can be an order of magnitude larger than variability in TOA fluxes, and their effects are teleconnected globally. It is therefore not possible to quantify the cloud feedback with a purely local analysis.

  • More robust methods show no discrepancies between models and observations

    In TFOW, we compute correlations and regressions between tropical SSTs and top-of-atmosphere (TOA) longwave, shortwave and net radiation using a variety of methods. LC09 found the observed behavior to be opposite from that of 11 atmospheric models forced by the same SSTs and conclude that the models display much higher climate sensitivity than is inferred from ERBE. However, in our analysis comparing these relationships with models, we are unable to find any systematic model bias. More importantly, the nature of these relationships in models bears no relationship to simulated sensitivity. That is, the metric developed by LC09 is entirely ineffective as a proxy for simulated sensitivity.

  • LC09 have compared observations to models prescribed with incomplete forcings

    The AMIP configuration in the model simulations used by LC09 have incomplete forcings. The AMIP protocol started off a test only of how an atmospheric model reacts to changes in ocean temperatures, and so models often only use the ocean temperature change when doing these kinds of experiments. However, over the period of this comparison, many elements – greenhouse gases, aerosols, the sun and specifically, volcanoes changed the radiative fluxes, and this needs to be taken into account. Some models did this in these experiments, but not all of them.

    For instance, the dominant source of variability in the reflected solar flux arises from aerosols associated with the eruption of Mount Pinatubo in June of 1991 yet all but 2 model simulations examined by LC09 omit such forcings entirely. Other radiative species are absent from the models altogether. It is thus obviously inappropriate to expect such model simulations to replicate observed variability in TOA fluxes.

  • LC09 incorrectly compute the climate sensitivity

    By not allowing for the black body radiation (the Planck function) in their feedback parameter, LC09 underestimate climate sensitivity. Using the correct equations, LC09 should obtain a feedback parameter and climate sensitivity of -0.125 and 0.82 K, respectively, rather than their values of -1.1 and 0.5 K.  In contrast, TFOW results yield a positive feedback parameter and greater sensitivity estimate, though we also caution that this approach is not a valid technique for estimating sensitivity, as a closed and therefore global domain is essential (though not by itself sufficient). Lastly, LC09 fail to account for variability in forcings in estimating sensitivity.

While climate models are known to struggle with many aspects of tropical climate, especially in regards to its coupled variability, the problems claimed by LC09 are not among them. Forster and Gregory [2006] and Murphy et al. [2009] address changes in the energy budget with surface temperatures for a much larger domain and present a much more complete and defensible analysis and discussion of issues. They demonstrate that recent observed variability indeed supports a positive shortwave cloud feedback. So the feedbacks from processes other than the Planck function response are clearly positive in both observations and models, in contrast to LC09’s conclusions. Moreover, it is not appropriate to use only tropical SSTs and TOA radiation for feedback analysis as the transports into the extratropics are substantial. Any feedback analysis must also recognize changes in ocean heat storage and atmospheric energy transport into and out of the tropics which are especially large during ENSO events. While the tropics play an important role in determining climate sensitivity, simplistic and arbitrary analyses of tropical variability can be grossly misleading.


Forster, P. M. F., and J. M. Gregory (2006), The climate sensitivity and its components diagnosed from Earth Radiation Budget Data, J. Clim., 19, 39–52
Lindzen, R. S., and Y.-S. Choi (2009), On the determination of climate feedbacks from ERBE data, Geophys. Res. Lett., 36, L16705, doi:10.1029/2009GL039628.
Murphy, D. M., S. Solomon, R. W. Portmann, K. H. Rosenlof, P. M. Forster , and T. Wong (2009), An observationally based energy balance for the Earth since 1950, J. Geophys. Res., 114, D17107, doi:10.1029/2009JD012105.
Trenberth, K. E., J. T. Fasullo, Chris O’Dell, and T. Wong, (2010): Relationships between tropical sea surface temperature and top-of-atmosphere radiation. Geophys. Res. Lett., 37, doi:10.1029/2009GL042314, in press.

61 Responses to “Lindzen and Choi Unraveled”

  1. 51
    H. Tuuri says:

    Chris, thank you for looking at this matter. Unfortunately, I do not have a copy of your paper. But I found seminar slides that may explain the confusion:

    I understood from the slides that Wong et al. do NOT use the 7-month smoothing to the long wave flux. (LC09) say at one point:

    “Instead, the moving average with a 7-month
    smoother was used for the SWR anomalies alone;”

    But later (LC09) say:

    “Table 1 compares net DFlux/DSST for intervals for
    which DSST exceeded 0.1, 0.2 K,. . . , for 3, 5, and 7 month
    time smoothing, for all monthly intervals.”

    Since the variation in the long wave flux is quite big from month to month, it makes perfect sense to use the 7-month smoothing for it, too. Using unsmoothed values at endpoints would produce huge variations for the slope from month to month.

    My naked eye analysis yesterday ‘smoothed’ the ERBE graph in Figure 1a and compared it to the SST graph, looking for correlation. Thus, it looks like (LC09) have used 7-month smoothed values for both the short wave flux and the long wave flux, and that makes perfect sense.

    Now that the this confusion is settled, we can look at the what kind of delta-Flux (total flux) / delta-SST we will get from the graphs. The biggest events in delta-SST happened in years 1986 – 1990, and in 1997 – 1999.

    A naked eye analysis of Figure 1a in (LC09) for the 1986 – 1990 event gives roughly: delta-Flux = 4 W/m^2 and delta-SST = 0.6 K.

    A naked eye analysis of Figure 1a in (LC09) for the 1997 – 1999 event gives roughly: delta-Flux = 4 W/m^2 and delta-SST = 0.7 K.

    For both events, delta-Flux / delta-SST is around 6 W/m^2 per K. I have not yet checked what the naked eye says about the graphs in Wong’s seminar slides nor can I look yet at the graphs in your paper.

    Is there delta-Flux data for 2000 – 2009? That could act as a test for the (LC09) discovery. There was a major La Nina event two years ago.

  2. 52
    Completely Fed Up says:

    “The only way LC09 could be wrong is that their ERBE graph is incorrect.”

    Or that’s a result of selection bias.

    Not uncommon: TGGWS did that a lot.

  3. 53
    Ray Ladbury says:

    H. Tuuri says, “If LC09 are correct that delta-Flux per delta-SST is 6 W/m^2 per K, that is a surprising result. A simple black body radiation model would give just 4 W/m^2 per K. That is, the Earth would resist warming even stronger than a black body, which in turn would mean that a doubling of CO2 might warm the Earth less than 1 K.”

    I rather doubt the atmosphere is lasing. If they are seeing more than black-body radiation, that in and of it self trips my BS meter.

  4. 54
    Hank Roberts says:

    > doubt the atmosphere is lasing

    Well, for _earth_’s boring old atmosphere.
    But there are more things in heaven …. (grin)

  5. 55
    H. Tuuri says:

    Ray, (comment #53) there is no paradox in having delta-Flux/delta-temperature bigger than for a black body. Earth’s surface temperature is on the average +15 C, but for a satellite, Earth appears as a black body whose temperature is -15 C. This difference is exactly the greenhouse effect caused by Earth’s atmosphere. If 1 degree increase in the surface temperature lifts the apparent temperature of Earth more than 1 degree, that is, higher than -14 C, then a satellite will observe delta-Flux bigger than for a black body. This would mean that the greenhouse effect diminishes as the surface temperature increases.

    If Earth would be without an atmosphere, then the surface temperature and the apparent temperature would be equal. In that case, delta-Flux/delta-temperature could not be higher than for a black body.

    Lindzen and Choi have tried to measure the effect of a tropical sea surface temperature rise on the greenhouse effect of the atmosphere. El Nino offers us a rare opportunity to measure it, as El Nino suddenly warms tropical seas up to 0.7 K. El Nino acts like a massive heater which is suddenly turned on. LC09 studies the effect on the radiation from tropical latitudes. To get a more complete picture, we should get radiation measurements from all latitudes. Tropical latitudes only cover about 1/3 of Earth’s area. Also, note that El Nino only allows us to study climate feedbacks that take at most months to happen.

  6. 56
    H. Tuuri says:

    Chris, (comment #50) I have now looked at the graphs in where you are a co-author.

    You seem to be right that LC09 may have used a wrong sign in their shortwave delta-flux.

    Let me analyze the 1986-1990 and 1997-1999 events based on the graphs on page 7 of your seminar slides. I use a naked eye analysis where I do a 6-month average to smooth the fluxes, AND I smooth also the SST over 6 months. Both should be smoothed so that we compare apples to apples.

    A crucial question: how reliable are the flux graphs? In 2006 there was an orbital correction made, and the older graphs in LC09 still lack some data. Are the latest graphs reliable now? Another question is how high did the long wave flux in 1998 shoot? It looks to surpass the scale.

    The graph on page 7 of your seminar slides shows that the short wave flux (that is, sunlight reflected from the atmosphere) slightly decreased at the top of the 1987 El Nino event. Maybe there were more clouds then? That offsets some of the gain in the long wave flux. During the 1998 massive El Nino, the short wave flux varied but, on the average, stayed at the normal level.

    Looking at the 1986-1990 event, the NET flux roughly follows the SST graph, where the variation of delta-NET flux is roughly from +2 W/m^2 to -1 W/m^2 when delta-SST changes from +0.35 K to -0.3 K. We get:

    delta-NET flux / delta-SST = 3 W/m^2 / 0.65 K = 4.6 W/m^2 per K.

    The 1997-1999 event is more complicated, as there is a short huge spike in the NET flux in 1998, while SST changes in a more smooth way. We could interpret the spike as a delayed ‘negative feedback’ that suddenly hits with a big force when a threshold in warming is surpassed. To compare apples to apples, we need to smooth that huge spike away so that the SST graph and the NET flux graph roughly follow each other. My naked eye would then give the top of delta NET flux at +2 W/m^2 in 1998, and the bottom at -2 W/m^2 in 1999. This corresponds to a delta-SST change from +0.6 K to -0.2 K. We get:

    delta-NET flux / delta-SST = 4 W/m^2 / 0.8 K = 5 W/m^2 per K.

    For both events, the value above exceeds the black body delta-flux/delta-SST, which is about 4 W/m^2 per K. Thus, the above qualitative analysis suggests that the Earth ‘climate sensitivity’ in the tropics is lower than for a black body. Please take into account the caveats that the tropics only cover 1/3 of the area of Earth, and that only the feedbacks that materialize in a few months have time to affect the results.

    This plot shows that the El Nino / La Nina of 2007-2008 might give us more data:
    Are there flux measurements for that period?

  7. 57
    Chris O'Dell says:

    H. Tuuri, you can get ERBE flux data averaged over various spatial and time scales at this website:
    (click item 8 to get the actual data)
    I recommend registering as a Langley user though if you do this.

  8. 58
    H. Tuuri says:

    Chris, thank you. I am now looking at the data at your link. The net flux of heat from the tropics to cooler areas is an interesting variable. A very quick look suggests that El Nino / La Nina have little effect on that heat flux. That would mean that the tropics handle themselves the extra heat coming from El Nino. In a sense, the tropics are a ‘closed system’ with respect to the extra heat emanating from El Nino.

  9. 59
    H. Tuuri says:

    Chris, I have now analyzed the monthly tropical (20 S – 20 N) data at

    I imported the data to a free statistics program called OpenStat. Since I was not able to find a numeric table of the Reynolds and Smith OISST v2 product, I ‘digitized’ by hand the graph on page 8 of

    I used OpenStat to compute moving averages over 12 months (I used option ANALYSES -> Autocorrelation in OpenStat). With shorter moving averages, the NET flux graph contains too much noise, though the delta-SST graph is smooth also with shorter moving averages.

    I analyzed the 1986-1990 sequence of El Nino/La Nina, as well as the 1997-1999 massive El Nino.

    1) For the 1986-1990 event, the variation in the 12 month moving average of the NET flux is from 345 W/m^2 to 347 W/m^2. The variation of the 12 month moving average of delta-SST is -0.18 K to 0.16 K. We get:

    delta-NET flux / delta-SST = 2 W/m^2 per 0.34 K = 6 W/m^2 per K.

    2) For the 1997-1999 event, the 12 month moving average of the NET flux at the start of 1997 is 344 W/m^2. It rises to a local maximum of 345 W/m^2, and later spikes at 346 W/m^2. That is, even the 12 month smoothed Net flux is not smooth at all, but has a double maximum. The 12 month moving average of delta-SST rises from -0.03 K to +0.22 K. We get:

    delta-NET flux / delta-SST = 1 W/m^2 OR 2 W/m^2 (depending on which of the double maxima we choose)
    per 0.25 K = 4 W/m^2 OR 8 W/m^2 per K.

    Conclusion: we get results that agree with Lindzen and Choi in LC09. We had to use a very long moving average of 12 months to smooth the NET flux enough for our graphical analysis.

    I used OpenStat to compute also the regression between delta-SST and the transfer of heat from the Tropics to higher latitudes. The correlation coefficient is -0.005 and the slope is -0.32. That is, there is essentially no correlation. An El Nino event in the Tropics does not cause the transfer of heat to increase. That is, the extra heat from El Nino is handled locally in the Tropics.

    I will next do further research on the NET flux from 60 S to 60 N, and its correlation to HadCRUT3 monthly global temperature anomalies. I will check if I can repeat the results of Forster and Gregory:

    The data and the programs that I used are available by email from me, if someone wants to repeat my calculations.

  10. 60

    38 David B. Benson,

    Thanks for your useful links and comments. Sorry to have been distracted so as to be late to acknowledge your help.

    I especially appreciate that you heard my question.

  11. 61
    Fred Moolten says:

    Perhaps this comment will come too late to be read, but I think it worth posting. Much of the LC09 analyses derives from ENSO changes, and I would argue that such changes are largely irrelevant to climate sensitivity to CO2. In fact, feedbacks to El Nino warming might well be lower than those to GHG warming, and even net negative.

    There are many possible factors involved, including the complexity of ENSO events, which involve changes in wind and ocean currents as well as energy exchanges between tropics and subtropics. Rather than invoke all of these, let me suggest one major principle that implies a likely disparity between sensitivity to El Nino warming and warming mediated by CO2. When SST warming is an initiating event, occurring in the absence of preceding atmospheric warming, the warmed water will lead to increased evaporation and a consequent increase in relative humidity while atmospheric temperature is still lagging in response to the elevated SST (typically months), leading to increased low cloud formation – a phenomenon known to be associated with El Nino. The resulting increase in cloud albedo can explain an increase in outgoing SW radiation of the kind described in LC09, and serve as an important negative feedback. Consider, however, a warming event originating in the atmosphere – e.g, through an increase in CO2. In this case, the atmospheric warming will tend to drive RH in the opposite direction, thereby tending to reduce low cloud formation and the associated albedo. This will be counteracted by warming of the ocean, leading to increased atmospheric water vapor, but the evaoporation will be a “catch up” phenomenon that might restore RH toward earlier values but not exceed those values. The evidence for long term reduction in low clouds over the course of decades of global warming is controversial, but some data in fact support such a reduction, and none I’m aware of imply an increase. In this case, the additional atmospheric water will principally serve to contribute a positive feedback to the warming, in concordance with climate models and observations.

    Although the particular difference emphasized here – between an El Nino initiated warming with consequent RH increase, and a GHG initiated warming with a decrease or minimal change in RH – will not be a complete explanation, it illustrates the principle that the two events cannot be expected to exhibit similar sensitivity responses, particularly in regard to SW flux, which was a critical element of the LC09 analysis.

Switch to our mobile site