Welcome to the new version of European Tribune. It's just a new layout, so everything should work as before - please report bugs here.

The Tip of the Hockey Stick

by Nomad Tue Feb 7th, 2006 at 04:47:17 PM EST

About one year ago, January 2005, an article of Steve McIntyre and Ross McKitrick was published in the Geophysical Research Letters with harsh criticism on the symbol of our changing climate: the Hockey Stick graph. It was a defining moment: after an uphill struggle their work had at last been accepted in a well respected science magazine for climate research. Although I've followed the ensuing debate with a half eye throughout the past year, I've been curious to where it was going and how well the Hockey Stick of Mann et al. would fare. The briefest synopsis: it's a trench-war out there.

Back from the front page.


Introduction
Although I suspect many on this forum are familiar with the Hockey Stick and its history, I provide a generalized (and incomplete) introduction. The Hockey Stick is the adopted pet-name of a scientific graph from the team revolving around climatologist Micheal Mann, based in Penn State University (Pennsylvania). The Hockey Stick graph, constructed on the basis of climate indicators (called proxies) such as tree rings or pine cones, shows a dramatic increase in the mean temperature of the Northern Hemisphere with time, and especially in the last part of the 20th century. These findings vindicated the long held hypothesis that the earth was rapidly heating up by anthropological influence, meaning, us and our industrious industries. In fact, the Hockey Stick graph took it one step further: the Earth had not been experiencing such high temperatures since nearly 800 years! An alarming figure, especially combined with the fact that the Earth is a system that has a slow response to changing climate conditions: hence, the worse was yet to come. For many years, the Hockey Stick was the defining symbol for climate research after it was showcased in the 2001 report of the UN's Intergovernmental Panel on Climate Change (IPCC) - the highest consensus on Climate Change. Lamentably, politicians only offered the Kyoto protocol as the solution to stem the tide.

Because of its widespread acceptance, the Hockey Stick quickly became a symbol, an icon which could be taken down - which contrarians never managed to. Their saving grace came with the emergence of McIntyre and McKitrick -two men without any previous professional experience in climate science - and their 2005 publication in the Geophysical Research Letters.

McIntyre, a statistician with a working history in mining industry, once remarked that he took an interest in the Hockey Stick since the graph reminded him of the figures that financial managers were wont to present to him: in general, they were too upbeat to be true. What they discovered was startling: several aspects of the methodology by Mann's group, namely how they had treated the data statistically, were dramatically flawed. It really wouldn't have mattered what data the team of Mann would've entered into their statistical model - the population decline of pandas, the stock market (although they didn't try those) - there would be a hockeystick figure as outcome nearly every time. Pandas would never decline and the stock market would not go down - and of course that isn't realistic. To make matters worse, McIntyre and McKitrick also were doubtful about the proxies that were used, specifically certain series of cone pines (link is to McIntyre's blog where he has several critical posts on cone pine series). The cone pines series particularly (but others as well) showed strong hockeystick trends which could also be explained by effects other than increasing temperatures - e.g. increased fertilisation or even logging.

One Year Later...
We are one year further since the publication of McIntyre and McKitrick, whose original publication of last year can be found here (pdf!): McIntyre & McKitrick. I thought it was time to make an assessment of the health of the Hockey Stick of Mann et al. Is it Hockey Stick of Hockey Schlock?

In tackling the debate about the Hockey Stick, I should begin with a few starting statements which already zoom in on the troubles surrounding the debate.

  1. For the layman, the actual arguments are virtually incomprehensible. For those trained in Earth Sciences (like myself), the debate is a similar hard nut to crack, as the main battle of the Hockeystick debate lies within statistics. I believe it will ultimately be professional statisticians that will decide this issue.
  2. A second point I wish to highlight is the reporting in the press. As was commented by myself and others previously on this forum, in reporting science the daily press are, frankly, a disaster. Science reporters have a tendency to simplify the problem or re-phrase an item so it gets drawn out of context - especially on a topic such as the Hockey Stick where sound bites just won't cut it. If you'd believe the papers, McIntyre and McKitrick no longer matter.
  3. And finally, the Hockey Stick debate has not only raised fine points concerning the Hockey Stick itself, but also about the way how climate science, or science in general, should be done with proper checks and balances. The problems with the Hockey Stick should've been spotted far, far earlier - before their publications in Nature and the IPCC report - and yet they weren't. The Hockey Stick debate is also a debate how climate science policy should change. This, however, I'd like to keep a discussion for another day.

Principally in science, the best hypothesis wins. In so many words: you're only right for as long as you're not disproved. It took a while before the gravity laws of Newton were replaced by an even better model describing the universe, but in the end, Einstein knocked them out. This just goes to show that the longer your hypothesis stand, the more credibility you gain. In this, the criticism of McIntyre and McKitrick gathers more impetus. After one year of receiving heavy flak, they're still around and the first signs indicate that 2006 will become as momentous as was 2005.

Despite the victorious boasts on the Real Climate website that the Hockey Team is the only team still standing in the field, the truth is not so narrow. It should be strongly kept in mind that the Real Climate website is created and fully run by -the authors- that wrote all the articles surrounding the IPCC Hockey Stick graph. Therefore, anything said on Real Climate concerning the attack on the Hockeystick should be approached with a critical mind in place. Partly in response to RealClimate's constant snide, McIntyre set up his own website, Climate Audit.

Critics on the criticism
In the course of 2005, at least 4 remarkable papers appeared in the literature which relate back to the debate around the Hockey Stick. All these papers were written by different authors from different institutes and seemed to have no previous connection to Mann et all., or McIntyre & McKitrick. Two papers were specifically aimed to reconstruct the criticism of McIntyre and McKitrick. I will briefly pass through three of them.

A first paper which tested the methods of McIntyre and McKitrick was written by geophysicist Peter Huybers, based in the Woods Hole Oceanographic Institute. His preliminary article before publication can be found here (pdf!).

Huybers addressed specifically two points, both statistical issues: the method of calculating principal component and RE statistics. Although I've read the basis behind these methods, I am far from being an authority to explain them to others, and so I won't. By and large, however, Huygens corroborated the criticism of McIntyre and McKitrick, although he regarded their claim somewhat "exaggerated". Remarkably, the response at RealClimate was rather jubilant:

...further calculations will take time to assess, but of the original claims in MM05, the first (the PC normalisation issue) demonstrably makes no difference to the reconstruction, and the second (the calculation of the significance of the RE statistic) was just wrong. So for this round at least, it looks like 'Hockey Team: 2, MM: 0'.

But that isn't what Huybers says, though. His summary reads:

MM05 (nb, McIntyre and McKitrick) show that the normalization employed by MBH98 (nb, Mann et al. in 1998) tends to bias results toward having hockey-stick-like graphs, but the scope of this bias is exaggerated by the choice of the normalization and errors in the RE critical value estimate. Those biases truly present in the MBH98 temperature estimate remain important issues,...

A conclusion from Huybers should be that, the methodology of the Hockey Team is biased and should be properly investigated. The different statistical approach didn't rule that out and doesn't justify the "rightness" of the chosen statistical model by the Hockey Team. So far for the Real Climate interpretation.

Fiercer were Eugene Wahl and Caspar Ammann in their more recent re-examination:

Our examination does suggest that a slight modification to the original Mann et al. reconstruction is potentially appropriate for the early 15th century (~ +0.05°), which leaves entirely unaltered the primary conclusion of Mann et al. (as well as many other reconstructions) that both the 20th century upward trend and high late-20th century hemispheric surface temperatures are anomalous over at least the last 600 years. Our results are also used to evaluate the separate criticism of reduced amplitude in the Mann et al. reconstructions over significant portions of 1400-1900, in relation to some other climate reconstructions and model-based examinations. We find that, from the perspective of the proxy data themselves, such losses may be smaller than those reported in other recent work.

The trouble for the non-mathematicians, again, is that the discussion revolves around arguments which are so specialised that there's a small chance for the lay-man to understand. Even so, last week McIntyre and McKitrick released their response (pdf) to Ammann and Wahl, and reading through it, even for the lay-man it is clear that they were merciless in breaking down Ammann and Wahl brick for brick. And never mind that Ammann has worked closely together with Mann.

In many ways, Ammann and Wahl repeated the points of Huybers, but also pulled observations of McIntyre and McKitrick out of context which then resulted in a larger article. (Larger doesn't mean better.) Similarly to Huygens, the arguments of Ammann and Wahl do not specifically address the problems that McIntyre and McKitrick point out in their 2005 paper. Actually, McIntyre and McKitrick turn the table on them: when the Hockey Stick data is tested after the criteria of Ammann and Mahl, the Hockey Stick data fails - and is shown unfit to use once more.

Robustness
If that weren't enough to provide some backing behind the arguments of McIntyre and McKitrick, a previous IPCC heavyweight Gerd Bürger, together with Ulrich Cubasch, pitched in recently with their Geophysical Research Letter publication, last December. Their article describes how they test the "robustness" of the statistical model of the IPCC Hockey Stick. By this, they mean that they are testing how sensitive the statistics of the Hockey Team are when exposed to small variations in their methodology. The robustness is an important point for a climate model, and an issue that relates back to the earliest criticism of McIntyre and McKitrick on the 1998 Hockey Stick. It turns out that Bürger and Cubasch find quite an enormous spread in the results, signifying that the statistical approach of the Hockey Stick team for their Hockey Stick is not robust at all.

A figure from their work:

The black line in the figure, marked MBH in the index, is the famous Hockey Stick graph. The figure shows that there is an enormous variation in the data plots when the statistical methods are tweaked slightly. And once again, Bürger and Cubasch criticise Mann et al for their statistical approach concerning rescaling.

The response of the Hockey Team is feeble at best, as they counter that the arguments which Bürger and Cubasch raise are now no longer relevant and no longer of current concern as they've recently tested other methods laid out in a different publications.

And this leads me to a last and rather tiresome aspect in the Hockey Stick debate: the way how the debate is held. In regard to the criticism of Bürger and Cubasch and the honest points raised in the Daily Kos thread, the authors of Real Climate dodge the raised criticism, they don't deflect or parry. Yet this is an approach which is starting to get noticeable and, personally, looks to me as a one trick pony to ignore the actual debate. There is a certain moral arrogance to it, especially in combination with the haughty tone, and it doesn't slot well with the scientist in me. And again, the expressed doubts about the proxy series still have not been taken into serious regard.

Similarly, McIntyre has expressed his irritation on his own blog about the way how the Real Climate team often rewrites the real criticism. Instead of addressing the point in case, the Hockey Team re-cast what they think is the addressed criticism and then triumphantly torch that down. The RealClimate responses on the critical Daily Kos bystander reflect that as well. In this way they seem to win each debate, while they're not even addressing the issue at hand. It's a trick politicians know as well; it's called spin. And on a gut-level it doesn't bide well for the Hockey Stick team.

So far, the war in the trenches. It's pretty grubby down there: this is a war wherein one group proclaims that their method is perfectly right, and the other side says that it is not. The lesson here is: don't try to mingle unless you're a Marine. And by Marine, I mean a Mathematician or Statistician. Bottom-line, though: this debate is not over. For anyone who is (scientifically) critical, the Hockey Stick figure can no longer be regarded as carved in stone. In this respect, 2006 has every potential to be another stormy year.

Display:

A final note. Whatever you may think, this debate is never about the denial of global warming. Personally, I think that the current warming of the earth is pretty much established at all possible fronts. The Hockey Stick debate places the current global warming in a historic context and asks a legitimate question: Compared to our history, what is the scale of the recent global warming? The Hockey Stick says, "Oh boy, it's bad." McIntyre & McKitrick say, "Not so fast!" And in that regard it remains a thoroughly relevant discussion. However you may feel about our rampant CO2 production and despite my personal feelings it should be prudent to bring that monstrosity to a halt, McIntyre and McKitrick raise good questions which need good answering. Something that hasn't been provided thus far.
by Nomad on Thu Feb 2nd, 2006 at 07:35:15 AM EST
I did some read-up and checked out some links of yours.

For now I have one criticism: what you quote from RealClimate on Huybers is actually a joint comment on that article and one by von Storch. (Elsewhere, Storch states: "We are speaking about the shaft of the hockey stick, not the blade.")

*Lunatic*, n.
One whose delusions are out of fashion.

by DoDo on Thu Feb 2nd, 2006 at 09:36:34 AM EST
[ Parent ]
Would've been the fourth article of interest to dissect, but it made this diary too repetitive and I finally dropped it. I did suspect someone would pick it up why I only discussed three articles and left the fourth one out. I do not wish to tell what others can tell much better, and that's my excuse for the link-fest that has become this reply.

Though it should be said that, similarly to Huybers, also von Storch corroborated the work of McIntyre & McKitrick, right down into their press release (pdf). More importantly, McIntyre does not agree with the statistics Von Storch used to emulate the Mann et al data - rendering the results of Von Storch moot.

His official reply, in the Geophysical Research Letters, is here (pdf). He elaborates further on it on Climate Audit, in posts such as these.

by Nomad on Thu Feb 2nd, 2006 at 10:05:00 AM EST
[ Parent ]
Further criticism: McIntyre has an education in mathematics, but he is better characterised as a businessman (indeed was so characterised by himself). Also, he has good relations to the climate sceptics, and his critics seem to include authors not connected to Mann (see here).

Also, while the RealClimater's tone in the DailyKos thread is improperly emotional, I can understand his point after I looked up the 2005 article s/he refers to (pdf!).  "juangdsi" was pointed to a graph (Figure 2) as proof that the comparison s/he missed was in that article, to which "juangdsi" responds that that figure doesn't explain much to him/her - however, from page 2320 right bottom (pdf page 13), you'll find a subchapter focusing on exactly that issue. (I won't attemt to evaluate it for now, I'd have to mull over the original McIntyre article first, but you may want to comment it.) And McIntyre on his blog makes it appear as if the RegEC method was only used by Mann et al from 2005, tough references go back to 2003 and 2001; and McIntyre states at the end of this that he hasn't even read the 2005 article yet(!) and other, earlier material sent to him(!!) and claims the 2005 article uses the same criticised dataset tough page 2321 (pdf page 14) explicitely explains otherwise(!!!).

*Lunatic*, n.
One whose delusions are out of fashion.

by DoDo on Thu Feb 2nd, 2006 at 10:39:01 AM EST
[ Parent ]
McIntyre has an education in mathematics, but he is better characterised as a businessman (indeed was so characterised by himself). Also, he has good relations to the climate sceptics, and his critics seem to include authors not connected to Mann

I don't see what criticism there would be in working in business. They make excellent scientists afterwards. Some don't, I reckon, but that's not the point. And in the diary I already mentioned that most of the criticisms I use here was produced by scientists not connected to Mann. I later discovered that Ammann might be a possible exception, so perhaps I should've used Von Storch instead. My mistake.

As for the DailyKos response, I do have a problem with tone from the Real Climate people. They know their stuff better than the bystander; they might as well direct juangdsi to the information instead of sending him/her off to one figure. I find it highly unprofessional to start snubbing, especially when you're better trained. But this has nothing to do with the science itself.

and claims the 2005 article uses the same criticised dataset tough page 2321 (pdf page 14) explicitely explains otherwise(!!!).

No, he doesn't. First of, McIntyre already plunged into the Rutherford 2005 article on which he already created critical posts on Climate Audit. Secondly, as McIntyre also pointed out to me by E-mail, he refers to the specific criticism levelled by Bürger and Cubasch - which specifically talk about the methodology and the statistical approach. Again, this discussion is two-pronged: 1) the reliability of the dataset and 2) the unleashed statistical methods to makes sense of them. Lies, lies, statistics. You know the story.

And now I'm off for a while, but please go on...

by Nomad on Thu Feb 2nd, 2006 at 11:26:30 AM EST
[ Parent ]
Regarding the businessman/statistician issue, I was responding to what I saw as an argument from authority from you - it wasn't meant as an ad hominem (hence my reference to his education). I agree that outsiders can produce great science (this is especially true in my former field astronomy; and let's also recall something from your field, that geology was revolutionalised by Wegener, a meteorologist).

I also agree (and already indicated so) that the RealClimater's tone was unacceptable. And not only for a professional - RealClimate.org is supposed to communicate science to the public, so s/he should have patience even if being right.

Your last paragraph is not a reply to the point you quoted, but a previous one, but upon re-reading I submit you are correct on that McIntyre remark and I didn't read with enough attention. Having read more of the debate since, I also share the suspicion that there are unresolved yet unacknowledged problems of both the methodological and reliability of dataset types. On the other hand, from my reading, of the Bürger & Cubasch criticisms, only that the tree ring dataset goes well beyond the calibration range has continued validity (but that one seems a strong one).

*Lunatic*, n.
One whose delusions are out of fashion.

by DoDo on Fri Feb 3rd, 2006 at 12:15:25 PM EST
[ Parent ]
Here is the 200-year version of the comparison of the latest published simulations from Wiki:

The link gives the references to all. What I would like to see evaluated by someone more closer to the debate (e.g. you, juangdsi or someone else lurking) is:

  1. Which of the authors of these are not part of the "Hockey Team"?
  2. How do the models not involving Mann or Jones (especially the last three: the red-orange, red and dark-red ones on the graph) fare in terms of methodological prudence?
  3. How do the datasets used for the models not involving Mann or Jones  (especially the last three) differ from those involving Mann or Jones?
  4. Does this graph miss other recent (post-2001) published simulation results which still show a Medieval Warm Period warmer than today?


*Lunatic*, n.
One whose delusions are out of fashion.
by DoDo on Fri Feb 3rd, 2006 at 12:43:24 PM EST
[ Parent ]
1) Well, I don't know Esper et al (2002) but for the rest beside the three last ones: I think all of them.

2.) I can't tell you. I've read the Moberg article, but it was a while ago. And even if I had read it recently, I can't tell you. I'm not expert enough myself to make sincere judgements about their methodology, and I am not aware of experts who have made them - although you could bet they are around. For this diary I wanted especially to focus on the Hockey Team vs McIntyre & McKitrick, so I left other articles in the drawer to continue suck up dust...

3.) Good question. That would take some time to research, but I can try to do that. Keep in mind though that it took me more than 3 weeks to analyse the articles I wanted to use for the above diary. I have a preliminary answer for Oerlemans, though, since he's from the IMAU in Utrecht, practically three doors down from where I was: he's specifically into ice-cores. I believe he also used isotope data (not too sure) but that would bring a whole new house of cards with it.

4.) That's quite a specific question and again one I can't answer since the Hockey Team finds so many simulations with a very low MWP.

Some superfluous notes. Although I don't think I need to point this out to you (DoDo), I want to draw attention to the exceptional large variations present in the wiki graphs. The lack of overlap between different authors is quite astonishing. Secondly, these spider diagrams make it incredibly hard to keep track of one simulation. I really recommend to study each one separately as well and you can note then how much the temperature anomaly deviates in time. Thirdly, of the last three listed authors Huang and Oerlemans can't answer the question on the Medieval Warm Period as their data doesn't go back far enough. Of course they do confirm the global warming trend.

Finally, a question to you: with your background in astrophysics, do you have some comments about the growing evidence that the sun's activity has constantly increased at least the past 100 years, and possibly longer?

by Nomad on Sat Feb 4th, 2006 at 05:04:40 AM EST
[ Parent ]
exceptional large variations present in the wiki graphs

Indeed, tough with respect to the central issue of the debate, the hockeystick blade, they are all similar. I note Rutherford (2005) mentions a Gonzalez-Rouco (2003) article as one that got a result significantly different from theirs, I wonder if you can find their graph.

This leads me to another thought regarding Bürger & Cubasch. They made the point about interpretation of proxy data well beyond the calibration range in the context of their finding of lack of robustness. However, may it be that the deviations outside of the calibration range, f.e. for tree rings, are predominantly in the direction towards sizes corresponding to colder local temperatures? I.e., that the model error is large when the real temperature was lower?

with your background in astrophysics, do you have some comments about the growing evidence that the sun's activity has constantly increased at least the past 100 years, and possibly longer?

This is more a field mixing astrophysics with nuclear science and geology (isotopes in ice cores). The last studies I am aware of (checking my record, the latest is Solanki et al 2004 and comments to it in the German GEO magazine) tracked solar activity back into the latter part of the first millennium in a detailed way and less detailed back to the last ice age, showed that last 70 years' activity is exceptional, and showed correlation with the temperature models (in particular the MWP), but the correlation broke down in the last 30 years. However, I may not be up to date. (After a first cursory search, I found that another team using another isotope found the recent activity less exceptional, i.e. similar levels were reached not thousands but hundreds of years ago, while Solanski et al. think that research was problematic.)

*Lunatic*, n.
One whose delusions are out of fashion.

by DoDo on Sat Feb 4th, 2006 at 06:42:03 AM EST
[ Parent ]
While climate change is a small world, and several authors cross-linked in different publications, it is far more interesting to look at the differences in results.

After MBH98, the first publication which differs in amplitude of the other temperature reconstructions was for borehole temperatures by Huang ea., 2000. This revealed a much cooler LIA of app. -1 degr.C, vs. the -0.2 degr.C in the shaft of the hockeystick of the MBH98 reconstruction. This is important, as that has consequences for the impact of natural changes (especially solar) on current and future temperature changes. Mann ea., 2003 optimised the Huang data (but corrected this partly in 2004), but Pollack and Smerdon protested against this optimisation.

The second reconstruction with a larger variability (and a high MWP) was from Esper ea., 2002. He used a different method for (regional) calibration of tree rings (see Cook ea., 2004), which retains long-term trends better than the method used by MBH98, but also a different calibration period. The effect of this can be read in Esper, Wilson and Briffa, 2005. Mann ea. reacted on the Esper publication, and Cook and Esper responded to this critique.

While Esper's reconstruction is only based on extratropical NH tree ring sites, Moberg ea., 2005 used different non-tree ring proxies for the long-term temperature trend and tree rings only for superposed short term temperature variations, effectively downplaying the influence of tree rings on the total reconstruction.

The reconstructions of Esper, Moberg and Oerlemans, 2005 (the latter based on glacier length records), all show a difference of app. 0.8 degr.C between the LIA and the mid-2000 temperatures (and a MWP app. at the same height of around 1950) vs. a difference of only 0.2 degr. C in MBH98 and other reconstructions. This is discussed by Esper, Moberg, Luterbacher and others, 2005 for its importance towards future climate expectations. The variability in the pre-industrial period was mainly from two sources: volcanic and solar. Volcanic gives a long-term average variation of maximum 0.1 degr.C over the past 600 years (see Fig. 6 from Briffa ea., 1998), that means that solar changes had an influence between 0.1-0.9 degr.C (the latter for the borehole reconstruction). This makes a hell of a difference, as the recent upswing since 1850 is in large part GHG driven (with a 0.1 degr.C solar past) or solar driven (with a 0.9 degr.C solar past), as in both cases, the instrumental temperature variation must be explained with a different ratio between GHG/aerosols at one side and (enhanced) solar at the other side (of course besides the accuracy of the instrumental temperature record). Thus we still are in need of (far) more accurate reconstructions of the past millennium to have a better view on what can be expected in the future...

by FerdiEgb on Sun Feb 5th, 2006 at 10:05:17 AM EST
[ Parent ]
In fact, I'll go on and suggest you create your own dairy about this. I'm not too sure how happy Jerome will be when we start de-railing ET into yet another climate focused website, but you are given some exceptional insight into this issue and seem better informed than I am. After DoDo's previous reply on solar activity I had already decided that it was definitely another topic I wanted to go into. You've just facilitated that promise. Thanks.

When I've fully digested your post, I think this diary is scrolled off the frontpage, though. Hence again, I urge you to create a separate diary entry - although you may want to wait after the Cartoon Controversy has died out a little...

by Nomad on Sun Feb 5th, 2006 at 12:39:52 PM EST
[ Parent ]
OK, I will wait a few days and start a new diary on the solar-climate connection and what climate models make of that...

Ferdinand

by FerdiEgb on Sun Feb 5th, 2006 at 04:02:53 PM EST
[ Parent ]
Excellent. I'll eagerly anticipate it.
by Nomad on Sun Feb 5th, 2006 at 05:10:15 PM EST
[ Parent ]
Look, MBH98 is under fire. As a reaction, M&B&H (but mostly M) simply refer to RegEM as being the up-to-date method. As a proof, they present Figure 2 which shows both methods producing similar results. Don't you think there is a missing link here in the logic? - Why did they switch to RegEM? And what will come next?

Juangdsi

by juangdsi on Fri Feb 3rd, 2006 at 04:32:53 AM EST
[ Parent ]
My past experience in another field (astrophysics) is that you can switch from using one model to another because either of

  • more precise results,
  • easier-to-handle programme (faster computation, more adjustable parameters, more potential patterns captured etc.) while results are similar,
  • to explore what a new method can do.

From the words of your ill-tempered correspondent at DailyKos, as well as some remarks in the 2005 Rutherford et al article ("perform
well even in the presence of nonstationary climate forcing", "This is easily done in the REGEM method by treating those values as missing, something that could not have been done in Mann et al. (1998)"), it seems to me it was mostly a combination of the latter two. Chapter 4 of Schneider (2001) also implies so (italics and bolding mine):

Conventional techniques for the imputation of missing
values in climate data, such as [...] and Mann et al. (1998, 1999) impute missing values in surface temperature data, differ from the regularizedEM algorithm in two respects.
First, the conventional techniques neglect the interdependence of the imputed values and the estimated statistics of the data. If the statistics of the data are estimated from only a small portion of the records in the dataset, it is possible that in filling in missing values with imputed values, long-term climate variability is underestimated. The regularized EM algorithm reduces such an underestimation of long-term climate variability by allowing for the dependence of the estimated statistics of the data on all available values in the dataset [...]
Second, the conventional techniques regularize [...] by one or the other form of truncated principal component analysis, in which typically a single truncation parameter, or a single discrete regularization parameter, is chosen for an entire dataset. [...] In the regularized EM algorithm, ridge regression regularizes the ill-posed or ill-conditioned estimation of regression coefficients, which, as argued in section 3a, might offer advantages over regularization methods that are based on truncated principal component analyses. Moreover, the continuous regularization parameter of ridge regression is chosen adaptively by generalized cross-validation, such that the regularization adapts to the density and the pattern of the available values in each incomplete record.
In that the above-cited conventional techniques estimate the statistics of the data under consideration only from a subset of the available data and regularize the ill-posed regression problems nonadaptively, the conventional techniques can be viewed as approximations to the regularized EM algorithm. Thus, on theoretical grounds one would expect that the regularized EM algorithm yields imputed values that are at least as accurate as the values imputed with one of the conventional techniques.

This is followed by the much-discussed quote about the lack of a general justification for calling the RegEM method "optimal" in a general sense relative to others and the need for a test in practice. After doing a simple test with only 3.3% missing data, Schneider extrapolates:

...that the regularized EM algorithm already in this relatively simple test led to more accurate imputed values than the technique of Smith et al. suggests that in more complex tests the regularized EM algorithm would also perform better than conventional noniterative imputation techniques...

As I read it, the first batch of error estimates and following model comparisons in the Rutherford article were an example of a more complex test (which may or may not be based on false assumptions).

In the DailyKos thread, you quote Schneider to question whether his simulations can be extrapolated to cases with much more missing points. However, I find your Schneider quote was a specific conclusion from the problem of inputation error underestimation. Indeed the very next paragraph claims the applicability of RegM just or what MDH(98) did:

The regularized EM algorithm can [...] be used to construct historic surface temperature datasets from proxy data. [...] Some of the variables in a dataset might represent surface temperature values at the nodes of a spatial grid and other variables might represent proxies of the surface temperature (cf. Mann et al. 1998).


*Lunatic*, n.
One whose delusions are out of fashion.
by DoDo on Fri Feb 3rd, 2006 at 03:23:57 PM EST
[ Parent ]
...what would happen if the same criteria of Bürger and Cubasch are tested on the RegEM model. A new method that improves on some points of the former one does not imply that it needs automatically to improve on the flaws of the prevrious one. (See, for instance, Windows ;)
by Nomad on Fri Feb 3rd, 2006 at 05:24:45 PM EST
[ Parent ]
That last sentence was horrible. I had a long day.
by Nomad on Fri Feb 3rd, 2006 at 05:25:42 PM EST
[ Parent ]
I have not followed your discussion but I have to say that the Expectation Maximization algorithm is quite intuitive, robust, and has good convergence properties.

A society committed to the notion that government is always bad will have bad government. And it doesn't have to be that way. — Paul Krugman
by Migeru (migeru at eurotrib dot com) on Fri Feb 3rd, 2006 at 05:38:22 PM EST
[ Parent ]
Ah, at last a mathematician around who has the experience!

If it doesn't take up too much of your time, would you read for us just the Bürger & Cubasch article, and give us your assessment whether and how it would be applicable to regularized EM too?

*Lunatic*, n.
One whose delusions are out of fashion.

by DoDo on Fri Feb 3rd, 2006 at 05:59:15 PM EST
[ Parent ]
I'll try to find time tomorrow.

What the EM algorithm is not is model-independent.

A society committed to the notion that government is always bad will have bad government. And it doesn't have to be that way. — Paul Krugman

by Migeru (migeru at eurotrib dot com) on Fri Feb 3rd, 2006 at 06:02:59 PM EST
[ Parent ]
Yep, that may well be the case (I put "which may or may not be based on false assumptions" in because of that), I would have to further immerse myself into that. But two scientific articles not in my field in one day are enough :-)

*Lunatic*, n.
One whose delusions are out of fashion.
by DoDo on Fri Feb 3rd, 2006 at 06:04:29 PM EST
[ Parent ]
The thing is I am fully happy with RegEM as a nice method to fill-in missing data (although not being half as nice as EM). But at Daily Kos it served as a "deus ex machina" for the mounting critique of MBH98. In that context, only the first of your 3 possible reasons to switch is valid. One would have to define "more precise results" though, for example, by "keeping the error small". BTW, the RegEM algorithm is much more complicated than MBH98.

In terms of parameter load (which is known to inflate the model error) both are comparable and are thus subject to the B&C critique at GRL. That paper suggests, as a possible remedy, "more sophisticated regularization schemes". But sophistication alone is certainly not a warrant against data processing troubles.

As for the "complex test", just calculate the missing ratio of the Rutherford and Schneider tests and compare it with the ratio of the millennial application.

by juangdsi on Fri Feb 3rd, 2006 at 06:25:06 PM EST
[ Parent ]
But at Daily Kos it served as a "deus ex machina" for the mounting critique of MBH98. In that context, only the first of your 3 possible reasons to switch is valid.

Well, if 2003 was its first published application by the Mann team, the switch was certainly not deus ex machina and was not motivated by a mounting critique of MBH98.

BTW, the RegEM algorithm is much more complicated than MBH98.

Only the algorithm itself, or its handling too? (One example I had in mind when I wrote the three criteria was a new celestial mechanics simulation, a rather complicated iterative one, but errors reduced faster than with classical methods.)

Your other criticisms certainly sound valid, and not addressed by RealClimate.

*Lunatic*, n.
One whose delusions are out of fashion.

by DoDo on Fri Feb 3rd, 2006 at 06:48:50 PM EST
[ Parent ]
Lets put it this way: If they had switched in 2003 because of provably greater robustness, I would have welcomed that argument at Daily Kos. Since that is not the case, it is a deus.

What do you mean by "handling"? - I would love to see errors being reduced faster for RegEM; but till now we have to live with some vague handwaving. But due to the nonlinear character of RegEM the derivation of error bounds (on the model coefficients!) is, I'm afraid, out of reach, at least for the current proponents.

by juangdsi on Sat Feb 4th, 2006 at 08:15:51 AM EST
[ Parent ]
Lets put it this way: If they had switched in 2003 because of provably greater robustness, I would have welcomed that argument at Daily Kos.

My point exactly.

Juangdsi, from your posts I get the impression you've been working on RegEM. I did some searches on RegEm, but could you direct me perhaps to some sites with some detailed description of what it does and how it is build, if you know any? I'd appreciate it.

by Nomad on Sat Feb 4th, 2006 at 08:52:16 AM EST
[ Parent ]
The Schneider article (IIRC chapters 3-4) detail it.

*Lunatic*, n.
One whose delusions are out of fashion.
by DoDo on Sat Feb 4th, 2006 at 10:47:49 AM EST
[ Parent ]
Thanks, it's on my list...
by Nomad on Sun Feb 5th, 2006 at 12:45:08 PM EST
[ Parent ]
Thanks for pitching in, juangdsi. It's much appreciated.
by Nomad on Sun Feb 5th, 2006 at 12:47:17 PM EST
[ Parent ]
Can someone post a link to the DailyKos thread you all are referring to please?

Thx
Luke

-- #include witty_sig.h

by silburnl on Fri Feb 3rd, 2006 at 01:07:31 PM EST
[ Parent ]
It is the last link in the diary, but here is it again.

*Lunatic*, n.
One whose delusions are out of fashion.
by DoDo on Fri Feb 3rd, 2006 at 03:28:47 PM EST
[ Parent ]
As someone who often uses exponential acceleration (information, innovation, population growth, automobiles - you name it) as a flippant lead-in to my little provocative seminar piece on the moral dilemmas facing the West today, I avoid jolly hockey sticks like a black and white striped ref intervening in a dispute on the ice.

I prefer to quote the Maya - whose knowledge was woven into threads (shades of ET) that no statistician could unravel. But they did mark out 2012 as the time when it will all come to a head, and the planet will pass into the Age of Light - whatever that might mean...

 But, Nomad, I shall be using the Mantle analogy ;-)

Good diary - don't let my flippancy disturb you, I'm recovering from driving home through a blizzard with too much 'meeting coffee' inside me.

You can't be me, I'm taken

by Sven Triloqvist on Thu Feb 2nd, 2006 at 11:48:14 AM EST
Interesting review and discussion of this issue in the kind of detail that is so helpful but will require more time for me to digest. In the meantime I wondered how this relates to a short piece I read some time ago in The Economist, sorry it may be an example of the poor science reporting you refer to it's hard for me to judge. I'll try to get the main points quoted here but I'm including the link to the whole article as well:


Heat and light
Aug 11th 2005
From The Economist print edition

An unexplained anomaly in the climate seems to have been the result of bad data

.... one of the most curious uncertainties of all is the apparent discrepancy between what is happening to temperatures at the Earth's surface and what is happening in the troposphere--the lowest layer of the atmosphere, and thus the part that is in contact with that surface....

Three papers published in this week's issue of Science suggest that [something wrong with the data.]...

The first of these studies, conducted by Steven Sherwood of Yale University and his colleagues, examined data from weather balloons....

data from radiosondes come with built-in inaccuracies. For example, their thermometers, which are supposed to be measuring the temperature of the air itself (that is, the temperature in the shade) are often exposed to, and thus heated by, the sun's rays. To compensate for this, a correction factor is routinely applied to the raw data. The question is, is that correction factor correct?....

Dr Sherwood and his colleagues hit on a ruse to test this idea. Because weather stations around the world release their balloons simultaneously, some of the measurements are taken in daylight and some in darkness. By comparing the raw data, the team was able to identify a trend: recorded night-time temperatures in the troposphere (night being the ultimate form of shade) have indeed risen. It is only daytime temperatures that seem to have dropped. Previous work, which has concentrated on average values, failed to highlight this distinction, which seems to have been caused by over-correction of the daytime figures. When the team corrected the erroneous corrections, the result agreed with the models of the troposphere and with records of the surface temperature. The improvement was particularly noticeable in the tropics, an area that had previously appeared to have high surface temperatures but far cooler tropospheric temperatures than had been expected.

The second piece of work looked at satellite measurements of tropospheric temperatures.... Dr Mears and Dr Wentz plugged this observation into a model, and the model suggested that the apparent cooling the satellites had observed is indeed such a spurious trend. Correct for orbital decay and you see not cooling, but warming.

The third paper, by Ben Santer of the Lawrence Livermore National Laboratory in California and his colleagues, argues that it is, indeed, errors in the data that are to blame for disagreements between the predictions of computer models about how the troposphere should behave and what the weather balloons and satellites actually detect. Dr Santer's team compared 19 different computer models. All agreed that the troposphere should be getting warmer. Individual models have their individual faults, of course....

by Alexandra in WMass (alexandra_wmass[a|t]yahoo[d|o|t]fr) on Thu Feb 2nd, 2006 at 12:36:31 PM EST
Hi Alexandra,
Thanks for the reply and the article; I don't read The Economist, so this was something new to me. The article itself seems fine and not something that would throw me into a tizzy. I've been worked up about press articles, but certainly not this one.

I was aware of the work of Sherwood and for more from him on the particular study, I'll refer back Real Climate (not all is bad over there) where they gave Sherwood their own platform in this post. I didn't know the other two studies, when I've time I'll see whether I can look them up.

Yet as to how it relates to this diary: hardly any. The reporting in the Economist is all about defining the modern global warming of today. All these measurements would be included to shape the tip of the hockeystick - which is also based on direct, modern measurements. The debate of McIntyre & McKitrick and Mann c.s. deals with the "blade" of the hockeystick - something DoDo hinted on in an above response - since the blade is formed by data from proxies. The question is: how does that (unmeasured) data relate to our current well measured global warming? It is the connection between the two that's the big issue.

Climate science is hard enough as it is when people want to do measurements today. Yet when you factor in the component of relative or absolute time, you quadruple the complications. Nor does it help that this branch of science has become politicised to an absurd degree. But this is me grumbling.

by Nomad on Thu Feb 2nd, 2006 at 06:33:58 PM EST
[ Parent ]
Glad the references might come in handy even if it's just the tip of the hockey stick we're talking about here.
by Alexandra in WMass (alexandra_wmass[a|t]yahoo[d|o|t]fr) on Fri Feb 3rd, 2006 at 09:21:00 AM EST
[ Parent ]
wow, nomad, that was a lot of legwork, thanks...

I have to admit that (as an engineer rather than a mathematician, fixated on solution rather than theory) I don't find the debate all that deeply engaging.  it seems to me we have plenty of evidence that:

a) we are already in trouble wrt glaciers, coral reefs, plankton populations, snowcaps, storm intensity, drought, etc. -- we don't have to wait for the blade, the shaft of this here hockey stick is already getting too warm for comfort.

b) hysteresis in the global climate system is enormous, so any remedial action we take will be slow to get a grip;  therefore regardless of the actual slope we can expect several decades of worsening conditions before our braking action is felt.

and therefore

c) it is pretty darned urgent that we Do Something About It.

I don't really see how the hockey stick grudge match outcome alters the case -- (c) still seems a pretty solid conclusion to me, so long as (a) and (b) stand.

comment?

The difference between theory and practise in practise ...

by DeAnander (de_at_daclarke_dot_org) on Thu Feb 2nd, 2006 at 09:33:04 PM EST
I thoroughly hope I don't sound too condescending with this one, I'll go out on a limb somewhat. See, I think there's a good reason why comedy writer Terry Pratchett displays his wizards as a male-only society, and has witches find solutions through logic. Actually, the older I get the noticeable it becomes that men are hobnobbing until they're white around the nose or until they all are agreed. More often than not, the latter never happens. And in exact sciences this is (sadly?) even more true. I also suspect that the concept of democracy is fatally flawed if there aren't enough women representatives, but that in an aside.

There are different sorts of philosophies to tackle the phenomenon of global warming. The dispute around the hockey-stick is a purely academic one. Yours sound the more practical one - and I don't mean to say it's not based on academics, it is. The practical approach sounds, to me, as the pro-active one: yes, it's starting to look bad, and yes, we might be a factor, so let's do something.

However, this discussion relates back to that other big problem in climatology: to what degree is our climate changing under the influence of anthropogenic CO2 and to what degree to other forcings? It's another dispute, for another day, but the Hockey Stick ties in with it. The Hockey Stick by Mann, if true, is giving us an idea how much forcing humans already have added to the earth. If, however, as McIntyre and McKitrick argue, this is unreliable and we see large variations throughout our recent history, the answer becomes completely ambiguous. Humans are reduced back to ants, only able to tip the scale a little, until Mom and Dad return home. And when that happens, then we're in real trouble. But the above is a case for not overreacting and looking for different and better methods to reconstruct the CO2 and temperature history. I fully agree with that. But then, I'm a male.

In another aside, I'm still with you when it comes to reducing CO2 - but for another reason. In a recent and thought-provoking essay in which you contributes as well, rdf wrote that all models of our economy are, in the end, finite and that the earth will pay for it if we don't change that. I think you've argued yourself that we should not produce a certain amount of waste materials per head - and I agree. It's unnatural, or if it is natural, parasitic at best. So, in that line of philosophy, I think we agree.

by Nomad on Fri Feb 3rd, 2006 at 06:24:44 AM EST
[ Parent ]
It's Headology :-)  and I almost named my boat Granny Weatherwax, so you've definitely struck a cultural chord here.  There is something rather Ponder Stibbons-y about the hockey stick wars :-)

Anyway... I think there are many debates going on at once over climate change...  it is not single issue, though wrangles like this one take place down at the single-issue level.  Many enviros have leapt at global warming as the only major environmental issue to get any press (thanks partly to maddeningly awful materials like that "Day After Tomorrow" film, ugh) -- anything to get Joe Public to wake up and smell the coffee, as it were.  So it gets played up as if it were the Only Big Issue.

Then there is enormous motivation to pick holes in arguments of climate urgency -- if it's the Only Big Issue then debunking it takes the heat off and we can all just drive our SUVs and live hoggishly ever after, right?  Which suits the agenda of profiteers and marketeers and ordinary people who don't want to know about bad news, so there's strong bias in that direction counteracting the natural human tendency to love a good dramatic crisis.  And generates disproportionate interest and press for scholarly wrangles at high levels of detail that might "invalidate this" or "debunk that", thus toppling the Big Issue.  (and incidentally enabling us to ignore all the other  Big Issues lined up right next to it).

I'm stating the obvious here, but Big Science tends to be reductionist because it's the only way we can get at the detail necessary for solid conclusions... as far back as the late 1800s Kipling affectionately mocked the learned gentlemen of the Royal Society who spent their lives earnestly attempting to slice into thinner and thinner sections the left eye of the female mosquito [approximate quote].  We are torn between our real and urgent need for Hockey Stick wrangles, for correct methodologies and minute, scrupulous review of our data sets and our inferences -- and our real and urgent need for long views, big ideas, and major paradigm challenges.

The difference between theory and practise in practise ...

by DeAnander (de_at_daclarke_dot_org) on Fri Feb 3rd, 2006 at 08:57:11 PM EST
[ Parent ]
Headology, that's the one. I was noddig to myself all the way along your paragraphs up until your last one, so I'm not sure how much I can add. You put in a nut-shell why I've my love-hate relationship with the science reporting in the press (perhaps it's more hate than love) and how people respond to it. In geology (that's my basis), when you set young students loose on an outcrop, they will always focus on the pebbles, the fossils, the things that tend to catch the eye: the dramatic part - while those things generally form only 10% of the rock. The other 90% doesn't get that attention. You really need to train yourself to look further than that flashy 10 percent.

As for the last paragraph, I stopped nodding because you put a new thought in my head. I think much of modern science goes exactly the way as the Hockey Stick wrangle: the slow, relentless tinkering exposed to the criticism of thousands. But long views, big ideas and new paradigms are exciting and tickle our fantasy pleasure spot - on which the other variant also can lodge again. Good things to ponder about on a Saturday morning.

by Nomad on Sat Feb 4th, 2006 at 04:13:48 AM EST
[ Parent ]
There is no greater compliment than to be told one has put a new thought in anybody's head -- heartfelt thanks for that kind word Nomad.  I try to put new thoughts in my own head as often as possible, too :-)  though sometimes the poor old head does get tired.

The difference between theory and practise in practise ...
by DeAnander (de_at_daclarke_dot_org) on Sat Feb 4th, 2006 at 09:14:42 PM EST
[ Parent ]
In the global warming debate, the following quote looks appropriate:

Oh, I don't worry about America. They always do the right thing - after exhausting every other possibility.

(Winston Churchill)

by das monde on Fri Feb 3rd, 2006 at 02:57:44 AM EST


Display:
Go to: [ European Tribune Homepage : Top of page : Top of comments ]