1) there are somewhere in the region of 4500 files in the download, about 1000 of which are mail, and 3500 data files. From the contents of one of these data files there are suggestions that one of the climate models has 11,000 data files, so at best this is probably a subset of data.
2)It has been reported that the email has been confirmed as genuine. However the genuineness of the data files should have its status reserved until it has been thoroughly checked. And for it to be considered real, that is going to take some time. Firstly it has to be established when the server was first improperly accessed, and secondly it has to be checked against backup copies to asess the integrity of the datasets . If someone has had access, then we know that they have removed files, they may have also changed files without the knowledge of reseasrchers.
3)From looking at the corespondence, Climate scientists appear no more or less riven with factions, infighting, feelings of sleights, and outright lunacy than any other group of academics. If anyone thinks that this faction of climate examiners is particularly bad is asked to think wether every single email of their own would pass scrutiny.
4)I am somewhat suspicious of the date of release of this information. 2 weeks before the Copenhagen climate change conference, almost guarantees that Climate change scientists will be far busier discussing the leak rather than preparing for the conference. A cynic or conspiracy theorist could almost come to the conclusion that it was planned to cause the most disruption. Any idiot can face a crisis - it's day to day living that wears you out.
The populist consensus has shifted from being avidly anti-corporate and pro-green, or at least broadly sympathetic, to seeing climate issues as an evil conspiracy by evil scientists who want to raise your taxes.
Outside of a few middle class holdouts, there's no longer any support at all for climate change management, or carbon rationing.
This is the bullet in the head for climate change measures.
Most people don't know or care about the scrappiness or internal politics of science. What they'll take home from this is that the deniers are right - the scientists have been deliberately lying to them.
Of course the timing isn't a coincidence. But it doesn't matter - as of now, we have no hope at all of an effective international climate change policy.
And for this generation the evil science meme is now so embedded that we may not have a policy again, even after the cities start flooding.
I hadn't even begun to think about it at this level. To continue the health analogy, I perceived it more as one more continuous fester for the future.
So: The popular narrative on climate change is destroyed because the popular narrative of scientists' integrity is destroyed.
If the work is so good, so reliable why not making it FULLY PUBLICLY AVAILABLE? Wasn't it payed with public funds?
I mean, get the source code out for public inspection. Parameters, fitting, etc...
Lets shed some light on the inner workings of the models.
I am just advocating for public scrutiny of what they do.
Note that in many cases, these projects even fail peer-scrutiny as the code, procedures are not made available for peer-review. And even if it was, it would be impossible for a couple/half-a-dozen reviewers to understand the model implementation. So full openess would allow complete peer-review (other peers than only journal reviewers) and public scrutiny.
Who is afraid of that?
As for model software, have you ever waded through a significant chunk of code? Are you going to verify the operating systems they run on as well: if not, why not? How abour the instrumentation in satellites launched 30yr ago? The firmware that controlled them?
And what are the scientists supposed to do for the next 50yr whilst a host of hostile ignoramusses trundles through their work making insistent non-stop demands for detailed explanations and historical trails, while refusing to learn a god-damned thing?
The request is absurd on its face.
Then arrange a permissions structure so that it is initially released only to reviewers, and released to everybody upon publication.
That would only take care of things going forward, but once the structure is established it might be possible to secure outside funding for porting "back issues."
The record will, of course, be incomplete - some data will have been lost, some code will have been modified beyond retrieval. But it's a step in the right direction.
- Jake If you only spend 20 minutes of the rest of your life on economics, go spend them here.
2nd step: Do a evaluation of past performance of predicative quantitative science. While some predictions are for the future, some can already be verified:
4th step: Re assessment of previous publications by scientists that are non-peer. An example: In malaria lots of maths is used for modeling. Other people using maths as a tool (but not in malaria) could read and give an opinion on the maths. It is very difficult for peers to point out errors post-publication... without creating enemies.
I can dream.
In the mean time, things like this "email issue" will probably happen in the future, putting the credibility of current scientists where it deserves to be.
The current trend, however, is that science journals make full transparency a prerequisite for publication, and data will then only be available through science journals access. Hence increasing their dominance on science publications.
Smart institutions hopefully will move ahead with structures like the one you propose.
Ultra-competition, style over substance, egotistic posturing over actual discovery are not necessarily inherently scientific. They're certainly features of academia, but I'm not convinced their effects can't be minimised to the point where they're no longer a key driver of the culture.
As for peer review and data sharing - from the climate denialist point of view, this is missing the point. Even if the scientific community agreed consistently, peer reviewed all models, shared data religiously, and created a clear consensus, the denialists would find one tenured kook and plaster them all over the front pages and the wacko blogs to 'disprove' the scientists.
This is not about evidence or honesty, it's about story-telling and persuasion.
There are certainly things scientists could do, but in terms of political rather than scientific effectiveness, improved transparency comes pretty low on the list.
If you have to release your data after the "preliminary investigation report" there'd be an incentive to delay publishing until you have a paper that you think will actually be cited by anybody outside your own department and close friends.
If universities have to make all data completely public, corporate attempts to hide, fabricate or spin results would be in direct conflict with the prestige of the participating scientists. Which is a rather more compelling incentive to refrain from participating in a project than vague concerns about academic ethics.
So you really have to have a group of Big Dicks who have both enough prestige to demand transparency and enough suitably inventive and painful punishments for the people who fail to comply with that demand.
This is an area where the European Union could do a lot of good. If the EU were to demand that all publicly funded research must be published in journals that demand full disclosure (to the general public, not just to the journal), the ripples would be felt worldwide.
I also know some people involved in climate modeling software development.
SO I PRETTY DAMN WELL KNOW WHAT I AM TALKING ABOUT.
But even if I did not knew anything I could make the following assertion: Any political decision in an open society which is supported in a technical and scientific process should make that process open to the general public.
In this case it is possible. At least for some of the models that are used, I am pretty sure it is.
And, I personally could not care less that you are a "senior scientist". Please present rational arguments and not arguments of authority. I know what I am talking about from proven first hand experience, and you, why should we trust you?
Like yourself, I have considerable experience in working with, and writing, fairly sizable pieces of computer simulation software in a variety of languages. None of what I have worked on is remotely comparable in complexity to a major climate model, of which I have merely used the outputs...which was plenty enough work on it own.
The very thought of making my data and model available in any usable form makes me quiver. I have tried to do this once or twice, and it takes a huge amount of effort. I just can't see making such availability a requirement for all scientific working groups. Their actual research productivity would grind to a halt.
At least, that is my opinion, based on my experience, which may well be less than yours.
I am in the last steps of preparing a paper and I am, for the first time, undecided if I am going to make the software available (the data I won't, as it can be generated from the software with not much computing power). Just bundling the software is a major pain and I am pretty sure no one will care to repeat my stuff, so I probably will skip it this time. If I submit to PLOS Comp Biol, they will probably force me, but other journals, I very much doubt.
They should force me.
So yes, I know very much well what I am talking about. And I could talk hours and hours and hours about this.
Not so much about climate modeling (still I know a few things). But about predicative science in general.
I would imagine lots of people in climate prediction are using old code (building on top of), which they cannot convert and really they dont know what the code does (like it was written in the 70s by people who are DEAD and left no documentation)
Considering the kind and quality of physicist code (and - in particular - the documentation of said code), this is not only probable - it is a virtual certainty (you should excuse the pun).
Ideally, you'd want a computer code cleanup staff on permanent retainer at all major universities. But that would not be cheap.
