As for model software, have you ever waded through a significant chunk of code? Are you going to verify the operating systems they run on as well: if not, why not? How abour the instrumentation in satellites launched 30yr ago? The firmware that controlled them?
And what are the scientists supposed to do for the next 50yr whilst a host of hostile ignoramusses trundles through their work making insistent non-stop demands for detailed explanations and historical trails, while refusing to learn a god-damned thing?
The request is absurd on its face.
Then arrange a permissions structure so that it is initially released only to reviewers, and released to everybody upon publication.
That would only take care of things going forward, but once the structure is established it might be possible to secure outside funding for porting "back issues."
The record will, of course, be incomplete - some data will have been lost, some code will have been modified beyond retrieval. But it's a step in the right direction.
- Jake If you only spend 20 minutes of the rest of your life on economics, go spend them here.
2nd step: Do a evaluation of past performance of predicative quantitative science. While some predictions are for the future, some can already be verified:
4th step: Re assessment of previous publications by scientists that are non-peer. An example: In malaria lots of maths is used for modeling. Other people using maths as a tool (but not in malaria) could read and give an opinion on the maths. It is very difficult for peers to point out errors post-publication... without creating enemies.
I can dream.
In the mean time, things like this "email issue" will probably happen in the future, putting the credibility of current scientists where it deserves to be.
The current trend, however, is that science journals make full transparency a prerequisite for publication, and data will then only be available through science journals access. Hence increasing their dominance on science publications.
Smart institutions hopefully will move ahead with structures like the one you propose.
Ultra-competition, style over substance, egotistic posturing over actual discovery are not necessarily inherently scientific. They're certainly features of academia, but I'm not convinced their effects can't be minimised to the point where they're no longer a key driver of the culture.
As for peer review and data sharing - from the climate denialist point of view, this is missing the point. Even if the scientific community agreed consistently, peer reviewed all models, shared data religiously, and created a clear consensus, the denialists would find one tenured kook and plaster them all over the front pages and the wacko blogs to 'disprove' the scientists.
This is not about evidence or honesty, it's about story-telling and persuasion.
There are certainly things scientists could do, but in terms of political rather than scientific effectiveness, improved transparency comes pretty low on the list.
If you have to release your data after the "preliminary investigation report" there'd be an incentive to delay publishing until you have a paper that you think will actually be cited by anybody outside your own department and close friends.
If universities have to make all data completely public, corporate attempts to hide, fabricate or spin results would be in direct conflict with the prestige of the participating scientists. Which is a rather more compelling incentive to refrain from participating in a project than vague concerns about academic ethics.
So you really have to have a group of Big Dicks who have both enough prestige to demand transparency and enough suitably inventive and painful punishments for the people who fail to comply with that demand.
This is an area where the European Union could do a lot of good. If the EU were to demand that all publicly funded research must be published in journals that demand full disclosure (to the general public, not just to the journal), the ripples would be felt worldwide.
I also know some people involved in climate modeling software development.
SO I PRETTY DAMN WELL KNOW WHAT I AM TALKING ABOUT.
But even if I did not knew anything I could make the following assertion: Any political decision in an open society which is supported in a technical and scientific process should make that process open to the general public.
In this case it is possible. At least for some of the models that are used, I am pretty sure it is.
And, I personally could not care less that you are a "senior scientist". Please present rational arguments and not arguments of authority. I know what I am talking about from proven first hand experience, and you, why should we trust you?
Like yourself, I have considerable experience in working with, and writing, fairly sizable pieces of computer simulation software in a variety of languages. None of what I have worked on is remotely comparable in complexity to a major climate model, of which I have merely used the outputs...which was plenty enough work on it own.
The very thought of making my data and model available in any usable form makes me quiver. I have tried to do this once or twice, and it takes a huge amount of effort. I just can't see making such availability a requirement for all scientific working groups. Their actual research productivity would grind to a halt.
At least, that is my opinion, based on my experience, which may well be less than yours.
I am in the last steps of preparing a paper and I am, for the first time, undecided if I am going to make the software available (the data I won't, as it can be generated from the software with not much computing power). Just bundling the software is a major pain and I am pretty sure no one will care to repeat my stuff, so I probably will skip it this time. If I submit to PLOS Comp Biol, they will probably force me, but other journals, I very much doubt.
They should force me.
So yes, I know very much well what I am talking about. And I could talk hours and hours and hours about this.
Not so much about climate modeling (still I know a few things). But about predicative science in general.
I would imagine lots of people in climate prediction are using old code (building on top of), which they cannot convert and really they dont know what the code does (like it was written in the 70s by people who are DEAD and left no documentation)
Considering the kind and quality of physicist code (and - in particular - the documentation of said code), this is not only probable - it is a virtual certainty (you should excuse the pun).
Ideally, you'd want a computer code cleanup staff on permanent retainer at all major universities. But that would not be cheap.
