The European Tribune is a forum for thoughtful dialogue of European and international issues. You are invited to post comments and your own articles.
Please REGISTER to post.
Peer review is a joke most of the time, data or no data. It is rare that a reviewer will actually check a paper's calculations for errors.
En un viejo país ineficiente,
algo así como España entre dos guerras
civiles, poseer una casa y poca hacienda
y memoria ninguna. -- Gil de Biedma
Peer review assesses the appropriateness of the data collected and the method of collection as reported, the validity of the statistical analysis as a test of the hypothesis, and checks that the discussion and conclusions actually follow logically from the results reported. Some reviewers will do some mental assessments of the tabular or summar statistics reported and may flag apparent anomolies for revisions or further explanation.
But the suggestion that we "check their calculations" is absurd beyond any possible practice in any discipline except perhaps in mathematics.
If it's not amenable to checking, then it needs to be re-written, because then it won't be readable for the target audience either.
The simulations are a different kettle of fish, of course.
Friends come and go. Enemies accumulate.
Note that I agree: reviewing complex papers by repeating the calculations is asking too much. When software is made available I tend to download it and evaluate it. But with mathematical formulae even reading is a big pain.
I know of a top scientist in population genetics that says that when he sees lots of maths in papers is because authors are trying to make something difficult to detect. ;)
But this only exposes how the current process is flawed: peer-review can only go so far. And that "far" is not enough to detect even gross mistakes.
Science is also riddled with Dunning-Kruger effect. For instance I work in biology/medicine with a CS background. Most people developing software in bio/med think that because they are so good bio/med people they immediately become fantastic programmers. And then you see people devising results done with software which has the quality coming from a high school student. Don't even try to suggest that they are completely ignorant in the subject of programming.
I once had a discussion with a top scientist which does only theoretical modeling on the advantages of indenting code. This person doesn't even indent code. And why? "Well, with 8/9 levels of code, indenting makes it unprintable. And it is impossible to break the code in less levels of indentation"
Another suggested that an optimization algorithm always finds the maximum if the algorithm is stochastic (whereas if it is deterministic only local maximum can be found). Code was made, published and used like this.
Probably, any major collaborations in simulation and data analysis should have a statistician and an algorithms guy and a programmer on board. Just try funding that in my world.
It's not that I object in principle to making code and data available. I rely on other peoples' code and data all the time. It just there is no mechanism for funding the systematic provision of what you are asking for, or or rewarding the scientists who take the time to do it. Fixing this is no simple matter. You must know well the time and effort required to produce, for example, a reliable distribution of even a minor piece of code.
If everybody is on the same level (ie everybody has to make the software available) then competition is fair.
Your new field, ecology is fairly bad field in terms of secrecy and closeness (but not bad in terms of predicting BS, me thinks). I have a completely ridiculous paper on an ecology journal (but at least the source code is available ;) ).
Half of my PhD is conservation genetics and people are really secretive. We should have a paper submitted very soon to Molecular Ecology and it is predicative (fall of expected heterosigosity over 200 years). It will probably be used to change policy. I actually intended to make a diary about it (after it is published).
I mainly work with population genetics simulations (effective population size and selection) and spread of drug resistance (selection, pharmacology). I have some "part/time" with phylogenetics also.
In some sense I became proud of being a CS guy at the origin. The problems in CS are less bad. I plan to return to CS (if I stay in science - which I probably wont).
I recently "got out of the hole" through becoming a cynicist: I don't really want to give a nice contribution to society anymore, at least through science I think it is impossible.
I just want to live the quiet life and "do no harm" is now enough for me.
I need a beer.
by Oui - May 15 25 comments
by Frank Schnittger - May 17 34 comments
by gmoke - May 17
by IdiotSavant - May 15 3 comments
by Frank Schnittger - May 4 19 comments
by Oui - May 9 6 comments
by Frank Schnittger - May 5 16 comments
by Frank Schnittger - Apr 30 6 comments
by gmoke - May 17
by Frank Schnittger - May 1734 comments
by Oui - May 1525 comments
by IdiotSavant - May 153 comments
by Oui - May 101 comment
by Oui - May 96 comments
by Oui - May 75 comments
by Frank Schnittger - May 516 comments
by Frank Schnittger - May 419 comments
by Frank Schnittger - May 334 comments
by Oui - May 212 comments
by Frank Schnittger - Apr 3016 comments
by Frank Schnittger - Apr 306 comments
by Oui - Apr 289 comments
by Frank Schnittger - Apr 2849 comments
by Oui - Apr 271 comment
by Frank Schnittger - Apr 2512 comments
by Frank Schnittger - Apr 2257 comments
by Oui - Apr 207 comments