The European Tribune is a forum for thoughtful dialogue of European and international issues. You are invited to post comments and your own articles.
Please REGISTER to post.
Peer review assesses the appropriateness of the data collected and the method of collection as reported, the validity of the statistical analysis as a test of the hypothesis, and checks that the discussion and conclusions actually follow logically from the results reported. Some reviewers will do some mental assessments of the tabular or summar statistics reported and may flag apparent anomolies for revisions or further explanation.
But the suggestion that we "check their calculations" is absurd beyond any possible practice in any discipline except perhaps in mathematics.
If it's not amenable to checking, then it needs to be re-written, because then it won't be readable for the target audience either.
The simulations are a different kettle of fish, of course.
Friends come and go. Enemies accumulate.
Note that I agree: reviewing complex papers by repeating the calculations is asking too much. When software is made available I tend to download it and evaluate it. But with mathematical formulae even reading is a big pain.
I know of a top scientist in population genetics that says that when he sees lots of maths in papers is because authors are trying to make something difficult to detect. ;)
But this only exposes how the current process is flawed: peer-review can only go so far. And that "far" is not enough to detect even gross mistakes.
Science is also riddled with Dunning-Kruger effect. For instance I work in biology/medicine with a CS background. Most people developing software in bio/med think that because they are so good bio/med people they immediately become fantastic programmers. And then you see people devising results done with software which has the quality coming from a high school student. Don't even try to suggest that they are completely ignorant in the subject of programming.
I once had a discussion with a top scientist which does only theoretical modeling on the advantages of indenting code. This person doesn't even indent code. And why? "Well, with 8/9 levels of code, indenting makes it unprintable. And it is impossible to break the code in less levels of indentation"
Another suggested that an optimization algorithm always finds the maximum if the algorithm is stochastic (whereas if it is deterministic only local maximum can be found). Code was made, published and used like this.
Probably, any major collaborations in simulation and data analysis should have a statistician and an algorithms guy and a programmer on board. Just try funding that in my world.
It's not that I object in principle to making code and data available. I rely on other peoples' code and data all the time. It just there is no mechanism for funding the systematic provision of what you are asking for, or or rewarding the scientists who take the time to do it. Fixing this is no simple matter. You must know well the time and effort required to produce, for example, a reliable distribution of even a minor piece of code.
If everybody is on the same level (ie everybody has to make the software available) then competition is fair.
Your new field, ecology is fairly bad field in terms of secrecy and closeness (but not bad in terms of predicting BS, me thinks). I have a completely ridiculous paper on an ecology journal (but at least the source code is available ;) ).
Half of my PhD is conservation genetics and people are really secretive. We should have a paper submitted very soon to Molecular Ecology and it is predicative (fall of expected heterosigosity over 200 years). It will probably be used to change policy. I actually intended to make a diary about it (after it is published).
I mainly work with population genetics simulations (effective population size and selection) and spread of drug resistance (selection, pharmacology). I have some "part/time" with phylogenetics also.
In some sense I became proud of being a CS guy at the origin. The problems in CS are less bad. I plan to return to CS (if I stay in science - which I probably wont).
I recently "got out of the hole" through becoming a cynicist: I don't really want to give a nice contribution to society anymore, at least through science I think it is impossible.
I just want to live the quiet life and "do no harm" is now enough for me.
I need a beer.
by ARGeezer - Aug 20 8 comments
by Frank Schnittger - Aug 16 42 comments
by Oui - Aug 18 28 comments
by Frank Schnittger - Aug 13 13 comments
by Frank Schnittger - Aug 8 125 comments
by Frank Schnittger - Aug 6 26 comments
by Oui - Aug 8 16 comments
by Frank Schnittger - Aug 1 115 comments
by ARGeezer - Aug 208 comments
by Oui - Aug 198 comments
by Oui - Aug 1828 comments
by Frank Schnittger - Aug 1642 comments
by Frank Schnittger - Aug 1313 comments
by Oui - Aug 816 comments
by Frank Schnittger - Aug 8125 comments
by Oui - Aug 75 comments
by Frank Schnittger - Aug 626 comments
by Oui - Aug 431 comments
by Oui - Aug 34 comments
by gmoke - Aug 21 comment
by Frank Schnittger - Aug 1115 comments
by Oui - Aug 19 comments
by Oui - Jul 3023 comments
by Oui - Jul 2823 comments
by Frank Schnittger - Jul 2711 comments
by IdiotSavant - Jul 255 comments
by john_evans - Jul 2580 comments
by Oui - Jul 256 comments