Welcome to the new version of European Tribune. It's just a new layout, so everything should work as before - please report bugs here.

LQD: Why Most Published Research Findings Are False

by Melanchthon Wed Apr 17th, 2013 at 02:20:08 AM EST

See comments for discussion of Reinhart/Rogoff [ed comment by afew]

An old, but interesting paper:

Why Most Published Research Findings Are False - Public Library of Science (PLOS)  Medicine

Summary

There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

front-paged by afew


Published research findings are sometimes refuted by subsequent evidence, with ensuing confusion and disappointment. Refutation and controversy is seen across the range of research designs, from clinical trials and traditional epidemiological studies [1-3] to the most modern molecular research [4,5]. There is increasing concern that in modern research, false findings may be the majority or even the vast majority of published research claims [6-8]. However, this should not be surprising. It can be proven that most claimed research findings are false. Here I will examine the key factors that influence this problem and some corollaries thereof.

...

Display:
And a recent paper about the same problem in neurosciences:

Power failure: why small sample size undermines the reliability of neuroscience

A study with low statistical power has a reduced chance of detecting a true effect, but it is less well appreciated that low power also reduces the likelihood that a statistically significant result reflects a true effect. Here, we show that the average statistical power of studies in the neurosciences is very low. The consequences of this include overestimates of effect size and low reproducibility of results. There are also ethical dimensions to this problem, as unreliable research is inefficient and wasteful. Improving reproducibility in neuroscience is a key priority and requires attention to well-established but often ignored methodological principles.


"Dieu se rit des hommes qui se plaignent des conséquences alors qu'ils en chérissent les causes" Jacques-Bénigne Bossuet
by Melanchthon on Mon Apr 15th, 2013 at 11:28:30 AM EST
The scraps of cash thrown at the neurosciences is pathetic.  Without funding it is impossible to get enough test subjects to obtain a valid statistical universe.  

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Mon Apr 15th, 2013 at 01:49:20 PM EST
[ Parent ]
All of science is being de-funded. It's the end of the sponsorship model as we have come to know it over the past 70 years.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Mon Apr 15th, 2013 at 02:08:40 PM EST
[ Parent ]
I once took part in a study where subjects' brains were zapped using magnetic coils. The only part of me that noticed anything was a neck muscle. When asked how many people were taking part, the researcher said studies like these were usually done with around 11 people!?

Schengen is toast!
by epochepoque on Tue Apr 16th, 2013 at 02:21:39 PM EST
[ Parent ]
Yup.

The "best" study ever done on the comparative anatomy of hominid brains, a much needed study, used something on the order of 7 human, 5 chimpanzee, 3 gorilla, and 2 orangutang examples as the investigation Universe.

pfui, in other words.

The US government spends, roughly, $124 billion on "Research" but only $32 billion a year for basic research, across all departments.  The National Science Foundation, "supports fundamental research and education in all the non-medical fields of science and engineering," has a whacking (snark) total of $6.1 billion to invest.  For the areas covered by their Research Directorates the NSF is the sole significant US funder.

The money necessary to adequately study things like "zapping brains with magnetic coils" simply isn't there.  Thus they are studied inadequately, when they are studied at all.  


She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Tue Apr 16th, 2013 at 03:11:44 PM EST
[ Parent ]
IIRC, the experiments described in The Golem regarding studies on transfer of memories by training mice and feeding their brains to rats ended after a biologist out-budgeted the behaviourists in sheer number of mice and rats. Concluded there was an effect too, but failed to gain proselyts, maybe having something to do with the field being heavily into mincing animal brain.

Sweden's finest (and perhaps only) collaborative, leftist e-newspaper Synapze.se
by A swedish kind of death on Tue Apr 16th, 2013 at 06:12:26 PM EST
[ Parent ]
i think they spend plenty on human brain research, we just aren't told about it because of national security.

like how to make humans switch off pain, do without sleep, and that's without even getting to the various, um compounds they use to try and make prisoners talk, soldiers cultivate aggression and conscience suppression etc etc.

cultivating sociopathy ain't cheap...

'The history of public debt is full of irony. It rarely follows our ideas of order and justice.' Thomas Piketty

by melo (melometa4(at)gmail.com) on Fri Apr 19th, 2013 at 12:04:26 AM EST
[ Parent ]
epochepoque:
I once took part in a study where subjects' brains were zapped using magnetic coils.

i don't even have that excuse... unless you count guitar pickups!

also i note you didn't specify whether you were zapper or zappee, lol

'The history of public debt is full of irony. It rarely follows our ideas of order and justice.' Thomas Piketty

by melo (melometa4(at)gmail.com) on Thu Apr 18th, 2013 at 11:58:54 PM EST
[ Parent ]
zappee. they wanted us to 'see stars' by stimulating the primary visual cortex. After much zapping (and not seeing anything) the coils became appreciably warm and had to be shut down. "Don't worry, half the people don't see it."

Schengen is toast!
by epochepoque on Tue Apr 23rd, 2013 at 03:56:41 PM EST
[ Parent ]
Neuroscience needs its Einstein - Salon.com

Researchers at the National Institutes of Health are diligently mapping all 100 trillion neural connections between our 86 billion neurons, and President Obama has just announced a $100 million Brain Research Through Advancing Innovative Neurotechnologies Initiative that will study all those networks in action.

The neurologist Robert Burton is skeptical, to say the least. His new book, "A Skeptic's Guide to the Mind," is a scathing indictment of reductionism in all its guises, and a stirring call to consider whether scientists are even asking the right kinds of questions.

... Given our brain's tendency to seek out patterns and avoid ambiguity, we are physiologically condemned to question the very entity that is responsible for generating the questions.

[...] We can study brain function and ultimately arrive at a good working model of how the brain works. However, there is no scientific method to address subjective experience...

[...] Believing that knowledge of brain wiring can tell us the nature of consciousness is like predicting what sound will come out of a set of speakers by looking at wiring diagram of the component parts.


A conceptual neurology theory will still be mainly a brain story rather than a rigid statistical-empirical conclusion.
by das monde on Mon Apr 15th, 2013 at 02:20:43 PM EST
[ Parent ]
Biology is not thermodynamics.  Biological systems cannot be turned into x/y graphs with a binary independent and dependent relationship.  Biological systems are inherently flexible, adaptive, Complex, and unpredictable.    

As the Harvard Law of Animal Behavior states:

Under carefully controlled experimental circumstances, an animal will behave as it damned well pleases.

Can't mindlessly apply the tools and techniques, e.g., Probability, of the physical sciences in the biological sciences.  They don't hold the phenomenena.  

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Mon Apr 15th, 2013 at 03:09:27 PM EST
[ Parent ]
There are thermodynamic aspects in biology. In particular, there should be adequate flows of energy and materials for animals to behave as they really please. Adaptations to boundaries of those flows are little investigated yet. Linking details of various agencies and processes should be more important than any statistics.
by das monde on Tue Apr 16th, 2013 at 04:46:52 AM EST
[ Parent ]
There is the field of Biophysics where they do study such things.  

Most of the time, for most Biological systems, it doesn't matter: there's that great fusion reactor 8 minutes away.  For crucial aspects of Biology, say species divergence, Geography is more to the point.  And Biologists don't study that either.  It's good enough to observe two similar critters, living in adjacent mountain ranges, diverged because there is a big whacking desert between them.  

 

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Tue Apr 16th, 2013 at 12:29:51 PM EST
[ Parent ]
Biology is only possible feeding off the flows in an out-of-equilibrium thermodynamical system.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Tue Apr 16th, 2013 at 04:48:48 AM EST
[ Parent ]
And the only reason there is non-equilibrium thermodynamics is Broca's Area, Wernicke's Area, the Primary Auditory Cortex, & etc.  So Physicists should study neurolinguistics?

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Tue Apr 16th, 2013 at 12:19:07 PM EST
[ Parent ]
Does.not.compute.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Tue Apr 16th, 2013 at 12:22:15 PM EST
[ Parent ]
Try saying anything about non-equilibrium thermodynamics without using Language.

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Tue Apr 16th, 2013 at 12:34:19 PM EST
[ Parent ]
What I'm doing here is privileging Biology in the same manner Physicists privilege Physics.  

Annoying, isn't it?  :-)

The big difference between the fields is Biologists generally concede Physics has something to tell them.  Physicists generally think Biology has nothing to tell them.

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Tue Apr 16th, 2013 at 01:31:08 PM EST
[ Parent ]
Biology doesn't have much to tell physicists about physics.

Then again, any field which suspects that observables are observer-dependent but has no idea what an observer is or how to define one is going to have one or two epistemological issues.

by ThatBritGuy (thatbritguy (at) googlemail.com) on Tue Apr 16th, 2013 at 04:16:11 PM EST
[ Parent ]
...With the rider that it's still better than making random shit up for cash, like economists do.
by ThatBritGuy (thatbritguy (at) googlemail.com) on Tue Apr 16th, 2013 at 04:17:59 PM EST
[ Parent ]
For my experience there is no such thing more arrogant than Physicists (by a long stretch).

They are wrong, of course. They are able to understand about nature what their brains can understand about nature. Nothing more.

Biology (and Sociology) has a lot to tell Physicists: on their cognitive limits, on how their socially constructed biases might be affecting their decisions.

Hell, Science Fiction has a lot to tell physics: can you prove we do not live in the Matrix? The first article of faith is that we can trust our senses. And that is nothing more than that: an act of faith.

by cagatacos on Wed Apr 17th, 2013 at 05:32:07 AM EST
[ Parent ]
You're leaving economists out - LOL.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 05:40:05 AM EST
[ Parent ]
In this case it was unintentional.
by cagatacos on Wed Apr 17th, 2013 at 06:04:15 AM EST
[ Parent ]
Hell, Science Fiction has a lot to tell physics: can you prove we do not live in the Matrix? The first article of faith is that we can trust our senses. And that is nothing more than that: an act of faith.

Solipsism is a fun philosophical exercise. So is Descartes' spiel about the "great deceiver". Which he could only break by appeal to faith in God being all-powerful and good.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper

by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 05:44:39 AM EST
[ Parent ]
All I'm saying is you cannot have structure formation in thermodynamic equilibrium. It's pretty trivial, really.

But having an aquarium at home has really brought it home how important it is for real existing river ecosystems that there is a massive throughput of new water flowing through them, bringing in free energy and carrying away entropy.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper

by Migeru (migeru at eurotrib dot com) on Tue Apr 16th, 2013 at 06:19:13 PM EST
[ Parent ]
And what I'm saying is, when studying living systems, Thermo doesn't necessarily get you anywhere.  It's possible to assume it, and move on.

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Wed Apr 17th, 2013 at 12:57:08 PM EST
[ Parent ]
Reminds me:

Series 4 Episode 03 - The Zazzy Substitution | Big Bang Theory Transcripts

Sheldon: I brought Amy here to show her some of the work I'm doing.

Amy: It's very impressive, for theoretical work.

Sheldon: Do I detect a hint of condescension?

Amy: I'm sorry, was I being too subtle? I meant compared to the real-world applications of neurobiology, theoretical physics is, what's the word I'm looking for? Hmm, cute.

Leonard and Howard together: Oooh!

Sheldon: Are you suggesting the work of a neurobiologist like Babinski could ever rise to the significance of a physicist like Clarke-Maxwell or Dirac?

Amy: I'm stating it outright. Babinski eats Dirac for breakfast and defecates Clarke-Maxwell.

Sheldon: You take that back.

Amy: Absolutely not. My colleagues and I are mapping the neurological substrates that subserve global information processing, which is required for all cognitive reasoning, including scientific inquiry, making my research ipso facto prior in the ordo cognoscendi. That means it's better than his research, and by extension, of course, yours.

Leonard: I'm sorry, I'm-I'm still trying to work on the defecating Clark Maxwell, so...

Sheldon: Excuse me, but a grand unified theory, insofar as it explains everything, will ipso facto explain neurobiology.

Amy: Yes, but if I'm successful, I will be able to map and reproduce your thought processes in deriving a grand unified theory, and therefore, subsume your conclusions under my paradigm.

Sheldon: That's the rankest psychologism, and was conclusively revealed as hogwash by Gottlob Frege in the 1890s!

Amy: We appear to have reached an impasse.

Big bang theory. Quite funny show which gets an impressive amount of nerdy details right (even though this quote is a bad example as I have not before seen Maxwell referred to by Clerk-Maxwell).

Sweden's finest (and perhaps only) collaborative, leftist e-newspaper Synapze.se

by A swedish kind of death on Wed Apr 17th, 2013 at 01:08:16 PM EST
[ Parent ]
Heh.

James Clerk Maxwell - Wikipedia, the free encyclopedia

James Clerk Maxwell was born 13 June 1831 at 14 India Street, Edinburgh, to John Clerk, an advocate, and Frances Cay.[9][10] John Clerk-Maxwell was a man of comfortable means, of the Clerk family of Penicuik, holders of the baronetcy of Clerk of Penicuik; his brother being the 6th Baronet. He had been born "John Clerk", adding the surname Maxwell to his own after he inherited a country estate in Middlebie, Kirkcudbrightshire from connections to the Maxwell family, themselves members of the peerage

Generally known as Maxwell, it's true, but without his father's snobbishness, would have been called Clerk.

by afew (afew(a in a circle)eurotrib_dot_com) on Thu Apr 18th, 2013 at 06:34:10 AM EST
[ Parent ]
Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research.

It is somewhat satisfying to have my surmise confirmed. There should be a required disclosure preceding the abstract that sets out the sources of funding for the research, with financially interested parties listed in bold, but that would just get gamed. Two thirds of the proposals probably should never have been approved. Perhaps an independent proposal review process should be put in place.

"It is not necessary to have hope in order to persevere."
by ARGeezer (ARGeezer a in a circle eurotrib daught com) on Mon Apr 15th, 2013 at 04:30:32 PM EST
Such disclosures are mandatory. As I recall, most of the stuff Ioannidis examines is NIH-funded.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Mon Apr 15th, 2013 at 04:40:00 PM EST
[ Parent ]
Are NIH funds the exclusive funding sources for those using them?

"It is not necessary to have hope in order to persevere."
by ARGeezer (ARGeezer a in a circle eurotrib daught com) on Mon Apr 15th, 2013 at 04:42:37 PM EST
[ Parent ]
Depends on the type of grant, the type of study, and how far back and forward in time you want to go and how many degrees of separation you want to look at.

You can play six degrees of Kevin Bacon with literally anybody who has ever done anything remotely remarkable and "uncover" "suspicious funding."

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Mon Apr 15th, 2013 at 05:01:26 PM EST
[ Parent ]
I was referring to the studies in the report. I haven't had time to read the paper, but it would seem that NIH has a problem with its research funding system if the results reported are meaningful. It could be political pressure. Don't know. It could be that a number of the papers were co-sponsored by interested parties.

"It is not necessary to have hope in order to persevere."
by ARGeezer (ARGeezer a in a circle eurotrib daught com) on Mon Apr 15th, 2013 at 07:15:52 PM EST
[ Parent ]
Not necessarily. Certain departments, like NCCAM, have substantial problems with funding stuff on the basis of inadequate preclinical research.

But a lot of the time, what Ioannidis examines is small pilot studies. You would expect most small pilot studies to be wrong. The fact that you would expect most small pilot studies to be wrong is the reason we do progressively larger follow-up studies.

But the larger follow-up studies appear in the second-tier journals, not the first-tier journals, because the first-tier journals have a novelty requirement.

Now, imagine that you have a molecule design that you believe could be a good chemotherapeutic drug. What do you do? You don't rush straight out to do a 1000-subject phase III trial. For one thing, that would be unethical. For another, it would be stupid, because you know that most molecules designs that look promising on paper turn out to be, in fact, not good chemotherapeutic drugs.

So you start with an animal model. Success. Publish.

You do a few more, better animal model studies, because you want to eventually feed this stuff to humans, and they'd better not keel over and die. Because that would be bad. Success. Three publications.

So you move on to test for maximum safe dose in humans (phase I trial). Success - they didn't die. One publication.

Now you do a small pilot trial (phase II trial). Success. One publication.

So you apply for, and get funded, a large phase III trial. Your drug turns out to not work. One publication.

Looking back over the literature this development process spawned, it yielded six false and one true publication (and that's if you do only one phase I and II trial before moving on to resp. phase II and III).

What, precisely, is wrong with this picture?

IMO, the take-aways from Ioannidis are:

  1. Prior probability is important when evaluating statistical results.

  2. Replication is important.

  3. A single paper proves nothing. You have to look at the whole literature.

- Jake

Friends come and go. Enemies accumulate.
by JakeS (JangoSierra 'at' gmail 'dot' com) on Tue Apr 16th, 2013 at 03:34:16 AM EST
[ Parent ]
Researchers Finally Replicated Reinhart-Rogoff, and There Are Some Serious Problems. | Next New Deal

In 2010, economists Carmen Reinhart and Kenneth Rogoff released a paper, "Growth in a Time of Debt." Their "main result is that...median growth rates for countries with public debt over 90 percent of GDP are roughly one percent lower than otherwise; average (mean) growth rates are several percent lower." Countries with debt-to-GDP ratios above 90 percent have a slightly negative average growth rate, in fact.

This has been one of the most cited stats in the public debate during the Great Recession. Paul Ryan's Path to Prosperity budget states their study "found conclusive empirical evidence that [debt] exceeding 90 percent of the economy has a significant negative effect on economic growth." The Washington Post editorial board takes it as an economic consensus view, stating that "debt-to-GDP could keep rising -- and stick dangerously near the 90 percent mark that economists regard as a threat to sustainable economic growth." 

Is it conclusive? One response has been to argue that the causation is backwards, or that slower growth leads to higher debt-to-GDP ratios. Josh Bivens and John Irons made this case at the Economic Policy Institute. But this assumes that the data is correct. From the beginning there have been complaints that Reinhart and Rogoff weren't releasing the data for their results (e.g. Dean Baker). I knew of several people trying to replicate the results who were bumping into walls left and right - it couldn't be done.

In a new paper, "Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff," Thomas Herndon, Michael Ash, and Robert Pollin of the University of Massachusetts, Amherst successfully replicate the results. After trying to replicate the Reinhart-Rogoff results and failing, they reached out to Reinhart and Rogoff and they were willing to share their data spreadhseet. This allowed Herndon et al. to see how how Reinhart and Rogoff's data was constructed.

They find that three main issues stand out.

by afew (afew(a in a circle)eurotrib_dot_com) on Tue Apr 16th, 2013 at 12:25:01 PM EST
Researchers Finally Replicated Reinhart-Rogoff, and There Are Serious Problems. | Next New Deal
Herndon-Ash-Pollin find that they exclude Australia (1946-1950), New Zealand (1946-1949), and Canada (1946-1950). This has consequences, as these countries have high-debt and solid growth.

Choice of endpoints. Classic cheating.
And the bucketing is another fiddle.
But the best is yet to come : error in the friggin' spreadsheet excluding certain countries!

Researchers Finally Replicated Reinhart-Rogoff, and There Are Serious Problems. | Next New Deal

If this error turns out to be an actual mistake Reinhart-Rogoff made, well, all I can hope is that future historians note that one of the core empirical points providing the intellectual foundation for the global move to austerity in the early 2010s was based on someone accidentally not updating a row formula in Excel.

So what do Herndon-Ash-Pollin conclude? They find "the average real GDP growth rate for countries carrying a public debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent as [Reinhart-Rogoff claim]." Going further into the data, they are unable to find a breakpoint where growth falls quickly and significantly.

Score one for transparency. Somebody tell the ECB.

It is rightly acknowledged that people of faith have no monopoly of virtue - Queen Elizabeth II

by eurogreen on Tue Apr 16th, 2013 at 12:40:08 PM EST
[ Parent ]
The excel issue is pretty frightening for people considering the peer-review edifice.

The others are just as serious and frankly more worrying from a bias point of view.

What it highlights most of all is that the assumptions at work in processing the data have deep implications. Which is a big problem even if the profession were more diligent (unlikely as that is in economics.) It's a big problem because journalists are never going to be more diligent. So policymaking is always going to be vulnerable to this kind of misleading work...

by Metatone (metatone [a|t] gmail (dot) com) on Tue Apr 16th, 2013 at 12:40:46 PM EST
[ Parent ]
Metatone:
The excel issue is pretty frightening for people considering the peer-review edifice.

Well, their data weren't released, at least in any form that would allow peer review to spot that amazing blunder.

What's frightening is that the authors themselves didn't see it.

by afew (afew(a in a circle)eurotrib_dot_com) on Tue Apr 16th, 2013 at 12:47:53 PM EST
[ Parent ]
It's a trend in academic and research papers for the authors to not include their data.  For any non-trivial conclusion the results, as we see, can be disastrous.  Ditto for the inclusion of neonicotinoids (an insecticide) in field crops which seems to be the final contribution factor to Bee Colony Collapse, the effects of which Monsanto propagandists scientists didn't bother to report, IF they bothered to investigate the matter -- which is unclear, to me.  

My personal opinion is study data is not included since, if it was, it would be immediately apparent the authors did the equivalent of shooting an arrow in the air and then painting a bulls eye wherever it happened to land.

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Tue Apr 16th, 2013 at 01:27:45 PM EST
[ Parent ]
I was going to post here the Reinhart and Rogoff issue and start of  a discussion, but it is good to see this is well advanced. I would like to add a few points:

  1. Having the data publicly available is good, but note that for most scientific projects, replicating the analysis can be very difficult. With Reinhart and Rogoff there are people interested in doing it, but with 99% of papers published nobody will care.

  2. In a paper with lots of maths/computational issues, no reviewer will have time to go through all the details, not even close. Maybe in maths, maybe in physics (but  I doubt) but certainly not in biology. Last week I reviewed 2 papers and 4 conference abstracts. A proper review would probably take me the whole month for one of the papers. And I get nothing from reviewing (remember that I still have to do my own work in a cut-throat environment - "publish or perish").

  3. Even if I find the errors when reading a paper, what  can I do? I can inform the author, but if he ignores it? I could eventually complain to the journal editor, gaining an enemy for life (somebody that will, in the future, peer-review my papers and grant requests). No way, Jose.

A bright note though: in some areas in biology there is a great drive to make data available even pre-publication (e.g. 1000 genomes project).

The more general comment is that, from a technical perspective the edifice is broken: computational/mathematical issues are (very) prone to human errors. The whole peer-review structure is not capable of dealing with the complex mathematical/computational infrastructure that is currently required.

Notice that I did not discuss in the paragraph above political or ethical issues. But, from point 2 on the list you can easily see that that is also a big issue. Something I will not discuss even under a pseudonym. And that would be beyond depressive.

Some 8 years ago I made a career change into science (because I wanted to "do good", or an equally naive thought like that). What I discovered in the last 8 years has made my cynical beyond belief.

by cagatacos on Wed Apr 17th, 2013 at 05:51:17 AM EST
[ Parent ]
Reinhart and Rogoff's paper is a classic example of telling the wealthy what they want to hear. It provided a strong argument for why sovereign debt reduction was needed, which provided a strong reason why social spending, which most of the rich hate, must be rolled back. I have no doubt that the acclaim this paper has received has been very good for both Reinhart and Rogoff, but I doubt that the paper by Thomas Herndon, Michael Ash, and Robert Pollin will receive anything like the attention received by Reinhart and Rogoff's paper. I predict few in a career decision making position in academia will consider any controversy over Reinhart and Rogoff's paper to be anything that should impeach their academic credentials. Justifying to wealthy, powerful people things they want to do is not usually a bad career move.

"It is not necessary to have hope in order to persevere."
by ARGeezer (ARGeezer a in a circle eurotrib daught com) on Tue Apr 16th, 2013 at 07:41:26 PM EST
[ Parent ]
Are Reinhart and Rogoff now professionally dead, after such an egregious error?

It is rightly acknowledged that people of faith have no monopoly of virtue - Queen Elizabeth II
by eurogreen on Tue Apr 16th, 2013 at 12:45:03 PM EST
[ Parent ]
Remember the 90% result has not actually been published!

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Tue Apr 16th, 2013 at 12:49:46 PM EST
[ Parent ]
Ah, but is considered unquestionable science.
by afew (afew(a in a circle)eurotrib_dot_com) on Tue Apr 16th, 2013 at 12:51:52 PM EST
[ Parent ]
Economics is medieval scholastics, at best.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Tue Apr 16th, 2013 at 01:18:55 PM EST
[ Parent ]
Axiom:

People have rational preferences among outcomes that can be identified and associated with a value

In other words:

Assume Cow = Spherical

THEN ...

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Tue Apr 16th, 2013 at 01:34:14 PM EST
[ Parent ]
People have rational preferences among outcomes that can be identified and associated with a value

Supports my interests = True

Then:

Reward researcher with money and or career advancement

"It is not necessary to have hope in order to persevere."

by ARGeezer (ARGeezer a in a circle eurotrib daught com) on Wed Apr 17th, 2013 at 05:47:49 PM EST
[ Parent ]
Don't be silly.

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Tue Apr 16th, 2013 at 01:15:52 PM EST
[ Parent ]
A small story: There is this paper on Nature about mitochondrial diversity. Essentially stating that, in a population, it is independent of the population size.

Most people disagree with this, therefore the paper has lots of citations opposing the view.

Now, there is no concept of "good" or "bad" citations. Therefore, for the author (and the journal) this is a highly important paper with lots of citations. A paper on Nature with hundreds of citations (good or bad) can make a career.

by cagatacos on Wed Apr 17th, 2013 at 06:01:19 AM EST
[ Parent ]
Well, they've just made the front page of the SZ, with the Excel error in the lead paragraph.
by gk (gk (gk quattro due due sette @gmail.com)) on Wed Apr 17th, 2013 at 07:36:23 AM EST
[ Parent ]
I'm assuming, since (as Mig noted) it hasn't been pubished, they may be immune from discipline by the institutions.

I probably wouldn't look for them in the big journals any time soon though.

Be nice to America. Or we'll bring democracy to your country.

by Drew J Jones (pedobear@pennstatefootball.com) on Wed Apr 17th, 2013 at 12:48:49 PM EST
[ Parent ]
None of the econometrics journals. But those aren't considered top-tier anyway.

The "top-tier" economics journals are almost entirely filled with evidence-free theoretizing. They'd fit right in.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Wed Apr 17th, 2013 at 01:14:32 PM EST
[ Parent ]
This is stunning, and disguting.

I have sent a note to Martin Wolf (of the FT), maybe he can give this a bit more publicity. Has Krugman written about this (not that it will make a difference)?

Wind power

by Jerome a Paris (etg@eurotrib.com) on Tue Apr 16th, 2013 at 04:00:07 PM EST
[ Parent ]
by IM on Tue Apr 16th, 2013 at 04:12:04 PM EST
[ Parent ]
It seems rather pompous to dignify the Excel cock-up as a "coding error", but I suppose that's the appropriate term of art.

It is rightly acknowledged that people of faith have no monopoly of virtue - Queen Elizabeth II
by eurogreen on Wed Apr 17th, 2013 at 07:51:22 AM EST
[ Parent ]
Got to be at least minimally serious and polite.
by Colman (colman at eurotrib.com) on Wed Apr 17th, 2013 at 08:02:51 AM EST
[ Parent ]
I do a lot of Excel work these days, and I always consider it coding work.

Coding with a crap interface, good I/O options, total garbage for memory and CPU use, but the same basic workflow and logic.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Wed Apr 17th, 2013 at 01:18:59 PM EST
[ Parent ]
Someone even programmed a video game (it sounds like it's actually a roguelike game) in Excel. See here. It seems to be without bugs (except that it doesn't run on a Mac), but then it was written by an accountant, not an economist.

It could be worse. See here for one of the most bizarre papers I've ever seen, a 1982 paper proposing the use of \( \TeX \) as a programming language, to save engineers and secretaries from having to learn two different languages.....

by gk (gk (gk quattro due due sette @gmail.com)) on Wed Apr 17th, 2013 at 02:37:41 PM EST
[ Parent ]
Excel is Turing complete. What's the debate about? That it's a piece of bloatware? That it cannot read in a csv file written by itself? That its math library is substandard? That its random number generator is piss-poor?

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 02:42:01 PM EST
[ Parent ]
The ribbon UI is bullshit.

Be nice to America. Or we'll bring democracy to your country.
by Drew J Jones (pedobear@pennstatefootball.com) on Wed Apr 17th, 2013 at 03:04:05 PM EST
[ Parent ]
The Turing and Post machines also have a 'ribbon' user interface :D

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 03:07:22 PM EST
[ Parent ]
Hey hey, I've written applications in Excel too. When I was younger, perhaps more patient, and had no influence on the choice of tools. Now I only debug other people's macros when I absolutely have to.

All of your points are well-taken : it's a coding error.

It is rightly acknowledged that people of faith have no monopoly of virtue - Queen Elizabeth II

by eurogreen on Thu Apr 18th, 2013 at 03:47:15 AM EST
[ Parent ]
As you say, it's also a wrong choice of tool. Any statistics done in Excel are automatically suspect because the judgement of the analyst is already under question.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Thu Apr 18th, 2013 at 03:49:05 AM EST
[ Parent ]
To be fair, Stata and SAS require taking out second mortgages.  And all of five people on Earth know how to use R.

Be nice to America. Or we'll bring democracy to your country.
by Drew J Jones (pedobear@pennstatefootball.com) on Thu Apr 18th, 2013 at 07:52:13 AM EST
[ Parent ]
But did they do more sophisticated statistics than just computing averages? You could do that easily in C.....
by gk (gk (gk quattro due due sette @gmail.com)) on Thu Apr 18th, 2013 at 07:56:16 AM EST
[ Parent ]
I would say that an advantage of them doing it in Excel is that that particular fault gets easily understandable for a wide audience. The weighing of different periods takes a bit to explain to your average person and the exculsion of high-debt, high-growth years after wwII is explained by RR as data they did not have, but not pulling a column all the way down in Excel is something most people get. And policy based on that gets easy to ridicule.

Sweden's finest (and perhaps only) collaborative, leftist e-newspaper Synapze.se
by A swedish kind of death on Thu Apr 18th, 2013 at 08:20:17 AM EST
[ Parent ]
What about SPSS?

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Thu Apr 18th, 2013 at 12:37:25 PM EST
[ Parent ]
I think that's the most expensive of all of them.

Be nice to America. Or we'll bring democracy to your country.
by Drew J Jones (pedobear@pennstatefootball.com) on Thu Apr 18th, 2013 at 12:56:46 PM EST
[ Parent ]
I didn't know either so I looked it up.

Costs range from $4,660 to $26,500 for the "Professional" kit and $2,320 to $13,200 for the Standard kit.

If I understand correctly, use is non-exclusive but the buyer has to pay a yearly licensing fee, with support included ... which isn't that bad a deal for a company, university, institution, etc.  

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Thu Apr 18th, 2013 at 01:20:33 PM EST
[ Parent ]
Not an excuse. They work for major research universities. A major research university that does not have a single specialized software tool capable of performing by-the-books time series analysis is a contradiction in terms.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Thu Apr 18th, 2013 at 02:01:54 PM EST
[ Parent ]
Very many academic statisticians use R. Or the commercial version, S-plus.

The "problem" with R is that the S language is functional, so it requires a bit of brain rewiring. On the other hand, working with data frames (tables) is very intuitive.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper

by Migeru (migeru at eurotrib dot com) on Thu Apr 18th, 2013 at 02:10:04 PM EST
[ Parent ]
To be fair, I do a lot of light-duty number crunching in Excel, because for input/output the interface is so much more convenient than a proper programming language.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Thu Apr 18th, 2013 at 02:03:00 PM EST
[ Parent ]
I do light-duty number crunching with R. I give you that Excel has a convenient tabular interface for data entry. But so does R if you insist...

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Thu Apr 18th, 2013 at 02:08:09 PM EST
[ Parent ]
I have never learned to use R. All the stuff I know how to code in - other than Excel - has various onerous loading calls from external files.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Thu Apr 18th, 2013 at 02:11:38 PM EST
[ Parent ]
If you're reading from external files, even Excel is onerous.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Thu Apr 18th, 2013 at 02:13:33 PM EST
[ Parent ]
You can only be "stunned" if your view of scientific institutions is totally naive.
by cagatacos on Wed Apr 17th, 2013 at 05:53:16 AM EST
[ Parent ]
I'm trying to maintain some modicum of idealism and hope once in a while... I'm cynical enough already.

(I fled academia after writing my PhD. The little I saw in 3 years was enough to depress me - and I was in the privileged position that I had a guaranteed stipend for 3 or 4 years and did not have to beg for anything. It also turned out, by a stroke of luck, that what I worked on (Ukrainian gas pipelines) was knowledge that had some marketable value...

Wind power

by Jerome a Paris (etg@eurotrib.com) on Wed Apr 17th, 2013 at 08:55:51 AM EST
[ Parent ]
I understand.

I fled IT in order to do "interesting things" in academia. I simply have no more energy to start from scratch once again.

I simply accept that academia is what I do to earn a salary and try to devise fulfilment for other things in life. I just learned to live with the idea that what I do for a living has nothing to do with what I find interesting.

by cagatacos on Wed Apr 17th, 2013 at 09:03:38 AM EST
[ Parent ]
Interesting, because I've been considering a PhD - not with any interest in becoming an academic, but because it would be an excuse to spend a few years working exclusively on something that interests me, with some possible commercial applications.

Unfortunately the open days I've been to have all made me think 'WTF' - not least after an extremely weird and hand-wavy mini-lecture about the theology of bird song at an internationally respected music department. (Was Darwin a closet theist? Do birds sing because they're full of the joy of creation? How about 'no'?)

It's possible there are departments where 90% of what happens isn't a waste of everyone's time and energy, but I'm not convinced I've found one of those yet.

Neo-liberalism seems to have pushed academia towards Scholasticism 2.0. There's a whole of academicking going on, but the hard sciences were mostly raped by money, and the humanities have turned into some kind of ghastly philosophical corpse packed full of words like 'discourse' and 'embodied' and 'semiotic' and 'normative'.

So economists who lie with Excel (accidentally, of course, one supposes) are just another fragment of Capitalist Gothic Post-Intellectualism.

On the other hand there's the Internet. So it's not all bad news.

by ThatBritGuy (thatbritguy (at) googlemail.com) on Wed Apr 17th, 2013 at 09:34:50 AM EST
[ Parent ]
Your mileage might vary.

I have noticed that local environments can vary a lot. For instance, where I did my PhD: my supervisor was an extremely liberal person, gave me a lot of freedom. My department was a collection of fascist lecturers (save my supervisor). Other departments in the school were OK (indeed most of my brain sharing was with other departments). I have been into lots of places and I think the rule holds: massive variation.

So, if you have knowledge on who you are going to work with...

Another source of variance is the field: the mood, intellectual liberty, honesty can vary a lot. For instance I still collaborate a lot with conservation geneticists: those I know - many top in their fields - are exactly what you would expect for a benign vision of things (intellectually strong, honest, open to discussion, ...). My field of work (poverty diseases): blargh... I actually did a career change inside my field also because of my perception of the community (I now study disease vectors, not parasites).

Remember: academia is very atomized. You might have zero work connections with the guy working next floor. This makes for the possibility of vastly different working cultures. Some are not a disaster... The problem is that it is difficult to know before you get in.

by cagatacos on Wed Apr 17th, 2013 at 09:56:14 AM EST
[ Parent ]
Tragedy is there's some important issues hidden in the fog of poly-syllabic gibberish.  

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Wed Apr 17th, 2013 at 12:25:06 PM EST
[ Parent ]
And there are a lot of even more important issues elsewhere.
by ThatBritGuy (thatbritguy (at) googlemail.com) on Wed Apr 17th, 2013 at 12:51:29 PM EST
[ Parent ]
Jerome is lucky, he likes his work and his work is meaningful.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 10:56:58 AM EST
[ Parent ]
Migeru:
Jerome is lucky, he likes his work and his work is meaningful.

you can take that to the bank!

'The history of public debt is full of irony. It rarely follows our ideas of order and justice.' Thomas Piketty

by melo (melometa4(at)gmail.com) on Fri Apr 19th, 2013 at 12:16:50 AM EST
[ Parent ]
Reinhart, Rogoff Admit Excel Mistake, Rebut Other Critiques

The road of excess leads to the palace of wisdom - William Blake
by talos (mihalis at gmail dot com) on Wed Apr 17th, 2013 at 11:04:44 AM EST
[ Parent ]
Their main defense is funny.
It is utterly misleading to speak of a 1% growth differential that lasts 10-25 years as small.  If a country grows at 1% below trend for 23 years, output will be roughly 25% below trend at the end of the period, with massive cumulative effects.
So instead of growing 2.5 times that GDP merely doubles in the same time, in 20 or so years. That is exactly the growth (or time span) whereby demands of financial services start to outstrip feasible productive growth. Show me persistent 3% annual growths over 25-30 years that did not end up in tears.
by das monde on Wed Apr 17th, 2013 at 02:24:58 PM EST
[ Parent ]
I'd be inclined to give them the benefit of the doubt if it were just the Excel fuck-up.

However, arbitrarily throwing out data and employing (without justification) non-standard methodology -- and having both bias the results in favor of your hypothesis (hello!) -- says to me that Harvard and U of Maryland need to have a little chat with these two.

Be nice to America. Or we'll bring democracy to your country.

by Drew J Jones (pedobear@pennstatefootball.com) on Wed Apr 17th, 2013 at 12:42:32 PM EST
[ Parent ]
I hope Martin does cover this.

This really does need to be out there as much as possible.  If econ is going to be a field for serious research, this horseshit has to stop.  The politicization of macro has to end.  (That means Lucas, Prescott and the whole lot of them need to go EABOD.)  The economics-as-math-for-people-who-need-to-be-fitted-for-helmets has to end.  

The Dark Age of Macro really has to end.

Be nice to America. Or we'll bring democracy to your country.

by Drew J Jones (pedobear@pennstatefootball.com) on Wed Apr 17th, 2013 at 06:33:11 PM EST
[ Parent ]
(on the web, at least).

(Though they get it wrong about the post-war data exclusion.)

They point out that R&R were cited extensively in the preamble to France's budget bill of 2010. Excuse me. I need to find a punching bag.

It is rightly acknowledged that people of faith have no monopoly of virtue - Queen Elizabeth II

by eurogreen on Thu Apr 18th, 2013 at 04:05:25 AM EST
[ Parent ]
Jerome:
Has Krugman written about this (not that it will make a difference)?
Krugman, Dean Baker and others are linked in this thread from 4 weeks ago, speaking about "the 90 percent zombie".

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 07:25:35 PM EST
[ Parent ]
Well, I have not seen anyone commenting on it yet, but for a great laugh just look at the Figures 3 and 4 of the Herndon et al. paper. Doing a regression out of something that looks entirely random ... In my untrained eye it shows that no correlation should be actually inferred between the two variables (PUBLIC Debt/GDP and GDP growth rate). I am almost certain that the actual correlation value is so low it would be laughable in any physical science.

I am guessing that the hidden story is where total debt is involved and the correlations will start to show when the game of peak oil musical chairs starts to play out. Once it is understood that the current amount of debt (and wealth) cannot be materialized period as it exceeds in multiples the planet's resources.

Orthodoxy is not a religion.

by BalkanIdentity (balkanid _ at _ google.com) on Wed Apr 17th, 2013 at 04:26:12 AM EST
[ Parent ]
I think you are confusing physical wealth with money (debt), me thinks.

You can solve the debt problem in a peak-oil environment. Inflation.

(If the economists here disagree, I would really like to have their view).

by cagatacos on Wed Apr 17th, 2013 at 05:55:54 AM EST
[ Parent ]
Actually, that is precisely my point in the second paragraph. I think  current financial system is making that assumption. I am pointing out the confusion. Inflation yes, will solve the problem but at the cost of massive misallocation of investment until it starts to happen.

To put it simply, and I think most of us agree on it here so it is more like preaching to the choir, a financialized economy cannot survive in a resource-constrained world but the point of its collapse is pushed further into the future than it should and this reduces substantially the efforts necessary for a pain-minimization transition strategy.

Orthodoxy is not a religion.

by BalkanIdentity (balkanid _ at _ google.com) on Wed Apr 17th, 2013 at 06:30:22 AM EST
[ Parent ]
A financialized economy cannot survive long-term, because a financialized economy depends for its wealth on colonial tribute, which it cannot sustain without an industrial plant sufficient to produce the armaments required to maintain the empire which provides said tribute. This would be the case even if the world were infinite.

Economists confuse the collapse of financial sandcastles with real resource constraints at the peril of talking nonsense.

Of course real resource constraints are real. But they're not what's driving the present depression.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Wed Apr 17th, 2013 at 06:51:30 AM EST
[ Parent ]
A financialized economy cannot survive long-term, because a financialized economy depends for its wealth on colonial tribute,

Financial institutions cannot survive, in their majority. But a few entities might be able to continue extract tributes for all that leveraging of the 2000s, economy and resources might be dammed. Yeah, they will "suffer" in absolute wealth terms. But what are rational choices for a universe master? He won't be better off releasing his prerogatives and sharing what is left with everyone. While most people are desperate for money, while (those few) universe masters are in friendly terms between themselves and hold all "policy making" power, their position is safe. Inflation threats, financial crises are just stories that keep their nearest competition working hard, blindly.

by das monde on Wed Apr 17th, 2013 at 07:40:37 AM EST
[ Parent ]
Of course real resource constraints are real. But they're not what's driving the present depression.

I wonder what part of this depression is already partially influenced by resource constraints? A week ago or so Krugman was talking about the inability of his model to predict the mild inflation (where he was predicting deflation). I would offer the following explanation: even with low demand on the "Western world", some resources (e.g. oil) area already so constrained (growth in BRICs + ongoing depletion) that resource scarcity is imposing a measure of inflation. Case in point: the relative price of oil in this depression is much higher than in previous recessions.

by cagatacos on Wed Apr 17th, 2013 at 08:11:52 AM EST
[ Parent ]
That's because Krugman is using an inflation model that does not account for changes to the terms of trade.

A rookie mistake, but one that is quite common among even economists who ought to know better.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Wed Apr 17th, 2013 at 08:33:37 AM EST
[ Parent ]
That borders on the inexcusable for a guy whose [Bank of Sweden] Nobel [Memorial] Prize is for work in international trade.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 08:37:24 AM EST
[ Parent ]
No, it's completely par for the course.

The neoclassical international trade literature is such utter contemptible, pseudoscientific cargo cult garbage that it shocked even me when I took that class.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Wed Apr 17th, 2013 at 08:54:46 AM EST
[ Parent ]
utter contemptible, pseudoscientific cargo cult garbage


guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 08:59:18 AM EST
[ Parent ]
Heh. My model failed. That can only mean two things: Not enough people are unemployed or downward sticky wages.
by generic on Wed Apr 17th, 2013 at 09:48:26 AM EST
[ Parent ]
by afew (afew(a in a circle)eurotrib_dot_com) on Wed Apr 17th, 2013 at 09:39:31 AM EST
[ Parent ]
Col-o-nial
by afew (afew(a in a circle)eurotrib_dot_com) on Wed Apr 17th, 2013 at 09:39:55 AM EST
[ Parent ]
What cagatacos said, but stronger:

A modern industrial economy requires, judging by the historical experience of the 20th century, around 5-7 per cent annual nominal growth.

It does not really require any real growth. So if the real economy can only deliver -1 per cent annual real growth, then inflation will have to be on the order of 6-8 per cent.

Sucks to be a bank shareholder in that world, but nobody ever promised that the world would be nice.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Wed Apr 17th, 2013 at 07:00:28 AM EST
[ Parent ]
A modern industrial economy requires, judging by the historical experience of the 20th century, around 5-7 per cent annual nominal growth.

Newbie question...

Why? What is there structurally in an modern economy that requires such?

by cagatacos on Wed Apr 17th, 2013 at 08:17:47 AM EST
[ Parent ]
There are good reasons why nominal growth should be necessary for a money economy. They basically all come down to the fact that if you do not have a larger aggregate cash flow tomorrow than you did today, then you will go into a debt-deflation spiral if there is any net cashflow leakage: People paying down more on their mortgages than they take out in new mortgages, people putting money into their mattress, a current account deficit unmatched by a public deficit, and so on and so forth and etc. Having a bigger nominal cash flow tomorrow than you had today gives you a buffer against this sort of things.

Why precisely 6-7 per cent per year? No idea, but industrial economies that go below that figure seem to blow up with rather distressingly increased frequency.

("Judging by historical experience" is econospeak for "damfino why, but whenever we've tried to not-do this Shit Blew Up.")

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Wed Apr 17th, 2013 at 08:32:34 AM EST
[ Parent ]
Inflation is financially stabilizing. At lower nominal growth, normal cash flow fluctuations will trigger insolvency cascades.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 08:33:14 AM EST
[ Parent ]
News is hitting the Tech Press.

Microsoft Excel: The ruiner of global economies?

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Wed Apr 17th, 2013 at 02:09:36 PM EST
[ Parent ]
No, wait. The bug wasn't in Excel, it was in the formula. Guns don't kill people. People kill people.

No, wait. Excel should be banned.

No, wait. Researchers should undergo a background check and psychological profiling before being issued with an Excel permit.

It is rightly acknowledged that people of faith have no monopoly of virtue - Queen Elizabeth II

by eurogreen on Thu Apr 18th, 2013 at 03:56:54 AM EST
[ Parent ]
While I tend to agree with your assessment in the broader picture it's nice to see some push-back, making it more likely it will become A Thing.

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Thu Apr 18th, 2013 at 01:46:19 PM EST
[ Parent ]
The Weird and Very Real World of Excel-Error Research

"This type of stuff probably happens all the time. What's unusual here is that someone checked it," says Ray Panko, a professor at the University of Hawaii and the elder statesman of small coterie of research academics who study--yes--Excel errors. Panko's meta-analysis of several such studies, he claims, indicates that 84 percent of spreadsheets contain some kind of materially significant error. But fellow Excel guru Stephen Powell, at Dartmouth's Tuck School of Business, counters that it's hard to know what the exact figure is, but he agrees that Rogoff's recent error is not anomalous--Powell says he's examined "hundreds, if not thousands," of spreadsheets from companies with errors just as egregious.

There are ample anecdotes to support Panko and Powell's claims. In 2012, JP Morgan discovered that it had been grossly miscalculating a risk-management measure--possibly for years--due to a mischievous Excel formula. Last summer, Olympics ticketholders were baffled when the London 2012 Olympic Games committee oversold the synchronized swimming event by thousands of tickets--a hapless clerk had entered "20,000" instead of "10,000" in a spreadsheet cell.  A year prior, the town of Framingham, Massachusetts, announced that it would request $600,000 in state aid to cover a budget shortfall caused by miscalculating "monstrous spreadsheets." You can even find huge copy-paste errors in Fed Reserve spreadsheets if you comb through them.

by das monde on Fri Apr 19th, 2013 at 08:19:52 AM EST
[ Parent ]
This, boys and girls, is the reason you always-always-always validate your model output against actual practical experience.

No, I don't care how onerous that is for you. Do it anyway.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Fri Apr 19th, 2013 at 10:39:25 PM EST
[ Parent ]
See this comment thread on Cyprus...
Speaking at conference [ECB Board member Joerg Asmussen] said abandoning the tough reforms and raising spending instead would not solve the debt crisis and would merely shift the problems to the future. He said:
It is an illusion to think that more debt is the answer to this debt crisis. Recent research has shown that high public debt levels in the euro area hamper growth, with a serious negative effect starting when debt exceeds 90% of GDP.
With links to Dean Baker, Paul Krugman, and en explanation of how the research was published but not in a peer-reviewed publication.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 02:31:51 PM EST
[ Parent ]
Fatal Sensitivity  James Kwak   Baseline Scenario

While the spreadsheet problems in Reinhart and Rogoff's analysis are the most most obvious mistake, they are not as economically significant as the two other issues identified by Herndon, Ash, and Pollin: country weighting (weighting average GDP for each country equally, rather than weighting country-year observations equally) and data exclusion (the exclusion of certain years of data for Australia, Canada, and New Zealand). According to Table 3 in Herndon et al., those two factors alone reduced average GDP in the high-debt category from 2.2% (as Herndon et al. measure it) to 0.3%.*

In their response, Reinhart and Rogoff say that some data was "excluded" because it wasn't in their data set at the time they wrote the 2010 paper, and I see no reason not to believe them. But this just points out the fragility of their methodology. If digging four or five years further back in time for just three countries can have a major impact on their results, then how robust were their results to begin with? If the point is to find the true, underlying relationship between national debt and GDP growth, and a little more data can cause the numbers to jump around (mainly by switching New Zealand from a huge outlier to an ordinary country), then the point I take away is that we're not even close to that true relationship.

In their response, Reinhart and Rogoff also argue that it is correct to weight by country rather than by country-year. Their argument is basically that weighting by country-year would overweight Greece and Japan, which had many years with debt above 90% of GDP. Herndon et al. recognize this point:

   "RR does not indicate or discuss the decision to weight equally by country rather than by country-year. In fact, possible within-country serially correlated relationships could support an argument that not every additional country-year contributes proportionally additional information. . . . But equal weighting by country gives a one-year episode as much weight as nearly two decades in the above 90 percent public debt/GDP range."

Both weighting methods are flawed. (Country-year weighting is only flawed if there is serial correlation, which there probably is; if the U.K.'s nineteen years with debt greater than 90% of GDP were independent draws, then it should be weighted 19 times as much as a country with only one such year.) But this brings me to the same point as above: if your results depend heavily on the choice of one defensible variable definition rather than another, at least equally defensible definition, then they aren't worth very much to begin with.



"It is not necessary to have hope in order to persevere."
by ARGeezer (ARGeezer a in a circle eurotrib daught com) on Sat Apr 20th, 2013 at 01:30:05 AM EST
[ Parent ]
...if your results depend heavily on the choice of one defensible variable definition rather than another, at least equally defensible definition, then they aren't worth very much to begin with.

Which could be another reason it was a good idea for R&R to present their article in a proceedings venue.

"It is not necessary to have hope in order to persevere."
by ARGeezer (ARGeezer a in a circle eurotrib daught com) on Sat Apr 20th, 2013 at 01:32:43 AM EST
[ Parent ]
Coppola Comment: Reframing Reinhart & Rogoff
Suffice it to say that Reinhart & Rogoff were wounded by a paper that showed that their data was flawed. They defended themselves, and were also defended by numerous economists around the world who argued that although the data might be flawed, the economic analysis justified their conclusions. But the next day, the Rortybomb blog delivered the killer punch. Econometric analysis by Arindrajit Dube demonstrated that even with good data, the economic analysis was flawed and the conclusions unjustifiable. High public debt cannot reliably be shown to cause low growth. But low growth can reasonably reliably be shown to cause high public debt.

This is a no-brainer, actually. There are two reasons for this.

Frances Coppola is always worth reading.

by afew (afew(a in a circle)eurotrib_dot_com) on Sat Apr 20th, 2013 at 02:07:28 AM EST
[ Parent ]
This has more to do with (economic) science than the R&R controversy per se:

What leaps out of his General Theory is that it is entirely unencumbered by empirical evidence. Keynes thus helped to promote an ideal of the economist as a brilliant man capable of solving problems from his armchair by dint of superb intellect*. What got undervalued in this was the importance of the mundane grunt work of careful fact-gathering.

Reinhart and Rogoff's errors, I suspect, reflect a culture which prizes brilliance - and no-one doubts that Rogoff is brilliant - over dull pedantry.

by cagatacos on Wed Apr 17th, 2013 at 12:04:31 PM EST
Oh, come on, that ideal is mostly due to Samuelson and Debreu. Keynes, being a trained mathematician, had a distaste for the "mock precision" of using "differential calculus" to do economics...

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 12:19:25 PM EST
[ Parent ]
While I do not have an opinion on the subject, it seems that you are confusing a penchant for quantitative precision with being theoretical. Two completely different things.

You can be highly theoretical while scorning quantitative precision. This makes sense in many areas where you cannot measure (or even know) all the variables influencing a certain parameter of study. So you simplify and try to make your models a good QUALITATIVE approximation.

Now, I never read much of Keynes (something that is bound to change very very soon), but it seems that he was clearly not on the "quantitative precision" camp. If he cared to check is models for empirical evidence, that I do not know and is a completely different affair.

by cagatacos on Wed Apr 17th, 2013 at 12:27:55 PM EST
[ Parent ]
If he cared to check is models for empirical evidence, that I do not know and is a completely different affair.
As Minsky said, an elementary criterion for an economic theory is that actually observed phenomena need to be possible outcomes of the theory. Keynes didn't need any more empirical justification than "look around, we're in a depression", just like Irving Fischer. Classical theory did not allow a depression because it assumed full employment. Neoclassical theory pretty much suffers from the same disease.

Also, econometrics was in its infancy in the 1930s. Simon Kuznets had only recently been working on US series of GNP.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper

by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 12:40:44 PM EST
[ Parent ]
Okay, some General theory:
THE PROPENSITY TO CONSUME: I. THE OBJECTIVE FACTORS

...

That it is investment, rather than net investment, which emerges from the statistics of output, is brought out forcibly and naturally in Mr Colin Clark's National Income, 1924-1931. He also shows what a large proportion depreciation, etc., normally bears to the value of investment. For example, he estimates that in Great Britain, over the years 1928-1931, the investment and the net investment were as follows, though his gross investment is probably somewhat greater than my investment, inasmuch as it may include a part of user cost, and it is not clear how closely his 'net investment' corresponds to my definition of this term:

[Data table]

Mr Kuznets has arrived at much the same conclusion in compiling the statistics of the Gross Capital Formation (as he calls what I call investment) in the United States, 1919-1933. The physical fact, to which the statistics of output correspond, is inevitably the gross, and not the net, investment. Mr Kuznets has also discovered the difficulties in passing from gross investment to net investment. 'The difficulty', he writes, 'of passing from gross to net capital formation, that is, the difficulty of correcting for the consumption of existing durable commodities, is not only in the lack of data. The very concept of annual consumption of commodities that last over a number of years suffers from ambiguity'. He falls back, therefore, 'on the assumption that the allowance for depreciation and depletion on the books of business firms describes correctly the volume of consumption of already existing, finished durable goods used by business firms' On the other hand, he attempts no deduction at all in respect of houses and other durable commodities in the hands of individuals. His very interesting results for the United States can be summarised as follows:

Several facts emerge with prominence from this table. Net capital formation was very steady over the quinquennium 1925-1929, with only a 10 percent

[Data Tables]

increase in the latter part of the upward movement. The deduction for entrepreneurs' repairs, maintenance, depreciation and depletion remained at a high figure even at the bottom of the slump. But Mr Kuznets' method must surely lead to too low an estimate of the annual increase in depreciation, etc.; for he puts the latter at less than 1½ per cent per annum of the new net capital formation. Above all, net capital formation suffered an appalling collapse after 1929, falling in 1932 to a figure no less than 95 per cent below the average of the quinquennium 1925-1929.

Thus,
What leaps out of his General Theory is that it is entirely unencumbered by empirical evidence.
is bollocks. Which appears to be par for the course in economic commentary.

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 12:50:20 PM EST
[ Parent ]
For an authoritative (and sensible) econometrician, take Mark Thoma: Empirical Methods and Progress in Macroeconomics (April 17, 2013)
The blow-up over the Reinhart-Rogoff results reminds me of a point I've been meaning to make about our ability to use empirical methods to make progress in macroeconomics. This isn't about the computational mistakes that Reinhart and Rogoff made, though those are certainly important, especially in small samples, it's about the quantity and quality of the data we use to draw important conclusions in macroeconomics.

...

The point is that for a variety of reasons - the lack of experimental data, small data sets, and important structural change foremost among them - empirical macroeconomics is not able to definitively say which competing model of the economy best explains the data. There are some questions we've been able to address successfully with empirical methods, e.g., there has been a big change in views about the effectiveness of monetary policy over the last few decades driven by empirical work. But for the most part empirical macro has not been able to settle important policy questions. The debate over government spending multipliers is a good example. Theoretically the multiplier can take a range of values from small to large, and even though most theoretical models in use today say that the multiplier is large in deep recessions, ultimately this is an empirical issue. I think the preponderance of the empirical evidence shows that multipliers are, in fact, relatively large in deep recessions - but you can find whatever result you like and none of the results are sufficiently definitive to make this a fully settled issue.

I used to think that the accumulation of data along with ever improving empirical techniques would eventually allow us to answer important theoretical and policy questions. I haven't completely lost faith, but it's hard to be satisfied with our progress to date. It's even more disappointing to see researchers overlooking these well-known, obvious problems - for example the lack pf precision and sensitivity to data errors that come with the reliance on just a few observations - to oversell their results.




guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 12:24:01 PM EST
[ Parent ]
Reinhart and Rogoff's errors, I suspect, reflect a culture which prizes brilliance - and no-one doubts that Rogoff is brilliant - over dull pedantry.

Evil it is then?

Be nice to America. Or we'll bring democracy to your country.

by Drew J Jones (pedobear@pennstatefootball.com) on Wed Apr 17th, 2013 at 12:33:59 PM EST
[ Parent ]
How about Gross Intellectual Incompetence?

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Wed Apr 17th, 2013 at 01:01:41 PM EST
[ Parent ]
'Twould have been dull pedantry to get the row number right.
by afew (afew(a in a circle)eurotrib_dot_com) on Wed Apr 17th, 2013 at 02:50:56 PM EST
[ Parent ]
by Metatone (metatone [a|t] gmail (dot) com) on Fri Apr 19th, 2013 at 10:57:13 AM EST
[ Parent ]
Great Leap Forward: NO, ROGOFF AND REINHART, THIS TIME IS DIFFERENT! SLOPPY RESEARCH AND NO UNDERSTANDING OF SOVEREIGN CURRENCY (April 17th, 2013)
We noticed that their data just did not add up. Leave to the side the silliness of simply aggregating across 8 centuries of experience, and adding up debt ratios of countries as disparate as the USA today or, say, Greece in 1932, let alone some feudal state operating on a gold standard a couple of hundred years ago. As I've remarked, any real historian would find the methodology ludicrous.

More importantly, they have no idea what sovereign debt is. They add together government debts issued by states on gold standards, fixed exchange rates and floating rates. They aggregated across governments that issue debt in their own currency and states that issue debt denominated in foreign currency. It is not even possible to determine from their book exactly what is government debt versus private debt.

When we couldn't make sense of their results, Yeva wrote to them to get the data. After all, their book touted their contribution to good research by proclaiming they were accumulating all this data for the good of humanity. They ignored our request. I have heard from several other researchers that Rogoff and Reinhart also ignored their repeated requests for the data.



guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper
by Migeru (migeru at eurotrib dot com) on Wed Apr 17th, 2013 at 03:58:39 PM EST
Arin Dube, an econometrician, has a guest post up at Next New Deal with some quick evidence of what most of us expected: Reinhart and Rogoff indeed appear to have it backwards.

UMass > Harvard.

Be nice to America. Or we'll bring democracy to your country.

by Drew J Jones (pedobear@pennstatefootball.com) on Thu Apr 18th, 2013 at 12:55:36 PM EST
From the link:

... while a nonlinearity may be expected at high ratios due to a tipping point, the stronger negative relationship at low ratios is difficult to rationalize using a tipping point dynamic.

sigh

A non-linear Emergent Property tipping point can occur in either a positive or negative summation (aka, non-dissipative) Feedback Loop.  

There's a serious flaw in the Model if it cannot handle such things.

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Thu Apr 18th, 2013 at 01:44:34 PM EST
[ Parent ]
There is no model, this is econometrics.

Also, if you want to infer "reverse causality" from a regression... why not switch the roles of the predictor and the signal variables?

guaranteed to evoke a violent reaction from police is to challenge their right to "define the situation." --- David Graeber citing Marc Cooper

by Migeru (migeru at eurotrib dot com) on Thu Apr 18th, 2013 at 02:06:52 PM EST
[ Parent ]
There is a Model.  There has to be ... unless they are just screwing around with numbers -  a postulate that would, for me, be easily proven :-) - but it's unquestioned, unexamined, and untested.  

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre
by ATinNM on Thu Apr 18th, 2013 at 02:47:04 PM EST
[ Parent ]
Would you lot be quiet and let us win this one?

/curses self for sleeping through stats

Be nice to America. Or we'll bring democracy to your country.

by Drew J Jones (pedobear@pennstatefootball.com) on Thu Apr 18th, 2013 at 06:11:40 PM EST
[ Parent ]
lol

Oh, all right.  Me shut up.

She believed in nothing; only her skepticism kept her from being an atheist. -- Jean-Paul Sartre

by ATinNM on Thu Apr 18th, 2013 at 06:31:06 PM EST
[ Parent ]

Guest Post: Reinhart/Rogoff and Growth in a Time Before Debt | Next New Deal
As is evident, current period debt-to-GDP is a pretty poor predictor of future GDP growth at debt-to-GDP ratios of 30 or greater--the range where one might expect to find a tipping point dynamic.  But it does a great job predicting past growth.


It is rightly acknowledged that people of faith have no monopoly of virtue - Queen Elizabeth II
by eurogreen on Fri Apr 19th, 2013 at 07:21:20 AM EST
[ Parent ]
Harvard has a reputation for poaching academics with big names in popular culture, and giving them cushy teaching free jobs to keep them.  It does not have a reputation for excellence in research.  The thing it does really well is undergraduate education, something which these big names have nothing to do with.
by Zwackus on Thu Apr 18th, 2013 at 11:35:26 PM EST
[ Parent ]
Are Reinhart and Rogoff Right Anyway? - The Baseline Scenario
GDP growth vs Public debt photo GDPgrowthvsPublicdebt.png

Yes, we still have that downward slope. But two things:

1. There's no cliff at 90 percent, which was the central finding of the original paper. This is the second sentence of that paper:

"Our main result is that whereas the link between growth and debt seems relatively weak at `normal' debt levels, median growth rates for countries with public debt over roughly 90 percent of GDP are about one percent lower than otherwise; average (mean) growth rates are several percent lower."

2. Aren't we only supposed to be interested in empirical results that are significant? What the figure from Herndon et al. says, in their words, is this:

"Between public debt/GDP ratios of 38 percent and 117 percent, we cannot reject a null hypothesis that average real GDP growth is 3 percent."

I find it hard to agree with Reinhart and Rogoff when they say, "We do not, however, believe this regrettable slip affects in any significant way the central message of the paper."



"Dieu se rit des hommes qui se plaignent des conséquences alors qu'ils en chérissent les causes" Jacques-Bénigne Bossuet
by Melanchthon on Thu Apr 18th, 2013 at 07:33:31 PM EST


Display:
Go to: [ European Tribune Homepage : Top of page : Top of comments ]