Display:
Because firstly, most of their results are frankly not clinically interesting. In most cases (and for many kinds of woo-woo in all cases investigated), the improvement felt by the individual comes down to confirmation bias, regression to the mean and various and sundry other well-known mental short-circuits.

Realigning your Chakras will cure a common cold in seven days. Doing absolutely nothing will cure a common cold in a week.

Secondly, attempting to claim equivalence between bioelectric potentials and Chi flows is just plain silly. This is precisely the kind of rhetorical slight-of-hand and "gotcha" games that makes people who actually spend some time studying science exasperated with woo-woos.

Thirdly, even if a religious ritual happens to chance upon something that works above and beyond confirmation bias, placebo effect and what have we, it does not necessarily validate the underlying model.

To illustrate this, let me tell a little story:

1000 people went picking mushrooms one by one. Each found two mushrooms - one of which is instantly lethal when consumed, the other is harmless - and each of them flips a coin to decide which to eat for dinner.

The next day, the remaining five hundred go out into the woods and find two mushrooms - one dangerous and one harmless. Again, each flips a coin to decide which one to eat.

This repeats itself on the third day.

On the fourth day, the remaining 125 people meet for a Grand Council where they relate their recent experiences in the craft of mushroom picking.

They then conclude that flipping a coin to decide which mushrooms to eat makes perfect sense - after all, it's worked for all of them for three days in a row, which cannot possibly be a coincidence.

Attending the council meeting is a botanist, who happens to have a handbook on mushroom picking. He courteously explains that his professional opinion is that using the handbook to discern between poisonous and edible mushrooms is superior to using the coin-flip method.

The coin-flippers tell the botanist that they already know which mushrooms are edible: The coin told them, and what the coin told them matches what the botanist's science tells them. So obviously the two must be equivalent. Or, in fact, the coin-flipping method is slightly superior on account of having found the edible mushrooms first and faster and with less fuss and bother. So they'll stick to the coin-flipping method, thankyouverymuch.

The council is dissolved and each coin-flipper goes his own ways to search for mushrooms.

Ten days later, the botanist is the only one attending the meeting who is still alive.

The End.

- Jake

If you only spend 20 minutes of the rest of your life on economics, go spend them here.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Mon Oct 20th, 2008 at 07:54:20 AM EST
[ Parent ]
LOL

A pertinent comment in Finland at this autumn mushroom-picking time ;-)

However, you forgot to add to your story the alternative botanist who carefully collected and consumed Amanita muscaria, and who, for short periods of time, regarded the Council and its members as a cosmic energy flow in another dimension.

You can't be me, I'm taken

by Sven Triloqvist on Mon Oct 20th, 2008 at 08:05:44 AM EST
[ Parent ]
Can't rightly take credit for the story - it's a rip-off of one of Orac's old posts, where he uses day-traders as the example. Too lazy to dig it out, though.

- Jake

If you only spend 20 minutes of the rest of your life on economics, go spend them here.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Mon Oct 20th, 2008 at 08:16:25 AM EST
[ Parent ]
JakeS:
Because firstly, most of their results are frankly not clinically interesting. In most cases (and for many kinds of woo-woo in all cases investigated), the improvement felt by the individual comes down to confirmation bias, regression to the mean and various and sundry other well-known mental short-circuits.

This isn't actually true. It's probably impossible to investigate woo woo without confirmation bias on either side, and I can certainly think of at least one disconformation study with a methodology which would have been rightly ripped to shreds or ignored if it hadn't produced a negative result.

In the placebo studies I mentioned, the results were statistically significant. In fact they were more significant than the results of mainstream drug studies - not that that's a high standard, necessarily. But even so.

by ThatBritGuy (thatbritguy (at) googlemail.com) on Mon Oct 20th, 2008 at 09:32:09 AM EST
[ Parent ]
You always get bad papers in any discipline, on any subject. The question is whether you get good papers on the subject or not.

- Jake

If you only spend 20 minutes of the rest of your life on economics, go spend them here.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Mon Oct 20th, 2008 at 11:34:57 AM EST
[ Parent ]
Or how easy it is to get them researched in the first place, and published in 'serious' journals if they challenge conventional wisdom.

But that just highlights the double standard - this study was quoted by 'serious' researchers as if it was definitive and utterly professional, when in fact it was a very small data set collected under less than strictly controlled conditions as a high school student science fair project.

by ThatBritGuy (thatbritguy (at) googlemail.com) on Mon Oct 20th, 2008 at 05:25:31 PM EST
[ Parent ]

Display:
Login
. Make a new account
. Reset password
Occasional Series