How To Lie With Numbers 1½

by JakeS Mon Oct 22nd, 2007 at 04:31:39 PM EST

The honest use of bar graphs and a warning note

Last time, I warned the reader to beware of bar graphs, especially when they are used to support claims of correlation. What I did not include was an example of legitimate use of bar graphs - because I didn't have one close at hand and I was too lazy to search the web for it.

Fortunately, the kind people over at Scienceblogs (specifically Matt Nisbet) has provided a ready-made example, that both presents honest use of bar graphs and permits me to drive home another point that I've been wanting to make.

Diary rescue by Migeru

Why, one might ask, is Nisbet's use of bar charts more honest than Svenskt Näringsliv's use? Simple: Nisbet's purpose is to rank different responses to a survey according to the number of replies. And bar graphs are very good for that, because they provide a direct link between the size of each bar - which is easy for the human mind to evaluate to fairly high precision - and the value of the entry it represents.

(The first figure in Nisbet's post is something of another story - connecting the points in a time series is not what I'd call unethical or sloppy, but I find it aesthetically displeasing. That, however, is somewhat more a matter of taste.)

The other point I was hinting at can be introduced by way of the comments section on his post. Here Nisbet is challenged to justify the attention he gives to the data. This goes to show that presenting your data honestly is not the be-all-end-all of making (valid) numerical arguments. It is the minimum entry requirement. The numbers that you present have to actually be relevant to the conclusion you're making.

Furthermore (and this is another thing that's brought up in the comments over there), in some cases data ages more gracefully than in others. For instance, data on what the climate was like a century ago is still relevant to climate models today - data on the weather at some specific location at a specific time during the last century is little more than a curiosity today (unless it was fairly extreme, such as the flooding of the Low Countries in 1954).

Conclusions:

Bar graphs are properly used to compare quantities - (naturally, such quantities as are compared must be comparable). This makes them particularly useful to present the results of polls, surveys and elections.

That someone isn't lying doesn't mean he isn't wrong - just because you can't catch someone red-handed in manipulating data is no excuse to disengage your other critical thinking processes.

An Aside: Since bar graphs provide a perfectly good way of presenting the results of surveys and elections, I wish people would stop using pie charts for that - they are inferior to bar graphs in almost every way I can think of.

Previous Conclusions:

Beware of bar graphs - if someone tells you that X causes Y and presents you with bar graphs, scrutinize them carefully. The proper graph to show correlation is in most cases a scatterplot. If he's using something else, chances are he's trying to pull a fast one on you.

Especially beware of highlighting - I'm sure highlighting single data points has legitimate uses, but off the top of my head, I cannot think of a single one. A very good indication that Someone Is Up To No Good.

The Entire Series:

How To Lie With Numbers - a short guide to politics and other things - introduction - bar graphs - highlighting.

How To Lie With Numbers 1½ - more bar graphs - a cautionary note

European Tribune - How To Lie With Numbers 2 - Laffer Nonsense From The WSJ - scatterplots- fitting methods - data grouping

- Jake

How To Lie With Numbers 1½ | 8 comments (8 topical, 0 editorial, 0 hidden)

Re: How To Lie With Numbers 1½ (4.00 / 2)

Thanks for this. It's good to have a basic guide to how statistics are manipulated. Frequently people just flat out distrust statistics because they know that they can be manipulated. Here you ar supplying the non statisticians among us with the necessary tools to separate the wheat from the chaf.

Any idiot can face a crisis - it's day to day living that wears you out.

by ceebs (ceebs (at) eurotrib (dot) com) on Mon Oct 15th, 2007 at 07:58:57 PM EST

Re: How To Lie With Numbers 1½ (none / 0)

Is it very wrong to use wrong statistics to "prove" something truthful? Are we not restricting ourselves too much by insisting only on formally correct argumentation? Especially in a debate environment, you frequently need to win time by not going into boring right details no one would follow, or by "asking" the opponent to spend his time on debunking if he wishes.

Some magician has said that the biggest magic is to create an illusion adequate to the reality. If so many people live in illusions, some magic might be necessary to wake them up...

by das monde on Mon Oct 15th, 2007 at 10:48:13 PM EST

I would say yes and no to that (4.00 / 2)

It depends on the venue. If the venue is a blog or a debate book - or even in many cases a newspaper article - I think that it is morally incumbent upon the author to at least not be outright dishonest. I also think that it is strategically wise.

If you are right, then reasoned argument will mean that your side carries the day. Thus, if we are more right than the bad guys, we will tend to win arguments more often if the argument is reasoned. Therefore, since I think that we are more right than wrong and more often right than the bad guys, I think that there is a long-term advantage in promoting reasoned debate.

In the short term, however, there are certainly platforms where reasoned debates are simply impossible. Television interviews come forcibly to mind. While I might personally wish that those platforms were less prominent, and while I think that it would be a worthwhile goal in and of itself to lessen the impact of those platforms, the unpleasant reality is that they exist, and any political movement must - at present - have a presence there if it wishes to be successful.

Given those circumstances, it is important to learn how to manipulate those platforms, and even I accept that some dishonesty may be required. I think that simply being less dishonest than your opponent is a viable recipe for using those media that do not allow for fully honourable discussion, without compromising your integrity. It's certainly not in violation of Kant's imperative: If being less dishonest than the other side is made a universal principle, dishonesty would disappear, although Kant would probably protest against this interpretation of his imperative...

More to the point, however, this series is not intended as a guide on how to debate, or even as a guide on ethical debating. The purpose of this guide is to allow the spectator - the man on the fence - to separate the wheat from the chaff, as ceebs put it.

This will (hopefully) tend to accomplish two things, if it succeeds: First, it will diminish - albeit in a minor way - the influence of those media that do not allow for honest presentation of statistics. Second, it will hopefully bias the now more knowledgeable citizen against people making bad arguments - therefore, if we obey the principle of lying less than our opponents (and if our case is stronger, which I think it is), we will come out ahead, if only in a minor way. Both seem to be worthwhile goals.

Finally, providing citizens with the tools to understand the arguments underpinning the various sides in the public debate(s) seems to me to be eminently worthwhile in and of itself.

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Tue Oct 16th, 2007 at 05:58:22 AM EST
[ Parent ]

Re: I would say yes and no to that (4.00 / 4)

The education purpose is very important - humans have enough brains to get basic statistics understanding, bluntly speaking. But some apparently like to thrive on public ignorance, and would always keep people lazy or detesting towards statistical inquiries. How to negate that?

I do not think that the situation on TV is inherently hopeless. Although I did not really follow Western evolution for long, I suspect a sharp devolution of TV intelectual standards in the last decade, catalyzed by the US de-reguliarizing 1996 Tele-Communications Act. It can be said what you see on TV today almost anywhere is what one person (Rupert Murdoch) wants you to see.

In electoral politics, the US coverage became tragicomical (and sadly spreading anywhere else). Just look at the electoral debates - the circus not worth the civilization. The media anti-Gore revolution of 2000 is very revealing - and evolving fast.

The progressives need to get some control of public discourses, for the beginning. They need to re-inforce more rational rules somehow. Giving up and just following stupid scripts of modern millionaire TV hosts is certainly wrong way to go. They need to use rational argumentation to keep it alive on TV - along with necessary sexed-up tactics. Not only rational but emotional brain channels have to be activated as well. The opponents have to be forced to demonstrate decent logic sometimes - and here some provoking methodology slips can play a role. The opposition has already demonstrated how to defend illogical positions momentarily - we would need only a few tricks to take over.

Most importantly, following only strict logic makes us very predictable, while the opposition has well developed tools to dismiss rational argumentation. In this conflict, the progressives need to be more creative in presentation, so to overwhelm opponents a little and take initiative. It is like in Sun Tsu's "The Art of War".

by das monde on Tue Oct 16th, 2007 at 06:48:38 AM EST
[ Parent ]

Re: I would say yes and no to that (4.00 / 3)

The progressives need to get some control of public discourses, for the beginning. They need to re-inforce more rational rules somehow. Giving up and just following stupid scripts of modern millionaire TV hosts is certainly wrong way to go. They need to use rational argumentation to keep it alive on TV - along with necessary sexed-up tactics. Not only rational but emotional brain channels have to be activated as well. The opponents have to be forced to demonstrate decent logic sometimes - and here some provoking methodology slips can play a role.

Agreed. As for how to do it, though, I'm an academic, not a politician, so I am afraid that I will have to defer that part to someone who, unlike me, is actually competent in that area.

However, to deal with what you rightly call 'well developed tools to dismiss rational argumentation' I think that basic numeracy is of paramount importance (alongside, of course, the rhetorical tactics developed to deal with such denialism - but as I said, that falls outside my area of competence).

The unfortunate fact - and a point I usually harp on at considerable length to anyone who cares to listen - is that lack of numeracy in the citizenry means that numerical arguments become essentially arguments from authority. Not only is that unlikely to change minds, it also reduces what should be civilized discourse to the level of tribal shamans pointing to their favoured totem poles and shouting, what is for all the viewer knows, magic incantations at each other.

Thus, highly technical subjects of vital importance to the continued well-being of the citizens are reduced to a question of who has the more convincing witch doctor. And quacks, by virtue of not being bound by such things as facts or ethics, will tend to have an advantage in such a shouting match.

I think that we are largely in agreement here (though I may be wrong about that, of course), but approaching the problem from different angles. Which is well and good - we need a diverse strategy, after all.

---

I need to read Sun Tzu myself. I have a copy, but I never got around to reading it... For that matter, I suspect that Clausewitz would also have some insights applicable to politics - such as the focus on destroying the enemy's capacity to prosecute war against you over and above the simple occupation of territory. That, at least, seems to be the strategy employed by the right in general and the neocons in particular...

- Jake

Friends come and go. Enemies accumulate.

by JakeS (JangoSierra 'at' gmail 'dot' com) on Tue Oct 16th, 2007 at 07:23:40 AM EST
[ Parent ]

Re: I would say yes and no to that (none / 0)

That is a very interesting idea.

We have met the enemy, and it is us — Pogo

by Carrie (migeru at eurotrib dot com) on Mon Oct 22nd, 2007 at 04:29:05 PM EST
[ Parent ]

Hans Rosling (4.00 / 2)

What do you think of Hans Rosling, his GapMinder software, and his two presentations at TED:

Debunking third-world myths with the best stats you've ever seen

New insights on poverty and life around the world

?

Truth unfolds in time through a communal process.

by marco on Tue Oct 23rd, 2007 at 08:54:39 AM EST

Re: Hans Rosling (none / 1)