Welcome to European Tribune. It's gone a bit quiet around here these days, but it's still going.

Public opinion - a brief rant on market research

by PeWi Sat Jan 28th, 2006 at 09:11:00 AM EST

I love Market research. i financed parts of my degree with being a guinea pig and pretending to be a heavy smoker, who had just had a dental operation so I could not smoke during the interview (I hate smoking) - So yes, I am highly cynical about Market research, but I don;t bypass a chance to tick boxes and confuse them... O.k I know enough about statistics that the outliers are usually eliminated, but still, it pays for the sandwich.


Anyway. on this page (don;t go) I came across this survey

 talk about directed questioning.
My collar exploded when they asked:

Which of the following is the principal source of your views on the development of life on Earth?
    - Religious beliefs
    - Science
    - The positions of religious or political leaders
    - The ideas expressed in news media
    - Other/ Not Sure/ Prefer not to say

then I saw their logo. Opinion Outpost. a post with arrows pointing in both directions - to create the impression of a cross.

Poll
Should guided questions in Market research be outlawed
. No - you can always freep the poll 33%
. No - if you have more than one option to answer 0%
. No - they pay for your lunch 0%
. No - where would we be without guidance from above 33%
. No - the researcher is pretty 0%
. No - but actually I don't really want to answer this 33%

Votes: 3
Results | Other Polls
Display:
Some opinion polls are also hidden ways of promotion a policy point. I remember several mails that presented themselves as polls but were in fact trying to get you to question a particular political candidate or issue in favour of another.

People love polls. Think about all the magazines with reader polls. Many pool don't tell you anything more then the opinions of the 10 people who took the poll and as your story illustrates sometimes they don't even tell you that much.

As for the reliability of the poll results that's another question. Leading questions or answer options are one issue but how the poll is conducted, who the respondents are, what the sampling frame is (i.e. how representative is the group you are interviewing compared to the population you are making statements about)are all questions to ask to assess the relevance of any poll results.

by Alexandra in WMass (alexandra_wmass[a|t]yahoo[d|o|t]fr) on Sat Jan 28th, 2006 at 11:33:15 AM EST
Another promotional use of polls involves leading questions. By producing an artificially high/low number in support of a particular policy you can push the media to:

(a) reopen a debate

and/or

(b) start building rationalisations for why people feel so strongly about the issue and why other polls didn't show this before.

Now, this approach isn't always successful, but especially as part of a broader campaign it's a good way of poisoning debate.

by Metatone (metatone [a|t] gmail (dot) com) on Sat Jan 28th, 2006 at 01:47:36 PM EST
[ Parent ]
Why favorite opinion poll question is "do you favor welfare" which always get a low response.

However, if you ask "do you favor giving money to the poor, ill and others unable to care for themselves" you get a high response.

What this shows is that most people's opinions are based upon a superficial response to buzz words. Recently a US poll asked which party has the majority in congress. The result was slightly above 50% correct. Given that there are only two choices, that's not saying much.

For an organization that tries to do a better job look up PIPA.

Policies not Politics
---- Daily Landscape

by rdf (robert.feinman@gmail.com) on Sat Jan 28th, 2006 at 01:14:43 PM EST
OK, first... what the hell were you doing on that site?  Yeah, I know you warned us, but... yeeech.

Second, and more on topic... no survey conducted on the Internet is going to be mistaken for a serious one by anybody who knows what they're talking about.  It's a decidedly unscientific self-selected sample.  Online polls are for novelty purposes only and should not be taken as indicators of anything generalizable.

My problem with opinion research has more to do with the inept way it is often covered in the media.

Even if this were a scientifically conducted poll, it would be rendered invalid based on the directed wording of the question.  It is certainly possible to word questions in a way that they are designed to elicit specific answers, but it's also possible to word them in a way that makes it much more likely to elicit the respondent's real opinion.

Any reporter attempting to write about a survey needs to see the wording of the questions being answered, and if they're asked in a leading way, the research needs to be either disregarded or at least acknowledged for what it is, which is a poll designed to produce a certain result.

I partly financed my graduate education by conducting telephone surveys (what can I say, I have a nice voice), some of which were political (if the election for Senator were held today, would you vote for....) and some commercial (how many cans or glasses of soft drink have you had in the past week?).  I also have some experience in analyzing public opinion research.  There are growing and serious problems with its reliability, especially related to the great unreported secret, the rejection rate.

When it comes to telephone surveys, this is partly the result of the over-polling of certain populations (e.g. Pennyslvania) who then just don't want to take part in any more surveys, and partly a result of technology, e.g. caller ID, which allows people to decide which phone calls to answer.  There's a lot more to it than that, but it's something the industry doesn't like to talk about because it threatens its very existence -- if you can't produce reliable surveys, what's the point of doing them at all?  (Assuming you're one of the people who really wants reliable opinion data....)

Telephone surveys are also, of course, completely useless in societies where most people don't have landlines, e.g. sub-Saharan Africa.

by the stormy present (stormypresent aaaaaaat gmail etc) on Sun Jan 29th, 2006 at 01:25:49 PM EST
I am a regular on this: http://www.memeorandum.com/ site and get my daily dose of wingnuttery from there. That's how I ended up at that particular site.

I did register and they ask you more personal questions - income range, postcode, and some such to probably weigh your answers better, but also to give you more targeted questionnaires.

I got an email from my brother today about this survey that revealed that more than 50% of British subjects don;t believe in evolution. He quoted it from a German publication that hadn't bothered to look at the original survey, otherwise they would have been able to make a slightly better analysis - but that would have meant doing what you suggested, actually looking at the questions and not just at the interpretation in the summary...

Anyway - that survey has been thoroughly debunked - just cannot find anything about it in the moment, so you have to take my word for it...

by PeWi on Sun Jan 29th, 2006 at 01:48:00 PM EST
[ Parent ]
No, I believe you.  I heard about that survey and thought it sounded decidedly unlikely to be reliable.

Actually, one of my favorite (scary) public-opinion stories involves the US survey that re-worded the First Amendment and found that most respondents would oppose having it put into law.  I can't find the link now either, sorry.

by the stormy present (stormypresent aaaaaaat gmail etc) on Sun Jan 29th, 2006 at 02:04:49 PM EST
[ Parent ]
by Samir on Mon Jan 30th, 2006 at 07:44:48 AM EST
[ Parent ]


Display:
Go to: [ European Tribune Homepage : Top of page : Top of comments ]