Why most opinion polls are crap: abstractions v. specifics

Posted by AzBlueMeanie:

As someone trained in political science who has spent my entire adult life reading opinion polls and studies about polls, this report comes as no surprise to me. What I find hard to believe is how the media villagers waste so much time talking about their polls but never discuss the reason why their polls are crap: they test abstractions, not specifics.

Could it be because discussing this would undermine the Beltway media villager "conventional wisdom" that this is a center-right country? Abstractions produce the desired result for their "conventional wisdom." But specifics (reality) undermines their so-called "conventional wisdom."

Practice tip: one can always produce the desired result by how one frames the questions and limits options. This is how the media and their pollsters manipulate public opinion.

Ezra Klein posts today, Still true after 40 years: Voters prefer cuts in theory, spending in practice:

In 1967, the political scientists Lloyd Free and Hadley Cantrill wrote
that Americans were “ideological conservatives” but “operational
liberals.” What they meant was that when asked broad questions about how
government should work and what it should do, voters responded like
conservatives. But when asked operational questions about which programs
should be cut and which services should be eliminated, they responded
like liberals. Voters like big cuts and smaller government in theory,
but they don’t want to actually cut anything in practice
.

Forty years later, this observation is, in political science circles,
utterly banal. But in Washington, D.C., it’s almost entirely ignored.
Which brings me to a particularly amusing poll from the Hill newspaper
today. Their headline
is: ”Voters prefer GOP ideas on budget, but dislike the Republican
Party”. My headline would be: “Voters prefer spending cuts in theory,
spending in practice.

* * *

Here, then, is a more accurate summary of the poll. When asked
whether they’d prefer a budget with huge spending cuts and a balanced
budget to one with tax hikes and no balanced budget, voters say they’ll
take the spending cuts, thanks. When asked whether they prefer spending
cuts to tax hikes, voters say, again, they want the spending cuts. When
asked whether they prefer the budget gets written by the party that
support huge spending cuts or the party that supports tax hikes, they
name the party that supports tax hikes. When asked whether Obamacare
should be fully repealed, most voters say “no.” And finally, when asked
if maybe we can just cut spending by getting rid of a piddling little
program like White House tours, voters overwhelmingly say no.

There’s nothing to marvel at here. Voters have been like this since
at least the 1960s. The wonder is that anyone still commissions polls
asking Americans whether they support theoretical spending cuts and then
trumpet the response as if it means something
.

In February, the Pew Research Center for People and the Press
decided to drill deeper. They asked Americans whether they’d like to
increase, decrease, or simply sustain spending in 19 different areas.
There was literally not one area — no, not even foreign aid — in which
“decrease” won out over increase and sustain:

Pew-poll-spending

The past few years have been a war between conservatives who want to keep the budget debate at a high level of abstraction [bumper sticker slogans] and liberals who want to focus on the actual programs and services affected [specifics]. We saw that fight play out in the election, as Republican presidential candidate Mitt Romney and his running mate Rep. Paul Ryan fought to keep their tax plan vague, while the Obama administration fought to make the likely cuts to tax expenditures specific. We’ve seen it in each budget debate as the Obama administration tries to name the cuts implied by the Republican budget while the Republicans deny those cuts. We’ve seen it in the sequester, as the Republicans have embraced the theoretical approach of across-the-board cuts, but fought energetically against the one specific cut to get any attention — the White House tours, of all things. We’ve even seen it on entitlements, where Republicans swear that Medicare is the problem, but also run against the Obama administration’s Medicare cuts.

I have always resented that the media wastes so much time on their crappy polls instead of doing investigative journalism and hard reporting that might actually make a difference in people's lives.

0 responses to “Why most opinion polls are crap: abstractions v. specifics

  1. I don’t think we are disagreeeing, we’re saying the same thing in a different way: “Sure, many polls are useless and those so called polls run by TV and radio shows that rely on volunteer call-ins are even dangerous because some people think they are meaningful. But polls run by reliable polling organizations can serve a useful function.” And “Well constructed and properly administered scientific polls are essential tool for good government.”

    I never said “all” polls, I thought it was clear from the “most” in the caption.

  2. I really have to disagree with the notion that most polls are crap, especially when you use a poll to support your arguments.
    Sure, many polls are useless and those so called polls run by TV and radio shows that rely on volunteer call-ins are even dangerous because some people think they are meaningful. But polls run by reliable polling organizations can serve a useful function. We elect “representatives” who should both reflect the wishes of their constituents and lead them to well thought out positions. It’s obvious that meeting with small groups with special interests, attending rallies with supporters, receiving a barrage of emails or phone calls, or getting advice from experts can each inform an elected representative of her/his constituents’ positions but none of these are fool-proof. Well constructed and properly administered scientific polls are essential tool for good government.
    It’s also true that polls can be misused. The media is often guilty and individuals often misunderstand the results, but that doesn’t make all polls “crap”.