Some words of wisdom about presidential polling

Posted by AzBlueMeanie:

Polls are for media consumption to fill hours and hours of air time and columns and columns in newspapers (and pixels online) with idle speculation and conjecture by media villagers and Beltway bloviators about the supposed state of the "horse race." Almost none of them have any background in political science or statistics, nor do they have the slightest clue what the hell they are talking about. They are into "infotainment."

When I was studying political science a half of a lifetime ago and working on Democratic campaigns, a wise Democratic campaign operative told me "Kid, don't even bother looking at polls until the end of September. Whoever is ahead at the end of September is going to win on election day." (He was talking about presidential polling).

And sure enough, these words of wisdom have held true, with two notable exceptions, since 1936. Nate Silver wrote about this awhile back in his FiveThirtyEight column in late September. The Statistical State of the Presidential Race – NYTimes.com:

What did the polling look like at this stage in past elections, and how did it compare against the actual results?

Our polling database contains surveys going back to 1936. The data is
quite thin (essentially just the Gallup national poll and nothing else)
through about 1968, but it’s nevertheless worth a look.

In the
table below, I’ve averaged the polls that were conducted 40 to 50 days
before the election in each year — the time period that we find
ourselves in now. (In years when there were no polls in this precise
time window, I used the nearest available survey.)

The table
considers the race from the standpoint of the incumbent party
(designated with the color purple) and the challenging party (wearing
the orange jerseys), without worrying about whether they were Democrats
or Republicans. Mr. Obama’s position, for instance, is probably more
analogous to that of the Republican incumbent George W. Bush in 2004
than it is to the candidate from his own party that year, John Kerry.

This is an awful lot of data, but there are several reasonably clear themes.

First, the polling by this time in the cycle has been reasonably good,
especially when it comes to calling the winners and losers in the race
.
Of the 19 candidates who led in the polls at this stage since 1936, 18
won the popular vote (Thomas E. Dewey in 1948 is the exception), and 17
won the Electoral College (Al Gore lost it in 2000, along with Mr.
Dewey).

* * *

If you eliminate the candidates with double-digit leads, the
front-runner’s record is eight Electoral College wins in 10 tries, or a
batting average of 80 percent.

This a simple method — to the point of being crude. But it’s
interesting, nevertheless, that the 80 percent figure corresponds quite
well with the FiveThirtyEight forecast, which gave Mr. Obama a 78
percent chance of winning as of Sunday night, and with the odds on offer by bookmakers, many of whom list Mr. Obama as about a 4-to-1 favorite.

The second theme is one that we’ve brought up before. There has not been any tendency, at least at this stage of the race, for the contest to break toward the challenging candidate.

Instead,
it’s actually the incumbent-party candidate who has gained ground on
average since 1936. On average, the incumbent candidate added 4.6
percentage points between the late September polls and his actual
Election Day result, whereas the challenger gained 2.5 percentage
points.

Nate Silver has a wealth of statistical analysis in this post well beyond what I have included above. I just wanted to point out this simple method of prediction. With the exception of Thomas Dewey in 1948 (partly due to the existence of third party candidatef Sen. Strom Thurmond, who won 39 electoral votes, and partly due to the fact that Thomas Dewey was a terrible candidate and campaigner), every candidate leading in the polls at the end of September has won in November, by however slim a margin (Al Gore narrowly won the popular vote; he lost the electoral college in court by losing Florida, where Independent candidate Ralph Nader proved to be a spoiler as well). As the old saying goes, "the cake is baked."

So for all of you who are obsessing over every poll that is reported in the media — many of these polls are what I would term "narrative polls" that have a house bias in favor of one political party or the other, or they are using scientifically questionable "likely voter" screens (really just a "best guess" based upon supposedly comparable past elections) — just stop! Relax. Read a good book. Go to the movies. Stop obsessing.

I do not find any evidence in the current batch of polling data to suggest that this election is anything like 1948 or 2000. For one, there is no strong third party or independent candidacy this year. And two, Barack Obama has maintained a polling average lead, or is tied in Florida and Virginia, among the "swing states" that will decide this election. The tried and true formula that late September polling predicts the winner in November is holding true.

One response to “Some words of wisdom about presidential polling

  1. I am retired from the Federal government where I was a survey statistician. That’s my pedigree, if you will. The only polling I trust is Nate Silver. One more point: with the widespread use of cell phones – many in lieu of having a land line, the polling results are dicey, just as they were in 1936 when polls predicted Landon would beat Roosevelt. In that year, phone numbers were the source of selecting respondents. Just one problem with that: only the wealthy had phones, not virtually every household. So, sleep well, pray hard, VOTE!!! and don’t place a lot of reliance on polls – unless it is from Nate Silver!