NY Times, Nate Silver, G.I. and Me

by David Safier

Imagine my surprise as I was reading through NY Times' 9th Annual Year in Ideas in its Sunday Magazine, when I came across the item, Forensic Polling Analysis. It's about uber-statistician Nate Silver's analysis of polling data from Strategic Vision LLC, in which he comes to the conclusion that the company makes its results up. (The statistical strategy Silver used to reach his conclusion is much more understandable to the lay person in this version than in his more lengthy analysis. I get it this time. It's actually ingeniously simple.)

This story should sound familiar to regular BfA readers, because I've posted about the fact that the Goldwater Institute's July 4 study "showing" that high school students know nothing about civics (but private school students know a little more than public school students) was "conducted" by Strategic Vision LLC. In his analysis on his blog, FiveThirtyEight, Nate Silver used the survey for G.I. as one exhibit indicating that the survey company is basically little more than a bunch of post office boxes and a few guys who know how to fill out spread sheets. It sure saves on the phone bills, not to mention the labor costs, if you conduct a phone survey without actually phoning anyone.

Which brings me to mention this:

G.I. (and especifically Matthew Ladner), I haven't forgotten about you. The civics survey is a potential stink bomb hiding in your offices which could go off at any time. Your credibility, which I often question on BfA anyway, is kinda on the line here. There's reason to believe the survey company you used is less than reputable. If you let a possibly bogus study sit out there with your name on it without doing everything you can to assure your audience that you have done a thorough investigation of the conduct of the survey — and remember, this study got national attention — you don't look so good. You've got researchers there, and resources, and the right to question the company you paid money to. If you were taken for a ride, admit it. If the survey is valid, you need to present evidence to expert skeptics like Nate Silver and others that their suspicions are unfounded.

I plan to keep bringing this up on a semi-regular basis. I hope G.I. will get to the bottom of the matter before, say, next July 4 rolls around. Our Founding Fathers (and Mothers) deserve no less.