Matthew Ladner responds

by David Safier

Matthew Ladner commented on the concerns I have expressed, along with others, about the survey Strategic Vision LLC was paid to conduct for G.I. Here are his comments, in full.

David-

I believed you linked to a post last week where I explained that I am awaiting further evidence from Strategic Vision. I'll make a few additional comments here.

If this survey were a fraud, it was an elaborate fraud indeed as from the start SV provided me with not only cross-tabs but also with raw data. It wouldn't be impossible for someone to sit around and type in the thousands of records it would take to produce such a file, but it would be very strange to do so.

Second, SV also surveyed private school students in Arizona, and as you will recall the results of that survey on the civics exam were still terrible. If SV were sitting around in Georgia manufacturing data to please their clients, this would be an odd way to do so.

Third, I have been in contact with other firms which have used SV, one of which went to the expense of replicating surveys with another firm. The results held up.

Fourth, the students in the SV surveys did about half as well as freshmen students from Harvard did on a similar civics survey that was given in a multiple choice format. The SV survey replicated the U.S. citizenship exam and used an open answer format, which is inherently more difficult than a multiple choice format. You can examine these results here:

http://www.americancivicliteracy.org/2006/summary.html

Finally, I have requested receipts of purchase from SV from the marketing firms. The surveys involved buying phone lists of students. If SV were simply making up data, it stands to reason that they would have simply pocketed the money for the lists.

If SV does not provide these receipts, I will be inclined to purchase a new survey from a different firm. I wouldn't get too excited, the results are likely to do be very similar. If freshman students at Harvard score a D on a civics exam, what is almost certain to be found is yet more evidence that American schools do a terrible job of teaching civics.

MY COMMENT ON LADNER'S COMMENT: I want to make clear, I think this is a poor study for a number of reasons, even if the survey is on the level. I'm not defending the civics knowledge of high school students. My main concern is the way G.I., like many other conservative think tanks and politicians, misuse information to make their point. If the survey is a fraud, that just adds to the evidence. That being said, I'm far from convinced this survey is legit.

You say SV provided you with cross-tabs and raw data. Apparently, the company has not provided that information to others who question the legitimacy of the survey. Maybe you would be good enough to make that information available, along with the receipts from marketing firms, so people who understand polling better than I can look it over. Who knows, it may vindicate SV. What do you think?