by David Safier
Friend of the blog Matthew Ladner has done it again with not one, but two studies based on surveys using methodology so poor, no self respecting statistician would take the results seriously. And his conclusions? Let's just say, even if they were accurate, they're pretty underwhelming.

Ladner's two studies for the Goldwater Institute are titled, Better Citizens at a Lower Cost: Comparing Scholarship Tax Credit High School Students to Public School Students and Tough Crowd: Arizona High School Students Evaluate Their Schools. The links take you to summaries of studies maintaining that private school students are getting better educations because they are more politically tolerant, are more willing to volunteer and like their schools better than public school kids. Pretty weak tea if you ask me, even if they were valid conclusions based on solid evidence, which they're not. From the summaries, you can link to the studies themselves.

The Goldwater Institute folks are in a bit of a bind right now. They've seen one of their favorite government programs, tuition tax credits, take a well deserved beating in the press and know it will take a further beating from a bipartisan legislative group set up to investigate both the credits and the School Tuition Organizations administering them. G.I. can't defend the blatant abuses of the program — in fact, it has worked to position itself as part of the move to reform the system — but it desperately wants to create the impression that the program is worth saving and even expanding to a full blown voucher system, which is G.I.'s ultimate goal. And it wants to get the information out into the media –quick! — to defend tax credits against their detractors.

So G.I. is desperate to show private schools are better than public schools to justify the program. Because if private schools are better, the more students who attend private schools using tax credits and/or vouchers, the better educated our children will be.

But there's one big problem. All the recent studies comparing traditional public, charter and private schools indicate that none of them is superior to the others academically.

Bush's DOE funded a report that explored test scores at various schools and concluded, if you compare similar students, those in private schools perform no better on standardized tests than students at traditional public or charter schools. The only exception is, students at conservative Christian private schools score lower than everyone else.

Studies in Florida comparing voucher students with public school students showed no appreciable difference in test scores. The experiment in D.C. where a large group of students attended private schools on vouchers provided the closest thing to a controlled experiment we've ever had, and it also showed no significant difference in scores. And that's comparing public schools in D.C., which has one of the most dysfunctional school districts in the nation, to private schools. That's not the result voucher proponents were hoping for.

The only place they've been able to find a difference is in attitude. Students and their parents appear to like their private schools better than equivalent students and parents in public schools. That's all. Conservatives would make fun of liberals if they used touchy-feely information like that instead of hard test score data, but that's all they've got, so the conservatives are trying to make the most of it.

Ladner sets out to show that Arizona students in private schools like their schools better and have more positive attitudes toward tolerance and diversity than their public school counterparts based on a random survey of public and private school students. Since G.I. is an advocacy group masquerading as a think tank, the burden of proof that its study is objectively accurate is very high. But Ladner's study doesn't come anywhere near the necessary burden of proof a university would require of an objective researcher, or even a student writing a term paper.

WONK ALERT: The rest of this post, after the jump, is a dry-as-dust discussion of the problems with Ladner's methodology. Unless you're interested in that kind of thing, skip it. I hate to bore people. But it's important to show what a shoddy, shameless piece of propaganda G.I. has come up with, and that takes time.