G.I.s Matthew Ladner receives NEPC Education Bunkum Award

by David Safier

You know how I've been after Goldwater Institute's Matthew Ladner for his fact- and analysis-challenged "research" work? It looks like I'm not the only one who is noticing, as Ladner extends his reach, preaching the gospel of the "Florida Education Miracle" to state legislatures across the country.

The National Education Policy Center, which originates out of University of Colorado, Boulder but has lots of connections with ASU profs, reviews popular educational research and fact checks it, often finding huge flaws in the work.

It has just put out its Bunkum Awards 2010, and Matthew Ladner makes the list for his obsessive and moronic repetitions of Florida's 4th grade reading scores on the NAEP exam, then extrapolating those results to prove that letter grades for schools, holding back third graders and vouchers will solve all our educational woes.

I usually excerpt articles, but I'm putting up the whole thing in this case.

NOTE TO THE MEDIA: PLEASE READ THIS. IT'S IMPORTANT. LADNER IS A PROPAGANDIST, NOT A GENUINE SCHOLAR. EVERYTHING HE SAYS NEEDS TO BE CHECKED WITH EXPERTS.

The 'If I Say It Enough, Will It Still Be Untrue?' Award

Think Tank(s): The Heritage Foundation
"Review of Closing the Racial Achievement Gap" (Heritage Foundation )

While the Heritage Foundation receives this award for publishing Closing the Racial Achievement Gap, by Matthew Ladner and Lindsey Burke, it should really share this award with the think tanks in Colorado, New Mexico, Arizona, Wisconsin, Oklahoma, Utah, and Indiana, as well as the Hoover Institution and the Pacific Research Institute, all of which also published essentially the same report. Perhaps even the derivative articles by Dr. Ladner at National Review Online and Foxnews.com deserve some recognition.

But Ladner’s fecundity isn’t really what sets this work apart. It’s his willingness to smash through walls of basic research standards in his dogged pursuit of his policy agenda. In this case, the agenda is a passel of Florida reforms: vouchers funded by tax credits, charter schools, online education, performance-based teacher pay, test-score grading of schools and districts, test-based grade retention, and alternative teacher certification. He likes them. A lot. And he contends that other states should adopt the same package because, in his vision, they have clearly caused an improvement in Florida schools.

The problem, alas, is with the “caused” part. Nothing in the data or analyses of Dr. Ladner or the Heritage Foundation comes even close to allowing for a causal inference. Instead, the report offers uncontrolled descriptive data focused on NAEP fourth-grade reading scores. Those scores increased, so Ladner’s favored policies must be working. Our reviewer threw a few pails of cold water on that conclusion, noting first that other policies were also in effect during the time period studied. Florida has an excellent supportive reading program aimed at early grades, its accountability system strongly targets resources at lower-performing schools, and it has one of the nation’s most aggressive class-size reduction reforms. Ladner has steadfastly forsworn the possibility that these factors may have played a major role in any achievement improvements.

The reviewer also pointed out the reason why Ladner focused on fourth-grade reading and why NAEP growth in other subjects and grades wasn’t nearly as impressive: the researchers failed to properly address the state’s policy of retaining third-graders with low reading scores. This gave the bottom-scoring students another year to grow before they took the fourth-grade reading test – voilà. By analogy, consider growth in height instead of growth in test scores. If two states wanted to measure the average height of their fourth-graders, but one state (Florida) first identified the shortest 20% of third-graders and held them back to grow an additional year before measurement, the study’s results might be considered biased.

But Dr. Ladner has found his bone, and he’s not letting go. When confronted with the reviewer’s critique, he responded, "The change averse may wish to quibble over the details, or agonize over just what reform did how much of what," but the bottom line is that Florida’s fourth-grade scores increased dramatically. Indeed they did. And no research-bound egghead is going to mess up his good causal story.

FURTHER NOTE: To show this isn't some kind of lefty smear job (though it does go after lots of conservative think tanks, since they're notorious for their fast-and-loose research and conclusions), the NEPC Grand Prize in Bunkum this year went to the U.S. Department of Education for its shoddy research.