This review was originally published on Maureen Downey’s Get Schooled Blog in The Atlanta Journal-Constitution on December 2, 2020, at: https://www.ajc.com/education/get-schooled-blog/opinion-study-on-savings-from-private-school-tax-credits-is-stab-in-the-dark/ZLO2D35EPNEKXGR5ASHA2SHT6I/.

Professor Welner wrote it in response to a guest column posted on the same blog on November 30, 2020, at: https://www.ajc.com/education/get-schooled-blog/opinion-private-school-scholarships-are-win-for-students-and-taxpayers/KQB4CIBHYVFFZCZN7PGDPSD6ZI/.

This review offers a critique of a teacher effectiveness experiment conducted by investigators from the Mathematica Policy Group and published by the Institute of Education Sciences. The Mathematica experiment was designed to provide evidence about the effectiveness of teachers who were themselves high-achieving students and trained by either Teach for America (TFA) or the Teaching Fellows programs. 

In Teacher Unions and Students’ Long-Term Economic Prospects, Emma García and John Schmitt review The Long-Run Effects of Teacher Collective Bargaining, a paper by Michael Lovenheim and Alexander Willén presented at the 2018 meeting of the Allied Social Science Associations (ASSA) paper session on "New Evidence on the Effects of Teachers' Unions on Student Outcomes, Teacher Labor Markets, and the Allocation of School Resources."

This review was originally published on the Economic Policy Institute website at: https://www.epi.org/blog/teacher-unions-and-students-long-term-economic-prospects/

Normally, when somebody hears about an evaluation of an education program, they reasonably assume the evaluation will tell them whether the program is working or not. When reading an evaluation report, policymakers, parents, and educators hope the evaluation will tell them if the program is helping the participating students. These seem like obvious, uncontroversial points.

On Monday, June 4, researchers from NC State released a working paper, An Impact Analysis of North Carolina’s Opportunity Scholarship Program on Student Achievement, which purports to be an evaluation of North Carolina’s largest private school voucher program. The authors enthusiastically publicized and distributed the report, making sure to provide advance copies to media organizations and a pro-voucher advocacy group. The report has been highlighted by all of the state’s major media outlets.

But there’s a problem: the report fails to tell us whether the Opportunity Scholarship program is working. The researchers’ efforts tell us nothing about whether accepting an Opportunity Scholarship will help or harm a student’s education.

The Heritage Foundation’s report, Focusing on School Safety After Parkland, was published following the mass shooting at Marjory Stoneman Douglas High School. The report highlights a handful of cherry-picked, pro-gun research but depends mainly on anecdotal evidence to support its findings that teachers should be armed, schools hardened, family and social services expanded, and discriminatory discipline practices de-emphasized. Yet the report’s credibility is harmed by the limited and biased presentation of research evidence and the omission of educator and student perspectives. The problems with supporting research and the questionable methods prevent generalizability and call the paper’s usefulness into question. The end result is a poorly refereed, one-sided defense of positions taken by gun advocates. That said, the report does provide conservative perspectives that can serve as a springboard for broader discussions about the impact of value-laden policies on promoting a broader sense of safety in schools.

The New Teacher Project’s report on teacher retention argues that not all teacher turnover is created equal and that attention should be paid to the quality of teachers who are leaving the profession.  Di Carlo considers all of the report’s recommendations to merit discussion, but here he focuses explicitly on TNTP’s use of the word “irreplaceable,” suggesting that “the analysis would have been better had TNTP avoided the overblown, media-friendly characterizations of their performance categories.”  Independent of the report’s recommendations about retention, when it comes to the characteristics, attitudes and behavior among teachers who are “nearly impossible to replace,” he encourages readers to interpret the results of this particular report with a healthy dose of caution.

NCTQ’s evaluation of the elementary grades student teaching component of 134 university-based teacher preparation programs declared 101 of those programs to be “weak” or “poor.”  Benner’s critique finds fault at every level of the NCTQ evaluation, including development and interpretation of the standards of evaluation, sampling techniques, methodology, data analysis, and findings.  She concludes that despite this particular review being ill-conceived and poorly executed, there is much room for improvement in teacher education. 

Baker’s step-by-step analysis explains the complexities of comparing achievement gaps between higher- and lower-income students across states.  Raw comparisons are misleading because they rely on an incorrect assumption that the average of low-income children in one state is comparable to the average of low-income children in another.  New Jersey, for instance, comes in 5^th in a ranking of raw comparisons between states, but when the necessary corrections are made for the size of the income gap between higher and lower-income students, New Jersey comes in 27^th – with a smaller gap than in the average state.  

Bruce Baker’s critique of the Return on Investment (ROI) Index for K-12 public school districts, published by the Center for American Progress, finds CAP’s analysis sorely lacking: it support neither the general findings conveyed in the report, nor the policy implications and proposals that follow from them.

Bruce Baker responds to the general assertion that public education system is less cost-effective now than it was 30 years ago by arguing that comparisons in education spending over time are overly simplistic if they simply adjust for the value of the dollar. He specifies and discusses three major factors that influence the cost of education: changes in the incoming student populations over time, changes in the desired outcomes for those students, and changes in the competitive wage of the desired quality of school personnel.

Bruce Baker evaluates Petrilli and Roza’s fifteen recommendations for how to stretch the school dollar as “quick and easy – generally ineffective and meaningless, or potentially damaging – revenue-neutral school finance fixes.” In addition, he faults the brief for failing to provide support for or analysis of its recommendations.

Bruce Baker takes off from Robert Reich’s lament about the “new attack on American education” to make the case that in many states, the de-investment in education has been occurring for decades, to ruinous effect.

This review of the Leaders and Laggards report, republished here with permission of the author, was written by Bob Williams independently, not as part of the Think Tank Review Project. Williams focuses on aspects of the report concerning alternative teacher certification and teacher tenure. He notes that the report’s authors appear to begin with the premise that "improving education requires weakening teacher tenure and union influence while supporting alternative certification and national programs to place inexperienced people ... into teaching positions with minimal training." Williams explains in this commentary how the report distorts the data in order to create state-by-state ratings that fit the authors' pre-determined agenda.

A new report authored by Tom Loveless and published by the Fordham Institute misleads in an attempt to convince policymakers to maintain tracking policies. The report combines weak data with questionable analyses to manufacture a flawed argument against detracking. This review was written by Kevin Welner independently, not as part of the Think Tank Review Project. It is available to subscribers of the journal "Teachers College Record," at http://www.tcrecord.org/Content.asp?ContentID=15872

Welner’s review describes how the Loveless report combines weak data with questionable analyses to manufacture an argument against detracking. Better treatment of these same data would, in fact, likely show that high-achieving Massachusetts middle school students in heterogeneous, untracked classrooms do as well or better than those in tracked classrooms – certainly in language arts (English) and maybe even in mathematics. He concludes that the report misleads in an attempt to convince policymakers to maintain tracking policies.