Zachary Smith, the Plain Dealer‘s current data wonk, just published another article ranking Ohio’s schools, Ranking Ohio Public High Schools from 1 to 823, based on Ohio’s 2022 state school report card’s Performance Index.  A few weeks ago he ranked all of the state’s school districts by the same Performance Index.

Ohio released its annual state school report cards in mid-September and changed its ratings from A-F to a five star system.  At the time, the Plain Dealer‘s Laura Hancock reported that because the state’s schools were dealing with COVID all of last year and the federal government, therefore, eliminated demands for school improvement plans, the state would not calculate an overall summative rating for schools and school districts: “This year the Ohio Department of Education is not offering an overall rating for each school and district, due to the reprieve on sanctions. In future years, there will be an overall star rating.”

This year the stars were awarded in five categories: Achievement, Early Literacy, Graduation Rate, Progress, and Gap Closing.  A new category was added, “College, Career, Workforce and Military Readiness,” but there was were no stars assigned this year in this area due to ongoing COVID recovery.

While the state did not assign any overall summative grade for schools and school districts this year, reporter Zachary Smith discovered—in the state’s description of the category of “Achievement”—what he considers an overall way to rank the state’s schools. In its description of measuring Achievement,  the state lists a “Performance Index” number for each school and school district.  The state says it calculates the Performance Index based primarily on aggregate standardized test scores: “The Achievement component represents the number of students who passed the state tests and how well they performed on them. This component includes three additional performance indicators —the Chronic Absenteeism Improvement Indicator, End-of-Course Improvement Indicator and Gifted Indicator.” Based on Performance Index scores, Smith and the Plain Dealer have been ranking the state’s school districts and high schools.

The problem is that test scores are known to reflect a community’s family economics more than they measure the quality of a school or school district. I wish Smith would go back a couple of years to look at Rich Exner’s profound 2019 Plain Dealer article on Ohio’s school report cards. See How Closely Ohio School Report Card Grades Trend with District Income. Here is an example of one of Exner’s bar graphs, which profoundly depict the story.

Two weeks ago, Jack Schneider, a professor at the University of Massachusetts, Lowell and Joel Boyd, the superintendent of the Lowell Public Schools, explained that the correlation of standardized test scores with family income has been an issue from state to state through the past two decades since No Child Left Behind thrust us into school accountability based on standardized test scores: “As research indicates, test scores are highly indicative of the inequalities that afflict our communities, and are not a valid basis for determinations about overall school performance… Scholars have repeatedly shown (that) the leading predictors of student standardized test scores are demographic variables like family income and parental educational attainment.”

Schneider and Boyd show how test scores this year particularly are likely to reflect the disparate economic realities for families during the COVID pandemic: “Imagine that in one school community, students were insulated from the worst effects of the pandemic. Parents were able to work from home, oversee remote schooling, and offer additional support. Young people felt safe, and their families remained intact. Family resources were deployed for educational purposes and enrichment. The pandemic was a challenge, but one that was mitigated to a significant degree.  In the other school community, students felt the effects of the pandemic acutely. Family members became sick, were hospitalized, and may have even died.  Working in so-called essential fields drew caregivers away from home during the day. Internet was often slow and unreliable, and students competed for quiet space with siblings. Young people felt vulnerable, frightened, and isolated.”

The No Child Left Behind Act, formulated in 2001 and signed into law on January 8, 2002, restructured public education by demanding quantitative, standardized-test-based school accountability and by using sanctions to punish the public schools struggling to raise aggregate scores.  Gail Sunderman was one of NCLB’s early critics as the lead author of NCLB Meets School Realities, published for the Harvard Civil Rights Project in 2005, in collaboration with James Kim and Gary Orfield. Sunderman is now a research scientist in the Department of Teaching and Learning, Policy and Research and director of the Maryland Equity Project at the University of Maryland School of Public Policy.

Sunderman reminds readers that No Child Left Behind was replaced by the Every Student Succeeds Act in 2015, but the new law still requires states to rate their schools and put the lowest scoring schools on corrective action plans. Here are Sunderman’s concerns today about the damage wrought by state school rankings: “There’s… evidence that state school rating systems often reflect personal and ideological preferences of state leaders… States with a more liberal orientation… are more likely to incorporate indicators related to school quality and indicators of student success, such as growth measures, while states with a more conservative leaning maintain a focus on student test scores…  While school rating systems may be a practical means to a political end, their educational value is questionable. Despite the proliferation of school rating systems, there is very little peer-reviewed, empirical research on their effects on student performance, and school and teacher practices…  Summative ratings also tend to obscure the well-documented relationship between student achievement scores and demographic variables, most notably race and socioeconomic status. An analysis of the Maryland five-star rating system, for instance, examined why no high-poverty schools earned a five-star rating, but when the researchers adjusted ratings to account for economic disadvantage, the number of five-star schools increased.”

Sunderman concludes: This inability of summative school ratings to distinguish school performance from student demographic variances disproportionately harms schools serving marginalized children and inflates the quality of schools serving wealthy and white students.”

This blog previously covered Ohio’s 2022 school ratings here.