BOULDER, CO (September 7, 2023)—In a recent report, the Center for Research on Education Outcomes (CREDO) examined charter school students’ year-to-year test score growth over four years, and across 31 states. The report shows very small differences favoring charter schools and has been repeatedly trumpeted by charter school advocates as having policy significance—as weighing in favor of charter school expansion.
A review reveals, however, how the report’s main findings fail to meet the minimum baseline experts consider to be meaningful. In fact, CREDO itself has labeled these small differences as “meaningless” and “small”—back when they found charter schools to be on the losing end of those differences.
Joseph J. Ferrare of the University of Washington Bothell reviewed As a Matter of Fact: National Charter School Study III 2023 and warns that it should be approached with caution by policymakers, given the nonexperimental design that renders it unable to fully account for the factors that drive families to choose charter schools. Those omitted factors could account for the small measured differences.
The recent CREDO report finds that charter school students, when compared to those in traditional public schools, experienced small, positive impacts in reading and math (0.028 and 0.011 standard deviations, respectively), but with considerable variation between groups and types of schools. For instance, Black and Hispanic students in the study showed more positive outcomes than other racial and ethnic identities. Virtual charter schools, meanwhile, produced strongly negative outcomes.
The geographic scope of the report and the researchers’ access to detailed data sets offer the potential to provide policymakers with an expansive bird’s-eye view, comparing charter school students’ learning in reading and math to students in traditional public schools. Unfortunately, the CREDO researchers repeat mistakes of their past charter school studies, including the use of the “days of learning” unconventional metric, that leads to more confusion than clarity.
This and other problems have been pointed out by several past NEPC reviews: in 2009, when the small differences were flipped; 2013 and again later in 2013; 2015; and 2017. Similar and additional concerns were raised by Andrea Gabor in 2015 and by Carol Burris as regards the 2023 study. CREDO’s reports do have potential, but that potential will not be realized unless and until the researchers respond constructively to these sincere and important criticisms.
Find the review, by Joseph J. Ferrare, at:
https://nepc.colorado.edu/review/charter-study
Find As a Matter of Fact: National Charter School Study III 2023, written by Margaret E. Raymond, James L. Woodworth, Won Fy Lee, and Sally Bachofer and published by the Center for Research on Education Outcomes (CREDO), at:
https://ncss3.stanford.edu/wp-content/uploads/2023/06/Credo-NCSS3-Report.pdf