NEPC Resources on Standards-Based Reform
NEPC Review: School Accountability--Past, Present, and Future (Hoover Institution, November 2020)
A report from the Hoover Institution argues that state and federal officials should retain results-oriented accountability systems that use standardized assessments of students followed by consequences for not meeting performance goals. The report is problematic for a number of reasons. It ignores a plethora of literature on the deleterious impact of test-based accountability on outcomes that could provide a more nuanced understanding of these systems. It fails to explain why these systems should be extended to include more testing at more grades. It also provides no evidence on the efficacy of its preferred reform strategies for low-performing schools, which include combining external interventions with market-driven consequences. Rather than evidence, the report relies on unsupported theories of accountability and market-driven reform to provide a rationale for its conclusions and recommendations. For these reasons, policymakers, educators, and state education administrators should not rely on this report for guidance as they consider strategies for assisting low-performing schools and districts.
NEPC Review: Fewer Children Left Behind: Lessons From the Dramatic Achievement Gains of the 1990s and 2000s (Thomas B. Fordham Institute, October 2019)
A Fordham report highlights the historic academic progress of Black and Hispanic groups over the past two decades, at the elementary school level, on the NAEP exam. From this, the report offers the major claim, based on its author’s eyeball test, that the academic progress of students of color is attributable “mostly” to poverty reduction. The report, however, also acknowledges that correlation is not causation and calls for systematic statistical analysis to test the author’s proposition. This review responds to that call by examining the validity of the report’s arguments around progress and causes, looking to expanded data sources, including both family income and school expenditures. The review notes uneven patterns of achievement among grade levels and refutes the report’s claim that flat achievement trends among twelfth graders are a result of dropout reductions.
NEPC Review: The Opportunity Myth (TNTP, September 2018)
A TNTP report aims to expose what it labels the “opportunity myth” in American education: that while schools purport to prepare students well, they don’t deliver. It paints a dramatic picture of American students being misled by false promises of opportunity, when they could make significant learning gains if they experienced grade-level content, strong instruction, deep engagement, and high expectations. The report contends that these negative experiences are primarily the result of educators’ daily decisions and are magnified for students of color and low-income students. Though the report presents an array of qualitative and quantitative data, some of its particular claims are not fully supported by evidence, and it is unclear how key constructs are measured. Importantly, in describing educators’ decisions, the report does not sufficiently account for larger systemic and societal impediments to opportunity that serve to establish and maintain many of the obstacles and problematic patterns the report observes.
NEPC Review: Lessons From State Performance on NAEP: Why Some High-Poverty Students Score Better Than Others (Center for American Progress , January 2016)
This Center for American Progress report examines whether states’ adoption of standards-based policies predicts low-income students’ NAEP achievement trends in fourth and eighth grade math and reading throughout the 2003-2013 decade. The report claims to analyze changes across five separate two-year intervals, but it only reports findings for 2009-2011, with no explanation of why or any documentation of the representativeness of that single interval. The reported finding for the selected interval is that state adoption of standards policies positively predicts fourth (but not eighth) grade math NAEP and eighth (but not fourth) grade reading NAEP. Even these selected positive results are statistically significant only at the generally unacceptable 0.10 level of significance. The report includes effect sizes but nothing about the percent of the variance explained in their model. In short, the report does not adequately describe variables or analytic methods or completely report findings, and the data and methods used do not allow for any causal findings. They use state standards adoption across grade levels and subject areas as well as selected accountability mandates as predictors but fail to assess their quality or fidelity of implementation. Yet based on these very problematic and limited analyses, the authors conclude that their analysis “strongly supports the potential of the Common Core to drive improvements in educational outcomes.” The study simply does not support this conclusion or the set of recommendations that follow.
State Sen. Alberta Darling's Recovery District Idea is Designed to Fail
Review of Turning Lightning into Electricity
This report presents the findings of research on parent organizing within what the report calls Education Reform Advocacy Organizations (ERAO), such as Stand for Children and Parent Revolution. The ERAO agendas focus on standards, test-based accountability, teacher tenure reform, and parent choice. The report recognizes that ERAOs are often criticized as astroturf organizations that mobilize parents behind their agendas instead of building authentic parent power; it then presents some evidence that many groups are committed to long-lasting parent engagement, and it reports on the community organizing practices they use to build parent leadership. Unfortunately, the report’s presentation of research methods is so weak that the research cannot be relied upon without a better idea of the rigor or lack of rigor of its approach. The findings may be valid for the groups studied, but the selection is biased towards ERAOs that work to build sustainable forms of parent participation; it is unclear if those groups are representative of the broader field of ERAOs. While the report suggests some community organizing strategies can be used to advance the ERAO version of education reform, this approach undermines an understanding of community organizing as a democratic practice through which organizations and agendas emerge out of the concerns and through the actions of indigenous community leaders working to build a more inclusive public democracy.
The Lost Opportunity of the Common Core State Standards
How Many Decades Before 'Reform' Becomes 'Status Quo'?
NEPC Review: Do High Flyers Maintain Their Altitude? (Thomas B. Fordham Institute, September 2011)
The research report reviewed here concludes that many initially high-achieving students are falling further and further behind over the course of their years in school. The report intends to raise the alarm and to advocate for improved programs for these students. It is, however, a false alarm due to biased methodology and misleading arguments. The report’s norm-referenced framework guarantees “losers” as well as “winners,” regardless of any true improvement made by the students. Also, the “regression to the mean” effect produces a false illusion of a tradeoff of over-progress by low achievers at the cost of under-progress for high achievers. Finally, its prescription for stronger school accountability for high-achieving students under NCLB does not follow research-based guidance on how to improve student learning. Other research, including that conducted by this reviewer, finds that students who are high achievers and low achievers make approximately equal academic progress in reading and math, while the achievement gaps remain large. Moreover, a substantial part of the variation in student progress is attributable to teacher and school effects beyond students’ initial status and background characteristics.
Two authors of the report published a written reponse on October 30, 2011. That response is available here, and Dr. Lee's brief reply is published and downloadable below.
NEPC Review: Learning About Teaching (December 2010)
The Bill & Melinda Gates Foundation’s “Measures of Effective Teaching” (MET) Project seeks to validate the use of a teacher’s estimated “value-added”—computed from the year-on-year test score gains of her students—as a measure of teaching effectiveness. Using data from six school districts, the initial report examines correlations between student survey responses and value-added scores computed both from state tests and from higher-order tests of conceptual understanding. The study finds that the measures are related, but only modestly. The report interprets this as support for the use of value-added as the basis for teacher evaluations. This conclusion is unsupported, as the data in fact indicate that a teachers’ value-added for the state test is not strongly related to her effectiveness in a broader sense. Most notably, value-added for state assessments is correlated 0.5 or less with that for the alternative assessments, meaning that many teachers whose value-added for one test is low are in fact quite effective when judged by the other. As there is every reason to think that the problems with value-added measures apparent in the MET data would be worse in a high-stakes environment, the MET results are sobering about the value of student achievement data as a significant component of teacher evaluations.
Suggested Citation: Rothstein, J. (2011). Review of “Learning About Teaching: Initial Findings from the Measures of Effective Teaching Project.” Boulder, CO: National Education Policy Center. Retrieved [date] from http://nepc.colorado.edu/thinktank/review-learning-about-teaching
NEPC Review: 2010 State School Report Card (October 2010)
This review examines the Heartland Institute's report ranking states on student achievement, education expenditures, and adherence to learning standards, as well as a ranking based on an average of the first three. The rankings are based on indices created by the report's authors, and the report highlights the top- and lowest-performing states for each of the indices. The report assigns letter grades to each of the states (plus DC), with a forced distribution: 10 states are assigned A's, B's, C's, and D's, and 11 states must get F's. The report explains how the indices were devised but does not cite any research or provide rationales to support the methodological approach used in their creation. The report acknowledges that it does not control for state variations in demographic or other factors. It nevertheless presents conclusions concerning quality, and it recommends school choice as a remedy. The report's policy recommendations are undermined by the flaws in the report's methodological approaches, its limited and partisan selection of research references, and a clear disconnect between the recommendations and the report's findings.
NEPC Review: Behind the Curtain: Assessing the Case for National Standards (February 2010)
President Obama and Education Secretary Arne Duncan have called for national "common core" curriculum standards. Some have argued that national standards are essential for reform, as they provide coherence, rigor, logic and organization. Others have contended they will narrow the curriculum, seize control from local districts and states, and distort the purposes of education. The Cato Institute’s Neal McCluskey argues that national standards will have only limited, if any, effect. The report contends there is only a weak theoretical case in favor of national standards and that the structure of schooling might be the real problem. It concludes that market models are the best way to reform education. While providing a useful summary and critique of the research on national standards, the non-sequitur in the report (standards do not work; therefore the free market will) presents readers with a conclusion not supported by the report’s evidence. Thus, the fundamental policy conclusions are not sustained.
Suggested Citation: Mathis, W.J. (2010). Review of "Behind the Curtain: Assessing the Case for National Standards." Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-Behind-the-Curtain
The Overselling of Growth Modeling
NEPC Review: The State of State Standards 2006 (Thomas B. Fordham Institute, August 2006)
A new Fordham Institute report assigns letter grades to each state for its academic standards. A review of this report finds that the method for determining those grades is flawed.
Suggested Citation:
Howe, K. (2006). Review of "The State of State Standards 2006." Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-the-state-state-standards-2006