NEPC Resources on School Policy and Organization
NEPC Review: The System-Level Effects of Denver’s Portfolio District Strategy: Technical Report (Center for Education Policy Analysis, University of Colorado Denver, December 2021)
This report analyzes changes in academic performance as measured by test scores and graduation rates in the Denver Public Schools versus comparable schools in Colorado over 11 years of the district’s experimenting with the “portfolio” approach to school district management. This approach includes central-office oversight of different school types (such as charter schools, innovation schools, and district-run schools), with widespread parental choice under a single enrollment system. The reported academic gains are dramatic and worth drawing attention to, but attributing them specifically to the portfolio reforms seems premature and thus not useful in showing how other districts could replicate that success.
NEPC Review: Use of Personalized Learning Platforms in One Pandemic-Era Microschool: A Case Study (Center on Reinventing Public Education, June 2022)
Families across the U.S. experimented during the COVID-19 pandemic with new educational models in response to school closures. This review examines the Center on Reinventing Public Education’s two recent reports on two related strategies that exemplify such experimentation: microschools and learning pods. The first report analyzes engagement patterns and success rates of a digital platform used at one microschool in Nevada. The second report praises learning pods’ staffing features and argues that these approaches might be adopted in traditional schools. Both reports paint the new strategies in positive light, but both have methodological and analytical shortcomings that limit their utility for policymakers.
NEPC Review: The Third Way: A Guide to Implementing Innovation Schools (Progressive Policy Institute, October 2020)
A report from The Progressive Policy Institute is a “how-to” guide for entities seeking to develop innovation schools in urban communities. The guide highlights case examples of states and localities that have “successfully” implemented innovation schools, with a focus on test score data and student demographics. It argues that equitable educational opportunity is achievable when schools have complete autonomy and strong accountability to increase academic performance, adopt diverse learning models, and expand school choice. However, many of this convoluted guide’s long list of 53 detailed recommendations are improbable and overlook potential disadvantages of innovation schools. These recommendations are highly complicated, largely unexamined, and likely infeasible, especially if a district’s goal is to serve all students and their families equitably.
NEPC Review: Beyond the Mirage: How Pragmatic Stewardship Could Transform Learning Outcomes in International Education Systems (June 2019)
A report, Beyond the Mirage: How Pragmatic Stewardship Could Transform Learning Outcomes in International Education Systems, prescribes a shift in the leadership role of education ministers – from providers and guarantors of education to pragmatic stewards of education systems. Focusing on the organization of education sectors in the Global South, the report contends that this shift will address the need for higher quality education, rather than simply providing access to education. The “pragmatic stewardship” advocated in the report involves strategies that increasingly incorporate private actors. Accordingly, the report draws on four case studies of different types of private-sector involvement in education as examples of a broader shift by education ministers. However, each case contains limitations – some discussed, others not – that undermine their suitability as successful examples of divesting public education systems of their primary role as guarantors and providers of education. While the report claims to be “non-ideological” and “beyond the mirage” of the education privatization debate, the funders of the report (no publisher is listed) have a material stake in a main program cited as evidence, raising concerns about conflicts of interest. The use of questionable evidence and the conflicts of interest combine to render the report’s recommendations unsubstantiated.
NEPC Review: Fiscal and Education Spillovers from Charter School Expansion (Massachusetts Institute of Technology, July 2018)
A paper by two researchers at the Massachusetts Institute of Technology examines the consequences that follow from an expansion in the number of charter school places available for enrollment. The study uses data from Massachusetts, where charter school growth has been carefully managed and where there was significant excess demand for charter school places. In 2011, the state increased the cap on charter school enrollments in districts with low test scores, resulting in a large increase in charter school enrollment in some of these districts. The paper analyzes three outcomes: (a) changes across charter and non-charter public schools in funding (how much resource was available per student), (b) resource allocation (how schools spent their funds), and (c) achievement (how well students performed on academic tests).
NEPC Review: Leveraging ESSA to Support Quality-School Growth (Thomas B. Fordham Institute and Education Cities, March 2017)
A report offers a how-to guide for reform advocates interested in removing communities’ democratic control over their schools. The report explains how these reformers can influence states to use the Every Student Succeeds Act (ESSA) Title I school improvement funds to support a specific set of reforms: charter schools, state-initiated turnarounds, and appointment of an individual with plenipotentiary authority over districts or schools. While the report acknowledges that the research evidence on the effectiveness of these reforms as school improvement strategies is limited, it uses a few exceptional cases to explain how advocates seeking to influence the development of state ESSA plans can advance them anyway. As this review explains, support for the effectiveness of these approaches is simply too limited to present them as promising school improvement strategies. The report omits research that evaluates the models relative to other school reform initiatives, and it fails to take into account the opportunity costs of pursuing one set of policies over another. It also relies on test-score outcomes as the sole measure of success, thus ignoring other impacts these strategies may have on students and their local communities or the local school systems where they occur. For these reasons, policymakers, educators and state education administrators should be wary of relying on this report to guide them as they develop their state improvement plans and consider potential strategies for assisting low-performing schools and districts.
Review of Failure Is Not an Option
This Public Agenda report profiles nine high-poverty schools in Ohio that the authors believe have exhibited “sustained success.” It first lists 11 commonly accepted attributes they assert are demonstrated across the profiled schools. The report then offers six general recommendations for other schools to achieve and sustain success, although the connection between the attributes and the recommendations is unclear. How these “key attributes” and subsequent recommendations were derived from the interviews is not specified. The school selection criteria suggests sample bias. Six of the nine schools were from a state “schools of promise” list and three were not. Four of the schools’ poverty levels were near the state average, belying the high-poverty claim in the report’s title. The report’s biggest deficiency is that, while it is presented as addressing equity needs, and the interviewees pointed out that poverty related factors must be addressed, the recommendations fail to propose remedies or explicitly address these factors. This omission puts the report precariously close to the discredited “no excuses” genre. The common sense nature of the recommendations will likely be found acceptable to many readers, but the proposals are not sufficiently grounded in either the study’s own data or in the larger body of research. In sum, these shortcomings marginalize the work’s usefulness in advancing school reform and educational equity.
NEPC Review: U.S. Math Performance in Global Perspective: How Well Does Each State Do at Producing High-Achieving Students? (November 2010)
A report from Harvard’s Program on Education Policy and Governance and the journal Education Next finds that only 6% of U.S. students in the high school graduating class of 2009 achieved at an advanced level in mathematics compared with 28% of Taiwanese students and more than 20% of students in Hong Kong, Korea, and Finland. Overall, the United States ranked behind most of its industrialized competitors. The report compares the mathematics performance of high achievers not only across countries but also across the 50 U.S. states and 10 urban districts. Most states and cities ranked closer to developing countries than to developed countries. However, the study has three noteworthy limitations: (a) internationally, students were sampled by age and not by grade, and countries varied greatly on the proportion of the student cohort included in the compared grades; in fact, only about 70% of the U.S. sample would have been in the graduating class of 2009, which makes the comparisons unreliable; (b) the misleading practice of reporting rankings of groups of high-achieving students hides the clustering of scores, inaccurately exaggerates small differences, and increases the possibility of error in measuring differences; and (c) the different tests used in the study measured different domains of mathematics proficiency, and the international measure was limited because of relatively few test items. The study’s deceptive comparison of high achievers on one test with high achievers on another says nothing useful about the class of 2009 and offers essentially no assistance to U.S. educators seeking to improve students’ performance in mathematics.
Suggested Citation: Kilpatrick, J. (2011). Review of “U.S. Math Performance in Global Perspective: How Well Does Each State Do at Producing High-Achieving Students?” Boulder, CO: National Education Policy Center. Retrieved [date] from http://nepc.colorado.edu/thinktank/review-us-math
Update: Paul Peterson, one of the report's authors, has posted a response to the review. The response can be found at: http://educationnext.org/no-matter-how-hard-you-try-you-cannot-deny-u-s…
Please also download and read Jeremy Kilpatrick's reply to that response, which is posted at the bottom of this page.
NEPC Review: The Impact of a Universal Class-Size Reduction Policy: Evidence from Florida’s Statewide Mandate (May 2010)
In 2002, voters in Florida approved a constitutional amendment limiting class sizes in public schools to 18 students in the elementary grades, 22 students in middle grades, and 25 in high school grades. Analyzing statewide achievement data for school districts from 2004-2006 and for schools in 2007, this study purports to find that "mandated [class-size reduction] in Florida had little, if any, effect on cognitive and non-cognitive outcomes." The study has four flaws that, taken together, invalidate it as an evaluation of class-size reduction: 1) The data used are drawn from grades 3 and 4 to 8, where the likelihood of finding class size effects is small. 2) The differences in class sizes of two comparison groups (treated and untreated) range from about 0.5 to about 3.0 students, all too small to make a difference educationally. 3) School and district average class sizes are used in the analysis rather than the actual sizes of classes in which students were enrolled. 4) The comparison is between two sets of districts, both with small classes, differing only in whether state funding was used in a focused or general way. This study actually found that administrative discretion in spending state class-size reduction funds did not affect students’ academic performance.
Suggested Citation: Finn, J. (2010). Review of "The Impact of a Universal Class-Size Reduction Policy: Evidence from Florida’s Statewide Mandate." Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://nepc.colorado.edu/thinktank/review-class-size-florida
NEPC Review: Teacher Layoffs: Rethinking "Last Hired, First Fired" Policies (National Council on Teacher Quality (NCTQ), February 2010)
This recent brief from the National Council on Teacher Quality is concerned with the question of what factors should be considered when school districts must decide which teachers to lay off during periods of tight budgets. Most districts, according to the brief, base these decisions primarily on long-standing "Last Hired, First Fired" teacher seniority policies. The main point of this brief is to argue that seniority is not a fair, useful, or cost effective criterion; instead, teachers' quality and performance could and should be the main criteria used to make these employment decisions. The brief's arguments and recommendations are straightforward, reasonable and commonsense. However, proposals to measure, recognize and reward differences in teacher quality and utilize these in employment and promotion decisions are neither new nor unique. As the history of education reform has shown, implementing such proposals is challenging and often reform attempts have met little or no success. To its credit, this brief recognizes some of the many hurdles and difficulties that need to be overcome or addressed. A useful contribution of the brief is to document wide variations among districts in their layoff criteria and mechanisms and to summarize specific options and concrete alternatives used in particular districts.
Suggested Citation: Ingersoll, R. & Merrill, L. (2010). Review of "Teacher Layoffs: Rethinking 'Last Hired, First Fired' Policies." Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-teacher-layoffs
NEPC Review: The Shaping of the American Mind: The Diverging Influences of the College Degree & Civic Learning on American Beliefs (December 2009)
The Intercollegiate Studies Institute report, The Shaping of the American Mind: The Diverging Influences of the College Degree and Civic Learning on American Beliefs, suggests that college is failing to provide an adequate education in civic knowledge and is also influencing graduates to become less supportive of American values. Desirable "civic learning" about American values is associated with positions and attitudes that are anti-abortion, pro-free-market economics, consistent with fundamentalist Biblical interpretations, and otherwise generally associated with contemporary conservative political positions. The bulk of the report focuses on contrasts of college graduates and non-graduates on these political values, based on a civics test administered as a telephone survey. "Civics knowledge," as they define it, is presented as of greater value than college education. The study suggests that more educated people are more liberal, yet the omission of basic information about the researchers’ polling methods, their fundamental analytic techniques and their actual results renders any such conclusions insupportable. Consequently, the report offers no valid information that could inform policy makers or the public.
Suggested Citation: Marchant, G. J. (2010). Review of "The Shaping of the American Mind: The Diverging Influences of the College Degree & Civic Learning on American Beliefs." Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-shaping-American-mind
Reviews Worth Sharing: Leaders and Laggards: A State-by-State Report Card on Educational Innovation (November 2009)
This review of the Leaders and Laggards report, republished here with permission of the author, was written by Bob Williams independently, not as part of the Think Tank Review Project. Williams focuses on aspects of the report concerning alternative teacher certification and teacher tenure. He notes that the report’s authors appear to begin with the premise that "improving education requires weakening teacher tenure and union influence while supporting alternative certification and national programs to place inexperienced people ... into teaching positions with minimal training." Williams explains in this commentary how the report distorts the data in order to create state-by-state ratings that fit the authors' pre-determined agenda.
Reviews Worth Sharing: Tracking and Detracking: High Achievers in Massachusetts Middle Schools (December 2009)
A new report authored by Tom Loveless and published by the Fordham Institute misleads in an attempt to convince policymakers to maintain tracking policies. The report combines weak data with questionable analyses to manufacture a flawed argument against detracking. This review was written by Kevin Welner independently, not as part of the Think Tank Review Project. It is available to subscribers of the journal "Teachers College Record," at http://www.tcrecord.org/Content.asp?ContentID=15872
Welner’s review describes how the Loveless report combines weak data with questionable analyses to manufacture an argument against detracking. Better treatment of these same data would, in fact, likely show that high-achieving Massachusetts middle school students in heterogeneous, untracked classrooms do as well or better than those in tracked classrooms – certainly in language arts (English) and maybe even in mathematics. He concludes that the report misleads in an attempt to convince policymakers to maintain tracking policies.
NEPC Review: Reroute the Preschool Juggernaut (June 2009)
Reroute the Preschool Juggernaut is a new book published online by the Hoover Institution and authored by the Fordham Institution’s Chester Finn. It is an inaccurate and poorly reasoned attack on the movement to secure all children a good preschool education. The book cherry-picks a few weak studies to fit its preconceptions, and it builds the case for targeted programs based on errors, exaggeration, misrepresentation, and logical inconsistency.
Suggested Citation: Barnett, W.S. (2009). Special Review of “Reroute the Preschool Juggernaut.” Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/Special-Review-Reroute-Preschool-Jugger…
NEPC Review: The High Cost of High School Dropouts in Ohio (February 2009)
A new report published by the Buckeye Institute for Public Policy Solutions is a minor variant on six similar reports published by the Friedman Foundation over the past three years. The new report repeats some of the errors in the previous reports, and it follows a parallel structure, arguing that the costs of dropping out are dramatic for the state of Ohio, and that last-chance charter schools for dropouts can increase graduation and address the dropout problem. However, the report’s claims about graduates for the 23 illustrative schools are inconsistent with the data reported by the state of Ohio for the year chosen, resulting in a dramatic overstatement of the graduation rates at the charters. The report also largely ignores the existing research literature on the personal and social benefits of educational attainment, the achievements of charter schools, and the factors associated with either completing or dropping out of high school. Further, the report fails to compare the alleged benefits of last-chance charter schools with plausible alternatives. State policymakers interested in increasing graduation would be better served by seeking out the available, well-researched scholarship on the topic.
Suggested Citation: Dorn, S. (2009). Review of “The High Cost of High School Dropouts in Ohio.” Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-the-high-cost
Fellows’ Education Letters to the President
NEPC Review: Education Olympics 2008: The Games in Review (Thomas B. Fordham Institute, August 2008)
This review examines the recently released Thomas P. Fordham Institute report, Education Olympics: The Games in Review. Published just after the completion of the 2008 Beijing Summer Olympics, Education Olympics strategically parallels the international competition by awarding gold, silver and bronze medals to top performing countries based on indicators including scores from international assessments in reading, mathematics, and science. The report contrasts American students’ unimpressive performance on international assessments with the United States’ success in the Olympics. However, the report fails to substantiate its primary claim: that American students’ relatively low rankings on these tests will weaken the U.S. economy and jeopardize its future global standing. It also fails to substantiate secondary claims, set forth throughout in various sidebars. The report recognizes its numerous methodological weaknesses, but it nonetheless bases its conclusions primarily on findings produced by this flawed process. In addition, the research meant to bolster the report’s position is very limited. Ultimately, its conclusions lack a basis of argument or evidence, and its attempt to link test scores to the nation’s economic standing fails.
Suggested Citation: Fierros, E.G. & Kornhaber, M.L. (2008). Review of “Education Olympics 2008: The Games in Review.” Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-education-olympics
NEPC Review: Markets vs. Monopolies in Education: A Global Review of the Evidence (September 2008)
The Cato Institute report examines international evidence on outcomes from public and private education. The paper makes three key claims: private schools outperform public schools in “the overwhelming majority of cases”; private schools’ superiority is greatest in countries where the education system has more market features; and “the implications for U.S. education policy are profound.” Each claim is problematic. The first is based on an atypical method of summarizing academic literature and excludes two important research studies. The claim also fails to adequately take into account selection bias due, for instance, to parents choosing private schools because of an academic focus on their children. The second claim oversimplifies a very complex issue, namely the optimal application of market forces to improve education. And the third claim is dubious as well: even if the report’s first two claims are legitimate (based on international evidence), there may be no practical implications for U.S. education policy.
Suggested Citation: Belfield, C.R. (2008). Review of "Markets vs. Monopolies in Education. A Global Review of the Evidence.” Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-markets-vs-monopolies
NEPC Review: The Effect of Special Education Vouchers on Public School Achievement: Evidence from Florida's McKay Scholarship Program (Manhattan Institute, April 2008)
A new report published by the Manhattan Institute for Education Policy, The Effect of Special Education Vouchers on Public School Achievement: Evidence from Florida’s McKay Scholarship Program, attempts to examine the complex issue of how competition introduced through school vouchers affects student outcomes in public schools. The possible contributions of this report, however, are outweighed by research design problems, failure to take into account alternative explanations, and unsubstantiated assumptions about the direction of possible selection bias. Together, these problems call into question the findings and render the conclusions drawn from those findings highly suspect.
Suggested Citation:
Yun, J. T. (2008). Review of “The Effect of Special Education Vouchers on Public School Achievement: Evidence from Florida’s McKay Scholarship Program.” Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/thinktank/review-effect-of-special