The Gap Between School Reform Rhetoric and Reality in 3 Cities

From Washington Post:

A new report looks at the results of school reform in three major cities and finds that reformers’ claims about success don’t exactly match reality. Here’s a piece on it by Elaine Weiss, the national coordinator for the Broader Bolder Approach to Education. This appeared on The Nation’s website.

By Elaine Weiss

Across the country, a wave of school reform based on market principles has taken hold, championed by leaders like former District of Columbia Public Schools and New York City Public Schools Chancellors Michelle Rhee and Joel Klein. Dozens of states fulfilling grant requirements under Race to the Top, or implementing those plans even in the absence of a grant, are, essentially, trying to replicate the purported success of then-Chicago Public Schools Chief Executive Officer Arne Duncan’s signature Renaissance 2010 initiative. But school administrators and city leaders rushing to mimic Rhee, Klein and Duncan should take a step back and reconsider. A new report from the Broader, Bolder Approach to Education finds their real-life prospects, in the best of circumstances, to be pretty poor.

Since the onset of their respective reform agendas, all three districts – New York City, Chicago, and Washington, D.C. — have participated in the Trial Urban District Assessment (TUDA), which provides district-level scores from the National Assessment of Educational Progress (NAEP). All of the TUDA districts, 10 in 2003 and now are 21, are large, high-poverty and heavily minority, so the study enables apples-to-apples comparisons of growth over time using a reliable, consistent measure of learning.

The lack of a solid evidentiary basis for these reforms made us suspect that the results would be less rosy than what reformers publicly touted. As we explored the NAEP data, however, we were surprised by how bad the outcomes were. Test score growth in math and reading at both the fourth and eighth grade levels tended to be lower in these “reform” districts than in their “non-reform” counterparts.

That pattern of poor performance was compounded by the fact that what little growth school districts enjoyed accrued heavily to higher-income and white students, not to the low-income and minority students who were supposed to benefit. Closing schools neither improved student outcomes nor saved districts money, and even the strong charters in these three cities delivered more mixed benefits than commonly reported. This is a critical story that, despite the public data, will come as a surprise to many: these supposedly successful reforms are doing little good and more than a little harm.

In Washington, D.C., test scores among most student groups had been rising prior to Rhee and Fenty’s arrival, some of them rapidly. After 2007, however, test scores largely stagnated or declined, and those that had not been rising fell still more. Moreover, D.C. schools stands out among the three districts studied for the most disproportionate accrual of gains to high-income, non-minority students.

This was particularly true for fourth grade reading scores, a key indicator of subsequent student engagement and success. 2003-2007 gains, especially among low-income and minority students, suddenly reversed course after 2007. White students lost a three-point gain. Hispanic students, too, lost three points, but had gained so much prior to Rhee that they still came out ahead. Black students managed to hang on to a six-point 2003-2007 gain, but saw no further improvement at all. Assessment based on income level reveals a similar pattern.

Perhaps most troubling, in some cases, no students really gained ground; rather, increases were due simply to changes in the composition of the student body. Disaggregation of the data showed this to be the case with respect to both reading and math scores among fourth graders.

Chicago Public School students of all groups made smaller gains in reading than their peers in other urban districts during Duncan’s tenure, and higher-than average math gains were driven by high-income and white students, patterns initially identified not by BBA, but by that city’s most respected education research institution, the Consortium for Chicago School Research.

Fourth grade reading was, again, a particularly troubling spot. Among all urban cities, white fourth graders gained 6 points in reading between 2003 and 2011, and black fourth graders gained 9; in Chicago, in contrast, white students gained 2/3 as much as their peers – 4 points – but black students gained just 1/3 as much as their peers – 3 points.

The only exception to this trend is in eighth-grade math, where, as we note in the report, CPS students of all races made larger-than-average gains. Even there, however, white students’ 20-point gains far outpaced those of black (15) and Hispanic students (13), so the gap widened.

We knew that Mayor Bloomberg’s claim to have cut the race-based achievement gap in half between 2005 and 2011 was greatly exaggerated but, again, did not understand the extent of the hyperbole until we dug into the data. When Columbia University professor Aaron Pallas averaged math and reading scores across fourth and eighth grades, he found that, rather than cutting the gap by 50 percent, the mayor had cut it by 1 percent. (Substituting NAEP for state test scores actually widened the gap by a couple of percentage points.)

White fourth graders made three times the gains of their black counterparts in reading (9 versus 3 points), with a similar divide based on income. And the gap between white and Latino students (who comprise 40% of the district’s student body) widened in all instances but fourth grade math, in contrast to narrowing gaps in other cities.

Gains in New York City were more evenly distributed across racial and income groups than in the other two districts. However, New York City public schools students’ gains averaged across both subjects and grades were less than half the study average; only than Cleveland’s students gained less – one point. As such, both minority and non-minority students in New York City gained less ground than their peers in other urban districts.

These outcomes are understandably disappointing to those who promote policies that focus on student test scores as a means of holding teachers and schools accountable. That disappointment might explain critiques from groups like EdTrust, which points to one exception in each city to distract from the clear pattern and try to paint the findings as mixed or “complex.”

As we note, however, these “reforms” are quickly becoming the new status quo; defending them, especially in the face of increasingly solid evidence of their failure, is thus hardly reform-minded. The intent is not to suggest that reformers do not feel an urgent need to improve the odds for minority or low-income children. Rather, we hope both to call attention to what is not working, and to shift investment to what does.

The report also highlights policies that have helped disadvantaged students in particular but have gotten little attention, such as Bloomberg’s new small schools, Duncan’s smart teacher recruitment and retention initiative and his multifaceted approach to increasing minority students’ college attendance and success, and D.C. schools’ high-quality pre-K program. These demonstrate that more holistic approaches to reform hold real promise.

Indeed, they may have contributed to high school graduation rates that increased substantially in both New York and Chicago, gains similar to those seen in other large cities over that period. A new Century Foundation book likewise represents the flip side of this report’s coin. Beyond the Education Wars showcases two low-income urban school districts that have improved by using community-wide supports, rather than failing by pitting schools and teachers against one another.

If reformers focused on raising the visibility of their true successes, and were more realistic about the limitations of narrower ones, all students, and our education system, would be on a better path.