Diane Ravitch: Reflections on the Math Scores in New York City and State

Math scores are up in New York City on state tests in grades three through eight. They are up in every grade. That’s terrific news for kids and their teachers, who have been working hard to improve achievement.

Last year, math scores were down, so the increase this year was a welcome change.

Across the span of grades 3-8 in the city’s schools, there was a gain of 8.1 percentage points, as compared to lower scores in 2006. The state has tested grades 3-8 (in response to the federal No Child Left Behind law) for only two years, in 2006 and 2007, so it is impossible to compare the scores for grades 3-8 in 2007 to any other year except 2006. As a matter of record, the largest increase in a single grade in a single year in New York City occurred in 2003, when math scores in the fourth grade jumped by nearly 15 points. This was the last state test reported prior to the implementation of Chancellor Joel Klein’s reform program.

This is what happened in this year’s math tests:

Math scores are up across the state; the gains in New York City outpaced the state gains. In grades 3-8, 72.7% across the state met the standards in 2007, compared to 65.8% in 2006, a gain of 6.9 points statewide. In the same grades, the proportion of New York City’s students meeting the standards rose from 57.0% to 65.1%, a gain of 8.1 points. Very impressive gains indeed, for both the state and the city, especially the city.

When Chancellor Klein’s Children First reforms were launched in September 2003, 66.7% of children in fourth grade met the state standards in math. In 2007, 74.1% of fourth grade students in New York City met the state standards in math. That is a cumulative gain during the years of mayoral control of 7.4 points, or a shade less than 2 points per year.

Recently, the press department at the New York City Department of Education had been claiming credit for the huge gains of 2003, but these scores (14.7 points in a single year) were recorded before the implementation of Children First.

Apparently the press department now claims that the city has gained 27.8 percentage points in grades 3-8 since 2002, but that seems unlikely. For one thing, the state has been testing these grades for only two years (before then, only grades 4 and 8 were tested annually). The press office seems to have combined the results of the state tests for 2007 with city tests that were administered in earlier years and have since been abandoned. It is unlikely that any independent psychometrician would approve of mixing the results of these disparate tests, which were not based on the same standards nor equated for their reliability and validity.

In the eighth grade, the gains for the city were also impressive, since this has been a historically low-scoring grade, where only twice before have more than 40% of students met state standards. When the Klein reforms were launched in 2003, only 34.4% of eighth graders met the state standards; in 2007, 45.6% did, a gain of 11.2 points over four years. That is a demonstration of the power of intensive test-prep activities, in which Tweed has invested heavily.

To be sure, testing experts tend to be suspicious of big changes in large-scale assessment programs, whether they go up or down. When a city or a state or a nation reports large one-year gains or losses, experts tend to raise their eyebrows and wonder about the test itself or the way it was scored. Was it easier or harder?

Jennifer Medina of The New York Times wisely pointed out in her first-day story that a federal study released just a week ago found that New York state’s math tests in 2005 in fourth grade and eighth grade were easier than in many other states. Indeed, New York’s fourth grade test was ranked easier than those in 28 of 32 other states, while the eighth grade was ranked below those of 12 other states in rigor. The State Education Department claimed that it changed the test in 2006 and made it more rigorous, which explained the drop in math scores last year; this year, with the astonishing increases in districts across the state, the State Education Department claims credit for improvements in teaching, curriculum, alignment, teacher training, collaboration with higher education, and everything else imaginable. So, if scores go down, the test got harder, but when scores go up, it has nothing to do with the test!

When the National Assessment of Educational Progress (NAEP) reported the 2005 results for mathematics, New York state found itself in an embarrassing situation. According to federal data, New York claimed that an astonishing 87% of fourth grade students met state proficiency standards, but on NAEP tests in 2005, only 36% of fourth grade students were rated proficient. In eighth grade, the state claimed that 56% of students were proficient, but on the more rigorous NAEP, only 31% were proficient.

Once again, New York state has reported startling results, after last year’s dismal scores in math. The public can be relieved that its state and local leaders are on the job, raising scores diligently and boldly, doing all the right things in the classroom, the school, and the district.

I suggest that we wait patiently to see whether the recent gains on the state tests are reflected on the national tests when the results are posted in November 2007.

In the meanwhile, I suggest that Governor Spitzer think seriously about creating an independent agency to administer tests and report on test results, one staffed by top-notch psychometricians who take neither credit nor blame for test results in the state and local districts.

Diane Ravitch

For analysis of 2007 ELA click here and here