Sunday, August 1, 2010

So what's the story? Have NYC schools made progress in test scores, or not?

Today's Daily News reports on how NYC parents are reacting in the Bronx and in Manhattan to the revelation that the great progress claimed by Bloomberg and Klein in the past was a fraud: "The scores make you want to cry," said Monica Major, a Bronx parent and president of School District 11's Community Education Council. "For years, we've been bamboozled and lied to."

Or as as Diane Ravitch writes in an oped, "When the [state] scores were released, there was a sound of bursting bubbles across the state. What once were miracles turned into mirages.”

Yet in recent days, Bloomberg and Klein have tried to argue that the evidence is still strong that city students have made "dramatic progress" under their leadership.

First, they have repeatedly claimed that since the city's average scores on the state exams have been rising over time, the fact that the proficiency cut scores have now been drastically lowered is irrelevant.

Yet the city's average scores on the state exams are likely just as unreliable as its proficiency levels, since the questions on the test have gotten easier over time.

See an earlier posting, in which Steve Koss discusses a 2007 Daily News expose showing how the increase in "P" (or "probability) values on the field test questions indicated that the state tests had become simpler between 2002-2005. (The chart to the right was originally in the Daily News.)

To prove this, intrepid reporter Erin Einhorn even gathered a bunch of children to take both the 2002 and 2005 math exams, and out our 34 kids, 24 of them did better on the 2005 exam.

As one of them said, "the 2002 questions were more complicated than in 2005... In 2005, they kept it short, simple and sweet.”

Really, the only semi-reliable measure we have of student achievement over this period is the city's performance on the national exams known as the NAEPs. Klein has also made questionable claims in recent days in relation to the NAEPs.

Here is what he said to the Wall St. Journal: "In both fourth-grade math and reading, New York City now performs at the same level as the nation overall," the schools chancellor said."

Is this true? No. In no subject or grade does the achievement level of city students match that of the nation as a whole, including 4th grade reading or math; though the gaps are smaller in that grade than in 8th.

A more meaningful comparison would be to look at in the change in NAEP scores compared to other large cities from 2003-2009, to see whether there has been exceptional progress during the Bloomberg/Klein era .

First I looked at the change in the percent of students at or above basic over this period, in NYC compared to other large cities and to Atlanta, which has made exemplary progress.

Note that compared to other large cities, NYC has done a bit better on average in two categories and worse in two: 8th grade reading and 8th grade math.

In all subjects, however, NYC's gains have been far less than those of Atlanta, (which incidentally has neither mayoral control nor has it been awarded the Broad prize.)

I also analyzed NYC's record in 8th grade reading, to see how our schools have done compared to other large urban districts.

This is what I found: NYC comes in dead last among the ten cities tested since 2003 in this subject -- the only city to make absolutely no progress.

Then I looked at the achievement gap issue between racial and ethnic groups, which Bloomberg and Klein's repeatedly claimed to have narrowed over the past few years. At one point, Bloomberg even went before Congress and boasted that he had cut it in half. Is that true?

No, of course not
; In NYC schools, according to the NAEPs, the achievement gap is still very substantial, in all subjects and grades, from 23 to 35 points depending on the category.

But have these gaps narrowed over time? Well, the black-white gap has shrunk by one or two points in 4th grade reading and 8th grade math, which is relatively small in relation to the overall size of the gap; and the gap has increased slightly in the other two categories.

In terms of the white-Hispanic gap, Bloomberg and Klein's record is even worse. In three out of four categories, the gap has increased; and it has grown much larger in 8th grade reading and math.

So, when you look at the NAEP scores, the progress made in NYC under Bloomberg and Klein compared to other large cities is only a little better in some areas, and in some areas, like 8th grade reading or the Hispanic achievement gap, far worse.


Anonymous said...

Sooooo when will we see Klein being led out of Tweed with his jacket over his head???

Anonymous said...

I think there were upticks in instruction and administration, due to putting people under undue pressure and shaking things up, not due to any sort of reform or planning. This was accompanied by demoralization, many shattered or destabilized careers that could have or could still soar, a fraudulent culture of test prep, and a degrading of local public education. Successes and abuses re charter versus trad. public schools is largely a wash. This leaves things in highly negative territory; nets red as a beet! Get these people out of office. The Mayor should recuse himself of control and not even be allowed to speak to the topic of education. Have someone competent clean up the mess and responsibly manage the schools.

ed notes online said...

Next BloomKlein myth to fall - phony high school grad rates.

Anonymous said...

Ed, the high school grad rates are of course directly related to the tests. Teachers are told that the kid passed the test so they have "demonstrated mastery" and should be passed. Thus the graduation rate rises. It also helps them that they report a seven year rate and hold the schools to a four year. Parents please keep this conversation going we teachers are with you, we know there are problems in education and we want to find the answers together with you.

Anonymous said...

So let's see, at my school in the Bronx we tout a 90% graduation rate; however, I have noticed that classes are much smaller in the higher grades. The 11th and 12th grade teachers have half the kids in the classroom that the 9th and 10th grade teachers have; also there are fewer classes of 11th and 12th graders. For example, 9th grade has 3 English teachers and 5 sections of English while 11th grade has only one teacher and 2 sections. Where do the kids go? Are they all stuck in 9th and 10th grade. It doesn't seem like it as we are strongly encouraged to pass kids along; if not you the teacher are in trouble b/c kids cannot be successful in your class. Where are the kids going?