The NY Times issues a correction, too little and too late.

An article in the NY Times magazine two weeks ago written by Jonathan Mahler on April 10 , about Middle School 223 in the Bronx, contained the following passage describing student achievement gains under Joel Klein:

“Since 2006, the city's elementary and middle schools have seen a 22-point increase in the percentage of students at or above grade level in math (to 54 percent) and a 6-point increase in English (to 42 percent).”

Upon reading this statement, I immediately knew it to be untrue. There has been little or no improvement in student achievement in NYC since 2006 – or even since 2003, when the Klein first implemented his policies, according to the most reliable national assessments called the NAEPs. 

In fact, after the NY State Education Department recalibrated the exams, increasing the scale scores needed for proficiency this summer in response to overwhelming evidence that the state tests and their scoring had gotten much easier over this period, the percent of NYC students at or above grade level actually dropped precipitously compared to 2006. 

See these charts, for example, from the DOE’s own website, showing the sharp fall in the percent of students at proficiency in English Language Arts, as calculated by the state, following the apparent (but highly unreliable) increase that had occurred between 2007 and 2010:

And below in math:

I wrote to the reporter, Jonathan Mahler, to ask where he got his data.  Here is his reply:  “The DOE was my source, and the fact-checker went back over the numbers with them.”

That didn’t exactly satisfy me, so I sent in a request for a correction via email to the editors, reproducing the charts above, as well as their source.  

A few days, I received a phone message from Aaron Retica, head of the research department at the NYT magazine, saying “we aren’t wrong” but that he understood why I might have been “confused.” 

 I followed up with phone call, and Retica explained that they were doing an “apples to apples comparison” by taking the DOE's re-estimate as a more reliable comparison, assuming the scale scores needed for proficiency had not changed. In other words, they bought the DOE spin, hook line and sinker, as portrayed on this slide, from the same power point deck:

This is a slide that Shael Suransky presented throughout the city last fall, including during my debate with him at NY Law school, in an attempt to convince his audience that NYC achievement levels had actually increased over the last four years.   

Though this slide may have confused many, it convinced very few; that is, aside from (apparently) NY Times editors.  In small words at the bottom, the slide says, ”Starting in 2010, NYSED changed the scale scores required to meet each of the proficiency levels, increasing the number of questions students needed to answer correctly to meet proficiency.”  

The slide, in claiming that the proficiency of NYC students in ELA in 2010 had risen to 68.2% rather than 42.4%; and in math; to 83.3% rather than 54%, disputes the judgment of the State, and ignores the fact that the number of questions needed to achieve proficiency was increased, not out of some arbitrary motive to punish NYC or devalue the accomplishments of its students, but because the state -- as well as all independent experts -- realized that the questions had themselves gotten easier over time. 

Indeed, this recalculation reeks of an attempt to rig the results, to back up improbable claims of great gains when the NAEPs have shown negligible increases. (Of course the meaningless nature of this exercise should be obvious in that there are two very different versions of achievement gains displayed in the above slide, depending on whether 2009 or 2010 cut scores are used.)

Like the DOE, the NYT is hugely averse to correcting errors, even of the most egregious kind –as in a Steve Brill article published in the magazine section last year on Harlem Success Academy, some of whose errors I catalogued here. 

The larger problem is that for many years, the NY Times ignored and/or omitted any reporting of NY’s test score inflation, delaying until after the bubble had burst, and long after the scandal had been covered extensively by the Daily News,  Gotham Schools, and even the NY Post .

Instead, the Times editors seem chronically disposed to give credit to Bloomberg and Klein where none was due; as anyone who was paying serious attention to this issue, and to the relatively flat results of NYC on the national assessments called the NAEPs, would be aware.

See for example, this 2007 post on our blog by Steve Koss, relating an ingenious experiment carried out by Erin Einhorn of the Daily News, when she gave the 2002 and 2005 math tests to the same bunch of children, revealing how they got better results on the 2005 exam. (For some reason the original Einhorn article is no longer available online.) Or this follow-up News article, in which leading test experts called for an independent audit of the state exams, which did not occur until three years later.

Where was the NY Times amidst all these revelations? Absolutely nowhere.   

To the contrary, on August 4, 2009, at the very moment when Bloomberg was pressing for the extension of mayoral control of the schools, and two years following extensive and continuing coverage of test score inflation in the News and elsewhere, the Times published a credulous account that recounted the steep increase in state test scores and the apparent narrowing of the achievement gap, featuring this quote from Joel Klein:

“Mr. Klein, for his part, said he was confident that rising scores reflected real improvements. “No matter how you look at them,” he said, “the picture is one that shows that the city is making dramatic progress.”

As I wrote in August 2009 to the Times education editor, Ian Trontz:

“… there are many prominent administrators, researchers, teachers and principals who believe strongly that there has been rampant state test score inflation in recent years. … as has been widely reported in the Daily News and elsewhere … To leave this out of your story seems negligent at best…

(See my critique at the time of their August 2009 article, NY Times falls in line with the Bloomberg PR spin control; and the response from Trontz: The NY Times response, and my reply.)

Shortly afterward, Wayne Barrett wrote about the controversy in the Village Voice,

"The Times front page piece last week -- headlined "Gains on Tests in New York Schools Don't Silence Critics" -- failed to quote any real critics, but gave Klein six self-promoting paragraphs. It did bury a single questioning quote from two academics not known as critics of the test scores in the thirty-fourth paragraph, but the top of the story trumpeted success scores that would have silenced any critic. If, that is, they were true."

Two days after the Times article ran, the NY State Senate voted to renew mayoral control witout any checks and balances, essentially allowing Bloomberg to retain his stranglehold over the schools.  The "paper of record" could not have done a better job at burying the story that DOE's gains were illusory than if they had actually tried. 

After State Ed officially burst the test score bubble last summer, the Times finally covered the issue in a front page story in October, entitled "On NY School Tests, Warning Signs Ignored."  But here, too, the paper left its own deficient reporting off the hook, and refrained from mentioning any of the abundant exposes that had appeared over the last three years, not only in the Daily News, but in GothamSchools, the NY Post, and our blog; any of which should have alerted interested observers to the reality.   (Here is my account of the failure of the NYT to cover this story in their revisionist history.)

So what happened with this example of the NYT editors credulously swallowing the latest DOE’s test score spin? 

The editors did run a correction, but phrased in a very confusing manner.  Unless a reader received their information elsewhere, it would be very hard for them to interpret its meaning.  Here is the correction they printed today, with the specific passage in bold:

An article on April 10 about Middle School 223 in the Bronx misstated the reasons the school does not qualify for some state financing earmarked for poorly performing schools. It does not meet some of New York State’s criteria for failing schools and it is relatively successful on state tests and other measures. It is not because of the school’s report card from New York City. The article also omitted an attribution for the increases in percentages of students at or above grade levels in math and English from 2006 to 2010. Those figures came from the New York City Department of Education, which did its own analysis of state testing data using 2010 proficiency levels for 2006 test scores. (Without that adjustment, the percentage of proficient learners in both math and English actually dropped from 2006 to 2010.) (emphasis added)

Even worse, here is how they amended the offending passage in the article itself:

Since 2006, according to an analysis of state testing data by the city's Department of Education (which used 2010's recalibrated proficiency levels to compare 2006's testing data to 2010's), the city’s elementary and middle schools have seen a 22-point increase in the percentage of students at or above grade level in math (to 54 percent) and a 6-point increase in English (to 42 percent).

So in the article and the correction, they failed to explain the larger context and why the state had recalibrated the proficiency levels – to correct for the fact that the exams had gotten easier over time. 

Instead, their explanation of this matter could lead an unsuspecting reader to believe that the DOE’s “adjustment” was entirely appropriate, and provided reasonable evidence of improvement – while omitting any mention of more reliable evidence from the NAEPs of largely stagnant results.

On the same day his article was published in the magazine section, Jonathan Mahler wrote an article in the Week in the Review section of the Times, entitled “Deadlocked Debate over Education Reform.”  In it he wrote:

“The data can appear as divided as the rhetoric. New York City’s Department of Education will provide you with irrefutable statistics that school reform is working; opponents of reform will provide you with equally irrefutable statistics that it’s not. It can seem equally impossible to disentangle the overlapping factors…”

In the future, I would hope that the editors of the Times might try a little harder to look at the data with a more discerning eye, and not swallow the distortions of the Department of Education. 

Perhaps they might even refer to independent experts who could dissect the data if they find it too difficult to interpret it themselves.  They owe it to their readers.  There is little point in trying to cover the NYC public schools if they continue to be incapable of weighing the evidence objectively and presenting the facts with a more practiced eye, rather than simply regurgitating what is handed them by the spinmeisters at Tweed. 

After all, if the Daily News and the NY Post can do it, why not the Times?