Assembly Member Jim Brennan on the DOE Progress Reports

As reported here earlier, two members of the State Assembly testified at City Council hearings on the Progress Reports Tuesday. As the Assembly must decide whether to renew mayoral control or replace it with something else, their thoughts are always relevant. News accounts of the hearings did not mention their testimony, most likely because the press was covering the escape of Chief Accountability Officer Jim Liebman from the Council chambers that occurred simultaneously with the testimony. Here is the first part of Brennan's testimony:

Councilmember Jackson and members of the committee, it's a pleasure to share some thoughts with you about the Department of Education's progress reports. I'm Assemblymember Jim Brennan, representing the 44th Assembly district in Brooklyn. Currently, I chair the Assembly Standing Committee on Cities and I have served on the Assembly Education Committee for 23 years.

In a November 20th e-mail that Chancellor Joel Klein sent out to Department of Education faculty members, he stated, "After almost five and a half years as chancellor, I know you can't point to a single number, be it a test score or graduation rate, to prove success or failure. The whole picture is important." Yet, in his November 14, 2007 letter to the editor of the New York Times, he defended the school progress reports by stating, "Everyone knows what A and F mean. Summing up all relevant measure with a single, simple grade draws sharp attention to the great work at many schools and the stagnation that might otherwise escape notice elsewhere."

His first statement seems more applicable to the New York City school system. The DOE has reduced teacher, student and parent surveys, attendance, test scores, one-year changes in test scores, and weights these variables differently, to come up with one letter grade. This "reductionism" has produced results that, for many individual schools, seem on their surface to be utterly irrational and have caused bewilderment, confusion, and rejection.

The State and City of New York already have an assessment and accountability system. The State Education Department calls it the "status" model, and parents, educators and the general public all understand it simply as scores on tests, broken down into levels of proficiency or lack thereof. Whatever its shortcomings, it's generally understood. On top of this, we have the Federal and State "No Child Left Behind" standards, which already hold schools accountable for making adequate yearly progress for the whole school as well as Black and Hispanic children, Asian children, Native American children, English language learners, special education and economically disadvantaged students. The State has a system called School Under Registration Review and we also have School Report Cards, which provide lots of information for parents and educators. The Department of Education has a history of closing poorly performing schools, and principals currently already have evaluation standards in their contract that allow for their removal. The core standards in all of these accountability systems are, of course, test scores. The validity of this system has been under debate for many years.

Along come the DOE's progress report cards. Only 30% of the grade is based on the old performance test score, the remainder is based on criteria completely new to the City and State, 55% on one-year changes in test scores and 15% on surveys and attendance. The main component of the new model, the one-year test score change - the "growth model" - did not get vetted by any other public body other than the DOE. There was no public hearing on the inputs and assumptions into this statistical model, there was no vote by the Panel on Educational Policy, there was no review or approval by the State Education Department and its own division of Assessment and Accountability. I believe concerns had been expressed informally that New York City's new system would produce results inconsistent with the existing system that might cause confusion and consternation. It is my understanding that the SED has never seen the guts of the new statistical system - the growth model, its inputs, its assumptions, its complex weighting. We have no knowledge of whether the new system controls for the probability that measuring test results within one year would fluctuate within a certain range up or down and that therefore no conclusions school quality could be drawn from such results. Last week I sent a letter to the State Education Department asking they review the validity of the statistical models used for DOE's progress evaluations.

...

For full text click here and letter to State Education Commissioner Mills here