There’s no accountability for anyone at the top at DOE; and I can prove it

The testing companies Pearson and McGraw Hill can screw up over and over again – and  continue to get  million dollar contracts.  Principals rarely  get fired, and even those caught cheating are appointed “achievement coaches.”  Yet teachers and children are threatened with punitive sanctions – either with losing their jobs or being held back – if their performance does not improve fast enough.  

Almost two years ago, NYC DOE gave their crude and unreliable evaluations of teachers based on test scores -- called the teacher data reports -- to the press, though they had promised never to release them.  All the daily papers promptly posted them.  Klein claimed he had no choice because they had been FOILed – though it was well known that he had asked reporters to FOIL them.

So in February 2012 – about 16 months ago – after the DOE released the TDR’s, I FOILed for the performance evaluations of the Chancellor and the other top officials in the leadership team at Tweed for 2009, 2010 and 2011.  More on my original FOIL request here.

According to Robert Freeman of the NY State Open Govt. Committee, the performance evaluations of all public employees in NY State are available to the public through FOILs, except for police, correction officers and firefighters.  In June of last year, the Legislature passed a law, restricting the view of a teacher’s evaluation to a parent with a child in the teacher’s class.  Mayor Bloomberg went ballistic, and said he would  have every school call every single parent to tell them they had the right to this information, and would post the ratings on ARIS. 

After waiting for more than a year, the "final" response to my FOIL finally came back from DOE about a week ago: that there are NO written performance evaluations for ANY of the top officials at Tweed  for 2009, 2010 or 2011, including the Chancellor:

“Diligent searches and inquiries for responsive records have been conducted as to any Chancellor (and his/her Chief of Staff) Chief Academic Officer, Senior Deputy Chancellor, Chief Schools Officer, Deputy Chancellor, Chief Operating Officer, Chief Financial Officer, and General Counsel for the time period covered y your request.  It is my understanding that no such records were located, because no such records have been created.  Accordingly, there are no records to provide.”

 See the pdf here and the photo above for the complete letter.  DOE prides itself as operating like a business, but there is no corporation or organization that would fail to evaluate the performance of its top executives.

Instead, the educrats at Tweed are more like a petty dictatorship, whose workings are so dysfunctional and mismanagement so extreme that they would have been fired years ago, if there was any accountability at the top.

Why no one in his right mind should believe the school grades OR the teacher growth scores

UPDATE: Here is Gary Rubinstein's new graph of school ranks. this year compared to last, according to the Progress reports.  The "x" axis is 2011; the "y" axis is 2012. 
The new DOE school grades for elementary and middle schools, euphemistically called “Progress reports,” came out with much fanfare, with 217 schools potentially put on the closing list because they either received failing grades or three “Cs” in a row, more than ever before.  As parent leader Shino Tanikawa pointed out in the New York Post, DOE is using unreliable grades to implement terribly misguided policies. (If you'd like to see your school's grade anyway, you can find it here.)

Last year’s headlines ran like this: School report cards stabilize after years of unpredictability. This year again, reporters cited claims of DOE officials, who “highlighted the stability of this year’s reports.”

It wasn’t true last year and it isn’t this year either.  Here is a figure from Gary Rubinstein’s blog from last year, showing no correlation between the rank order of schools in 2010 and 2011. I suspect the Gary’s illustration will be similar for this year, once he gets around to making it.

As InsideSchools reported, 24 out of the 102 schools that received “D”s or “F”s this year had received top grades of “A” or “B” the year before. Other high-performing schools, such as PS 234 in Tribeca that received an “A” last year, fell precipitously to a “C” with the same principal, same staff and most of the same students.  The school plunged from the 81^st to the 4^th percentile, which would have meant a “D,” if not for the DOE rule that no school that performs in the top third citywide can receive a grade lower than C, as Michael Markowitz pointed out in a comment on GothamSchools.

According to the DOE formula, 80-85% of school’s grade depends on last year’s test scores on the state exams.  Most of that figure is based on supposed “progress,” i.e. the change in test scores from the year before.  As I have pointed out many times, including in this 2007 Daily News oped, “Why parents and teachers should reject the new grades”, experts in statistics have found that the change in test scores at the school level from one year to the next is highly erratic and up to 32-80% random. 

In recognition of this fact, Jim Liebman, who developed the school grading system, originally told skeptical parents that the system would eventually incorporate three years of test scores, which would lessen the huge amount of volatility, a promise that DOE has failed to abide by.  For proof of this promise, see Beth Fertig’s book, Why Can’t U teach me 2 read?:

“…[Liebman] then proceeded to explain how the system would eventually include three years’ worth of data on every school, so the risk of big fluctuations from one year to the next wouldn’t be such a problem. (p.121)”

To make things worse, the state exams last year and the scoring guides were riddled with errors, as many parents, teachers and students noted . Finally, test scores are not a good way to assess school quality, for myriad reasons, even if the tests were perfect and the formula based on multiple years worth of data, as I pointed out in this NYT Room for Debate column. 

LESSON: Anyone who believes in the accuracy of these school grades is sadly misinformed, and DOE’s attempts to dissuade parents from sending their kids to schools with low grades or to close schools based upon such unreliable system is intellectually and morally bankrupt.

In another untenable move, the teacher “growth scores,” also based on the one year’s change in test scores, but this time at the classroom level, have been released to principals outside NYC.  These growth scores will be incorporated in the new teacher evaluation system to be imposed statewide.  See this excellent column by Carol Burris about how the heedless use of growth scores is likely to damage the education of our kids. 

Here are the detailed results of the statewide survey by principals, including their comments, showing that the vast majority believe that these scores are NOT an accurate reflection of the effectiveness of individual teachers.  More than 70% of principals also said they were “doubtful” or strongly opposed to any use of growth scores in teacher evaluations.  Aside from the annual volatility in these scores, there are many other problems with relying upon such flawed measures of teacher quality:

The growth scores, developed for the state by the consulting company AIR, attempted to adjust only for the following demographic factors: 

• Economic disadvantage (ED), but without differentiating free lunch or reduced lunch students – very different, with very different expected outcomes;
• Students with disabilities (SWDs), but not types or severity of disability;
• English language learners (ELLs).
  

And even though AIR did attempt to control for the above factors, they still admitted that teachers who work at schools with large numbers of students who were poor or had disabilities tended to have lower growth scores. 

AIR made NO attempt to control for classroom characteristics such as class size, or the racial/ethnic background of students, or any other variable that is known to affect achievement. This is different from the NYC value-added teacher data reports, released last year by DOE, widely derided as unfair and unreliable, which at least attempted to control for many of these factors.

Even so, Bruce Baker, professor at Rutgers, found that while the teacher data reports claimed to control for class size, teachers at schools with larger class sizes were significantly more likely to be found ineffective than those who taught at schools with small classes, and as “class size increases by one student, the likelihood that a teacher in that school gets back to back bad ratings goes up by nearly 8%.”  The situation with these growth scores is yet worse, with no attempt made to control for class size at all.  Is it fair to deny NYC teachers tenure and/or risk losing their jobs because they are saddled with larger classes than teachers in the rest of the state?

AIR also found that teacher growth scores tended to be higher in “schools with students with higher mean levels of ability in both ELA and in Mathematics.” This shows that teachers who work in schools with large numbers of low-achieving students are more likely to be found ineffective. 

Apparently NYC principals have not yet been offered the opportunity to examine the growth scores of their teachers, unlike principals in the rest of the state. As far as I know, DOE has not explained why.  With far larger numbers of struggling students who are economically disadvantaged and crammed into larger classes, it is quite likely that a higher proportion of NYC teachers will be found ineffective than elsewhere in the state– and thus unfairly penalized for teaching disadvantaged students in worse conditions.

Here is the link where you can find the AIR technical manuals on growth scores.  

A NYC teacher's response to the publishing of her confidential -- and unreliable -- data report

Roe Wrubel is currently a 4th grade teacher.  My daughter was lucky to have Roe as her teacher in the 1st and 2nd grades. Her talent and commitment encapsulates everything that is great about the profession. For those who are interested, Roe's TDR came out far above average.  But that did not prevent her from being distressed about its release and publication.  She posted the below eloquent remarks next to her TDR on the NY Times website.  With her permission, I am reprinting them here:
 

I truly don't have the words to express the enormity of how this small act, this one 'report', this blip that's been added to the overwhelming archive that is the world wide web, encapsulates all that is wrong with how the public views the science and the art of the educator.  In my 24 years as a NYC public school teacher I have never been so disheartened, so demoralized, so utterly disappointed and felt so completely hopeless.

The release of the Teacher Data Reports on February 24 makes no sense to me. When information has been proven to be inaccurate, when sample sizes are statistically insignificant, and when 18,000 out of 75,000 teachers are singled out, yet the data is still released, the precedent that is being set undermines what tens of thousands of teachers have dedicated their lives to doing. The Value-added model also almost ignores student performance, focusing mostly on progress. If a student scores well on the third grade test, they and their teacher are penalized if they get even one less question correct on the fourth grade test, which, by the way, is significantly harder than the third grade test. Now if I’ve helped my students grow a year’s worth, why would I be penalized for that?

Most educators I know, and I know quite a few, are exceptional, dedicated, intelligent, diligent, caring and honest people. NYC public school teachers have been working under an expired contract, for wages that do not match neighboring school districts, at rates that do not even begin to approach inflation in these harsh economic times, and do so with a smile on their faces as their students enter their classrooms on a daily basis.

This cowardly and appalling act was my proverbial straw. We teachers put enough pressure on ourselves as we strive to do better for our students every, every, every single day. I know many of my colleagues feel the same way as I do. For all of us to be demoralized yet again, and so publicly, is utterly heartbreaking. We know 'what happens in Vegas stays in Vegas', but what happens on the Internet lives forever.

I often feel frustrated as a teacher because I can never do enough, be enough or give enough to my students, although I try. I spend hours and hours beyond the school day and every weekend doing everything I can to help my students grow as independent, responsible, inquisitive learners. I am always preparing lessons, corresponding with parents, making charts, hanging bulletin boards, developing and refining curriculum, reading and responding to the work of my students, and I do it all because it’s what I believe my students deserve. I commute over three hours a day to work with the exceptional teachers, staff and administrators in my school because I want to always learn and think and grow myself.

Why is no one talking about class size anymore? If I had fewer students in my class, I promise you that I could give them more attention, and help them move forward in their skills much more easily. Blaming teachers for the failings of the education system, the testing and scoring process, and the economic climate will not in any way help children!

Unless you live the life of an educator, even the most well-informed and well-meaning individuals just don't get it. We put our heart and souls into teaching the whole child, and yet are judged so arbitrarily, unfairly and now, publicly. I have no problem being evaluated by my administrators. And if any of them wanted to reveal what they think of me as a teacher to families, good or bad, I would welcome it. They know me, and they know my students, and they see the whole picture, and they know the effort I put into reflecting upon and refining my pedagogy every hour of every day. I wish there were more people out there advocating for teachers who really understood and could express the realities of what education and educators can and cannot be.

I am shocked that many of the media outlets and all of the politicians are ignoring the current super-sized testing that will be taking place in New York state in April of 2012 in favor of giving credence to the release of outdated and relatively meaningless data. While my fourth grade students are taking the ELA test, for example, they will be reading 13 passages (including the listening selection which requires note-taking), answering 57 multiple-choice questions, writing answers to 7 short-response questions and composing 2 extended-response essays over the course of three days of testing.

That testing will last approximately 175 minutes. While they are taking the State Math tests they will be asked to solve 62 multiple choice questions, solve and explain their thinking for 5 short response questions and carefully solve 4 extended, multi-step response questions, all covering multiple strands of mathematics. They will have 180 minutes, exactly three hours, to do so over the course of three days. Just thinking about all of that work in such a short amount of time makes me tired! The stamina involved is almost impossible for a nine or ten year old child to muster. I might fade under the same circumstances.

Not only that, but these scores will determine what middle school a child gets into. Now they will also determine whether or not a teacher is considered effective as well. There are approximately 59,000 minutes in a school year, not including lunch and recess. These tests cover 350 minutes in the life of a student. This is what determines the value of a child’s skills and now a teacher’s skills. The flaws are in the testing system and in the evaluation of that testing, not in the work of dedicated teachers.

I want students to learn and I want educators and education to get better for all students. Why can’t we find real solutions to real problems, instead of placing blame, instigating panic, and scoring political points on the backs of those who care the most and work the hardest?  I am still confused about how releasing this data, that was intended for internal use and meant to inform instruction, holds any value to parents. The information does not give a true picture of an educator; the inaccuracies actually perpetuate false information. How will that help a parent to support their child’s learning development? Parents also do not and should not choose who their child’s teacher is. With this made up ammunition, parents will be trying to have their child’s class changed, all for no particular reason.

The philosophical underpinnings of how I teach are antithetical to what has happened with this data being released to the public. I believe in child-centered, inquiry based, socially responsible and reflective learning to help create the kind of thinkers and citizens our society needs and deserves. Children have a natural quest for learning and a desire to be a part of a learning community. It is my job, no, my pleasure, to help usher my students towards living the lives of dedicated, engaged and thoughtful learners. The entirety of what I am tasked to do, and the reality of the circumstances of the daily lives of children and families, are really all I'm concerned about. Bravo to all of the teachers out there who continue to play the role of Sisyphus yet again. I for one will keep pushing that boulder up the hill until I get it to reach the other side.