Why the release of the Teacher data reports and adoption of a new statewide evaluation system will be bad for teachers and bad for kids

UPDATE: please sign our petition, also to the right , urging the newspapers not to publish this unreliable data.  

The UFT has lost its lawsuit and the DOE says it will release the teacher data reports to the media within weeks, with all the major newspapers expected to print them.  These reports, which rate 12,700 NYC teachers by means of  numerical ratings of 1 to 100, are based solely on the 2010 test scores of their students, filtered through a complicated value-added formula.  They are widely seen by many experts as highly unreliable, based on false or incomplete data and with huge margins of error -- even if you believe that standardized test scores alone are all that matter.

The city will release these data reports, despite the promise in 2008 by then-Deputy Chancellor Chris Cerf, now Acting NJ Commissioner of Education,  that the DOE would do everything in its power to keep them confidential.

Even the expert panel that DOE claimed supported their methodology pointed out that these figures cannot  be trusted.  The DOE had posted this statement in 2008, which is still online,  in defense of the Teacher Data reports:

“A panel of technical experts has approved the DOE’s value-added methodology. The DOE’s model has met recognized standards for demonstrating validity and reliability.”

Yet the opposite was true, as revealed by a document that I finally received in 2010, more than a year after I had FOILed it.  (See also this Juan Gonzalez column for more about this document and my FOIL.)  

Here are just a few of the comments from this panel of "technical experts" handpicked by Tweed, who refused to endorse the method's use "for accountability, promotion or tenure" purposes:

1) Test scores capture only one dimension of teacher effectiveness, and they are not intended to serve as a summary measure of teacher performance…

2) If high stakes are attached, there will be potential to game these measures by teaching to the test, selecting students, altering difficult-to-audit student characteristics, or outright cheating. …

3) To calculate expected test scores…there are likely to be additional factors not yet considered that influence student achievement. …etc. "

I can think of no other profession in the public or private sector in which this kind of unreliable and potentially damaging data is made public.  The only effect of this will be to further undermine teacher morale -- already at an all-time low in this city -- and to dissuade teachers from working in our public schools and with the highest needs children.  Yet so far, GothamSchools is the only media outlet that has pledged not to publish them. 

Meanwhile, the Governor is pushing a deadline of Thursday for the state and city teacher unions to agree on a statewide evaluation system, called APPR,  for the Annual Professional Performance Review, that will rate teachers 20-40% on test scores, and the rest on principal evaluations. Yet nearly one third of all principals in the state have signed onto a letter protesting this system, for reasons that are further explained here and here.  In the city there is even more discord, because the DOE refuses to give teachers the right to appeal a principal's negative rating to an independent arbiter, despite numerous documented cases in which NYC principals have arbitrarily delivered unsatisfactory ratings to teachers for political or personal reasons.

Below is a letter from eight esteemed Teachers of the Year, originally posted on the NY State Teachers website, sent to the NY State Board of Regents last spring, pointing out how the proposed APPR is likely to unfairly penalize many excellent professionals, especially those work with at-risk students.  Nevertheless, on Monday, the Regents voted to go full speed ahead with its NCLB waiver application to the US Department of Education, that will further commit them to this damaging evaluation system.

____
Dear Chancellor Tisch and Board of Regents,

It is with sadness, pain and frustration that we write this letter. We are deeply concerned about recent changes to the State Education Department's Annual Professional Performance Review system. These changes, while politically popular, will neither improve schools nor increase student learning; rather, they will cause tangible harm to students and teachers alike.

The changes to APPR will kill the spirit of collaboration that developed from NYSED and NYSUT working together. Evaluating teachers based on test scores is a huge paradigm shift. The fact that NYSUT was willing to work with NYSED to develop a fair evaluation process shows good will on the part of teachers across the state. To unilaterally change the terms of a jointly crafted law at the eleventh hour poisons the atmosphere. Without buy-in from practitioners in the field, this reform effort is unlikely to succeed.

We believe in appropriate use of data to improve student achievement.

Likewise, we believe that schools should develop rigorous systems to evaluate teachers and support professional growth; however, to allow 40 percent of a teacher's evaluation to hinge on a single standardized test score risks great harm to our schools and the people therein.

We could quote the research of educational experts like Diane Ravitch, Richard Rothstein and Jonathon Kozol as to why poverty and parental support affect test scores significantly more than any curricular changes a school can provide. We could refer to myriad child psychologists who have documented the deleterious effects of high-stakes testing on our nation's youth. We could call upon assessment experts who insist that standardized tests were not developed to evaluate teacher effectiveness. And we could examine the last decade of educational results that followed No Child Left Behind: rampant gaming of the system to provide the appearance of growth, narrowing of the curriculum, excessive teaching to the test and virtually no change in the achievement gap.

All of the above would lead the reasonable person to be skeptical about using standardized tests as the engine for school reform. Worse yet, we fear that the competition generated by this approach will reduce the collaboration necessary for true school improvement.

To illustrate the challenges of the new APPR system, we offer these stories from our schools:

1) Andrew has a severe learning disability. He is a hands-on learner who struggles on written exams. His resource teacher, counselor and mother thought he would be best-served taking a challenging science course, even though everyone knew he would fail the Regents exam. When 40 percent of a teacher's evaluation depends on that test score, will schools still make this sort of humane, pedagogically sound decision?

2) Jason missed two days of school this week for golf sectionals. He is a weak student and will struggle to pass the Regents exam. He will miss yet another day next week and perhaps more days if he advances to the state tournament. These golf matches were scheduled during school hours by officials representing New York State. Does the coach or sectional committee bear any responsibility for Jason's performance on the Regents exam?

3) Tranh moved to America in January to live with his uncle. He speaks very little English and missed half a year of instruction. Who is accountable for his standardized test scores?

4) Simone will miss school all next week because her parents are taking the family on vacation. She will miss five days of instruction for this illegal absence. Will her teachers get an asterisk placed next to Simone's test scores?

5) Emily finally told her doctor and her parents that she is struggling with depression. She is starting counseling and medication. Needless to say, her grades are suffering. As Emily's life hangs in the balance, how do we find the strength to show her compassion when we know her poor grades will negatively affect our evaluation?

6) Trudy is a veteran teacher. She volunteered to teach a class of at-risk learners because she has the skills to do so. Her passing rate on the Regents exam will be significantly lower than her peers teaching the stronger students. Under the new APPR, what motivation will teachers have to take on the most challenging students?

7) Marcia teaches art, Beth teaches special education and Craig is a guidance counselor. There are no standardized assessments attached to their jobs. They are gifted educators, but they - like many others in our profession - will not feel the same pressure as those teachers who have a high-stakes exam attached to their course. How do we deal with the divisiveness caused by this inequality?

8) Diane teaches fourth grade. She worked diligently to prepare her students for the ELA. She went to workshops to learn about standards and her passing rate suggests great skill as a teacher. Last spring, the cut scores were changed without warning. Suddenly both Diane and her students seem less-skilled. How do we ensure that the vagaries of testing don't harm people like Diane and her students?

All of the above issues are real and will take time to work out. That's why the new APPR system must be implemented slowly and thoughtfully. Increased time would allow schools to grapple with these thorny issues. Forcing schools to implement a plan without proper preparation will produce anger, stress and confusion, none of which will help kids.

We fully understand the desire to improve accountability. Using external assessments for a small part of a teacher's evaluation, as agreed to by NYSUT, seems fair and reasonable. Changing the law without warning seems less so.

On behalf of our colleagues across the state, we ask you to please reconsider the original plan that was agreed upon by all stakeholders.

This collaborative approach would ultimately provide the most benefit to our students.
Sincerely,

• Jeff Peneston, 2011 New York State Teacher of the Year
• Debra Calvino, 2010 New York State Teacher of the Year
• Vickie Mike, 2009 New York State Teacher of the Year
• Rich Ognibene, 2008 New York State Teacher of the Year
• Marguerite Izzo, 2007 New York State Teacher of the Year
• Stephen Bongiovi, 2006 New York State Teacher of the Year
• Elizabeth Day, 2005 New York State Teacher of the Year
• Dr. Patricia Jordan, 1993 New York State Teacher of the Year