A court decision on the teacher data reports that will hurt our kids

It is unfortunate that the day after a court decision held that NY teachers should be evaluated by use of multiple assessments, with student scores on state standardized tests only one minor factor, today, the appellate court said that the DOE could release the teacher data reports to the public, based only on these same test scores. 

Most testing experts agree that these reports are highly unreliable and reductionist, and they will unfairly tarnish the reputation of many excellent teachers:

1.     The state tests were never designed for such a purpose – and are technically unable to make year to year judgments on “progress” or value added. 

2.    Many studies have shown the extreme volatility of these measures, and how the results differ even from one sort of test to another.  See Juan Gonzalez’s column on how DOE consultants themselves believe these reports are highly unreliable; here are links to the original documents revealing this, obtained through a  FOIL.

3.    As John Ewing, former executive director of the American Mathematical Society, recently concluded, ”if we drive away the best teachers by using a flawed process, are we really putting our students first?  Mike Winerip reported on a top-notch NYC teacher who was denied  tenure in just this manner.

If NYC goes ahead and releases this data it would likely be the first school district in the country to do so willingly and enthusiastically; when the LA Times generated its own value-added data for Los Angeles teachers, the paper was widely criticized.  Chris Cerf, former deputy Chancellor and now acting State Superintendent of NJ schools, was originally in charge of creating the teacher data reports; he promised that they would never be used for teacher evaluations and that the DOE would fight against any effort to disclose them publicly. In a 2008 letter to Randi Weingarten, Cerf wrote: "It is the DOE's firm position and expectation that Teacher data reports will not and should not be disclosed or shared outside the school community."

Chancellor Walcott should think twice before releasing this data, if he cares about real accountability, the morale of teachers,  and the potential damage to our kids.

Here are some of the recent studies from experts on the unreliability of this evaluation method:

Sean P. Corcoran, Can Teachers be Evaluated by Their Students’ Test Scores? Should they Be? The Use of Value-Added Measures of Teacher Effectiveness in Policy and Practice. As  the author concluded from his analysis, “The promise that value-added systems can provide a precise, meaningful, and comprehensive picture is much overblown… .Teachers, policy-makers and school leaders should not be seduced by the elegant simplicity of value-added measures. Given their limitations, policy-makers should consider whether their minimal benefits outweigh their cost.”  

Economic Policy Institute, “Problems with the Use of Student Test Scoresto Evaluate Teachers.” 2010.

National Research Council, Henry Braun, Naomi Chudowsky, and Judith Koenig, eds., GettingValue Out of Value-Added: Report of a Workshop, 2010: “Value- added methods involve complex statistical models applied to test data of varying quality. Accordingly, there are many technical challenges to ascertaining the degree to which the output of these models provides the desired estimates.”  

John Ewing, former executive director of the American Mathematical Society, current president of Math for America;  MathematicalIntimidation: Driven by the Data; “Why must we use value-added even with its imperfections? Aside from making the unsupported claim (in the very last sentence) that “it predicts more about what students will learn…than any other source of information”, the only apparent reason for its superiority is that value-added is based on data. Here is mathematical intimidation in its purest form—in this case, in the hands of economists, sociologists, and education policy experts…And if we drive away the best teachers by using a flawed process, are we really putting our students first?"

Sean P. Corcoran, Jennifer L. Jennings, Andrew A. Beveridge, Teacher effectiveness on high- and low-stakes tests; April 10, 2011. " To summarize, were teachers to be rewarded for their classroom's performance on the state test or alternatively, sanctioned for low performance many of these teachers would have demonstrated quite different results on a low-stakes test of the same subject.  Importantly, these differences need not be due to real differences in long-run skill acquisition…

That is, teachers deemed top performers on the high-stakes test are quite frequently average or even low performers on the low-stakes test. Only in a minority of cases are teachers consistently high or low performers across all metrics… Our results… highlight the need for additional research on the impact that high-stakes accountability has on the validity of inferences about teacher quality. "

Tell your child's teachers how much you appreciate them!

Register your Account

Sign Up with us and Enjoy!

By registering with this blog you are also agreeing to receive email notifications for new posts but you can unsubscribe at anytime.

---

A password will be e-mailed to you.

It’s teacher appreciation week; and in a  letter in EdWeek , Arne Duncan attempts to convince teachers  that his policies actually reflect respect towards their profession.   The comments from teachers show they're not buying it.  Sabrina Shupe Stevens provides the perfect rejoinder:

Actions speak louder than words. Though you often have nice words to say about teachers, what you do is more important, and your actions thus far do not indicate that you respect, value, or support teachers and our profession as much as you claim.

Please read the rest of her post for all the evidence of how his policies have undermined the profession.

I would only add that if Duncan really respected teachers, he would honor their word that the best way to improve their effectiveness is to reduce class size, which is their response in numerous surveys, instead of supporting “selective increases” in class size, as he recently proclaimed in a speech before the American Enterprise Institute.

Indeed, rather than giving teachers the esteem they deserve, Duncan, Bill Gates and others are pushing for their performance and their job security to be to be judged primarily on the basis of unreliable  reductionist measures like value-added test scores.

See the excellent critique of value-added models, written by John Ewing, former executive director of the American Mathematical Society  and now president of Math for America, who points out that “making policy decisions on the basis of value added models has the potential to do even more harm than browbeating teachers” and  calls a recent Brookings report “fatuous”:

Why must we use value-added even with its imperfections? Aside from making the unsupported claim (in the very last sentence) that “it predicts more about what students will learn…than any other source of information”, the only apparent reason for its superiority is that value-added is based on data. Here is mathematical intimidation in its purest form—in this case, in the hands of economists, sociologists, and education policy experts.

And if we drive away the best teachers by using a flawed process, are we really putting our students first?

Whether naïfs or experts, mathematicians need to confront people who misuse their subject to intimidate others into accepting conclusions simply because they are based on some mathematics. Unlike many policy makers, mathematicians are not bamboozled by the theory behind VAM, and they need to speak out forcefully. Mathematical models have limitations. They do not by themselves convey authority for their conclusions. They are tools, not magic. And using the mathematics to intimidate— to preempt debate about the goals of education and measures of success—is harmful not only to education but to mathematics itself.

Especially this week but every week, parents should let their children’s teachers know how much they value their hard work, their caring, and their sacrifice, especially at a time that they have been treated so harshly by the oligarchy that has been engaged in a non-stop campaign to demean them, and then adds insult to injury by trying to convince them that they are really elevating the profession.

As anyone who has ever volunteered in a classroom knows full well, teaching is  one of the hardest jobs in the world, and they deserve better from our government and from the think thanks, the venture philanthropists, the privateers, and the hedge-fund managers who devalue their contributions every day.