Ten reasons to distrust the new accountability system

Welcome to a new school year in which all schools will be given grades of A through F -- 85% of which will be based on test scores. The principal's job and even the continued existence of the school will depend upon avoiding a failing grade. And yet there are obvious flaws in this system:

1- One of the primary metrics used to grade schools will be their "value-added" test scores; meaning that the growth or decline in scores for students enrolled at each school from one year to next. Yet the state tests are given in January (ELA) and March (math). Thus for ELA, there will be only half a year of learning for each grade in each school. Even in March you're not getting a full year's worth of learning.

2- It is well known that poor students lose more learning over the summer. Thus most experts believe that if you're going to do value-added measures to assess the learning taking place in the school, students should be tested at the beginning and the end of the year. The method the DOE is using could easily unfairly penalize schools with large numbers of poor children. DOE says they are controlling for the demographic background of students in elementary schools, but not in middle or high schools.

3- There is tremendous volatility and randomness in year to year test scores at the school level - which means that if the school's grade is based on one year's performance alone, the statistical error is large enough to make the results unreliable. That's why so many schools that get bonuses based on their test scores in states throughout the country rarely repeat.

4- The peer schools that schools will be compared with in order to determine grades are based on student test scores and demographics alone, and do not take account widely unequal school inputs like class size, overcrowding etc. which despite the arguments of DOE, are largely out of the control of individual schools. In fact, since Tweed continues to control enrollment, schools and principals are being judged based upon very different conditions and ability to succeed, leading to even more instability and turnover at the large overcrowded schools that continue to educate our neediest students.

5- The whole purpose of the accountability system as it is currently designed is basically punitive - even if the school grades were reliable, the administration doesn’t intend to use results either to help low-performing schools improve, for example, by giving them more resources or help them learn how to emulate more effective schools; but simply as a tool to decide which principals should be fired, and which schools should be closed. This will lead to more turnover and churning at our low-performing schools.

6- Many experts, such as Bob Tobias, former head of testing at DOE, say that because of the large standard error, or natural variability mentioned above, the interim assessments to be given five to six times a year cannot reliably measure short term gains in learning—which means much of the data churned out by ARIS will be garbage and all the expense and time taken up by the tests and the “data inquiry” teams in each school will be wasted.

7-Similarly, many testing experts say that the state tests are not designed (or validated) to do the grade by grade comparisons that the administration intends to use them for. As revealed in today’s Daily News, the difficulty of the state tests varies from year to year considerably – making the validity of these comparisons even more doubtful.

8.- DOE does not intend to measure critical negative student outcomes, no less include them in the accountability system for schools – outcomes that have already become more common– including the numbers. of students discharged to GED programs, long term suspensions, and/or transfers to other schools or sorts of programs. All of these negative consequences have become more common in recent years, and will likely become even more rampant in the future as principals feel even more pressure to improve test scores or risk losing their jobs -- and no school will be held accountable for how many kids are forced out in the process.

9.- By basing so much of the school’s grade on test scores, the incentive on the part of principals and teachers will also increase to cheat.

10- By excluding all other measures of a school’s success, one will not only get a very limited picture of what schools should be attempting to achieve, but the system itself may actually suppress all these other, important aspects of learning and enrichment.

Here is a good discussion by Henry Levin (in pdf) of this issue, in another context:: "By measuring only one or two school objectives...the result is that it may be impossible to determine what is being sacrificed...surely some area of learning is shortchanged by the shift in resources. As an example, [there is] cogent evident that non-cognitive aspects of school which are not even measured in school accountability systems may be the dominant determinants of education's effects on earnings, rather than cognitive aspects measured by tests....A similar issue is evident in high stakes testing where teaching efforts, curriculum and test preparation shift to what is tested from what is not. “

For all these reasons and more, the new accountability initiative may work to impoverish all of our students and schools – while punishing those who need our help the most.