Why the school progress reports and NYC education reporters deserve a big fat “F”

Gary Rubinstein, a math teacher at Stuyvesant, totally outclasses the numerous education reporters in this city in his analysis of the recent school grades.  

He shows via this graph to the right and discusses in his blog how there is little or no correlation in rank order for schools between last year or this year's progress reports. 

(Incidentally, in a subsequent posting, Rubinstein also points out that NYC charters are twice as likely to get "Fs" in progress than non-charters.)

Unfortunately the mainstream media continue to repeat without dispute Suransky’s claim that the progress reports were much more “stable” this year, even though 60% of schools changed grades.    

Not one reporter, to my knowledge anyway, has bothered to point out how experts have shown that 32-80% of the annual gains or losses in scores at the school level are essentially random – and yet 60% of the school grade is based upon these annual gains or losses. 

See the Daily News oped I wrote in 2007, in which I offer even more criticisms of the progress reports, including their inherent instability, “Why parents and teachers should reject the new grades”.  

 Researchers have found that 32 to 80% of the annual fluctuations in a typical school’s scores are random or due to one time factors alone, unrelated to the amount of learning taking place. Thus, given the formula used by the Department of Education, a school’s grade may be based more on chance than anything else.

Yet here is one typical headline from last week: School report cards stabilize after years of unpredictability. Here is the NY Times account:

“We have a really high level of stability this year, which is a good thing,” said Shael Polakow-Suransky, chief academic officer for the city’s Department of Education…. There is movement and that’s good because we are measuring one year of data and we expect schools will go up and down, but we don’t want to see movement caused by something that’s external to the kids,” Mr. Polakow-Suransky said, referring to changes in the state exams that caused incredible increases and then a drop-off in schools’ grades.

Of course, if one year’s movement up and down is primarily random, that by definition is “external.” 

Nor have any education reporters bothered to report that Jim Liebman, who designed the system, testified to the City Council when the grades were introduced that the DOE would improve the reliability of the system to incorporate three years worth of test score data instead– which both he and Suransky have refused to do.

Indeed, as recounted on p. 121 of Beth Fertig’s book, Why can’t U teach me 2 read,  Liebman is quoted as responding to Michael Markowitz’s observations that the grading system was designed to provide essentially random results this way:

“There’s a lot I actually agree with, he said in a concession to his opponent…He then proceeded to explain how the system would eventually include three years’ worth of data on every school so the risk of big fluctuations from one year to the next wouldn’t be such a problem.”

And yet no one, including Fertig, has mentioned this discrepancy and DOE’s lack of rationale for intentionally allowing an essentially single unreliable grade to determine a school’s future – with 10% of schools now guaranteed to be given a failing grade and thus liable to being closed, based largely on chance. 

Read my earlier 2010 column, Why the school grading system, and Joel Klein, still deserve a big "F".

More on the bogus school bonuses

All three dailies are onto the absurdity of the DOE granting huge bonuses to staff at low-performing schools that received "C" and "D" on their progress reports – including, according to the Daily News, at least one school that is closing.

The bonus program will eat up almost $40 million in precious education funds this year. Despite all the budget cuts already enacted at the school level, and more to come, leading to layoffs, larger classes, the loss of after school and enrichment programs , the DOE insists that these indefensible bonuses will be continued (NY1).

"It's always cited as one of the most novel, exciting programs. And I think New Yorkers want results. We are paying for results," said Schools Chancellor Joel Klein.

Yes, results based on inflated test scores, credit recovery, and widespread tampering with student test scores.

From the Post: “Somebody has to be complicit in this robbery -- it cannot be that DOE can simply appropriate public money as it sees fit," said Paola de Kock, whose son graduated in June from Stuyvesant HS in Manhattan.

Unfortunately, they can and they will continue to waste our taxpayer money -- as far into the future as the eye can see. Unless the City Council stands up for our children and stops them. Don't hold your breath.

See also the article in the Times.

Bloomberg administration blames parents for larger classes

See the article in today’s NY Times, Class Size in New York City Schools Rises, but the Impact is Debated, a follow up to the article on Wednesday, Class Size Makes Biggest Jump of Bloomberg Tenure.

Though it is one of those typical “on the one hand this, on the other hand that” pieces– citing research that is either outmoded or easily refuted -- it is important because it is the first in-depth article in our paper of record to have dealt with the issue of class size in at least five years.

Indeed, the Times has had a “black out” on class size through most of the Bloomberg administration – as the former education editor admitted in June of 2006 – though at that point, she promised “to explore the class size issue” soon after -- which has not occurred until now, almost three years later.

This omission has persisted, despite the fact that our public school students continue to suffer from the largest class sizes in the state, smaller classes have consistently been the top priority of NYC parents, and in subway and TV ads, the administration has claimed to be reducing class size while being repeatedly cited for misusing hundreds of millions of dollars of state aid meant for this purpose.

In today’s article, the administration once again tries to evade its own responsibility for failing to reduce class size, despite a state mandate passed in 2007. In the previous Times article, Garth Harries of DOE attempted to blame the economy– even though the state provided an additional $400 million this fall, with $150 million of that targeted for class size reduction. He also attempted to shift the blame onto principals, which Chris Cerf tries again in today’s article, without acknowledging that it is the DOE’s duty to see that these funds are spent appropriately.

But now, even more outrageously, they are trying to blame parents – with Harries actually arguing that large classes are the result of popular schools where parents insist on sending their kids.

As I pointed out to the reporter, the vast majority of children attend their neighborhood zoned elementary and middle schools– and DOE entirely controls the admissions for high schools, so blaming parents for the systemic problem of large classes is entirely unwarranted. Who will they blame next – our kids?

Indeed, at the same time that the administration goes around claiming that mayoral control means accountability, they are quick to shift the blame on everyone else when they fail to create more adequate and equitable learning conditions for our children.

The article also repeats the administration’s canard that there is a trade-off between teacher quality and class size, when the two factors are actually complimentary. Indeed, the main reason we have such a high teacher turnover rate here in NYC is that our teachers so often leave for a new profession or to work in suburban or private schools -- because their excessive class sizes do not provide them with a fair chance to succeed.

In a recent national poll, 97% of teachers responded that reducing class size would be an effective way to improve teacher quality – far above any other strategy, including raising salaries, instituting teacher performance pay, or providing more professional development. Indeed, the only way we will ever obtain a more experienced and effective teaching force here in NYC is by reducing class size.

But the most ridiculous part of the article is the “evidence” offered by the administration that smaller classes don’t matter, by referring to an unpublished (and probably unpublishable) internal DOE study that purported to show that the grades schools received on the “Progress reports” weren’t correlated with smaller classes. No mention is made of the fact that most experts have found that the grades schools receive are mostly random – with almost no correlation from one year to the next -- as an article by the same reporter in the Times pointed out last year.

In contrast, the Institute of Education Sciences, the research arm of the US Department of Education, has concluded that class size reduction is one of only four, evidence-based reforms proven to increase student achievement. (None of the policies that the Bloomberg/Klein administration has introduced are on the list, by the way.)

In fact, the DOE has devised another formula – a “value added” model to evaluate teacher effectiveness, in which class size is included as a “predictor”, the ONLY factor included in the model under the school system’s control. This is an admission that the larger the class, the less a teacher is expected to raise student achievement. All the other factors in the model pertain to characteristics of the students themselves, such as economic status, prior test scores, absences, etc.

See the model here – which includes average class size at both the classroom and school level, showing that both should be taken into account when assessing a teacher’s performance. The DOE also states that the model used “draws on 10 years of city-wide data (test scores, student, teacher, and school characteristics) to predict individual student gains.”

Check out the accompanying FAQ:

Is the DOE’s Value-Added model reliable and valid? A: A panel of technical experts has approved the DOE’s value-added methodology. The DOE’s model has met recognized standards for demonstrating validity and reliability. Teachers’ value-added scores from the model are positively correlated with both School Progress Report scores and principals’ perceptions of teachers’ effectiveness, as measured by a research study conducted during the pilot of this initiative.

Anyway, please send a letter to the Times at letters@nytimes.com with your name, address and phone number. Let them know what you think – and whether it’s fair to blame parents for the fact that NYC classes have remained the largest in the state, with no significant improvement under this administration.

Joel Klein gets mixed reviews down-under

The Australian media continue their refreshingly skeptical coverage of Joel Klein’s promotion of his educational policies during his visit down-under.

Expose bad schools, says US educator (Sydney Morning Herald)

The Education Minister, Julia Gillard, is pushing for states and territories to adopt a similar system of transparency in Australia. On Monday she softened the blow for disadvantaged schools by announcing $500 million in funding to help them to entice good teachers.

But the woman Mr Klein described today as "a bold and fearless leader" should be warned. The US model has come under attack for its narrow measure of what made a school good.

A 2006 study by the US Centre on Education Policy showed that so much emphasis was being put on reading and maths that it caused a decline in teaching of history, science and the arts.

A similar model in Australia could leave little incentive for schools to improve teaching in subjects other than the "basics" measured. Sport, music, art and foreign languages could suffer as schools sweat to meet indicators.

Big business dominates educational planning (Sydney Morning Herald)

Joel Klein is in Australia to "spruik" his business-friendly school reforms courtesy of the giant Swiss bank UBS, the recipient of a multibillion-dollar bail-out from Swiss taxpayers, and dubbed the "world's biggest subprime loser" by The Age.

The federal Education Minister, Julia Gillard, "welcomes the active involvement of UBS" in education reform. Since her recent US visit, she has been championing the "remarkable outcomes" she claims Klein has achieved in New York, where he is the chancellor of the city's education department.

Klein, who was previously chief executive of the international media company Bertelsmann (and who had an article on this page on Monday), believes schools should be run more like businesses, and is an enthusiastic promoter of "charter" schools, some of which are operated for profit. He told Fortune magazine, "We're converting the role of the principal into a CEO role."

On occasion, according to The Nation magazine, Klein has referred to children as cars in a shop, a collection of malfunctions to be adjusted. Teachers, he said, needed "to look under the hood" to figure out the origins of the pings. …In the US, as much as a quarter of the school year can be devoted to test preparation. The assessments are supposed to show where the students have gaps in their knowledge so lessons can be adjusted. For the first few years of any testing regime, as students get used to sitting standardised tests, as teachers learn how to teach to the tests and as schools narrow their curriculums, test scores tend to improve. So it is not surprising that New York students are getting better scores in the national standardised tests.

This enables Klein to claim great educational improvements even though a national study released earlier this year using 2004 data found that New York has one of the worst graduation rates in the US, 43rd out of 50 large cities. Gillard is preparing to adopt key elements of Klein's business approach for use in Australian schools on the basis of Klein's ability to improve student test results, without examination of what those test results really represent. Will she unquestioningly adopt the business mantra of "standards, assessment and accountability" in the face of opposition from education experts?

Education is not a business and corporations that have made such bad judgments with regard to their core business, like banks, shouldn't be poking their gnomic noses into our schools.

US educationist talks tough on schools The Age (Melbourne)

In his speech, Mr Klein said his controversial methods, including standardised testing, had transformed the culture of his city's schools from one of excuses to one of performance.

But Australians are divided over Mr Klein's approach. Vicki Froomes, one Melbourne educator who worked with the New York schools Mr Klein threatened to close, said teachers would "teach to the test".

Ratings scheme for schools fails the test for improving them The Age (Melbourne) EARLIER this week on ABC2, Virginia Trioli asked federal Education Minister Julia Gillard if she agreed with Rupert Murdoch, who, in his Boyer Lectures, called Australia's public education a disgrace. Murdoch had said: "The failure of these schools is more than a waste of human promise and a drain on our future workforce, it's a moral scandal." "I'd have to say I think Rupert Murdoch is making a lot of sense," Gillard said. …The only qualification that Murdoch has to judge our schools is that he owns about 70 per cent of capital city daily newspaper circulation. When billionaire media magnates speak, the rest of us listen. The same cannot be said for the other American citizen, New York schools chancellor Joel Klein, who Gillard has brought to Australia, "impressed" by his education reforms, especially school league tables, which had produced "remarkable outcomes". Rubbish. Internet comments on the test results show the improvement in school performance measurement comes from manipulating the tests by prepping students. Klein also makes claims about the results that cannot be supported by any fair analysis. Statisticians who have examined the results say they can be explained by random error. Klein, a corporate lawyer and political apparatchik, is here to spruik the virtues of Gillard's wacky plan to publish a rating system for schools. Critics point out that the system, based on experience in Britain and the US, "names and shames" poorly performing schools whose output is predictable based on socio-economic background and lack of funding. The schemes' great political virtue is that it allows governments without any real commitment to raising the standard of poorer schools to appear to be doing something. The NY Times blog also reports on Klein’s trip.

The absurd NYC school grade system

There is now abundant evidence to show that the grades that DOE awards schools – their so-called “progress reports” -- are meaningless and rely on chance, like throwing dice, because they rely predominantly on whether a school’s test scores last year were above or below the year before.

Sixty percent of the grade depends on so-called “progress”, which means annual gains or losses in test scores, 25% on the test scores themselves, and only 15% on survey results. As a result, this year, many schools went from failing grades to grades of A , and just as occurred last year, some highly regarded schools received “Fs”.

Last year, in a Daily News oped entitled “Why parents & teachers should reject the new school grades”, I pointed out that research shows that 30-80% of the annual gains or losses in test scores are random – and sure enough, this is what the grades turned out to be. Last week, Daniel Koretz, professor at Harvard and national expert on testing, wrote:

”My advice to New Yorkers is to … ignore the letter grades and to push for improvements to the evaluation system…It does not make sense for parents to choose schools, or for policymakers to praise or berate schools, for a rating that is so strongly influenced by error.”

And yet according to the DOE, eighteen schools that received a D or an F last year have new principals this year – showing how lousy decisions are being based upon these inherently unreliable measures.

As Ellen Foote points out, the principal at IS 89, a school that received a “D” last year despite being selected as the only NYC middle school to receive an award for its achievements from the federal government:

“Last year’s grades often reversed the state’s opinion of schools… She knew of one District 2 middle school that got a B last year but was on the state’s list of schools that need improvement. Under No Child Left Behind, many students in that school transferred to I.S. 89, a Blue Ribbon school. That meant the students transferred from a B school to a D school. This year, both schools received an A.

Indeed, as Eduwonkette has written, the proportion of NYC schools receiving an “F” that are in good standing according to federal or state government standards is larger than the proportion of schools receiving an “A”.

“How do you reconcile those discrepancies?” Foote said. “How do parents make sense of that? It just is all over the place — it’s such a disservice to schools and to parents…. It’s so simplistic at best and confusing and probably invalid at the worst, at a very high cost in terms of money, resources and morale.”

The smaller the school or number of students tested, the more likely annual gains or losses are invalid; and as this post in Gotham Schools reveals, the smaller the school, the more likely it received an extremely high or low grade this year.

In response to this sort of criticism, Jim Liebman, the head of the DOE accountability office, said last spring that this year’s progress score would be based on two years of data instead of one year– which would have improved its reliability, yet for some reason, he went back on his word.

Why? To my knowledge, the DOE has never explained.

Rather than making things better, Liebman made the grade even more unreliable by increasing the importance of the “progress” score to account for 60% of a school's grade, an even larger percentage -- up from 55% last year. Why? Again, this has not been explained.

Was this to take advantage of last year’s anomalously high increases in scores? Were Klein and Liebman themselves gaming the system to ensure that 80% of schools would receive “A”s or “B”s --so that they could claim great credit for their so-called but illusory improvements? Who’s to know?

The idea that 80% of NYC schools could be rated “A” or “B” is in itself absurd, given the fact that our school system has among the highest class sizes in the nation and among the lowest graduation rates. This is grade inflation of the highest order, apparently crafted so the administration can pat itself on the back.

I wish those reporters who uncritically reported on the $20 million teacher bonus program on Friday – which also rely on improving a school’s score in the unreliable progress category – would now dig a little deeper, to examine how these cash rewards were likely awarded randomly as well.

Charter School Report Cards: Apples, Oranges, and DNA

Yesterday’s glowing announcement of the “pilot test” release of School Progress Reports for 20 charter schools should hardly come as a surprise to anyone who has followed the DOE’s steady, five-year march toward privatization. “Charter Schools Outshine Others as They Receive Their First Report Cards,” hails Jennifer Medina’s report in the New York Times. Medina quotes the DOE’s Michael Duffy that, “As a group, they skew higher,” as he simultaneously took a backhanded slap at public schools by stating that accountability in charter schools is “baked into their DNA.”

The DOE’s pilot test Progress Reports rated 20 schools – eleven elementary or K-8 schools and nine middle schools. One school, Opportunity Charter, originally received an F, but the DOE has since retracted their Progress Report grade. They assigned letter grades to thirteen of the remaining schools: five A’s (middle schools all), five B’s (elementary or K-8 all), two C’s, and one F. Of the remaining six, their total scores as computed from the reported components (School Environment, Student Performance, Student Progress, and Closing the Achievement Gap (Additional Credit) suggest approximate letter grades of two more A’s, one B, one C, and two D’s. Overall, roughly 68% A’s and B’s, 16% C’s, and 16% D’s or F’s.

Leaving aside bias factors in the DOE’s demonstrated predisposition toward charter schools and the lack of class size data that might well correlate with and explain some of the skewing, a slightly deeper look at the DOE’s Progress Report data reveals several interesting facts.

-- School Environment comprises 15% of each school’s score, but for the charter schools, this measure consists entirely of Attendance. As a result, in seven of the 20 piloted schools, attendance accounted for over 20% of their cumulative scores, and in three of those schools, it accounted for well over 30%.

-- In seven of the nine middle schools, their peer indexes (the average of their students’ Grade 4 NYS ELA and Math exam scores) exceeded 3.0, and an eighth school came in at 2.97 (the lowest, Opportunity Charter, had a peer index of 2.54 and also happened to rate an F grade before it was retracted). Overall, these nine charter middle schools had an average peer index of over 3.13 (3.21 if we also drop Opportunity Charter out as the DOE did), suggesting that these schools succeed in part by siphoning away stronger students from other schools.

-- The four middle schools that rated letter grades of A had the following peer indexes: 3.45, 3.39, 3.36, and 3.34 (the fifth was 2.97). By comparison, the Clinton, Wagner, and Sun Yat-Sen middle schools in Manhattan had peer indexes of 3.53, 3.58, and 3.30, respectively, and the five middle school only (Grades 6 – 8) schools in District 4 had peer indexes of 2.82, 2.86, 2.87, 3.20 (Isaac Newton), and 3.65 (Manhattan East, on East 100th Street).

-- In what seems a near statistical miracle, the three KIPP schools each reported Attendance rates of 98.0%.

Given the obvious selectivity of the charter middle schools, the lack of intake information about charter elementary and K-8 schools, the absence of reporting on many charter schools, and the inaccessibility of charter school class size data, it seems disingenuous at best for the DOE to claim that charters “skew higher” from a sample of just 13 schools that the DOE itself chose for letter grade assignment.

All this is not to say that charter schools do not do good work in motivating and educating their students. Just once, however, it would be nice if the DOE would release enough data in format and content for independent voices to evaluate its public relations claims and provide a real basis for comparing apples to apples. Sadly, nothing in the past several years suggests that all those well-paid DOE executives recognize the differences between apples and oranges. Apparently, it’s all just fruit salad to them.

Update: here is an excel file with the charter school progress reports and their components.

Voices in opposition to the school grading system grow louder

Diane Ravitch has an oped in today's NY Sun about the new school grades:

Is the grading system accurate and reliable? Did the grading system identify the worst schools? Is the closure of the lowest-performing schools likely to improve public education? Could the Department have taken other actions that might have been more effective than closing schools?

The answers to all of these questions, she suggests, is no. Diane also provides an important critique of the whole notion that simply closing schools is the best way to make significant progress:

Nor is it enough to turn out the lights. Schools are not a franchise operation. They are deeply embedded community institutions. They should be improved with additional resources, smaller classes, and additional training for educators. The starting point in reforming schools is to have a valid evaluation system that correctly identifies the schools that need extra help. It may not be easy to transform the schools that are in trouble, but if we want a good public education system, there really is no alternative.

Indeed, this is an essential element of the school reform process for which Tweed no longer feels accountable -- their responsibility to provide the support and resources schools need to improve.

See the show on PBS about the NYC school grading controversy, including parents and principals at some of the schools that got low marks, and one that got high marks, talking about the meaning and impact of these grades. The show also includes an interview with the Chancellor, in which he attempts to explains the "F" that PS 35, the Staten Island neighborhood school received, despite having 98% of students at grade level in math, by comparing it unfavorably to Anderson School – a highly selective gifted and talented school.

The interviewer, Rafael Pi Roman points out that William Sanders, the father of value-added accountability systems, told him that the sort of one year’s test score gains that the NYC grades are based upon are not meaningful. Klein responds that nevertheless, the school grade is a positive motivational factor in getting schools to work harder on improving test scores.

You can also listen to audio clips from the City Council hearings on the school grades from December 10, now posted on You Tube:

Public Advocate Betsy Gotbaum , who says out that closing schools unilaterally, as the Chancellor has done, without first consulting Community Education Councils is potentially illegal.

City Council Education Chair Robert Jackson (Part 1, Part 2, and Part 3), who aggressively questions James Liebman on many issues, including whether the DOE reached out to parents sufficiently.

Council Member Lew Fidler of Brooklyn, who flunks the school grades for their lack of transparency. (Part 1 and Part 2.)

And Council Member John Liu , who is masterful in showing that these grades are derived primarily from the results of only two tests -- though Liebman keeps trying to argue that these are really "multiple assessments" given out over "multiple days." (Part 1 and Part 2.)

Finally, watch the Channel 2 news segment featuring the hearings and showing Liebman fleeing from parents, now also posted on YouTube.

UPDATE: see also this article in City Limits: PARENTS, COUNCIL STILL ANGRY ABOUT SCHOOL GRADES

James Liebman on the run

On Monday, at the City Council hearings on the school grades, James Liebman, the chief accountability czar and former law professor, faced fierce criticism from Council Members. No wonder; his testimony was evasive, full of misleading statements and outright errors.

Liebman went on to make many questionable statements, among them, that a school at which "hundreds of children on average lost 10 percent of a proficient level in a year almost certainly has a significant problem."

Instead, experts say that one year's gain or losses in test scores at the school level is 34-80% random, and unrelated to the amount of learning taking place.

Liebman also claimed that factors related to overcrowding and class size were taken into account when devising the grades, when they clearly weren't.

In his testimony and power point, he claimed that he had consulted with many groups and experts, including the United Federation of Teachers, the Council of Supervisors and Administrators (the Principal's Union), CPAC, Community Education Councils and the NY Performance Standards Consortium in devising these grades.

Ann Cook, the co-chair of the Consortium, later testified to the fact that this was untrue. Her group had asked for and gotten a meeting about the interim assessments, but the topic of the school grades never even came up.

Ernest Logan, President of the CSA also denied that he had ever been consulted, and laughed when Jackson asked him this question. (See this letter from Logan to the Chancellor, about the many flaws in the school grades.) The UFT VP, Aminda Gentile, said they had “conversations” with DOE about the school grades, but there was no consultation.

Betsy Gotbaum, the Public Advocate, also criticized the unreliability of the school grades, and said that the Chancellor's decision to close schools without consulting first with Community Education Councils is against the law. She cited the state law, (2590-h) , which says that the Chancellor has the authority to:

Establish, control and operate new schools or programs…or…discontinue any such schools and programs as he or she may determine; provided however, that the chancellor shall consult with the affected Community District Education Council before substantially expanding or reducing such an existing school or program within a community district. (The law is posted here.)

Yet, she added, this has not happened in this case. "And the truth is, I can't think of an example where it has happened."

When asked by the chair of the Education committee, Robert Jackson, Liebman admitted that CECs had not been consulted before the announcement to close schools. Instead, they had been consulted afterwards, "entirely consistent with the process that has applied for the last several years."

Did he believe that parents should be consulted? Liebman said that the process that was used "was sufficient and adequate and very comprehensive."

Jackson said this response was "totally unacceptable", and if this was the direction the chancellor is going, he is in "big trouble." Liebman also claimed that the method he used was very "transparent" with very "clear rules" and that the results of the Quinnipiac polls showed that parents understood the methods used. (!!)

Liebman kept returning to the results of this poll in his defense, though it turns out that only 143 public school parents were polled.

City Council Member Lou Fidler was concerned that stigmatizing schools with failing grades will likely accelerate the decline of these schools, rather than helping them improve. Melinda Katz said it best: In her 14 years as an elected official, she’s never seen an agency so sure they’re right, when all the parents she has spoken to believe they’re wrong.

John Liu was very effective, asking Liebman repeatedly if the 85% of each school's grade was not just based upon a single measure, the results of a test taken once a year. Liebman kept on evading the issue, saying these grades were not based on one measure but actually "many measures" from a "series of assessments" that take place over a "series of daysm" and that each assessment "cuts across many hundreds of different items, and many skill areas." Liu pointed out the fact that its still only one test!

Finally, Liebman blurted out, "Life is one test" and everyone booed. Liu concluded that not only was Liebman trying to obfuscate, but that that his entire testimony was an obfuscation.

At the end of Liebman's three-hour testimony, the Chair, Robert Jackson, politely requested that he step outside the hearing room to receive petitions from Time Out from Testing and Class Size Matters, signed by nearly 7,000 parents, calling for a halt to the school grades. (Thanks so much to those of you who signed.)

In preparation, we filed out in an orderly fashion, (see above photo from the NY Times) but rather than have to confront us directly, Liebman slipped out a side door, out the back exit of City Hall, and ran away from us like a thief in the night, as we tried to catch up. He then entered the private gates to Tweed, but refused to let us in.

Liebman’s flight from parents was captured on video on many of the nightly news shows. As Lisa Donlan was quoted as saying in the Daily News, all this is symbolic of DOE’s arrogant and dismissive attitude. "He wouldn't even stay to hear our questions ... after we sat for three hours and listened to his testimony."

Here is an excerpt from today’s Times story, “Defending School Report Cards, Over a Chorus of Boos”:

“Mr. Liebman, whose title is chief accountability officer of the Education Department, ducked out a side door, leaving parents to chase him out the back of City Hall to behind the Education Department’s headquarters at Tweed Courthouse.

There, several education officials ran in circles for several minutes to avoid Jane Hirschmann, the director of Time Out From Testing, an advocacy group, as well as parents and reporters.”

Later in a phone interview, Liebman claimed to Times reporter that “he had not deliberately avoided the parents.” This claim is about as trustworthy as the school grades themselves.

See also article in Daily News, Escape from NY parents, the CBS newsclip here and NY1 here.

The CBS story repeats the erroneous statement that Liebman has met with Time out from Testing “many times”; in fact, according to Jane Hirschmann, head of the group, he has refused to ever meet with them.

I also gave testimony posted here about how unfair, inaccurate and destructive these school grades are, and entered into the record the comments criticizing the school grades from many of you, including parents, teachers, and at least one retired principal, that were posted online at our petition.

Update: Erin Einhorn of the NY Daily News pointed out today in Only in NY schools can get an 'A' & 'F' that of the 26 SURR schools on the state failing list, nine got As or Bs.

"The city can do whatever they want to do, but at the end of the day, I think the public deserves better," said Merryl Tisch, the vice chancellor of the state Board of Regents and a longtime supporter of Mayor Bloomberg.

Update on School Report Cards and Testing

The DOE's school report cards, called "Progress Reports" continue to be the focus of sharp criticism.

Writing in the Gotham Gazette, David Bloomfield, of the Citywide Council on High Schools points out how the administration has not convinced us they have a credible system for rating schools. His article is titled Report Cards Flunk the Clarity Test. Here's an excerpt:

Klein and Bloomberg have arrived at a highly individual definition of a "good school," without any social consensus on that definition. No matter parents are confused. None of them would have mixed the ingredients in just that way were they to evaluate the school. So none of them should rely on the mayor or chancellor to determine where they send their children or how they behave toward poorly (or, for that matter, highly) graded teachers and administrators.

Also in the Gotham Gazette, Richard Kessler of the Center for Arts Education, explains how the Bloomberg administration's mania for testing harms arts education. Click here for his article.

Time out from Testing continues to gather signatures against the Progress Reports and the increased testing required to compile them. They are looking to complete their petition this week. Add your name here.

letter to Mayor from David Bloomfield about the school grades

Dear Mayor Bloomberg:

Sufficient questions have been raised regarding the grading of New York City public schools (except charters) that it is time to rescind all recently issued letter grades. The system's unreliable data; the subjective, crazy-quilt method of computation; and the politicization of scores make useful judgments about school quality impossible.

Further, the grades bear little or no relation to other evaluations of school quality, sowing confusion rather than clarity among students, parents, educators, public officials and the general public. Perhaps you want to see this as a noble experiment gone bad. Fine. Whatever the rationale, as today's New York Times editorial suggests, it is time to replace this system since it has so obviously failed any reasonable measure of public or professional confidence.

David C. Bloomfield
Program Head, Educational Leadership at Brooklyn College
NYC public school parent and member, Citywide Council on High Schools

The NY Times calls for ditching the school grades

Today, the editorial page of the NY Times comes out squarely against the new school grades.

The paper calls on the Mayor to “ditch the simplistic and counterproductive A through F rating system."

The editorial points out that "a single letter grade that does not convey the full weight of this approach and lends itself to tabloid headlines instead of a real look at a school’s problems.”

It also notes how the formula is statistically untenable, as it is based on one year’s test score gains, and that “a high-performing school was given an F and several low-performing schools — those actually on the state’s failing list — were given A’s and B’s.”

This editorial is especially notable as the Times very rarely comments on the radical changes going on our schools.

Today's City section also has an excellent oped by parents at a school that received a "D" --Central Park East I:

“Central Park East I and other “failing” well-regarded schools like it have become collateral damage in a clash of competing educational philosophies: waste time and energy training children to do well on a test and get the grade; or educate them to be creative in their problem-solving, but fail on the Department of Education’s report card. That’s just not an acceptable choice.

It’s not Central Park East I that deserves the D, it’s the educational specialists who can’t or won’t accept that testing young children is not the best way to judge how well an elementary school is doing.

Yes, we’re angry about the bureaucrats giving us a failing grade, but we’ll wear our scarlet letter — our D — with pride if it’s the price we have to pay to stand up and fight for a great education for our children.”

See also the many letters published in the NY Times yesterday, protesting the concept and the formulation of these grades.

Master file of school "peer index" scores here!

A list of schools sorted by their "peer index" scores is now available in an excel file -- along with their grades and rankings on quality reviews.

Each school's grade was determined in large part by comparing its test results, attendance and survey results to the 20 schools above and the 20 schools below. For high schools, the "peer index" score must be within 0.5 of theirs.

A low peer index for elementary schools reflects a low-need, higher demographic student body; for some reason, this is reversed for middle and high schools. Check out the list to see which other schools your child's school was compared to in order to determine its grade.

Here is DOE's explanation:

1) Elementary & K-8 - peer index is a value from 0-100. We use a composite demographic statistic based on % ELL, % SpEd, % Title I free lunch, and % Black/Hispanic. Higher values indicate student populations with higher need

2) Middle & High - peer index is a value from 1.00-4.50. For middle schools, we use the average 4th grade proficiency ratings in ELA and Math for all their students that have 4th grade test scores. For high schools, we use the average 8th grade proficiency ratings in ELA and Math for all their students that have 8th grade test scores. Lower values indicate student populations with higher need.

Have any observations? Leave them as a comment on the blog. Thanks to Jim Devor for the file.

Stop the new grading system! petitions and fact sheets available

Each school’s grade and “progress report” will be distributed to parents at the parent/teacher conferences next week.

Time Out From Testing has put together excellent petitions and fact sheets, calling for a halt to the school grading system, showing how it is likely to further degrade the learning conditions at our schools. It would be very helpful if parents set up tables at their schools next week, and collect signatures.

• Download the fact sheet
• Download the petition

I have also posted an easily downloadable copy of my Daily News oped to hand out, with more information about why the grading system is inherently unfair and destructive. Petitions can be returned at the address below and I will get them to Time Out from Testing. thanks!

Class Size Matters, 124 Waverly Pl. New York, NY 10011

See also the many letters to the NY Times today -- all protesting the many inadequacies of the formula and the grading system itself.

Eduwonkette on the "statistical malpractice" of this administration

Check out the telling critique of the new school grading system from Eduwonkette, an astute new blogger:

"Earlier this week, Mayor Michael Bloomberg flexed his muscles by threatening to close F schools as early as June. He quipped, "Is this a wake-up call for the people who work there? You betcha."

Through analyzing these data, I've concluded that the people in need of a wake-up call work not at F schools, but at the NYC Department of Education....There are five reasons the report cards might kindly be called statistical malpractice".

One statistical anomaly she points to: the grades received by the schools that run from 6-12th grades. Of the 33 schools, 22 have different grades at the middle and high school level -- many of them sharply different. Examples?

"Consider the Academy of Environmental Science - its high school got a C, but its middle school got an F. At Hostos Lincoln Academy of Science, the middle school got a D, but the high school got a B. At the Bronx School for Law, Government, and Justice, the middle school got an F, but the high school got a C."

Clearly the grades these schools received reveal nothing useful about the leadership or the overall quality of the school, as the administration would maintain.

Is this a wake-up call for the people who work there? You betcha.

How the school grading system is a fiasco

Check out my oped in today’s Daily News, calling the new school grading system a “fiasco” and pointing out the reasons why.

It ends this way:


The new grades are merely another simplistic, misleading attempt by this administration to look tough - and evade its duty to improve our schools.

(Click on the picture to the right to enlarge it.)

Not surprisingly, the NY Post ran an oped in support the new grades– written by a charter school operator. Interesting that DOE could apparently find no one involved with a traditional public school to back them up.

What’s especially ironic is that the Post also ran an article on p.2 revealing that charter schools have been exempted from receiving grades – supposedly because the DOE can’t require their parents and staff to fill out the surveys.

The Daily News ran an excellent article, entitled “How a good school fails.” It looked at PS 35 on Staten Island, in which 49% of the students live in poverty, yet 86% passed last year's state reading test, and 98% the state math test. Citiwide, this school was in the 91st percentile in reading, and 97th in math. But it got an “F”.

How can this be so? Because last year, only 35% of its students improved their scores over the year before in reading, and only 23% in math.

How absurd! As my oped points out, between 34-80% of the annual fluctuations in a school's test scores are random, or based on one-time events alone.

While many excellent schools like PS 35 received “F"s yesterday, a lot of struggling schools also received “A"s.

I went through the state's list of failing schools for nine districts: Districts 7, 8 and Districts 10-16. Of the 141 schools that received grades yesterday, 27 of them received As, 51 Bs, 44 Cs, 13 Ds, and 6 Fs.

So if a school was on the state failing list, it was far more likely to get an A than a D or F. The distribution of grades of schools on the failing list was remarkably similar to schools citywide: 19% As, 36% Bs, 31% Cs, 9% Ds, and 4% Fs. (According to the DOE press release, 23% of schools overall received As, 38% Bs, 25% Cs, 8% Ds, and 4% Fs.)

John Elfrank-Dana, a teacher at Bergtraum HS analyzed the grades in terms of the quality reviews that every school received last year – a more holistic approach:

Fully 80% of the schools that got F or D grades were rated as “proficient” or better in their quality reviews, while 37% of those that were rated unsatisfactory in their quality reviews got As and Bs on their report cards.

Here is a quote from the NY Times today from Emily Horowitz, a parent at Muscota school, which unfairly received an “F”:

“I find the methodology to be confusing, problematic and flawed” … Ms. Horowitz said that the grade “doesn’t mean anything” to her, and that parents were planning a rally this morning to “celebrate our school.”

Hurray for Emily and the parents at her school! See our previous posting of individuals who should be celebrated for standing up for reason during these difficult times,

Please nominate others for our Profiles in Courage series by leaving a comment here.

Update: see this NY1 story about the principal and parents at Muscota.