Protections against institutional cheating in NYC schools

Spurred by reports of widespread cheating on standardized tests in Atlanta; Washington, DC; Los Angeles; Pennsylvania; and elsewhere, New York State Education Commissioner John King announced last week the creation of a “high level working group” to address "the integrity of our testing system.” The Bloomberg administration responded that this was "a knee-jerk reaction to cheating scandals in other states."

Unfortunately, the announcement was marred by the state’s failure to reveal who was appointed to this task force. As the NY Post noted,

“Oddly, the announcement came two weeks after the formation of the group, and department officials couldn't say who or how many people were on it other than Executive Deputy Commissioner Valerie Grey."

Indeed, there is a crying need for more systematic protections against cheating, which were eliminated when Bloomberg and Klein took office, as pointed out in our book, NYC Schools Under Bloomberg and Klein. Here is an excerpt from the chapter on "Institutional Cheating" by Sol Stern and Andy Wolf, showing how multiple procedures were in place previously – even before there was such huge emphasis placed on test scores:

Before test documents were destroyed, the Board [of Education] routinely conducted several levels of analyses to detect cheating. Robert Tobias, former director of testing and assessment for the BOE, provided us with the following summary of the board’s actions to screen for possible cheating:

“One was an erasure analysis that identified classes and schools with a high incidence of answers that were erased and changed from wrong to right. A second was a gains analysis that identified schools where students showed extremely high increases in test scores over the previous year. The third was an item analysis that detected unusual scoring patterns, such as large numbers of students who answered difficult questions correctly but easy questions incorrectly. In addition to these forensic analyses, we collected information on allegations of cheating from District Assessment Liaisons and other informants.

“When this information raised credible suspicion, we placed the respective test answer documents in secure storage, referred the matter to the Office of Special Investigations, and did not destroy the test documents until the investigation was completed. In other instances, we were directed to send the test documents to the State Education Department or the Special Investigator for the NYC Public Schools to facilitate investigations of cheating allegations referred directly to them. These procedures were in place when I retired from the public schools in Nov. 2001.”

It was also the practice of the old Board of Education to dispatch district administrators to each school on test days to oversee procedures. They would check on whether the tests were stored in a secure place in unopened cartons, observe the opening of the cartons and removal of the shrink wrap on the exams, and monitor the distribution and collection of the test materials. Finally they would oversee the delivery of the completed test papers to the district office.

All this was eliminated when Bloomberg and Klein took office. In a 2009 audit, the NYC Comptroller’s office concluded that the city Department of Education had “engaged in sloppy and unprofessional practices that encourage cheating and data manipulation.”

As to this new, rather mysterious state task force, one can hope that it will propose meaningful reforms, though there are many reasons to be doubtful. The NY State Education Department (NYSED) has in the past been known as the murky graveyard of whistleblower complaints, as ineffective as the endlessly inconclusive investigations of the NYC Department of Education, which tend to trail on for years, like a replay of “Waiting for Godot,” with little or no results.

Indeed, one might argue that having NYSED rely on an internal working group to guard against cheating is a little like Rupert Murdoch putting Joel Klein in charge of News Corp’s internal investigation into the phone hacking scandal: neither can be fully trusted to pursue this issue aggressively since they are fatally compromised by their institutional desire to look good; in this case, to show that schools are improving when there may be no actual gains.

Note the way in which NYSED and the Regents have just acceded to DOE’s demands to deregulate virtual (online) learning, with no attempt to provide any sort of oversight or quality control, and no requirement that students even attend classes to graduate.

Instead, the new regulations are a blank check, enabling DOE to continue and expand its fraudulent use of online credit recovery – deregulated by NYSED in 2009 --except that now, high school students won’t have to fail courses initially to gain enough credits to graduate through this substandard educational delivery system.

Jackie Bennett has analyzed how credit accumulation has soared in in recent years, just as it became one of the central components of DOE’s high-stakes accountability system, determining which schools will remain open and which will be closed:

In this city, the number of credits awarded to students in high schools truly is high stakes. It counts as nearly one third of each high school’s Progress Report grade, and the Progress Report counts for just about everything, including the removal of principals and the closing of schools. Since the Progress Reports were introduced in 2006-2007, the percent of students earning 10 or more credits each year has leapt a (truly) incredible 16 percentage points citywide. For schools with the highest concentration of high need students (the schools most likely to be threatened with closure) the jump is 18 points.

At the same time as credit accumulation and graduation rates have risen, the number of NYC high school graduates needing triple remediation (in reading, writing and math) at CUNY community colleges has doubled.

On this blog, we recently featured a post by a teacher revealing how credit recovery was used -- or misused -- in her school, after DOE coached her principal on how to implement it. See also here, and here, for even more cases of fraudulent online credit recovery -- which  flourish with the open encouragement of the educrats at Tweed.

Yet rather than attempt to put the brakes on this growing scam, last month the Regents and the NY State Education Department agreed to eliminate all controls on online learning, including any attendance requirements, allowing schools to give credits to students whether they come to school or not. There are no longer any class size limits as long as the course of study is undertaken under the general “supervision” of a teacher, who can “oversee” any number of students, according to the new regulations.

In the end, however, even if the state was really committed to preventing cheating, nothing is likely to work as long as our high-stakes accountability system remains in force. Campbell’s Law (which Steve Koss wrote about in our book, and as far back as 2007 on our blog) predicts that the more high-stakes testing is used for decision-making, the more cheating and gaming of the system is inevitable. As Bob Tobias pointed out in a recent interview,

“the current emphasis on high-stakes accountability …encourages some people to do the wrong thing. So as you kick the stakes up, people are going to focus on the metrics that will be used to determine their fate. They’ll be looking for ways to elevate those metrics, and some people will try to take a short route.”

Yong Zhao, eminent professor at the University of Oregon, has explored the numerous ways in which the growing overemphasis on standardized exams is undermining our public schools. He used the proliferation of school cheating scandals as the jumping-off point for a brilliant five-part series on his blog, concluding that the nation’s public schools must “ditch testing.” In an interview with Education Week, he suggested that as an alternative, the country should move to a portfolio-based assessment system:

“You can’t fix this by changing internal security,” Mr. Zhao said. “If the stakes are so high that the teachers don’t even believe the measurement itself, they’re going to try to cheat.”

Two important education laws requiring more transparency from DOE passed today!

Today, along with the budget, the City Council approved two new important bills requiring more transparent reporting from the DOE.

According to the first bill, 354-A, the DOE will now be required to report each year by June 30 on the number of students discharged by individual schools and systemwide, as well as by discharge code, so we will know better what is happening to the thousands of students that continue to leave NYC high schools each year before graduation but are not counted as dropouts. 

Some students move out of state or transfer to parochial and private schools, but many transfer to GED programs or alternative schools where they do not have a chance to graduate with a real diploma, and currently we have no idea how many are in each category.  Moreover, the percent of 9^th graders who are discharged from high school has doubled under this administration, without any explanation. 

I am particularly proud of this bill because the work of Class Size Matters helped bring more prominence to this issue.  See the discharge rate report  that we released in 2009 by lead author Jennifer Jennings, this story in the NY Times which covered our findings, and our testimony in support of this bill in January.  An audit we asked for from the State Comptroller’s office, along with then-Public Advocate Betsy Gotbaum released in March found that 15-20% of the students reported as discharges by DOE should really have been reported as dropouts.

The other bill, 364-A,  just passed by the Council, will report on the fate of students at the closing schools; whether they are discharged, drop out, or are given credit recovery to graduate before the school closes its doors.  For more on this issue, see our blog posting about the hearings.  If students are behind credits at the closing schools – which many are – they have not been allowed by DOE to transfer to other degree-bearing regular high schools, and there have been sky-high discharge and dropout rates at many of these schools. Hopefully this bill will provide DOE more incentive to ensure that students are not left behind when their school phases out.

At the hearings in January, DOE officials made the absurd claim that these two bills violated FERPA, or federal privacy protections, and  threatened that they would not comply with the laws as written  even though they contain language that specifically gives them a pass if the number of students in any category is so small that individuals might be able to be identified. I hope they have changed their mind and will report this critical information responsibly and accurately.

Good news; the DOE says they will comply with these two laws, according to this report in GothamSchools:  Bills will hold DOE’s feet to fire on discharge, graduation rates

High school progress reports; more unreliable and unfair measures

Today the high school progress reports were released for 2009-2010; here is an excel spreadsheet.

Yet experts continue to have grave reservations as to their reliability, with DOE placing far too much emphasis on test scores and especially the "progress" component, which means one year's changes in test scores, credit accumulation, passing rates etc., which have found to be extremely erratic and statistically unreliable. For more on this see this study, and this previous posting.

We have also seen the extremely damaging effects of this high-stakes accountability scheme, causing principals to increasingly "game" the system, that is, encouraging their teachers to "scrub" the scores, and or raising the scores themselves to passing, and/or awarding credits to students who either didn't actually pass their courses or even take them. Two articles in the past few days have revealed this occurrence at high schools in Queens and Manhattan; these practices are disturbingly widespread, and often occur with DOE's knowledge.

Right now in the city's schools, under Bloomberg and Klein, there is a lawless atmosphere, with very little oversight. In this "wild west" environment, it is very easy to manipulate the data, especially since DOE officials do not appear to care how schools get results, as long as graduation rates and test scores improve

Finally, it is interesting to note that the twenty schools with the highest overall progress report scores had average class sizes of only 23.3 students per class, with only one of them averaging more than 25.7 students per class. Meanwhile, the twenty high schools with the lowest overall scores had an average class size of 24.9, with four schools averaging 29 students per class or more. (Click on charts to enlarge.)


As the controversial teacher data reports take class size into account as a limiting factor as to how much a teacher is expected to raise test scores; it would only be fair for the DOE to take this factor into account as well with the school progress reports, especially as many high schools are allowed to cap enrollment and thus class size at far lower levels than other high schools.

Gary Babad breaks his contract and is demoted to DOE's press office

In a violation of his exclusive contract with the NYC Public School Parent blog, Gary Babad has published a exclusive report on the Washington Post blog about the appointment of a new Superintendent of DC schools.

We dragged Gary out of obscurity and propelled him to fame and fortune, and this is how he repays us? (Okay, there was no pay involved, but still!)

Gary's photo is at right, at the Class Size Matters fundraiser in May, talking to famed attorney Norman Siegel and to me; an occasion in which he never even hinted at his future betrayal.

As punishment, we are sending Gary to the bowels of the Tweed building to head their PR department, otherwise known as the NYC Department of Education press office.

This department is short-handed because of much turnover in recent months; and it's obvious they need help with their spin control. (The new hires haven't even managed to subscribe to our NYC education news list yet, the first step in any genuine attempt to control the news.)

For example, last week, when a press spokesperson tried to explain the lack of action for five years in a DOE investigation into the misconduct of a principal, recently shown to have misused funds in an audit from the State Comptroller's office, the excuse was that "Our lawyers are very thorough."

More recently, a DOE press spokesperson fed the absurd line to an Ed Week reporter that the DOE "is poised to introduce online credit-recovery options for students this coming school year in 10 schools" and then, only when a teacher is present.

Meanwhile, we know from numerous news articles that credit recovery is pervasive throughout the city, and credits are being handed out like candy to students, through cut-and-paste online programs, even when they failed to attend their classes and had flunked their courses.

We also know through DOE's own job postings and RFPs that online "learning", as they call it, is being used in at least 42 schools this year, with a plan to spread the discredited practice more widely next year, through "multi-million dollar online learning technology development projects" to be utilized during and beyond the school day - from home, library, or anywhere students have access to online resources.""

And in today's Daily News, this is how the DOE press office explained their decision to grant tenure to a principal, two days after a girl had drowned on a field trip at his school, a trip which had occurred without adequate supervision and permission slips:

"In the heartbreaking days following Nicole's death our primary focus was not on the tenure status of Columbia Secondary School officials," said Education Department spokeswoman Natalie Ravitz..

As Steve Koss, contributor to this blog pointed out, "such a statement is ridiculous and belied by their own actions, since it certainly didn't take them long to focus on the employment status of the teacher (now fired) and the AP (now demoted from administration back to being "just" a teacher.)"

We're sorry, Gary, that we have to demote you to heading the press office at Tweed, but at least at DOE, you can be assured that poor performance and/or misconduct are not punished, but are rewarded with a higher salary and assurances of lifetime employment.

More on the bogus school bonuses

All three dailies are onto the absurdity of the DOE granting huge bonuses to staff at low-performing schools that received "C" and "D" on their progress reports – including, according to the Daily News, at least one school that is closing.

The bonus program will eat up almost $40 million in precious education funds this year. Despite all the budget cuts already enacted at the school level, and more to come, leading to layoffs, larger classes, the loss of after school and enrichment programs , the DOE insists that these indefensible bonuses will be continued (NY1).

"It's always cited as one of the most novel, exciting programs. And I think New Yorkers want results. We are paying for results," said Schools Chancellor Joel Klein.

Yes, results based on inflated test scores, credit recovery, and widespread tampering with student test scores.

From the Post: “Somebody has to be complicit in this robbery -- it cannot be that DOE can simply appropriate public money as it sees fit," said Paola de Kock, whose son graduated in June from Stuyvesant HS in Manhattan.

Unfortunately, they can and they will continue to waste our taxpayer money -- as far into the future as the eye can see. Unless the City Council stands up for our children and stops them. Don't hold your breath.

See also the article in the Times.

Bloomberg and Klein's personal credit recovery program

See the presentation given by Klein to business leaders yesterday. Some news accounts here: Times, GothamSchools, NY1, Post, Daily News.

As usual, the Chancellor tried to take credit where no credit is due -- for big increases in test scores that occurred in the winter and spring of 2003, despite the fact that his policies were not implemented until the following fall. Perhaps these retroactive claims are Klein and Bloomberg’s personal credit recovery program.

In fact, DOE officials and the mayor were quite subdued when there were big test score gains in 2003. For more on this, see our blog here.

As the Times pointed out when the state ELA scores were first reported on May 21, 2003:

“The city's positive results come at a time when Mayor Michael R. Bloomberg and his schools chancellor, Joel I. Klein, are trying to overhaul the public school system and impose a uniform reading and math curriculum at all but the highest performing schools.

…Experts said the sharp increase in test scores could prove problematic for Mr. Bloomberg and Mr. Klein, since it is too early for them to take credit and sets a benchmark for next year that may be hard to match….

The higher scores, particularly at schools that have showed sustained increases since 1999, also gave new ammunition to critics of Mr. Bloomberg's changes, who said that they might do more harm than good by disrupting existing programs….

This is how the state math gains were reported when they were released on October 22, 2003:

“Fourth graders across the state made stunning gains in their math scores last spring, with even sharper increases in New York City…In the city, news of the gains, which were particularly pronounced in the Bronx and in some of the poorest-performing districts, elicited cheers among teachers and principals. But not everyone greeted the news so enthusiastically:

The suggestion that city schools were on the upswing put Chancellor Joel I. Klein, who is overhauling them, in a tricky position. While the chancellor’s critics pounced upon the higher scores as evidence that the school system did not need such an overhaul, some of his allies acknowledged that he would now be under even more pressure to show gains next spring.

A more accurate and fair analysis would look at data only from 2003-2008; instead of claiming credit for gains that occurred before the Bloomberg/Klein policies were put into place.

2.. None of these gains is meaningful given the evidence of the NAEPs, which shows no significant progress in NYC since 2003 in any grade or subject except for 4th grade math. We know that the state tests have been inflated, and there’s been terrific amount of test prep in NYC and teaching to the test, unrelated to any real gains in learning.

3. See slide 13: as of 2004, still only 55% of NYC eighth graders who make a 700 --level 3 – graduate in four years. (not even counting the discharge rates which would bring their actual graduation rate even lower. ) This is a huge repudiation of the quality of our high schools. Even the fact that only 87% of 8th grade students who hit level four, the highest level, graduate on time is pretty awful.

4.No info is offered on the results of the science and social studies exams – which are low-stakes tests, like NAEPs, and thus more reliable. Unlike math and ELA scores, for which the DOE offers financial rewards to teachers, schools, and kids, with little or no protections against cheating, these are the exams where we could expect less distortion in results.

NYC continues to have some of the lowest scores in the state in those subjects, even in the largely middle class districts. 28 of 32 districts in NYC were in the bottom 10% in the state in Science; 26 of 32 districts were at the bottom 10% in the state in Social Studies.

I’m sure there’s more deception and spin in these slides; please take a look and leave a comment on the comment page!

And the winner is...

Thank you to all who nominated an entry to the contest "Tweed's Greatest Foul-Ups."

This was very difficult to judge because there were so many foul-ups, so many fiascos. It is hard to say which was the absolute worst: the school bus re-routing, the ARIS supercomputer, the credit recovery contribution to raising the graduation rate, the "end of social promotion," the report cards, the pre-K admissions mess, the middle school admissions mess, the gifted and talented admissions mess, the quality reviews, the mindless obsession with test scores, on and on.

All contributed to turning the New York City school system into a new and painful version of 52 Pick-Up. But this game is not funny. It is only funny when Gary Babad writes the press releases, even if they are fake.

Two nominations had to be excluded because while they were powerful, they were not the work of Tweed: one, Norm Scott's suggestion that turning the education system over to a politician was the original disastrous mistake; and two, the suggestion (by anonymous) that hiring a non-educator as chancellor was another disastrous error. The first, as Norm notes, was the decision of the Legislature; the second was the choice of the Mayor.

So, the fastest way to whittle down the list of finalists is to restrict them to those who signed their name to their choice. That makes for a very short list, which is indicative of the fear that people in this city have to openly criticize those in power. This in itself is indicative of the terrible change, the repression of open discussion, that the new regime has introduced into our civic life.

Faced with a very short list of people who were willing or able to sign their names, I award the grand prize to Diana Senechal, who selected the "workshop model" as her biggest blunder. I take it that the blunder was the effort to impose a single method of teaching, in the absence of any genuine curriculum. This blunder was itself indicative of the arrogance of power, the belief that these non-educators could tell every teacher in the system how to teach. From that same arrogance flowed all the other blunders and fiascos, all recognized by teachers and parents, but unacknowledged at 52 Chambers Street as errors.

-- Diane Ravitch