NYC DOE still putting out false discharge data and inflating the graduation rate

Throughout the Bloomberg years, when the administration would trumpet rising graduation rates, I noticed how the discharge numbers were very high and seemed to be increasing. Every student listed as a “discharge” rather than a “dropout” can inflate a school’s figures,  as he or she are no longer counted in the cohort  -- in either the denominator or the numerator for the purposes of calculating the graduation rate.

So in 2009, I co-authored a report with Jennifer Jennings, entitled High School Discharges Revisited: Trends in NYC’s Discharge Rates, 2000-2007.   Our analysis showed that discharge rates had increased over this period, especially among Black and Hispanic students, students with disabilities, and English language learners.  

 Between the classes of 2000 and 2007, the discharge rate for students with disabilities increased from 17 to 23 percent, including in the class of 2005 where it spiked at 39 percent.  The report provided evidence that a thousand students had been “moved” into the special education cohort that year, possibly in order so DOE could claim an increase in the overall graduation rate.  Finally, we pointed out how some of the students categorized as discharges according to the DOE codes, such as students who left school to attend GED programs or because of pregnancy, should have been listed as drop-outs instead, according to state and federal standards. 

Our report led the DOE to change its coding for some of the categories and the City Council to pass a law called Local Law 42, to require detailed and disaggregated discharge reporting each year.  The results of that reporting are here.

A related law, Local Law 43, was also passed required the reporting of discharge rates at closing high schools, shown here; in these schools the discharge and drop out rates increase sharply.  Here is my testimony in support of both these bills.

Also as a direct result of our report, Betsy Gotbaum, then the Public Advocate, asked the NY State Comptroller to audit DOE’s discharge rates.  When the results of that audit were finally released in 2011, they revealed that 14.8% of students who were labelled as discharged should have been identified as dropouts instead, and fully 20% of the special education students. Moreover, the auditors found that DOE had no evidence to show that more than half of their sample of discharged students weren't actually dropouts; taking DOE months to come up with documentation. 

Just a few weeks ago, on Sept. 9, 2014, the NY State Comptroller’s office released a little noted, follow-up report showing almost no improvement in this area. According to a DOE internal audit from 2012, as many as 14% of reported discharges still should have been reported as dropouts.  A subsequent audit from December 2013 continued to find unspecified errors in DOE’s discharge classifications. 

I have now FOILed DOE and the State Comptroller for these two audits; we will see how long it takes them to respond.  Yet it is disappointing that there has been so little progress in the accurate reporting of this data, whether out of sloppiness or to inflate NYC's graduation rate, especially given DOE’s claims of being a “data-driven” agency.

Two important education laws requiring more transparency from DOE passed today!

Today, along with the budget, the City Council approved two new important bills requiring more transparent reporting from the DOE.

According to the first bill, 354-A, the DOE will now be required to report each year by June 30 on the number of students discharged by individual schools and systemwide, as well as by discharge code, so we will know better what is happening to the thousands of students that continue to leave NYC high schools each year before graduation but are not counted as dropouts. 

Some students move out of state or transfer to parochial and private schools, but many transfer to GED programs or alternative schools where they do not have a chance to graduate with a real diploma, and currently we have no idea how many are in each category.  Moreover, the percent of 9^th graders who are discharged from high school has doubled under this administration, without any explanation. 

I am particularly proud of this bill because the work of Class Size Matters helped bring more prominence to this issue.  See the discharge rate report  that we released in 2009 by lead author Jennifer Jennings, this story in the NY Times which covered our findings, and our testimony in support of this bill in January.  An audit we asked for from the State Comptroller’s office, along with then-Public Advocate Betsy Gotbaum released in March found that 15-20% of the students reported as discharges by DOE should really have been reported as dropouts.

The other bill, 364-A,  just passed by the Council, will report on the fate of students at the closing schools; whether they are discharged, drop out, or are given credit recovery to graduate before the school closes its doors.  For more on this issue, see our blog posting about the hearings.  If students are behind credits at the closing schools – which many are – they have not been allowed by DOE to transfer to other degree-bearing regular high schools, and there have been sky-high discharge and dropout rates at many of these schools. Hopefully this bill will provide DOE more incentive to ensure that students are not left behind when their school phases out.

At the hearings in January, DOE officials made the absurd claim that these two bills violated FERPA, or federal privacy protections, and  threatened that they would not comply with the laws as written  even though they contain language that specifically gives them a pass if the number of students in any category is so small that individuals might be able to be identified. I hope they have changed their mind and will report this critical information responsibly and accurately.

Good news; the DOE says they will comply with these two laws, according to this report in GothamSchools:  Bills will hold DOE’s feet to fire on discharge, graduation rates

Statement on DiNapoli's discharge rate audit

We would like to thank the Comptroller Di Napoli and the Office of the State Comptroller for their hard work on today’s discharge rate audit, which was done in response to a request on April 30, 2009 from Betsy Gotbaum, then-NYC Public Advocate, following a report that we wrote with Jennifer Jennings on the continued high number of students reported as discharged from NYC public schools. (Our 2009 report is posted here)

Even after the OSC office allowed the DOE to provide “additional information and documentation” for randomly selected students who did not have sufficient evidence in their files, 15% of the general education students in the graduating class of 2008 who were reported as discharged, and 20% of those from the special education cohort did not have the required proof to show that they were not actually dropouts. Thus, all these students were improperly removed from the cohort, inflating DOE’s reported graduation rates.

The audit’s findings of an error rate of between 15-20% in reported discharge rates are cause for grave concern that the DOE is not providing sufficient oversight to ensure accurate graduation and discharge figures.

According to the OSC, this means that between 2000-3000 of the general education discharges and between 266 and 539 special education students were “probably incorrect and should instead have been classified as dropouts.”

That fully 20% of the special education sample was incorrectly identified as discharged would lower this groups abysmal graduation rate even further to between 8.9 and 9.3 percent, compared to 9.7 percent reported by DoE.

The OSC concludes: “When we statistically projected the results of our sample to the entire [general education] cohort, we found that the correct graduation rate for the cohort was probably between 62.9 and 63.6 percent, rather than the 65.5 percent reported by DoE. “

Yet as the DOE itself notes; the official overall graduation rate for the class of 2008, according to SED guidelines , was really 56.4%, not 65.5%, so it would have been better for the OSC to provide an estimate of how much lower the actual graduation rate would have likely been, with all students included.

The OSC statement that since the city’s graduation rate was likely less than 5 percentage points lower than originally claimed means that the reported graduation rate was “generally accurate” and that errors “would not negate the upward trends in graduation rates in recent years” is puzzling, since reported increases have only been about 2-3% per year, for general education students, and among special education students much less, so an error rate of 2-3%, as the OSC found, would be about the same as the reported increases.

In any event, a 15-20% error rate in discharges remains very high. This high rate of errors reflects DOE’s lack of proper oversight or any accountability mechanisms to verify that discharges are properly reported at the school level. In addition, as the audit pointed out, some of these errors are related to the DOE’s failure to align its discharge guidelines to those of the state, by counting students who have transferred to non-approved GED programs (outside DOE) as discharges instead of dropouts.

Some important questions remain unanswered:

There remains no explanation as to why the discharge rate of students in their first year of high school has doubled under this administration, as our report noted. There also is no explanation as to why so many of the students reported as discharged are the most at-risk students, including those who are male, ELL, Hispanic and/or African-American, if these figures are more or less accurate.

I urge the DOE to agree to clear up some of these mysteries, by regularly reporting discharges, disaggregated by age, code, special education status, and demographic background, as a recently introduced City Council bill would require, though the DOE has expressed its opposition to such reporting at Council hearings. If they have nothing to hide they will agree to report all this data on a regular basis.

The DOE should also release full graduation and discharge figures in their annual graduation reports, including data for special education students, which they have failed to do since we released our discharge report in April 2009, despite repeated requests.

The NYC Department of Education should report an annual graduation rate that includes all students, rather than continuing to report as its “official” rate just the general education cohort.

I remain concerned that students who have transferred to GED programs within the DOE system but do not receive GEDs should not be classified as discharges, because this artificially raises school graduation rates. Also that students who receive GEDs should not be reported as a regular graduates, because a GED is not a high school diploma, whether or not that conforms to state guidelines. Otherwise, with the pressure on schools to inflate their graduation rates, they will continue to have an incentive to push at-risk students into GED programs and the like.

Finally, the DOE should revise its excessively harsh and punitive accountability system, so as to minimize the incentive of schools to inflate their graduation rates either through increased discharges to GED programs and/or falsifying their reporting. Unless this occurs, schools will remain motivated to shade the truth, and “push out” or exclude our neediest students, because of fears that they will be closed down or have half their teachers fired if their graduation rates do not improve.

Here is one example, from the audit, of a student whom the DOE claimed was a legitimate discharge, because an attendance teacher said that he had confirmed with a neighbor that she had moved to the Dominican Republic, and that this “was later confirmed by the student directly.” (p.31) Yet as the OSC responded:

School officials discharged this student in January 2008 without proper documentation to support their discharge determination. School officials stated they made a home visit in January 2008 and were told by a neighbor that the family had moved to the Dominican Republic (DR). DoE officials provided a memo, dated May 1, 2010, in which the student confirmed having travelled to the DR on December 18, 2007. However, when we spoke to the student, she told us that she and her family had lived at the same address for many years and she had not been visited by any attendance teachers in January 2008. She did acknowledge that a DoE employee had visited her in May 2010 [presumably after the audit began]. She also told us she had traveled to the DR on December 18, 2007, to spend Christmas with her family. She said when she tried to attend school after her return to the USA, she was refused admittance because the school had already listed her as being discharged. Since school officials did not have the appropriate documentation required by SED, this student should have been classified as a dropout or should have remained on the school’s register and included in the calculation of the graduation rate for her cohort. (p. 47)

As long as schools continue to be punished for low graduation rates with the threat of closure, instead of provided with help to improve outcomes, they will continue to exclude students like this one. It is students like these who will suffer the most.

The "top" NYC high schools, with SAT scores, class sizes, and caveats

Check out the NY Post listing on the "top" 50 NYC high schools here: The top 10, 11 - 20, 21 - 40, 41 - 50. The data for all 400 plus high schools, including graduation rates, average SAT scores, etc. is available here.

The Post used the DOE progress reports, plus other relevant outcome data, to calculate this list. Like all such rankings, they have to be taken with a heavy grain of salt. In this case, the salt pile is immense, since so much of the DOE data are either unreliable or incomplete. For example, high school NYC graduation rates are often inflated, because they don't taken into account the numbers of discharged students, a huge loophole that many high schools use to boost their grad rates.

The Post says that the graduation rate in their listings reflects the percent of 9^th graders who end up graduating after four years plus a summer; but that is not true. Many high schools discharge significant numbers of students before they even reach the 12th grade, and many if most of these students end up dropouts, but are never counted as such.

Even in the case of one of top schools on the list (Bard), the reporter notes that “about 20 students in each class transfer out.” In the case of Bard, a highly selective school, they probably enroll in other regular high schools, but for other schools, discharged students often end up in alternative high schools, GED programs or sometimes nowhere at all. The DOE used to make the discharge data by each individual school available in their graduation reports, but no longer does, ever since Jennifer Jennings and I produced a report on the rising discharge figures under Bloomberg and Klein.

[Correction! Updated reports for the classes of 2008 and 2009 do contain discharge data by school, at least for general ed students; see Appendix B at the links above, which reveal egregiously high discharge rates at many schools, with twice the number of official "dropouts" in many cases. What the city no longer seems to report on are discharge rates for D 75 and self-contained students.]

The NY Post's listing does not include any data on the growing practice of credit recovery, which is another manner in which many schools are artificially inflating their grad rates, (see this article by the same Post reporter on the phenomenon.) The DOE refuses to release any data on credit recovery, so it is impossible to know just how widespread this practice is. The class size data are also are not fully reliable; and tend to underestimate the actual size of classes in many high schools, since inclusion (CTT) classes are commonly reported as two separate classes.

I also don't trust the college-going rates in the listings; and the SAT scores don't include information as to what percent of the class actually took the SATs. Finally, the ratings may reflect more than anything else the socio-economic background of the students rather than what the schools actually bring to the table.

Nevertheless, as parents have a right to see this information, I have now posted a spreadsheet with SAT average scores for every school, for 2008 and 2009, as well the schoolwide class size averages for 2009-10 school year, as calculated by the DOE (as opposed to class size averages in each school by grade and subject, that are available here.). Neither of these files are on the DOE website, as far as I know.

The ugly naked face of mayoral control

Tuesday night's marathon session of the Panel for Educational Policy at Brooklyn Tech that lasted till nearly 4 AM was one of the most inspiring and awful events I have ever witnessed.

Inspiring because there were thousands of people there to protest the closing of 19 schools, and hundreds spoke out, for more than eight hours: eloquently, angrily, passionately and intelligently, about why their schools should not be closed and why the administration's blind and reckless policies would hurt our most vulnerable children. These English language learners, special education students, poor and homeless, will likely be excluded from the new small schools and charter schools that will replace their schools, and will undoubtedly be discharged in huge numbers as these schools phase out, never to receive a fair chance at a high school diploma.

Parents, students and teachers cited facts and numbers, personal experience, trenchant analysis and damning evidence of the DOE's malignant neglect and botched statistics.

Though the testimony started at 6 PM and continued until 2:45 AM, it was never boring. Early on, there was even humor from Lisa Donlan and Jane Hirschmann, who put on an inspired puppet show -- excerpts of which are below.

It was inspiring because at long last, Joel Klein got the reception he deserved: booed, jeered, criticized, compared to the Bernie Madoff of educational policy, with his destructive Ponzi scheme of closing schools and shifting around high-needs students; a scheme that will soon collapse, when there is nowhere left for these children to go.

It was awful because nothing that anyone said made any difference in the final vote.

The PEP, which the legislature in their wisdom allowed to continue with its a supermajority of mayoral appointees, was intent on rubberstamping whatever flawed or ridiculous policies put forward by the administration.

Shamefully, there was not a word from the chair, David Chang, or any other mayoral appointee to any of the thousands of people who urged them to think twice. Only the independent members from Manhattan, Bronx, Queens and the Bronx voted no.

When Patrick Sullivan, Manhattan member, asked the mayoral "yes men" if they had anything to say to justify these closings, not one of them felt obligated to explain his or her vote.

This event should never have been allowed to occur in a city and a nation that calls itself a democracy; with all the power in the hands of one man to make the decisions for thousands of other people's children, but this is the ugly naked face of mayoral control.

See also City Panel Backs Closing of 19 Schools (NY Times), Public sentiment has turned against Mayor Bloomberg's dictatorial school reforms (Daily News); The School Closing Marathon (Gotham Gazette); School Vote Scene Report: Joel Klein Called "Racist," (Village Voice); City's reasoning for wanting to close Jamaica HS based on faulty statistics (YourNabe.com); Parents Battle for a Say in Educational Policy (Gabe Pressman, NBC); Panel Decides to Shutter 19 NYC Public Schools (NBC New York) NYC school officials vote to close 19 underperforming schools (7Online.com); Rage as 19 schools get the axe (New York Post); “Attack” on Brooklyn high schools (YourNabe.com.)

Why the administration should not close these schools!

Class Size Matters submitted comments on Friday on the proposed closings of 22 schools; check them out on our website here.

We point out how DOE’s Educational Impact Statements are profoundly flawed, with incomplete or inaccurate graduation rate data, and how DOE officials have refused to follow their own accountability standards in proposing these closings.

There is strong evidence that large numbers of students will drop out and/or be discharged in high numbers as these schools phase out, with no chance to graduate with a high school diploma, as has occurred in the past.

The NY Times just published a piece implying otherwise, about Columbus high school:

“… [DOE officials] say they make the closing process as painless as
possible. For the closing school, it is a gradual death, with current
students allowed to graduate if they do not fall behind, but no new classes
admitted. As space opens up, the new schools come to life, adding a grade each
year.”

"New schools come to life"? A "painless" process? A "gradual death"? "...current students allowed to graduate if they do not fall behind"?

Sounds very humane, like someone putting a dying animal out of its misery. Unfortunately, the reality is quite the reverse.

The fact is that most students at Columbus and the other high schools slated for closure are already "falling behind" in that they are not slated to graduate in four years. Many of them require special education services and/or are English language learners, and take up to five or six years to graduate. And as these schools phase out, it becomes more and more difficult for students to gain the necessary credits, as their schools no longer offer all the necessary courses.

Indeed, low four-year graduation rates are the prime reason the DOE cites in proposing to close these schools, and there is no plan provided to ensure that the majority of students currently in these buildings will ever have a chance to earn a high school diploma.

See this chart, from the discharge report I co-authored with Jennifer Jennings, showing how discharge rates skyrocket for students at NYC high schools - to up to 30% or more of the students in each graduating class for last two years before they phased out. These students are forced to leave for GED programs, or sometimes simply disappear off the rolls, never even counted as dropouts.

The schools targeted for closure have been flooded with the most vulnerable students in recent years, including disproportionate numbers who are homeless. Closing them will doom these students to failure.

Other problems with the impact statements include: the DOE has failed to acknowledge how these closings will likely lead to even worse overcrowding elsewhere, with several thousand high school seats lost the first year alone.

There is no mention of the fiscal impact these closings will have in an era of contracting budgets, with up to a thousand teachers put on Absent Teacher Reserve, and millions of dollars in start-up costs for the new small schools and charter schools.

Indeed, mass school closings are poor educational policy; and will likely lead to high numbers of dropouts and/or discharged students, more overcrowding, and higher class sizes at a time enrollment is increasing, and school budgets have been slashed to the bone. There is no evidence that the administration has made any efforts to improve these schools before closing them.

We are calling for a moratorium until the Independent Budget Office can prepare an analysis of the considerable fiscal impact of these proposals, their effect on class size and overcrowding, and what’s likely to happen to dropout and discharge rates at these schools as they phase out, as well as other issues insufficiently (or inaccurately) addressed in the Educational Impact Statements.

Finally, the DOE should consider developing a genuine turn-around strategy, implementing targeted improvements, including reducing class size and a host of other reforms proven to raise student achievement, boost graduation rates, and enhance the learning environment.

Please send in your own comments today to the members of the Panel for Educational Policy, and to the DOE. The PEP members’ contact information is on the right hand side of the blog. The DOE contact information is on the public notice page here; for high school closings, send them to HS.Proposals@schools.nyc.gov

All comments sent are supposed to be summarized, posted and presented to the PEP before the vote. And hope to see you tomorrow night at Brooklyn Tech High school; come at 5:30 PM to sign up to speak.

April 30 press conference on rising discharge rates

Here is video from our press conference at the Public Advocate's office, including an introduction by Betsy Gotbaum and a lucid power point given by Jennifer Jennings, the co-author of our report on the discharge rate.

Also check out Part 2, with the rest of Jennifer's presentation and some relevant comments from Kim Sweet, head of Advocates for Children, about how worrisome these figures are, particularly the extremely high discharge figures for full-time special education students at 24 percent, nowhere reported in the city's official graduation rate. Part 3 features Dianne Morales, Executive Director of the Door, who has seen increased numbers of students coming to her program after having been encouraged to leave their high schools over the past five years, and some recommendations that I offer about what should be done about this troubling phenomenon. Part 4 and Part 5 has us answering questions from reporters.

Here are some news stories about our findings: Number of Students Leaving School Early Continues to Increase (NY Times); Study looks at city discharge rates (Channel 7 news); Saying discharges are up, report demands grad rate audit (Gotham Schools.)

(Video thanks to David Bellel)

Discharge rates still rising; especially for students in their first year of high school!

There are literally thousands of NYC students who disappear off the rolls every year -- leaving high school without a diploma, but who never counted as dropouts in the DOE's official graduation rate or anywhere in its porous accountability system. There were over 20,000 students who entered our high schools in 2003, and should have graduated in 2007, but never did. These are the "desparacidos" -- the ones who disappear.

Check out the report released on Thursday -- written primarily by the brilliant Jennifer Jennings (a/k/a Eduwonkette) with minor contributions from myself -- showing that in NYC, the discharge rate has significantly increased between 2000-2007.

While the city reported an "official" graduation rate of 62% for the class of 2007, if all special education students were included this would drop to 58 percent. If GEDs were excluded, this would fall to 55 percent, and if discharged students were counted as dropouts, this would further fall even further -- to 44 percent. (click on the graph to the left to enlarge it.)

Most shocking is the fact that the rate and numbers of students discharged in their first year of high school literally doubled. This may be because these students moved out of the city or to parochial or private schools in larger numbers than ever before, yet analysis of census and enrollment data provide no evidence for a rising rate of migration or transfer to parochial schools.

More likely, these students are entering HS even more overage than before (due to multiple grade retentions), since no student can be legally discharged before the age of 17. Or perhaps many of these discharges are illegal.

What is especially tragic is little or nothing has been done to address the problem of discharged students since the problem of "pushouts" was first exposed by Advocates for Children in 2002. Perhaps this is because the higher the discharge rate, the higher the graduation rate by definition, since all these students are excluded from the denominator for the purposes of calculating the graduation rate.

In fact, in the report, we point out several features of the DOE's high-stakes accountability system which encourage schools to rid themselves of low-performing students as fast as possible through discharging them, instead of giving them the support, resources, and smaller classes they need to graduate.

This is a "black hole" of accounting which must be addressed. As a result of our report, and at the request of Public Advocate Betsy Gotbaum, the State Comptroller has agreed to audit NYC's graduation and discharge data. Hurray!

There's loads more interesting information in our report, including tantalizing evidence about possible data manipulation for the Class of 2005. This class had originally caused headlines when its graduation rate was first released in the Mayor's management report in February 2006, revealing a drop from the year before -- 53.4 percent compared to 54.3 percent for the previous class. According to the NY Times at the time:

The political touchiness over yesterday's numbers was evident in how the mayor's office chose to report them this year. Administration officials created a new category in the preliminary management report that had the effect of masking the decline in the four-year graduation rate. Although that rate continued to be reported separately, the new category factored in students who stayed on for an extra year of school, allowing the mayor's office to state, "More students graduated from high school in four years or are still enrolled in school for a fifth year."

Speaking to reporters yesterday, Mr. Bloomberg said it would take years before many of his changes, like grade retention policies that hold back elementary and middle school children largely on the basis of test scores, were reflected in improved graduation rates.

Then, in June, the DOE announced that the figure of 53.4 percent had been a mistake and that this class had really graduated at 58.2 percent. As the NY Times uncritically reported at the time, "because of a computer glitch, last year's citywide graduation rate was five points higher than previously reported — the highest on-time graduation rate in more than two decades.....Officials said the mayor was angry after learning of the mistake and intent on getting an accurate tally."

How did this more "accurate tally" happen? By looking at the data, and comparing DOE's original and "corrected" graduation reports , it appears that over 1000 students may have been recategorized as full-time special education and then discharged at the astonishing rate of 39%.

At the same time, the number of general education students who had entered four years before fell by over 2,000. All of these changes, unremarked at the time by any reporter, had the convenient effect of allowing the Chancellor to claim a sharp rise rather than a fall in the city's "official" graduation rate, (which includes only general education and part-time special education students), as well as proof of the efficacy of his reforms.
We also found that several of the categories that NYC calls "discharges" are more properly considered dropouts by the federal government -- including students who leave because they've turned 21, withdrawn because of pregnancy, were expelled, or had transferred to GED programs outside the DOE system. Check out our press release here.