MDRC report on NYC small schools and gaming graduation rates

 

The latest version of the Gates-funded reports from MDRC has been released, showing graduation rates for NYC students who were assigned to the new small schools through the supposedly random HS admissions process were 6 to 9.5 percent higher compared to those who “lost” the lottery and were assigned to large schools instead.  Overall, it has some of the same weaknesses that I noted – along with others in the previous report:  

1-The comparison groups are not necessarily equally matched – it does not separate out free and reduced lunch students, who are expected to have very different outcomes.

2 – The comparison also does not separate out special education students who are severely disabled – those that are taught in a regular classroom vs. those assigned to self-contained classes.

Indeed, though it claims comparable figures in both sets of schools, the authors admit that for both special education students and English language learners, “the sample sizes for these subgroups” at the small schools were so limited they could not compare outcomes between both sets of students.

3- Not all small schools held lotteries, as not all of them were oversubscribed – only presumably the more popular schools which may have been those with the most successful outcomes.

Assuming that there are real advantages to attending a small school, there could be many reasons for this higher rates unmentioned in the report:

•  Peer effects:  given that a student has to attend an information session and apply to a small school, this screens out many of the most at-risk kids, including recent immigrants, those with unstable home lives and “over-the-counter” students – those who do not enter the high school application process and who, according to most accounts, are usually assigned to the large schools.  The effect of being in a class along with other more actively engaged students or those with more ambitious parents may in itself bring substantial benefits – apart from any quality inherent in the school itself.

• Though the report cites no difference in 10^th grade average class sizes (28), stats apparently drawn from the state report cards, this is highly unlikely, as many large high schools feature classes at the union contractual maximum of 34.  Generally small schools have been able to cap enrollment and thus class size at lower levels than the large schools.  In fact, the report says that the new schools were chosen by DOE according to whether they planned to have their teachers “responsible for a manageable number of students” with a “reduced teacher load” –which is unlikely to occur with class sizes at the contractual maximum.

•  During the Bloomberg administration, there has been tremendous pressure placed on New York City teachers to pass at least 80 percent of their students, and to boost their scores on Regents exams – with schools graded and teachers evaluated on the results. Until this year, in fact, teachers graded the Regents exit exams of students at their own schools, with staff at small schools marking the exams of their own students. 

See the terrific book by John Owens who worked at one of the small schools, entitled  Confessions of a Bad Teacher – making clear the immense pressure imposed on teachers to graduate as many students as quickly as possible. Given that the small schools have younger teachers and those less likely to have institutional memories of the way things used to be, the more likely it is that they will be responsive to these pressures.  As the report notes, principals appreciated how their teachers were more “adaptable.”

• Yet another issue that the report fails to address is how the rapid increase of small schools that are more space intensive and capped their enrollments at lower levels exacerbated conditions at the large schools– making them even more overcrowded with the highest need students, and significantly diminishing their educational opportunities.  Although the report notes that graduation rates have increased to a lesser extent at the large schools as well, many schools that had been relatively successful soon found themselves on the failing lists – including Lehman HS and quite a few others.

•  Finally, data is a funny thing and graduation rates can be calculated in all sorts of ways.  The state and the DOE both claim that the city’s graduation rates for the class of 2010 was 61 percent. The MDRC report estimates graduation rates for the same year in their study for students at the small schools at 74.6 percent and 65.1 percent for the matched comparison group – suggesting that those who applied to small schools were a comparatively higher achieving group, even among those “randomly” assigned to large schools.

Yet the most recent analysis from Education Week published last June showed NYC graduation rates for the class or 2010 at 54 percent. Among the 50 largest districts in the nation, only Albuquerque, Denver and Detroit had lower rates; with NYC tied with Milwaukee. Why the difference? 

EdWeek’s methodology is complex, but it basically calculates the graduating rate by comparing the number of students that enter high school each year to those who are promoted to the next grade, and then graduate four years later with a regular high school degree – figures that are hard to fudge.  In contrast, the state and the DOE exclude all students who are said to have transferred out of district or to private or parochial schools or who are “discharged” – including those who leave for GED programs.  

The oversight for student transfers and discharges is notoriously lax.  In a report I co-authored several years ago, we found a very high and rising discharge rate in NYC schools.  This report led to a 2011 audit from the NY State Comptroller’s office, which stated that more than half of all discharges lacked proper documentation; and that even after extensive searches by DOE, 10-15 percent of discharges should have been reported as dropouts. There has been no audit since. The MDRC study reports that data for fully 19 percent of students who entered both small schools and large couldn’t be found four years later.

The EdWeek list of graduation rates of the largest school districts in the country for the class of 2010 are here and below.  Official DOE figures show that the rate has only declined slightly since then.  Almost as disturbing is that NYC is the only district listed among the fifty that failed to provide enough information for EdWeek to conclude whether its graduation rate is higher or lower than would be predicted, given the demographic background of the students.  So much for transparency at Tweed!

District Grad Rates 2010 from Education Week by CSM