Charter School Report Cards: Apples, Oranges, and DNA

Yesterday’s glowing announcement of the “pilot test” release of School Progress Reports for 20 charter schools should hardly come as a surprise to anyone who has followed the DOE’s steady, five-year march toward privatization. “Charter Schools Outshine Others as They Receive Their First Report Cards,” hails Jennifer Medina’s report in the New York Times. Medina quotes the DOE’s Michael Duffy that, “As a group, they skew higher,” as he simultaneously took a backhanded slap at public schools by stating that accountability in charter schools is “baked into their DNA.”

The DOE’s pilot test Progress Reports rated 20 schools – eleven elementary or K-8 schools and nine middle schools. One school, Opportunity Charter, originally received an F, but the DOE has since retracted their Progress Report grade. They assigned letter grades to thirteen of the remaining schools: five A’s (middle schools all), five B’s (elementary or K-8 all), two C’s, and one F. Of the remaining six, their total scores as computed from the reported components (School Environment, Student Performance, Student Progress, and Closing the Achievement Gap (Additional Credit) suggest approximate letter grades of two more A’s, one B, one C, and two D’s. Overall, roughly 68% A’s and B’s, 16% C’s, and 16% D’s or F’s.

Leaving aside bias factors in the DOE’s demonstrated predisposition toward charter schools and the lack of class size data that might well correlate with and explain some of the skewing, a slightly deeper look at the DOE’s Progress Report data reveals several interesting facts.

-- School Environment comprises 15% of each school’s score, but for the charter schools, this measure consists entirely of Attendance. As a result, in seven of the 20 piloted schools, attendance accounted for over 20% of their cumulative scores, and in three of those schools, it accounted for well over 30%.

-- In seven of the nine middle schools, their peer indexes (the average of their students’ Grade 4 NYS ELA and Math exam scores) exceeded 3.0, and an eighth school came in at 2.97 (the lowest, Opportunity Charter, had a peer index of 2.54 and also happened to rate an F grade before it was retracted). Overall, these nine charter middle schools had an average peer index of over 3.13 (3.21 if we also drop Opportunity Charter out as the DOE did), suggesting that these schools succeed in part by siphoning away stronger students from other schools.

-- The four middle schools that rated letter grades of A had the following peer indexes: 3.45, 3.39, 3.36, and 3.34 (the fifth was 2.97). By comparison, the Clinton, Wagner, and Sun Yat-Sen middle schools in Manhattan had peer indexes of 3.53, 3.58, and 3.30, respectively, and the five middle school only (Grades 6 – 8) schools in District 4 had peer indexes of 2.82, 2.86, 2.87, 3.20 (Isaac Newton), and 3.65 (Manhattan East, on East 100th Street).

-- In what seems a near statistical miracle, the three KIPP schools each reported Attendance rates of 98.0%.

Given the obvious selectivity of the charter middle schools, the lack of intake information about charter elementary and K-8 schools, the absence of reporting on many charter schools, and the inaccessibility of charter school class size data, it seems disingenuous at best for the DOE to claim that charters “skew higher” from a sample of just 13 schools that the DOE itself chose for letter grade assignment.

All this is not to say that charter schools do not do good work in motivating and educating their students. Just once, however, it would be nice if the DOE would release enough data in format and content for independent voices to evaluate its public relations claims and provide a real basis for comparing apples to apples. Sadly, nothing in the past several years suggests that all those well-paid DOE executives recognize the differences between apples and oranges. Apparently, it’s all just fruit salad to them.

Update: here is an excel file with the charter school progress reports and their components.