The NYC School Survey: Changes over time leading to possibly skewed results and how the survey could be improved

The following was written by the new research associate for Class Size Matters, Emily Carrazana. Take a look!

Since 2007, the NYC Department of Education has issued something called the “Learning Environment Survey,” administered to students, parents, and teachers to collect their views about each school's quality and the system as a whole. Every year between 2007 and 2014, when parents were presented with ten choices, class size came out as the top priority of 21% to 24% of parents when they were asked “Which of the following improvements would you most like your school to make (Choose ONE)?”  

In 2015, this question related to parents’ top priority was completely left out of the survey, possibly because then-Chancellor Farina seemed uninterested in listening to parents in general and especially on the need to lower class size. In fact, her peculiarly unique view was that class sizes in NYC schools were too small.   

  After significant pushback from Class Size Matters and others, this question was resurrected the following year, but with a few other "tweaks". While class size was still offered as one of the choices, options relating to more or less state test preparation were removed, and “a safer school environment” option was added. The phrasing on the choice of more enrichment programs was expanded from “Stronger enrichment programs” to “Stronger enrichment programs (e.g. afterschool programs, clubs, teams)”. 

This choice now became the only one that provided any additional descriptive information, including specific examples of what might qualify as an enrichment program. Before that point, the percent of parents choosing “enrichment programs” as their top priority had significantly fallen, from 19% in 2007 to 12% in 2014.  Unsurprisingly, in 2016 for the first time, the number of parents who chose enrichment exceeded those who chose class size –with enrichment the top choices of 23% to 27% of parents from that year onward, compared to class size at 20% to 21%.

(Take note of the sharp uptick in enrichment programs in year 2016, the same year that the wording changed)

This is not to downplay the importance that schools should place on improving their students’ access to extracurricular and afterschool programs, especially to parents, many of whom work a full day; but to highlight the way in which the question seems to have been altered that might have skewed the results.

For instance, it is reasonable to assume that had the option for smaller classes been elaborated on in a similar fashion, for instance - "Smaller class sizes (e.g. more small group instruction, one-on-one feedback from teachers,  resulting in a more cohesive class culture and fewer disruptions, all of which have been proven to result from class size reduction), more parents might have chosen this option as their highest priority. Admittedly, expanding each option into a long-winded description is not optimal or efficient for a survey design, and might have led to even more slanted results. Instead, all options should be presented in a manner that does not persuade respondents to distort their own priorities. 

Another change that may have resulted in smaller class sizes being knocked down from first place citywide was the rapid expansion of the preK program starting in the 2014-2015 school year, and the inclusion of the responses of thousands more parents whose children attended preK that year and the following one.  Moreover, only in 2015 did the DOE begin to include the responses in the survey results of parents at hundreds of private preK centers operated by community based organizations. 

PreK classes  are strictly limited by state law to class sizes of 18-20 students per class, depending on whether there are one or two teachers plus an aide; so it is not surprising that preK parents would not choose class size reduction as their top priority. The survey data also includes the responses from hundreds of thousands of parents of students with special needs who are in self-contained classes and/or attend District 75 programs, whose class sizes are limited to only 8 to 15 students.  

It is therefore understandable to see why these parents might not view lowering the size of their children’s classes as important as those whose children are crammed into classes of 25, 30 students or more; conditions which are quite common in NYC public schools.  To be more informative, it would be preferable to disaggregate and report separately the priorities of parents of students in District 75 programs and especially those enrolled in preK classes.

Information on how DOE officials analyze or use these survey results, if at all, is virtually untraceable. Any follow-up conducted by school administrators seem to be done on an entirely voluntary basis. While the DOE provides a few pre-populated worksheets and  PowerPoint decks with suggestions on how the overall results can be presented to and discussed by parents, nowhere in these documents can you find any mention that their priorities as reflected by their responses to this key question should be addressed. 

This is a disappointing omission, especially for an administration that claims to base their policies on a “Framework of Great Schools” centered on mutual respect and strong family ties,  and a Chancellor who often insists he wants to focus on “parent empowerment”.

If the Chancellor really wants this survey to be more useful, the actual structure of the question might also be changed. Parents are asked to choose from a list of nine options, all of importance, and select only one that they would most like to see improved in their child's school.  Instead of asking them to do the mental backflips needed to weigh the pros and cons of a safer school environment vs. more arts programs or hands-on learning, the question might allow them to assign corresponding priority levels to each area. For instance, 5 = Highest Priority, 4 = High Priority, 3 = Priority, 2 = Somewhat of a Priority, 1 = 1 Low Priority. This would be advantageous for administrators to inform decision-making at their schools.