A well-connected informant speaks out about the MAP exams

Teachers are speaking out on social media and elsewhere about the inordinate amount of time spent testing kids this month, using either iReady or MAP, whose uselessness I previously wrote about here. A former DOE administrator emailed me to say that despite their claims, the DoE did not allow a single high school to use a different screener than MAP exam.  This person also described how the results of these inherently unreliable exams will be misused.  Though my informant said that students are not allowed to opt out, we have heard from some parents that their principals are indeed honoring such requests this, so I urge you to go to NYC Opt out’s website for a sample letter/form to send to your school.   The message I received follows:

When principals were commanded to "select" a screener at the end of July (when probably 75-90% were on vacation), there was a "superintendent approved alternative" option.  I know that more than forty high schools selected that option.  Probably many, many more.  None were approved …  High schools had only one option, btw, except for kids with IEPs and ELLs.  Everyone else gets the MAP….

We learned from a DoE employee that there have been weekly meetings with Academic Policy, which is focusing on “academic recovery and interventions” using the results of the screeners.  The MAP data is being jointly managed by NWEA (which publishes MAP) and the DoE, which has also built a data system for its own use - superintendents (and everyone above) can use the system to compare schools within and across districts.

Instructional interventions are being prepared for use by schools based on the MAP data, starting with interventions for students with IEPs but then expanding for use by all students with lower scores.

[First Deputy Chancellor] Donald Conyers informed all superintendents that parents are not allowed to opt their children out of the MAP - if they don’t want to have their child tested, they must keep them home on test day, and, presumably, the test make up day. While this may not affect high school so much, it obviously makes it nearly impossible for elementary school parents to opt their children out.  This is a severe policy - the State Education Department recognizes the parental right to opt children out of state testing.  Why is the City being so strident?

 Taken together we believe it is becoming clearer and clearer that these tests will have serious consequences for all schools, even if they do not have high stakes attached for students. (The privacy issues in the data-handling remain considerable for students, as well.) 

If the system is being built so that central admin (sups and above) can compare school performance, it won't be long until principals are called on the carpet (Edustat/comstat style)….And that rolls downhill to teachers in schools with low scores.

My own belief, shared by most of those in my progressive education circles, is that these tests have almost zero value, especially in high school.  When Jesse Hagopian led the MAP boycott in 2013, one of the reasons was that the margin of error exceeds the expected growth for high school students, rendering the test invalid.  I'm 99% sure that is still the case.

The uselessness of the interim assessments that the DOE purchased for $36 million

The NYC Department of Education is mandating that all schools administer standardized interim tests three times a year, also called benchmark or formative assessments.  Here is the excerpt from the DOE handbook, “Instructional Principles”, describing these assessments, which are euphemistically call “screeners”. 

Alternative standardized assessments can be selected by schools but must be approved by DOE central.

According to the UFT, the first round of testing for grades K-10 will occur starting Sept. 27 and will continue through Oct. 22.  Students in grades 3-10 will be administered computer-based tests during this period, and K-2 kids will be assessed via “individual 15-minute question-and-answer sessions.” Given that many kids in grades 1 and 2 are in classes of 28 or more, these sessions could take a teacher seven to eight hours of class size over this three -week period.

This new regimen of commercially-prepared tests according to the DOE will cost $36 million – more than twice the amount they are spending for their “class size pilot”. 

Some of us remember how in the first phase of Children's First under Chancellor Joel Klein, similar interim assessments were required at every school, along with literacy and math coaches and a unified curriculum, similar to the regime of reforms that DOE is now attempting once again.  Here is an excerpt from a DOE 2005 press release, heralding these measures as leading to gains in state test scores:

Our new Citywide core curriculum, coaches, professional development, interim assessments, and aggressive intervention programs for struggling students, coupled with the enormously hard work of our teachers and other school staff, all contributed to these tremendous gains.

These same test score gains, however, were not seen on  the NAEPs,  the more accurate and reliable national assessments, and when the state exams were re-calibrated in 2010 to eliminate the rampant test score inflation that had occurred over this period, the gains disappeared.

This is not surprising.  As I wrote recently  in the Gotham Gazette, there is little or no research showing standardized interim assessments help students learn, or reliably diagnose their learning problems.

Few if any independent peer-reviewed evaluations have offered evidence for their validity or demonstrated that they have any positive impact on learning. One of the few randomized studies showed that administering the MAP exams had “no significant effect” on achievement. Many teachers have critiqued iReady exams, and many students despise them. 

As the late Robert Slavin wrote, director of the Center for Research and Reform in Education at Johns Hopkins University:

Research finds that benchmark assessments do not make any difference in achievement. …In a rational world, these findings would put an end to benchmark assessments, at least as they are used now. The average outcomes are not just small, they are zero. They use up a lot of student time and district money… Interim assessments fall into the enormous category of educational interventions that are simple, compelling and wrong. Yes, teachers need to know what students are learning and what is needed to improve it, but they have available many more tools that are far more sensitive, useful, timely, and tied to actions teachers can take.

Why are these assessments so relatively useless?  Because by and large, they are disconnected from the actual curriculum and work being done in classrooms. UFT President Michael Mulgrew pointed out  in 2019,when then-Chancellor Carranza first proposed imposing these tests, “How do you use a standardized formative assessment when you don’t have any sort of standardized curriculum….You don’t even know what you’ll be measuring.”

Indeed, after the first round of Children’s First in NYC, interim assessments were quietly eliminated after a few years, along with all those literacy and math coaches.   

Scott Marion, the Executive Director of the Center for Assessment  explains,

Interim assessments do not have a de facto place in balanced assessment systems. In fact, my colleagues and I argued in our Tricky Balance paper and policy brief that interim assessments more likely create unbalanced systems….Widely-used commercial interim assessments, in particular, generally are not tied to any specific curriculum and are not necessarily coherent with instruction and other assessments in the system…commercial interim assessments have a limited role, at best, in balanced systems of assessment, and any role must be supported by positive evidence that outweighs negative consequences.

In Seattle, the MAP tests were dropped in 2013 in all high schools after a successful boycott of teachers and students, and made optional in other grades.  Teachers pointed out how they didn’t test students on the material they were learning in the classroom.   

And last spring, after many Chicago students were found to be taking  hours and even days  to complete the open-ended MAP exams, the Chicago Board of Education dropped its contract with North West Evaluation Association, the company that produces them.

There are also serious privacy concerns with these exams.  The MAP exam attempts to measures student engagement and self-regulation by a student’s pattern of clicks and responses.   NWEA claims that “Teachers can use this information to identify students who may benefit from SEL (social and emotional learning) interventions.”  Parents have expressed concerns about the company’s use of this data, and the fact that NWEA hasn’t signed the Student Privacy pledge. 

Recently, a Brooklyn parent wrote me: “One of my sons already took a round of MAP when his middle school began administering them ….  The data still follows him, the test was so "ridiculous and meaningless, unrelated to what he was learning" (his words) and the predictive analysis so deeply flawed (forecasting that this honor roll student would not graduate HS or go to college on time).

She added: “I will be opting both of my HS students out of MAP and will continue to advocate for what our schools and all students really need to flourish and be safe, especially this year.”

Other NYC parents, students and teachers should seriously consider doing the same.

Watch Democracy Now's coverage of the successful Seattle boycott of these exams nearly a decade ago.