Saturday, May 25, 2019

Warning: stand-alone field tests ahead & how you can support opt-out bills in Legislature


Note: You can send letters to your state legislators in support of the Right to Opt Out of High-stake testing bills mentioned below, Senate bill S5394 and Assembly bill A7744, with the help of NYSAPE here.  The following is by Fred Smith, testing expert, focused on the upcoming stand-alone field tests. You can see which grade in your child's school is targeted for field tests and in which subject here.

As another test-burdened school year comes to an end, we still have extra exams to give massive numbers of children.  These tests are not mandated.  Parents don’t know about them.
They are known as stand-alone field tests (SAFT) and are being given so a commercial publisher can assemble next year’s ELA and math exams. 
The testing program and the need to find out how many Level 1s, 2s, 3s, and 4s there are have become inexorable.  Only the test results are vacuous.
It’s time to sound an alarm bell about SAFT again to alert New York City public school parents about them and explain why they should exercise the right to opt their children out.
The New York State Education Department, under Commissioner MaryEllen Elia, has identified 869 schools in New York City this year to give the field tests to 91,565 students.  Here’s how the statewide and citywide picture looks.

The scope of this effort is vast.  The field tests began on May 20, when schools assigned to give computer-based tests (CBTs) could choose a day to administer them.  All schools giving CBTs are self-selected. 
The paper and pencil versions of the tests (PBTs) are due to start this coming Tuesday, May 28,and may be given on any day through June 7. These assignments are made by NYSED in conjunction with the testing company, Questar Assessment, Inc.
Since most of tests that remain to be given are paper-based, let’s focus on them.  The following chart accounts for PBT schools and targeted students in New York City:

70,648 unwitting NYC students in 698 schools are the target of the upcoming PBTs.  They have been volunteered to take them by NYSED with the complicity of City Hall and Tweed which have not let parents in on the plan. 
A chart pinpointing the schools and number of students expected to take the PBTs is linked here to lets you in on the secret.  You can look up schools and the grades that are involved in your district, find out if your children are targeted and opt them out of the field tests if you choose.  The detailed chart was posted by NYC Opt Out with this admonition about the field tests:
Computer-based stand-alone field testing starts next week, with paper testing following a week later. (Exact dates vary by school.) Check the chart to whether any grades in your school have been slated for field tests. If they have, you can: (a) ask your principal to refuse giving them--there are principals who send the boxes of field tests back unopened year after year or (b) send in a note saying you refuse to let your child participate (and let other parents know to do the same).
Why? Stand-alone field tests are a further waste of our children's time. They are not very good at determining item difficulty, plus our children are not being compensated for what is essentially product research. There are already field test items embedded in the actual spring tests.
On May 16, the Daily News ran my opinion piece about the fundamental flaws in stand-alone field testing.  I indicated that neither the state nor City Hall had taken steps to ensure that parents are informed about the extra round of testing, which is arranged without their consent—and which is unethical and disrespectful. 
The Op-Ed refers to an SED expert conceding in 2009 that there are shortcomings in the stand-alone method.  It also cites a barely noticeable statement in the technical reports prepared by the test publisher.  The two discredit the use of SAFT in test development. 
The first points out that SED knew about the fundamental weaknesses of stand-alone field testing years before the current testing program began.  It contrasts the practice of embedding field test items among operational test items with stand-alone field testing instead, issued at a time when the annual test results were getting preposterously high.  
From Making state tests public may also make them easier by Maura Walz, Gotham Schools Sept. 24, 2009:
Field tests allow test makers to figure out how hard questions should be and set the scale used to judge students. Exams like the SAT include field-test questions folded into the actual exam, but students don’t know which questions won’t count toward their scores. The experimental questions are then used again on future tests to gauge their difficulty.
Unable to field test questions in this real-world setting, the state must rely on no-stakes tests given to a sample of students on a different date. [NYSED spokesman Tom] Dunn said that students who sit for the field tests are told that the exams are only experimental.
Top of Form Bottom of Form
But when students know they won’t get a grade for the field test, they might be less motivated to do their best, [Howard] Everson chair of the state’s Technical Advisory Group, an oversight committee that monitors state testing said.
As a result, field tests often suggest that questions are more difficult than they actually are. And because they’re used to set the scale by which the real tests are graded, the end result is an easier state exam, Everson said.
“This is not ideal,” [Harvard testing expert Daniel] Koretz wrote in an e-mail. “What we don’t know is how much of an impact this has had.”
The next one is from the New York State Testing Program 2017: English Language Arts and Mathematics Grades 3-8 Technical Report, Questar Assessment Inc. (p.7). It was posted by NYSED in April 2019 and acknowledges the preferred method for trying out items–embedding.
Embedded Field-Test Items
In 2010, NYSED announced its commitment to embed multiple-choice items for field testing within the Spring 2012 Grades 3–8 ELA and Mathematics Operational Tests. This commitment continued for the Spring 2017 administrations of the tests. Embedding field-test items allows for a better representation of student responses and provides more reliable field-test data on which to build future operational tests. In other words, since the specific locations of the embedded field-test items were not disclosed and they look the same as operational test items, students were unable to differentiate field-test items from operational test items. Therefore, field-test data derived from embedded items are free of the effects of differential student motivation that characterize stand-alone field-test designs. Embedding field-test items also reduced the number of stand-alone field-test forms during Spring 2017, although it did not eliminate the need for them.
This boilerplate appears in Technical Reports since testing was aligned with the Common Core Standards in 2012 and Pearson was the publisher. 
I emailed each member of the Board of Regents a copy of my Daily News op-ed on May 17, urging them to consider the problematic nature of stand-alone field testing and to take steps to suspend it before it was scheduled to begin on May 20.
It seemed to be the perfect time for the Regents to intervene—reinforced by the timing of two developments that address the need for parents to be notified in advance of all testing.
At the end of April, State Senators Robert Jackson and Jessica Ramos put forth a bill called the "right to opt out of high-stakes testing act" or Senate bill S5394. Their proposal was also introduced as Assembly bill A7744 by Assemblyman Harvey Epstein.  It has broad applicability and affirms two principles:  1- Parents must be informed about all upcoming tests; and 2) Parents may refuse to have their children participate (without fear of punishment or coercion) in testing.
And there was the fact that these principles were expressed by the Board’s esteemed Chancellor Betty Rosa in a statement she issued on April 5.
I felt it was imperative for the Board to be polled on the matter of directing NYSED to suspend the vast, un-mandated testing program until all parents had been given sufficient notification about the tests so that their right to opt out could be honored.  I appeal to the Regents to confer today and to act with dispatch.
There is certainly nothing sacrosanct about having to administer field tests at this late time in the school year. And we know that many, many schools in the rest of the state reject the field test assignment—often returning the test shipments unopened, making SAFT a farce.
An order from the Regents to NYSED to desist would allow ample time to study the merits of the stand-alone approach and to decide whether this method should be discontinued.  At the very least, giving these tests could be deferred until the fall.
I am not privy to what the Regents did.  But Commissioner Elia wrote a letter  in response to my oped that appeared in the Daily News a week later.
I’m not going to parse the irrelevancies and obfuscations in her response.  She fails to address the main issues—continued use of weak field testing methods to perpetuate a bad testing program and failure to notify parents about tests.  If the Regents don’t see through this, my conclusion is that they want to keep doing testing business as usual.  
Dear Readers, if any of you wish, please call or email your Regent member and ask where they stand.
---Fred Smith
Post Script: It is noteworthy and ironic that the percentage of students in NYC targeted for CBTs (23%) is far smaller than the percentage in the rest of the state (60%).  This is an indication of the gap that exists between us and other districts in technological readiness.  They are evidently more prepared to use computers in their classrooms as we lag behind.  This is significant because the original goal of NYSED was that all state testing would be done on computer by the spring of 2020. 

1 comment:

Anonymous said...

HS teachers are often forced to give these, too.