NYS educators agree: Flawed, confusing and misleading ELA exams

This week, NY students in grades 3-8 are in the midst of taking lengthy ELA exams, which in NYC, will help determine whether they will be held back, and what schools they will attend in the future, as well as what grades their schools receive and how their teachers will be evaluated.  See this article by Juan Gonzalez, about how our kids this year are  spending 270 minutes on the ELA exam and 270 minutes on the Math exam — 90 minutes over each of six days.  No wonder a growing number of parents are choosing to opt their kids out of these exams. ( For more on this, see With Test Week Here, Parents Consider the Option of Opting Out – NYT/SchoolBook and Parents keeping kids out of state reading exams - NY Daily News.)

What follows are comments from teachers and principals throughout the state about how this year's ELA exams are flawed, and contain many ambiguous and misleading questions.  Unfortunately, parents will NEVER be allowed to see these tests as the state is determined to keep them secret from this day forward.  And yet, NYS taxpayers are paying $32 million to Pearson for these exams.  If you are a NYC parent or a teacher, and want to get active on this issue, please email changethestakes@gmail.com  Please also add your comments below if you have thoughts or observations about these exams.

3^rd grade test:

From a NYC teacher: A couple of crazy things I've noticed: one really misleading fact/opinion question on the 3rd grade test (question asks "which sentence from the story is an opinion?" and the correct answer choice has the opinion embedded within a piece of dialogue. There's another that asks "what is the best way to remember what is in this ad?" that is highly subjective (different people have different strategies for recalling information, and each of the choices has some validity). 

NYC principal: The listening selection for grade 3 has MANY questions (multiple choice, short response, extended response) that follow this incredibly thin selection and aren't necessarily answered in the selection.

4^th grade test:

 From a literacy specialist [in a district outside NYC]:  I proctored the fourth grade test today. I thought that the test was terrible and not a true measure, in my opinion, of reading comprehension. First, some of the early passages in the test were very long (more than two pages) and meandering, making it difficult for 8/9 year-old readers to clearly discern the principal problem among several - or the problem the test-maker thought was the principal problem. These long passages put an undue burden on young reader's stamina during the early part of the test. Even though I am an adult who reads a lot (I am currently finishing my doctoral dissertation),  I found getting through the long passages and questions mentally tiring. This was in part due to the fact that the questions were convoluted and designed to "catch" students in test traps.

In addition, some of the test's print features were inconsistent (i.e., same exact phrases were bolded in some question and not others). The word choice both in the question stem and in the answer choices was meant to obscure meaning, choosing at times arcane vocabulary to refer to text information in the correct choices.  I have been a teacher for 19 years and a literacy specialist for 13, and I can say with some degree of confidence that this test was unfair and not a good instrument to measure students’ ability to read proficiently and use complex text to think critically and learn about the world. I feel sad for my wonderful and hard working students who sat for 90 minutes running through an unfair reading rat maze for political antics and for the benefit of corporate profiteers. I am afraid for the profession I love and for the future of public education.

From a principal, outside NYC:  This morning I had a fourth grader who told me that yesterday’s test was “hard.”  She then went on to explain that the stories were fine and the questions were easy, but that the answers didn’t match the questions.  Sometimes all the answers seemed right, other times all the answers seemed wrong, and sometimes the answers were just confusing.

5^th grade test:

NYC principal: As angry as I was before, seeing the tests today (which we are not allowed to quote in any way) has sent me over the edge!  I haven't even read all of them yet but the fifth grade test is unbelievable.  There were easy reading selections and lots of trick questions--more than I have ever seen before--that are absolutely no indication of any kind of 5th grade level reading comprehension.  My APs and I can't even figure out what answer they are looking for in some questions!  I think we absolutely need to fight that these tests be made public.  People will be shocked to see them.  

NYC teacher [at another school]: I completely agree with that principal.   Passages were dense, though reasonable.  What was irritating was how many questions were trick questions, and don't really test comprehension, they test your ability to answer tricky questions.   There were definitely questions in which my kids were just making silly mistakes all on their own.  But there were also plenty of questions in which the wording was meant to lead you astray, or there were 3 perfectly viable answers for which you had to use really developed reasoning to distinguish which was best, and honestly, I don't think a 9 year-old should be told they aren't worthy of passing fourth grade just because their reasoning hasn't reached an adult's level of analysis, or because they took a different perspective on a question than a test monger.

6^th-8th grade tests:

NYS middle school principal [outside NYC]: As I reviewed the exams for the sixth through eighth grade yesterday, I was appalled. I felt that sixth grade was the most difficult of the three exams, followed by eighth and with the fairest exam being the seventh grade. There were so many questions that contained answer choices that the ELA teachers said they could not decide which answer would be 'best' (By the way - weren't they getting rid of using that in the question stem?). I felt terrible for my children, especially for my English Language Learners and my special education students. They were extremely frustrated by the ambiguity of the answer choices and the questions that required them to synthesize several different pieces of information to come up with one answer that was mysteriously lurking among the four choices.

I had one student in an ESL class, who I was told was bright and could do well (whatever that means since the cut off scores are manipulated each year), tell me he was finished at 8:30 AM - the test started at 7:50 AM. As we strongly encouraged that he go back into the test to check his answers - his eyes began to well with tears. He was frustrated and gives up easily to not deal with the frustration. My heart broke. I can't imagine his willingness to now sit for another two days with each day bringing him more and more frustration. That's like me sitting for an IB language assessment. I'm motivated to learn the language, but I know I'm not proficient, I know I'm going to fail and I have to sit for it any way. Why should I try?

NYC test scores; small and unreliable gains

Yesterday, the state finally released school test scores; for NYC schools they are posted here.   Individual student test scores will only be made available August 17 – through the ARIS system, for which you will need your child’s OSIS number. 

Although the city showed gains of a few percentage points, the results were nothing to write home about: only 43.9 percent of city students in grades 3-8 met the standards in reading and 57.3 percent in math.

Though the  Mayor predictably claimed the city's gains of 1.5 percentage points in reading and 3.3 points in math showed great progress, actually the results are very mixed.  Only 35% of 8^th graders were proficient in reading.  Moreover, I believe the results overall are still highly unreliable.  Why? 

• The high stakes attached to test scores  in city schools will tend to lead to gains, because of excessive test prep, narrowing of the curriculum, and even cheating, rather than real learning.  (For more on this see our blog.)

• The same testing company is still writing them and the same “experts” are in charge at the NY State Education Department as in previous years, when there was tremendous test score inflation (the state intends to switch vendors next year.)

• Even if the exams were perfectly constructed and scaled, the city's gains are so small as to be likely statistically meaningless.

How else do we know the tests are still flawed?  Only 3.5% of students statewide received 4’s (or advanced) on the ELA exam, and only 2.7% in NYC. This is clearly a test which cannot distinguish performance at the upper levels.

Howard Everson, consultant to SED, claimed otherwise to GothamSchools, saying “that the gains under the new standards were small, they can be viewed as statistically significant because of the sheer number of students tested. He also said he trusted the state’s ability to track score trends even as the tests’ length, composition, and proficiency standards change.”    

Yet Everson told the New York Times in 2009 that the state tests were "about as good as we can build them," right before the test score inflation bubble burst and after it was clear to most objective observers quite the opposite. 

In addition, this year, some city schools, according to the NY Post, saw suspicious gains of up to 25 percentage points in both subjects.   Yet the  Mayor in his press conference said the precautions to prevent and check for cheating that were in place before he took office were too expensive to implement. He also said there was “no evidence of widespread cheating” (actually, in a Freudian slip, he said “no evidence of widespread teaching” !) 

Unfortunately, NYSED is no longer releasing the test questions, which will prevent anyone from discerning whether they were poorly or ambiguously worded as has occurred quite frequently in past exams; the state claims that this change “helps to ensure that preparation for the tests goes much deeper than simply reviewing past exams.”    But other states and reputable testing companies like the College Board still release the questions on their exams; why can’t NY State do the same?  The lack of transparency can only further diminish public confidence in the results.

There's a short clip of my views in last night's Fox-TV news story here.