Update on growing evidence of state test score inflation

Since I posted my blog yesterday on the growing evidence of state test score inflation -- and the inherently invalid claims from NYSED and the media of actual increases in student achievement:

• Articles today in the  NY Daily News and NY Post about the drops in the raw scores on the NY exams needed for proficiency.

• NYSAPE just released its analysis of this issue which is somewhat clearer than mine here:  Link to Press Release

•  In a comment on my blog, Andrea Gabor points out that comparing the test results last year to this is like comparing rotten apples to rotten oranges.

• Diane Ravitch provides a succinct summary and concludes:  "These are serious charges. It is now the responsibility of Commissioner Elia, the Board of Regents, and the State Education Department to demonstrate the validity and integrity of the tests."

Meanwhile, Chalkbeat attributes the (highly dubious) gains at least in part to the expansion of charter schools and the Common Core – no doubt Bill Gates’ favorite explanations as well.  (Be sure to read the comment about Success charters below the piece.)

Then check out the difference between NYSED's  discussion in 2015 and 2016 of this process of equating raw scores to different proficiency levels:

http://www.p12.nysed.gov/irs/pressRelease/2015800/documents/EquatingExplained2015.pdf

Q3: Why did the raw scores change?

A:The raw scores that map to each of the scale score cut scores change almost on a yearly basis. See

the "Raw Score to Scale Score Conversion Charts" at http://www.p12.nysed.gov/irs/ela-math. These

charts are available back to 2006.

Q4: How did the raw scores change in 2015?

A:On the 2015 tests, year-to-year raw score change is for Level 3 were small and varied by grade.

Raw scores went down slightly on 5 tests (indicating slightly harder tests in 2015 compared to 2014

for Grade 6 ELA and Grades 4, 5, 6, and 7 Math) and went slightly up on 5 tests (indicating slightly

easier tests in 2015 compared to 2014 for Grades 3, 5, and 7 ELA and Grades 3 and 8 math). Raw

scores stayed the same on two tests (Grades 4 and 8 ELA)

In 2014 NYSED had posted a similar discussion of equating, including this info:


Q4: How did the raw scores change in 2014?
 
A: On the 2014 tests, year-to-year raw score changes for Level 3 were small and varied by grade. Raw scores went down slightly on 6 tests (indicating slightly harder tests in 2014 compared to 2013 for Grades 3, 4, and 7 ELA and Grades 3, 5, and 6 Math) and went slightly up on 4 tests (indicating slightly easier tests in 2014 compared to 2013 for Grades 5 and 6 ELA and Grades 4 and 7 math). Raw scores stayed the same on two tests (Grade 8 ELA and Grade 8 Math).

Yet look at what they posted this year:

http://www.p12.nysed.gov/irs/pressRelease/20160729/documents/EquatingExplained2016.pdf

Q3: Why did the raw scores change?

A:The raw scores that map to each of the scale score cut scores often change with each subsequent

test administration. See the "Raw Score to Scale Score Conversion Charts" at http://www.p12.nysed.gov/irs/ela-math

The number of raw points necessary to achieve each performance level change due to several factors. These factors include equating and changes in number of raw points available on each test. For instance, the total number raw scores for ELA and Mathematics went down this year in all grades due to the shortening of the tests (i.e., removal of one reading passage and items).

That was it.  Unlike 2014 and 2015, there was no Q4  or answer this year because unlike past years, they couldn’t say in five tests the raw scores went up and five down OR six vs four tests, etc.  

Instead they would have to admit that in 11 out of 12 exams, not only the total raw scores but the PERCENT of raw score points out of the total fell for proficiency.  And this despite the other changes, including the tests being shorter and untimed, which all things being equal would make it likely that the percent of right answers would go up not down.