More bloviating from NYS and NYC on test scores, all over again

Update: see the statement from NYSUT here:"the State Education Department has failed to adjust the troublesome proficiency benchmarks to take into account changes in the state tests. “The state’s measuring stick for determining ‘proficiency’ remains badly flawed, painting a misleading and harmful picture of our public education system. In simple terms, they do not add up. The benchmarks must be fixed before any more students or schools are incorrectly labeled.”

Additional doubts expressed about the reliability of these results from Aaron Pallas of Columbia University in the Daily News: 

 "This week, we are seeing 2018 test results; but the state Education Department has still not released the technical report documenting how the scores for the 2016 tests, administered in April of that year, were calculated. The tests have been used for both high- and low-stakes purposes, but we still don’t know much about them.

Tests can be useful indicators of how individual students, schools and districts are doing, and whether they are improving, holding steady or declining over time. But the imperfections of the state’s testing system urge caution in making too much of any single test administration. It’s best to place the test scores alongside other performance indicators, and not to treat them as more precise or accurate than they really are."

 

So once again, in an eternal Groundhog day, the Mayor trumpeted the new state test scores, claiming great progress: 

Mayor de Blasio and Chancellor Carranza Announce NYC Students Continue to Outperform NY State on English Exams and Continue to Close Gap on Math Exams

“We came into office to shake the foundation of a system that neglected too many students, parents, teachers and schools,” said Mayor Bill de Blasio. “We now have a school system that is steadily improving before our eyes. We’ve seen steady gains across our students’ State math and English exams, proving that equity and excellence go hand in hand. I salute our students on their progress.”

And yet as clear as clear as day, on nearly every NYSED slide the following statement is made: 

"Due to the new two-session test design and performance standards, the 2018 Grades 3-8 ELA and Math results cannot be compared with prior results."

No comparisons can be made, because the exams were fundamentally changed, shortened to two days instead of three.  Also computerized testing was used in much of the state, the opt out rates remained high at 18%, and the entire system of scaling was altered.  Now the maximum score is about 651-661 in ELA and 644-656 in math instead of previously at the 402-428 in ELA and 387-401 in math. 

Why else can't we trust any claims of progress?  Because the trend lines on the more reliable national assessments called the NAEPs for the last four years have not matched the trend lines on the state tests.  While scores and proficiency levels on the state exams keep on going up, for both the state and the city, they are stagnant or falling on the NAEPs -- known as the "audit test" because its questions, scoring,  testing time, and scaling do not change much from year to year.  

The NAEPs are only given every two years, in 4th and 8th grade reading and math, so we can only look at the comparisons for those data points. In the charts below, instead of the apparent gains the city and state made between 2013 and 2017 on the state test scores, results in both 4th grade reading and math actually dropped since the Mayor took office. (NYC scores are in bold vs NYS scores; and the NAEPs are in orange below the state test scores in green.)

Also see this from the City Hall press release: ‘"New York City students outperformed the State on State English exams for the first time ever in 2016 and have now done so for three years in a row.”

But see above.  According to the more reliable NAEPs, NYC has never matched NY State in either average test scores or proficiency levels, and not in 2015 and 2017 when this happened in the 4th grade state reading scores, nor 2013 when it happened in 4th grade math.  (If you don't trust my NAEP graphs, check out the ones on DOE's website here.)

Just as during the days of Bloomberg/Klein, City Hall and Tweed prefer to inhabit an illusory bubble produced by state test score inflation & enabled by an incompetent and/or dishonest State Education Department. 

Why do I blame the state as well?  Because in many of the very same slides in which NYSED cautioned that no year-to-year comparisons can be made, they also claim the results show improvements in test scores compared to the previous year.  

See  slide 50 for example, in which the two sentences are included one after another.  

After saying, "Cannot compare 2018 scores with prior years because of switch from 3 to 2 -day testing", they add, "All race and ethnicity groups made progress, continuing to slowly close the achievement gap."    

Juxtaposing these two sentences is oxymoronic, and beggars belief.

Here are some additional questions that I would have asked the Commissioner and/or the Mayor if I'd had the chance:

1. How can NYSED or DOE or mayor claim progress has been made, if as clearly stated that as a result in the change in the tests, this year’s scores aren’t comparable to previous years?
2.  Why did they so radically change the scoring range, from a maximum of about 428 to about 651 this year?
3. Why does the state no longer report scale scores in its summaries, rather than proficiency levels which are notoriously easy to manipulate?
4. Where are the NYSED technical reports for 2016, 2017, and 2018 that could back up the reliability of the scoring and the scaling?
5. Why was the public release of the scores delayed though schools have had student level scores t for a month?
6. How were the state vs the city comparisons affected by the fact that opt out rates in the rest of the state averaged more than 18% while they were only about 4% here?
7. Finally, how can either the state or the city claim that these tests are reliable or valid, when neither the scoring nor the trends have been matched on the NAEPs, in which NYC scores have NEVER equaled the state in any category and results for the state & city have fallen in 4^th grade math and reading since 2013?

Though the Mayor apparently tempered his tone at this afternoon's press conference, according to Twitter he apparently claimed that he expects next year's scores to show significant gains because those 3rd graders will have had the benefit of Universal preK.
 

Sorry to say I won't trust the state test results next year either.  We will have take those scores with several handfuls of salt too -- and wait for the 2019 NAEP scores to judge their reliability.  

How Betsy DeVos much-criticized tweets echo earlier disinformation campaigns by Bill Gates and Arne Duncan

In recent days there has been much criticism of Betsy Devos for putting out this deceptive graph on Twitter that purported to show that school spending has no impact on student achievement as measured by the NAEPs:

The Nation’s Report Card shows that test scores continue to stagnate. This is not something we’re going to spend our way out of and not something we’re going to regulate our way out of. #RISE2018 pic.twitter.com/H6GHz8YpYg

— Betsy DeVos (@BetsyDeVosED) April 12, 2018

Here's a sample of the criticisms that she received:

Chart has some enormous flaws best refuted by research by @SchlFinance101 & @KiraboJackson Data in the hands of simpleminded education commentator is worse than ignorance itself because it will mislead others. But will fight fire with fire in my next post.

— Ed Law Blog (@DerekWBlack) April 13, 2018

Here's the response from economist Kirabo Jackson, showing his contrary analysis that indeed, money matters in improving student achievement:

Here is a similar graph without the nonsensical scaling.https://t.co/1CgoRZxlT1 pic.twitter.com/MZw6NfAP7p

— C. Kirabo Jackson (@KiraboJackson) April 12, 2018

Yet the sort of deceptive chart employed by DeVos has been disseminated for years by corporate reformers.  This includes corporate reform Sugar Daddy Bill Gates, who provided this chart while asserting to the Governor's Association in 2011 that  "Over the last four decades, the per-student cost of running our K-12 schools has more than doubled, while our student achievement has remained flat, and other countries have raced ahead."

 

In his speech Gates implicitly endorsed the successful effort subsequently undertaken by many Governors to defund their public schools - which our education system has still not recovered from.  

The chorus of boos that recently met Betsy Devos' tweet on school spending is similar to how another one of her tweets was received last month:

Does this look familiar? Students lined up in rows. A teacher in front of a blackboard. Sit down; don’t talk; eyes up front. Wait for the bell. Walk to the next class. Everything about our lives has moved beyond the industrial era. But American education largely hasn’t. #SXSWEDU pic.twitter.com/kyy2r7bTud

— Betsy DeVos (@BetsyDeVosED) March 6, 2018

This tweet got 6.7 thousands comments on Twitter, most of them scathing.

Yet just a few weeks before, Laurene Powell Jobs had made essentially the same claim to the New York Times, when her Emerson Collective LLC sponsored a nationwide television program that centered around the assertion that our high schools haven't changed in 100 years:

“For the past 100 years America’s high schools have remained virtually unchanged, yet the world around us has transformed dramatically,” intones the familiar voice of Samuel L. Jackson in a video promoting the TV event."

This contrast in how misinformation is received is also reminiscent of Arne Duncan's distressing record as Education Secretary, when he proclaimed to the American Enterprise Institute in 2010 that that budget cuts to education should be enthusiastically accepted as the “new Normal”:

“My message is that this challenge can, and should be, embraced as an opportunity to make dramatic improvements. I believe enormous opportunities for improving the productivity of our education system lie ahead if we are smart, innovative, and courageous in rethinking the status quo....

In our blueprint for reauthorizing the Elementary and Secondary Education Act, we support shifting away from class-sized based reduction that is not evidence-based. It might be that districts would vary class sizes by the subject matter or the skill of the teacher, or that part-time staff could be leveraged to lower class size during critical reading blocks.

I anticipate that a number of districts may be asked next year to weigh targeted class size increases against the loss of music, arts, and after-school programming. Those tough choices are local decisions. But it important that districts maintain a diverse and rich curriculum--and that they preserve the opportunities that make school exciting, fun, and engage young people in coming to school every day.

What did he suggest as the best most “smart” and innovative way to drastically cut budgets? That states and districts should adjust by allowing “smartly targeted increases in class size.”

Then again in 2011, Duncan told journalist Dana Goldstein that “Class size has been a sacred cow and we need to take it on."  Indeed, Duncan got his wish.   Schools all over the country laid off thousands of teachers during the recession, and most haven't recovered from the onslaught to their budgets and class sizes.  According to the Center for Budget and Policy Priorities, while the number of students increased by 1.4 million since 2008, the number of public K-12 teachers and other school workers fell by 135,000.

In 2012, when Mitt Romney was running for President, he visited a Philadelphia school and proclaimed that class size doesn't matter.  His views were greeted with catcalls by many of the same Democrats who had kept their mouths shut when Arne Duncan said essentially the same thing.  Indeed, Obama ran a campaign ad, raking Romney over the coals for his erroneous views on class size, ignoring how this had become a standard line purveyed by Gates-funded DC think tanks and his own Education Secretary.

My point is simple: what's bad for the goose must also be bad for the gander.  Don't let educrats or so-called philanthropists get away with their false claims and damaging policies, no matter what party they belong to or how much funding they offer your organization. Because their rhetoric can become the "New Normal" and hurt kids and our schools for generations to come.

Joel Klein's failed record as NYC Chancellor

Whatever favorable reviews Joel Klein has gotten for his new book, "Lessons of Hope" have mostly been written by people who did not experience his failed policies as NYC Chancellor.  If you want to read illuminating accounts, I recommend Diane Ravitch, Helen Zelon in the Observer, or Gary Rubinstein's blog, all of whom offer important reflections on Klein's record. 

I could write about the tremendous waste and corruption of the many multi-million dollar contracts he handed off so recklessly, for example to Wireless Generation for ARIS and/or Future Technology Associates.  I could write about his obvious mismanagement style, re-organizing the governance structure continually, creating more chaos and confusion than the steady hand the school district needed and deserved.  I could also write about his evident contempt for parents, teachers, and the law itself, with one of his favorite phrases when challenged being, "So sue me." 

But in the below I focus on the evidence that Klein himself would recognize as damning if he were to be intellectually consistent in his emphasis on test scores as the best measure of accountability:  the fact that NYC made less progress than any other city as measured by results on the federal assessments known as the NAEPs, except for Cleveland, over the course of his administration.

leonie7851