Showing posts with label test scores. Show all posts
Showing posts with label test scores. Show all posts

Tuesday, November 12, 2019

What's really behind the city, state and national drop in NAEP scores

The results of the biennual national tests called NAEPs were released on October 30, showing stagnant or declining test scores in reading and math in nearly all states in the decade since 2009. 
“Over the past decade, there has been no progress in either mathematics or reading performance, and the lowest-performing students are doing worse,” said Peggy Carr, the associate commissioner of the National Center for Education Statistics, which administers the NAEP. “In fact, over the long term in reading, the lowest-performing students—those readers who struggle the most—have made no progress from the first NAEP administration almost 30 years ago.”
The poor results are most likely a consequence of several factors, including the damaging double whammy experienced by schools in 2009-2011 – when the great recession hit, which led to thousands of teacher jobs lost and class sizes increasing sharply, and the imposition of the Common Core standards.
Concerning the recession, see the chart below from the Economic Policy Institute, showing a current shortfall of more than 300,000 public education jobs starting in about 2010:

Many states and districts, including NYC, still have not recovered from the sharp increase in class size that occurred starting in 2008.   Just as class size reduction benefits students of color and from low income families the most, increases in class size hurt their opportunities to learn the most, helping to explain the widening achievement gap over this period.
In addition, the corporate-style policies that proponents claimed would help narrow the achievement gap, including the Common Core standards and state exams aligned with those standards, adopted in nearly all states starting in 2010, likely contributed to the decline in performance on the NAEPs as well.
The Common Core emphasizes informational text rather than literature, and  “close reading” strategies, with students assigned to analyze short passages, often excerpts from literature, in isolation from any larger context.
In essence, Common Core led to a curriculum designed for test prep, but devoid of engaging relevance and content for many students. To make things worse, the assigned texts are often two or three Lexile grade levels above the actual reading level of the students to whom the reading is assigned, in a misplaced intent to provide more “rigor.”
Close reading involves analyzing and re-analyzing individual passages, focusing on details and interpreting the author’s particular choice of words, structure, and intent, without any reference to anything in the student’s own experience or prior knowledge: “Students go deeper in the text, explore the author’s craft and word choices, analyze the text’s structure and implicit meaning” etc..  It is a process that is more suited to a graduate seminar in literary criticism than elementary or even high school English classrooms, and has been imposed upon classrooms throughout the United States in a misguided effort to sharpen their analytic “skills”.  It is hard to imagine anything more boring, and more likely to turn off a young reader.
Here are some recent tweets from teachers around the country, in discussing the Common Core in relation to the latest NAEP scores.
From a second grade teacher in Louisiana:
And a special ed teacher in New York:
Strangely enough, the NY Times story on the NAEPs mentioned neither the recession nor the Common Core in attempting to explain why there has been no  progress since 2009. In a Twitter exchange with one of the reporters, she said no one had mentioned Common Core to her in years.   

I don’t doubt that few if any of its original proponents now mention Common Core  – given its abysmal failure to improve results in our schools -- but that doesn’t mean that millions of students and teachers aren’t still wrestling with its flaws every day in classrooms throughout the nation, as evidenced by the above tweets.

In any case, the last quote in the Times article was from Jim Cowen, the executive director of the Collaborative for Student Success, an organization established to promote the Common Core standards, decrying how the state tests -- those explicitly aligned with the standards -- have become too easy and there was a need for "accountability" -- but not apparently for those who promoted the flawed standards themselves.

Moreover, the other experts quoted decried the emphasis on short passages rather than allowing students read longer books with richer meanings and larger contexts, without specifically mentioning the Common Core.

Peter Afflerbach, an expert on reading and testing at the University of Maryland, called the eighth-grade declines “troubling” and “precipitous,” especially for the lowest-achieving students saying that  "too many schools have assigned elementary students short passages instead of challenging them with longer, thematically rich texts and books.The new eighth-grade results show the students haven’t developed the reading comprehension to deal with text complexity."

Compounding the bad news was a just-released report from the ACT, showing that College Readiness levels in English, reading, math, and science have all decreased since 2015, with English and math seeing the largest decline.  
A recent study has provided further evidence for the negative impact of the Common Core. In those states whose original standards differed significantly from the Common Core, the adoption of the new standards had a significantly depressing impact on test scores which has grown over time, with the sharpest negative effect on fourth grade reading scores and especially on the achievement of students with disabilities, English Language learners, and Hispanic students.
Altogether, the falling NAEP scores, the ACT report, and this study represent a devastating indictment of the Gates/David Coleman/Arne Duncan reform agenda -- and yet despite all the evidence against them, and the fervent critiques from teachers, most states are sticking with the standards and the flawed pedagogy they impose.  As Susan DuFresne, a teacher in Washington state, proclaimed:
One more trend may have contributed to the decline in reading scores over the last few years. There has been a sharp increase in the use of digital reading programs across the country – with a survey from Common Sense Media revealing that 94 percent of English/language arts teachers say that they used them for core curriculum at least several times a month. This is despite a wealth of research that suggests that reading comprehension suffers when reading is done on screens. 

An Ed Week analysis of the just-released NAEP data found that in both grades 4 and 8, students who spent more time on digital devices in English class scored lower on these exams. Look at this astonishing graphic - showing that 65% of students who scored below basic on the NAEPs spent four hours or more of classroom time on screens per day.  Whether this association is due to correlation or causation, it is a highly disturbing trend:
As for NYC in particular, the much ballyhooed upward trend in state scores has now been proven to be illusory, as I argued last year, as was the Mayor's claim that the achievement of NYC students has  matched or surpassed average achievement in the rest of the state.  
  
According to the more reliably scaled NAEPs, in no subject or grade do the NYC scores come close to the average in the rest of the state – even though the state scores too have stagnated over the last decade.  Also confirmed was my prediction in 2016 that we have entered yet another era of state test score inflation.
And while in 2003, NYC students scored  above the large city average in all four NAEP exams, we have now slipped behind that level in three out of the four categories– and only equal it in one: fourth grade reading.  The same pattern exists with NY state’s NAEP scores, which were once ahead of curve nationally and have now fallen below it. 
What’s especially disconcerting, though, is how little seems to have been learned from the failures of the past.  A few days after the NAEPs were released, the NY State Education Department announced it was hiring Achieve.org to summarize the research and the public feedback on whether and how to revise the state’s high school graduation requirements, which rely on students passing five high-stakes exit exams.  Achieve.org has been one of leaders in the Gates-funded push for the Common Core,
In addition, the NYSED public engagement process, which will involve a Commission and multiple forums, will be funded by the Gates Foundation, which has spent more than $400 million since 2009 on financing and goading states to adopt the Common Core, with hundreds of millions more spent to encourage the expansion of online learning.   
Eleven states out of 27 have in recent years dropped their high school exit exams, and many of them now allow high school diplomas to be retroactively awarded.  This trend follows research showing that the practice of requiring students to pass these exams leads not to higher achievement or college readiness, but instead to higher drop-out and incarceration rates.    

And yet for some inconceivable reason, the NY State Education Department has chosen to work with the primary funder of the Common Core, as well as  one of the organizations that set our nation’s schools on the wrong path, to help guide their deliberations on this important issue. 

Thursday, May 17, 2018

Would the new teacher evaluation bill fix the dysfunctional test-based system that currently exists? The answer is no.


UPDATE: James Eterno, NYC teacher has posted a petition, urging the Governor and the legislature to  repeal the state law which ties teacher evaluation to test scores.  This is unlike the current bill, which continues to tie evaluation to alternative assessments or the state exams, unreliable, unfair and statistically invalid ways of rating teachers.  Instead, NY schools would return to locally determined teacher evaluation methods that existed before Race to Top, Arne Duncan and Bill Gates managed to dupe the state and the teachers union into adopting the current dysfunctional system.
A new bill, passed by the NYS Assembly and being considered by the NY Senate as S08301, would change the teacher evaluation system in the state for (at least) the fourth time since 2010.  Despite the claims of NYSUT, the state teacher union, a careful reading of the bill does not indicate that it would de-link teacher evaluations from student test scores.  

Instead, teacher evaluations would continue to be partially determined by student “growth scores,” which in turn would be based on “alternate assessments” as approved by the NYS Education Department or where desired locally, still based on the state exams.  Thus, the concerns expressed by the NY State School Boards  Association, the New York Council of School Superintendents and other education groups, that this bill, if passed, could mean even more testing for students, appears warranted, since the state exams will continue to be given anyway, as mandated by federal law. 

More discussion of the teacher evaluation issue, which NY State can’t seem to get right, is in an column written by Gary Stern of LoHud News,  in which he calls the system “a ghastly mistake that won't die.” Diane Ravitch argues that the currently teacher evaluation law, called APPR, should just be repealed, and the decision how to evaluate teachers should go back to the districts, as it was before the promise of Race to the Top funds lured the state to create a new system based in part on student test scores.  My view? If the law is not to be simply repealed, there should be hearings, public input and careful consideration as to what should replace this complex and unreliable mess of a system, rather than the current bill.


The below detailed analysis was written by Deborah Abramson-Brooks of the Port Washington Advocates for Public Education and NY State Allies for Public Education.
_____
Can we PLEASE settle this once and for all?

The proposed legislation -- recently passed in the NYS Assembly with a "same as" bill now sitting in the NYS Senate -- is all about amending the current teacher evaluation law, NYS Ed. Law section 3012-d.  The proposed legislation affects only *some* of section 3012-d; those portions of 3012-d that are not addressed in the proposed legislation remain intact. Accordingly, it is IMPERATIVE that the proposed legislation be read in conjunction with existing NYS Ed. Law section 3012-d. If you like to use a ruler and red pen, feel free to put the documents side-by-side and start red-lining, or adding!

Myth #1: tests will be decoupled from teacher evaluations. FALSE.
I have seen/heard waaaaay too many comments suggesting that the proposed legislation permanently decouples testing from teacher evaluations.  Sorry, no. 

Teacher (and principal) evaluations remain segmented into two categories: student performance and classroom observations. According to the proposed legislation, student performance will continue to be based on some type of assessment, whether it's a state exam, or some "alternative assessment" that SED will approve. The difference is that the NY State Education Department will no longer be able to mandate that a district MUST use the STATE test scores for teacher evaluations; rather, this is subject to local collective bargaining discretion.

The following is from a NYS Assembly press release:

"THE LEGISLATION WOULD REMOVE THE MANDATE that state created or administered assessments be used to evaluate a teacher's or principal's performance. The Commissioner of Education would be required to promulgate regulations providing ALTERNATIVE ASSESSMENTS FOR DISTRICTS THAT CHOOSE NOT TO USE THE STATE ASSESSMENTS. The selection and use of assessments would be subject to collective bargaining. The bill would also eliminate the use of the state-provided growth model in a teacher or principal's evaluation. ALL TEACHERS WOULD BE REQUIRED TO HAVE A STUDENT LEARNING OBJECTIVE (SLO) consistent with a goal-setting process determined or developed by the Commissioner.


The legislation would also ELIMINATE THE USE OF CERTAIN RULES TO DETERMINE A TEACHER OR PRINCIPAL'S OVERALL RATING, and make permanent provisions that prohibit grades three through eight ELA or math state assessments scores from being included on a student's permanent record."  (my emphasis added)

TRANSLATION: 1) SED is precluded from mandating that districts use the grades 3-8 state exams and/or Regents exams to evaluate teachers; BUT NOW, districts can *choose* to use the state exams to evaluate a teacher/principal, or some alternative assessment approved by SED, via local collective bargaining.

Myth #2: no more growth models!  FALSE.
As mentioned above, *some* type of test/assessment must still be tied to the teacher evaluation.  Some districts may choose to use the state exams in grades 3-8.

Some districts may choose to use Regents exams for high school teachers.  (It remains to be seen if SED will allow districts to use Regents exams for all teachers, or only those whose classes culminate in a Regents Exam.) 

Other districts may decide to use one of the SED-approved alternative assessments, which come with their own growth models, i.e., NWEA MAP, and which are no more reliable for this purpose than the state exams.  Whichever assessment is used, some type of growth model will likely be required by SED, or else how would a district determine student growth, a requirement that remains mandated in the state law?  See NYS Ed. Law section 3012-d(2)(c): “Student growth’ shall mean the change in student achievement for an individual student between two or more points in time;” and NYS Ed. Law section 3012-d(4)(a) which also discusses student growth. 

Whichever assessment is used, expert statisticians have been sounding the alarm bells that *any* test-based growth model – whether VAM (Value-Added Model) or the SGP (Student Growth Percentile), or some other model -- used for high-stakes considerations should immediately be viewed with great suspicion.  Many experts, including Diane Ravitch, call these models “junk science.”  The AmericanStatistical Association has come out against this method of evaluating teachers, and there is an entire website called VAMboozled, written by expert statistician and Professor Audrey Amrein-Beardsley, who explains in detail why growth models based on student test scores are an unreliable and unfair way to evaluate teachers.

Finally, in the lawsuit filed by Great Neck teacher Sheri Lederman, Justice Roger D. McDonough of the New York State Supreme Court found that the portion of the NYS teacher evaluation statute that linked student growth scores to the teacher’s evaluation was irrational and produced results that were “arbitrary and capricious,” and ruled that the teacher scores and ratings that flowed from them were illegal. At that point, however, the Legislature had already voted on a moratorium against using state test scores in teacher evaluation until the school year 2019-20. 

The moratorium is coming to its end; thus this bill.

Now let's read the entirety of the current APPR law, NYS Ed. Law section 3012-d (a/k/a part of The Education Transformation Act of 2015), alongside the Legislature's red-lined/green additions version to THAT LAW, and see what else changes/stays/gets added.  Here is NYS Ed. Law section 3012-d: https://codes.findlaw.com/ny/education-law/edn-sect-3012-d.html

Subdivision 1 of 3012-d remains intact. That section provides: "Such ANNUAL EVALUATIONS SHALL BE A SIGNIFICANT FACTOR FOR EMPLOYMENT DECISIONS including but not limited to, promotion, retention, tenure determination, termination, and supplemental compensation. Such evaluations SHALL ALSO BE a significant factor IN TEACHER AND PRINCIPAL DEVELOPMENT including but not limited to coaching, induction support, and differentiated professional development." Translation: high-stakes employment decisions remain.

Subdivision 2, Definitions, remains intact.

Subdivision 3, Ratings, as in H-E-D-I (meaning Highly Effective, Effective, Developing, and Ineffective) designations, remain intact.

Subdivision 4, Categories; continue to define APPR by student performance along with classroom observations. Translation: tests continue to be linked to APPR.

What changes: *For the first student performance subcomponent: 1) "a teacher shall have a student learning objective (SLO) consistent with a goal-setting process determined or developed by the commissioner, that RESULTS IN A STUDENT GROWTH SCORE;" 2) the law continues to say: "for any teacher whose course ends in a state-created or administered assessment, such assessment MAY be used as the underlying assessment for such SLO" -- meaning the state tests can still be used as the underlying assessment for the SLO; but 3) the mandate to use a growth score based on the state exams is eliminated.

What else changes: *For the optional second student performance subcomponent: a district may locally select a second measure and be either: (A) BASED ON A STATE-CREATED OR ADMINISTERED TEST (again, meaning the state exams); or (B) based on a state-designed supplemental assessment (again meaning the state exams).  The state-provided growth model mandate is eliminated.

This language in 3012-d(4) remains intact: "The commissioner shall determine the weights and scoring ranges for the subcomponent or subcomponents of the student performance category that shall result in a combined category rating. The commissioner shall also set parameters for appropriate targets for student growth for both subcomponents, and the department must affirmatively approve and shall have the authority to disapprove or require modifications of district plans that do not set appropriate growth targets, including after initial approval. The commissioner shall set such weights and parameters consistent with the terms contained herein."  This gives a whole lot of power to Commissioner Elia, who has in the past put great reliance on unreliable standardized exams.

In addition, the entire provision regarding classroom observations remains intact.

Subdivision 5, Rating Determination. 1) The draft legislation completely removes what must happen to a teacher or principal who is rated using two subcomponents in the student performance category and receives a rating of ineffective for both. 2) BUT... it otherwise leaves the HEDI matrix intact, which in essence means student growth scores and teacher observation are weighted 50/50, and sometimes the student growth portion is given more weight.

Subdivision 6, Prohibited Elements, entire section left intact.  So, for anyone claiming that student portfolios could be used instead of standardized exams, check out subsection (a):

“The following elements shall no longer be eligible to be used in any evaluation subcomponent pursuant to this section: a. evidence of student development and performance derived from lesson plans, other artifacts of teacher practice, and student portfolios, except for student portfolios measured by a state-approved rubric where permitted by the department.” 

And for anyone saying that unit tests (meaning designed by the classroom teacher) could be used to measure student performance, check out subsection (d) that rules out the use of: “any district or regionally-developed assessment that has not been approved by the department.”

Subdivision 7 continues to ensure that the process by which weights and scoring ranges are assigned to subcomponents and categories is transparent and available to those being rated before the beginning of each school year. The draft legislation completely removes what must happen to a teacher or principal who is rated using two subcomponents in the student performance category and receives a rating of ineffective for both.

Subdivision 8 says a student may not be instructed, for two consecutive school years, by any two teachers in the same district, each of whom received a rating of ineffective under an evaluation conducted pursuant to this section in the school year immediately prior to the school year in which the student is placed in the teacher's classroom.... remains intact.

Subdivision 9 regarding the right to terminate a probationary (non-tenured) teacher or principal remains intact.

Subdivision 10 regarding the local collective bargaining representative negotiating with the district remains largely intact.

Subdivisions 11 and 12 remain intact.

Subdivisions 13 and 14, looping 3012-d to 3012-c (which outlines the subcomponent scores that align to H-E-D-and I) remains intact. NYS Ed. Law section 3012-c here: https://codes.findlaw.com/ny/education-law/edn-sect-3012-c.html

Subdivision 15 remains intact.

The draft legislation adds a new subdivision 16, to provide that 1) the grades 3-8 state tests "shall not be required to be utilized in any manner to determine a teacher or principal evaluation required by this section;" 2) SED (Commissioner Elia) is required to promulgate rules and regs providing alternative assessments that may be used; 3) the selection and use of an assessment is subject to collective bargaining law; 4) any unexpired collective bargaining agreement in effect on the date this proposed legislation takes effect (if ever) stands until the entry into a successor collective bargaining agreement, and a successor collective bargaining agreement must comply with applicable law and timelines, etc.

Translation: all new collective bargaining agreements must comply with the amended APPR law (if it passes) or, conversely, whatever assessments are used in the district’s teacher evaluation system must be negotiated with the union.

And finally, the draft bill discusses the grades 3-8 test scores vis-a-vis a student's permanent record. (FWIW, this issue is addressed in NYS Ed. Law section 305(45) – not NYS Ed. Law section 3012-d.)

NYS Ed. Law section 305(45) provides: "no school district or board of cooperative educational services may place or include on a student's official transcript or maintain in a student's permanent record any individual student score on a state administered standardized English language arts or mathematics assessment for grades three through eight, provided that nothing herein shall be construed to interfere with required state or federal reporting or to excuse a school district from maintaining or transferring records of such test scores separately from a student's permanent record, including for purposes of required state or federal reporting." 

 The most recent expiration date for that provision was December 2018; the recent  budget bill amended that expiration date to December 2019, via Chapter 59 of the Laws of 2018.  The draft legislation seeks to eliminate the expiration date altogether.  NYS Ed. Law section 305 here: https://codes.findlaw.com/ny/education-law/edn-sect-305.html

Happy red-lining!

***********************************

Saturday, August 24, 2013

Your child's state test scores are now posted; read this advice first from Tory Frye

Dear Parents and Guardians,
Your child's scores are now available on ARIS Parent Link.  For what it is worth, there are a few things that I think we should all do:
1) Ignore the test results and do not tell our children what their "number" is, no matter how high or low;
2) Know that these tests are part of a movement to privatize public education by convincing parents that our children are underperforming, compared with other countries, and a massive restructuring of the public education system is the only thing that will save the future of the US; this would involve higher standards, better teachers (which requires a de-unionized teaching force), more charter schools, vouchers for private schools, and market-based methods to make parents consumers of "public" education; they say that this is needed because of the "new" US economy, where because of economic and labor policies, we have an increasingly bimodal distribution of jobs (you are either a Walmart Greeter or a Scientist, with few solid working- and lower-middle class options left);    
3) Recognize that no elite NYC private schools use high stakes standardized tests in this way and that the country that is held up as a model of universal, high quality public education, Finland, also does not use test scores in this way; 
4) Know that our state has purposefully set up our children to fail in order to "shock" us into submission and turn on our children's teachers and their unions;
5) Resist the urge to pressure our children's teachers and schools to do better on these tests; this will only totally eliminate the arts, sports, sciences, recess and other activities that have been diminished in the pursuit of high scores and that children desperately need; these are the things that keep some of our most vulnerable children in school;
6) Be aware also that our children's test scores are being given by the state to inBloom, a private company that will store all of children's data in a "cloud" and offer it to other private companies to make more educational "products" that are typically "personalized" and computer-based and designed to further undermine face-to-face instruction, classrooms and human teachers.  See here for inBloom: https://www.edsurge.com/inbloom-inc  See here for criticisms of it: http://www.classsizematters.org/inbloom_student_data_privacy/
7) Consider opting out of the tests next year as a way of resisting corporate education reform and the monetizing of our children's school experiences.
So that is what I think we should do; feel free to agree or disagree!
Best,


Victoria (Tory) Frye, D6 parent