
Please take a look at my piece on computer scoring of the PARCC and SBAC exams in a
special report of the Network for Public Education; also posted at the
Washington Post AnswerSheet.
To summarize: neither Pearson, in charge of administering and scoring the PARCC exams, nor AIR in charge of the SBAC exams, have publicly told parents that their standard contracts call for half or more of their children's writing responses to be graded exclusively by robo-scorers, and never to be seen by a human being. Yet these states also have the option to have the exams scored by humans at an extra charge.
Why is this important?
No computer program has so far been shown to have the ability to distinguish total nonsense from coherent prose. Instead, robo-scorers grade prose according to its length and how much abstruse vocabulary is used. As Les Perelman has warned, an expert on computer scoring, the inability of machines to identify gibberish from substance may lead teachers and
test prep companies to train students to game the
system by writing verbose and nonsensical prose -- the opposite of what the Common Core standards are supposed to encourage.
This reality unmasks the claims of its proponents that the Common Core and its allied assessments
are really aimed at encouraging critical thinking and more advanced
skills. Instead as I conclude, "
the primary objective of Bill Gates and many of those promoting
the Common Core and allied exams is to standardize both instruction and
assessment and to outsource them to reductionist algorithms and
machines, in the effort to make them more efficient. Essentially, the point of this grandiose project imposed upon our
nation’s schools is to eliminate the human element in education as much
as possible."
One correction to the piece: Instead of my implication that ETS uses computers exclusively to score writing on the GRE exams, the company informed us that a trained human reader
and a computer scores every writing sample. If the human
and the computer scores closely agree, the average of the two scores is used .
If they disagree, a second human being is asked to assess the sample, and the final score is the
average of the two human scores.
We wrote every Commissioner from the PARCC and SBAC states to ask them if they were using automated scoring, and only those from the following states responded: Oregon, Massachusetts, Nevada, and West Virginia. These officials informed us that their states were not using automated scoring at this time.
Wyoming Superintendent Jillian Balow also replied to our letter that she shared our concerns about automated scoring and that her state was no longer using the SBAC exam. Colorado Commissioner Richard Crandall responded to a parent activist in his state,that the state was using the standard PARCC method of having the writing samples scored by a computer two thirds of the time, with only ten percent of those re-checked by a human being, and argued this was necessary to maximize "efficiency." (If you'd like to read them, their responses in full are linked to from
my report.)
Though Rhode Island Commissioner Ken Wagner, formerly NY Deputy Commissioner, did not respond to our letter, the same day he made the astonishing claim in a public forum that his state was also using computers to score two thirds of the writing samples on the PARCC exam, because research has shown that "
computers did a better job scoring than even expert trained teachers." He also added that “
SAT, GRE, GMA, those kinds of programs have been doing this stuff for a
very long time.”
Yet as mentioned above, every writing sample on the GRE is scored by both
a computer and a human being, and the College Board uses
trained human scorers exclusively on writing samples for the
SAT and
AP exams.
The following 18 states and districts have failed to respond to our
letter as to whether they are using computers
to score writing samples on their PARCC or SBAC exams: CA, CT, DE, DC,
HI, ID, IL, LA, MD, MI, MT, NH, NJ, NM, ND, SD, VT, and WA.
If you are a parent, teacher or advocate in one of these states, please send your Commissioner of Education these questions:
- What percentage of the ELA exams in our state are being scored
by machines this year, and how many of these exams will then be
re-scored by a human being?
- What happens if the machine score varies significantly from the score given by the human being?
- Will parents have the opportunity to learn whether their children’s ELA exam was scored by a human being or a machine?
Their email addresses are
posted here. And please let us know if you get a response by emailing us at
info@studentprivacymatters.org and/or posting their reply below. Thanks!