Please take a look at my piece on computer scoring of the PARCC and SBAC exams in a special report of the Network for Public Education; also posted at the Washington Post AnswerSheet.To summarize: neither Pearson, in charge of administering and scoring the PARCC exams, nor AIR in charge of the SBAC exams, have publicly told parents that their standard contracts call for half or more of their children's writing responses to be graded exclusively by robo-scorers, and never to be seen by a human being. Yet these states also have the option to have the exams scored by humans at an extra charge.
Why is this important? No computer program has so far been shown to have the ability to distinguish total nonsense from coherent prose. Instead, robo-scorers grade prose according to its length and how much abstruse vocabulary is used. As Les Perelman has warned, an expert on computer scoring, the inability of machines to identify gibberish from substance may lead teachers and test prep companies to train students to game the system by writing verbose and nonsensical prose -- the opposite of what the Common Core standards are supposed to encourage.
This reality unmasks the claims of its proponents that the Common Core and its allied assessments are really aimed at encouraging critical thinking and more advanced skills. Instead as I conclude, "the primary objective of Bill Gates and many of those promoting the Common Core and allied exams is to standardize both instruction and assessment and to outsource them to reductionist algorithms and machines, in the effort to make them more efficient. Essentially, the point of this grandiose project imposed upon our nation’s schools is to eliminate the human element in education as much as possible."
One correction to the piece: Instead of my implication that ETS uses computers exclusively to score writing on the GRE exams, the company informed us that a trained human reader and a computer scores every writing sample. If the human and the computer scores closely agree, the average of the two scores is used . If they disagree, a second human being is asked to assess the sample, and the final score is the average of the two human scores.
We wrote every Commissioner from the PARCC and SBAC states to ask them if they were using automated scoring, and only those from the following states responded: Oregon, Massachusetts, Nevada, and West Virginia. These officials informed us that their states were not using automated scoring at this time.
Wyoming Superintendent Jillian Balow also replied to our letter that she shared our concerns about automated scoring and that her state was no longer using the SBAC exam. Colorado Commissioner Richard Crandall responded to a parent activist in his state,that the state was using the standard PARCC method of having the writing samples scored by a computer two thirds of the time, with only ten percent of those re-checked by a human being, and argued this was necessary to maximize "efficiency." (If you'd like to read them, their responses in full are linked to from my report.)
Though Rhode Island Commissioner Ken Wagner, formerly NY Deputy Commissioner, did not respond to our letter, the same day he made the astonishing claim in a public forum that his state was also using computers to score two thirds of the writing samples on the PARCC exam, because research has shown that "computers did a better job scoring than even expert trained teachers." He also added that “SAT, GRE, GMA, those kinds of programs have been doing this stuff for a very long time.”
Yet as mentioned above, every writing sample on the GRE is scored by both a computer and a human being, and the College Board uses trained human scorers exclusively on writing samples for the SAT and AP exams.
The following 18 states and districts have failed to respond to our letter as to whether they are using computers to score writing samples on their PARCC or SBAC exams: CA, CT, DE, DC, HI, ID, IL, LA, MD, MI, MT, NH, NJ, NM, ND, SD, VT, and WA.
If you are a parent, teacher or advocate in one of these states, please send your Commissioner of Education these questions:
- What percentage of the ELA exams in our state are being scored by machines this year, and how many of these exams will then be re-scored by a human being?
- What happens if the machine score varies significantly from the score given by the human being?
- Will parents have the opportunity to learn whether their children’s ELA exam was scored by a human being or a machine?
