Comments on the SHSAT and the Chancellors Privacy regulations

Dec. 13, 2024

On Wednesday, night, the new Public Engagement Committee of the Panel for Education Policy, NYC’s school board, met to hear from the public on two controversial issues, a contract for Pearson to produce a new computerized version of the SHSAT, the entrance exam for the Specialized high schools, and also  DOE’s  proposed revisions to the Chancellors regulations A- 820, that govern student privacy.

The proposed Pearson five year contract for the SHSAT at cost of $17 million took up most of the time, with many parents concerned that any further delay in a vote to approve the contract that had already been postponed twice would threaten the ability of their children to attend one of the eight elite schools that decide admissions solely by means of that one exam.  

My comments follow, suggesting that the PEP only renew the contract for one year, and base any further renewal on specific conditions.  I also include my comments on the Chancellors regulations.  Shannon Edwards of AI for Families also offered excellent comments on this critical issue; you can read them here.

A video of the proceedings is here. Whatever happens, it’s real progress that the Panel for Education Policy seems interested in hearing from the public on these critical issues, rather than merely rubberstamping whatever the Mayor wants them to do.

 Statements on SHSAT

Thank you for holding this public session this evening.  As the American Psychological Society, the American Education Research Association, the National Academy of Sciences, and the testing companies themselves proclaim, no high stakes decision such as admissions to any school should be based upon test scores alone.

In fact, NYC is the only district in the nation that uses this unacceptable method for admissions to any single school, and yet we do it for eight schools, with a test that is non-transparent, scored in a highly unusual way, and is designed by Pearson, a company that has been shown repeatedly over more than a decade to engage in improper behavior and to  administer faulty tests and score them erroneously, year after year, starting with the infamous Pineapple questions on the 2012 state exam that not only  made news nationwide but  became a symbol of everything wrong about standardized testing.

 We have offered a timeline of these issues on our NYC Parent blog – none of which, by the way, did DOE report on in their Request for Authorization, as they should have been. .

We understand that the state requires a standardized exam, but we urge you to amend the terms of the contract so that it is renewed for only for one year, and condition any further renewals in the years to come on the following  three demands:

• DOE should use this test for admissions only for the three schools that are required to use a standardized exam by state law. 

• Consider whether another standardized test could be used instead for these three schools at lower cost or even for free, such as the state exam, as the law only says that a standardized exam should be used, and doesn’t specify which one, and also given how Pearson has a long history of errors and misdeeds;

• Finally, require full test transparency, including an independent analysis of the SHSAT for racial and gender bias, a formal validity study that shows whether the scores predict HS performance, and an analysis of how the results of the new computer-based adaptive exam compared to the earlier paper-based, non-adaptive exams.  

This sort of transparency has been requested for at least 16 years by researchers, but has not been provided.

 The importance of independently analyzing the exam for gender bias is paramount, as girls are accepted to the specialized HS at far lower rates than boys, and there is peer-reviewed, published research showing that those who are accepted do far better than boys who received the same scores.

 On the Chancellors A-829 proposed privacy regulations

Hi again, my name is Leonie Haimson and I am the co-chair of the Parent Coalition for Student Privacy.   We helped get a new  state student privacy law passed in 2014, Ed Law 2D, when we realized how ineffective the federal privacy protections for student data were in an era of ed tech expansion. 

I was appointed to serve on the state education department Data Advisory Committee, and  have  been advocating for years for the DOE to update their  Chancellors regulations A820, but was horrified to see that when this was finally done, they weakened rather than strengthened the existing privacy regulations, despite  widespread breaches and misuse of data that has occurred over the last few years. This includes the Illuminate breach that exposed the personal data of over one million current and former NYC students.

According to these regs, DOE and individual schools could disclose a huge range of student and parent info with anyone they please, and without any enforceable privacy or security protections, including but not limited to: their names; addresses; telephone numbers; e-mails; photographs; dates of birth; grade levels; enrollment status; dates of enrollment; participation in officially recognized activities and sports; weight and height and more.

They propose doing this by designating this info as “Directory Information”  -- an outmoded provision of FERPA from the 1970’s that allowed the disclosure of any information that would not be considered too risky  to divulge.

Yet this ignores the fact that there is NO mention in Ed Law 2D of Directory Information nor any language that would exempt any personally identifiable info from its mandated privacy protections.

Moreover, in this day and age, a child’s name, birth date and home address is sufficient for identity theft, as the NY Dept of State warns, which is especially valuable to fraudsters given that minors do not already have credit ratings.

Personal student data can also be used for predatory marketing by ad tech and social media companies, bombarding them with ads, and undermining their mental health, as noted in recent lawsuits launched vs Facebook, Instagram & TikTok by New York City and the State Attorney General

This data including photos could  also be used to threaten student safety, leading to sexual harassment, Deepfake porn or even abduction.

Providing student names, photos & addresses could also aid in the Trump administration’s efforts to deport migrant students, based on their residence in hotels or shelters.

As a result of over 3,000 emails sent by parents and teachers to DOE and  Panel members, as well as letters from several elected officials and UFT President Michael Mulgrew, the vote on the regs was repeatedly delayed, and on Nov. 19, the Chancellor held a meeting during which she promised to form a Working group that would collaborate on the regs. Yet we have not heard back from the DOE about this Working Group, [Note: The next day, on Thursday at a CPAC meeting, the Chancellor and her team confirmed that a Data Privacy Working Group would begin meeting next month to strengthen these regulations].

We hope that in the meantime, the PEP will refuse to approve any regulations such as these which so seriously threatens the health, safety, and privacy of NYC students.  Thank you for your time.

Timeline of Pearson's Errors, Misdeeds and Crimes

Below is a timeline of the various misdeeds and testing errors Pearson has made over the last decade or more.  Pearson is up for a five-year DOE contract to deliver a computerized version of the SHSAT, at a cost of $17 million.

During a recent Talk out of School podcast,  Akil Bello, testing expert and critic, explained how the SHSAT is a very weird and controversial exam that has never been independently validated or assessed for racial or gender bias, and as a result, far fewer Black, Latino and female students are admitted to the specialized high schools. This year, just 4.5% of offers went to Black students, and 7.6% to Latino students, though the two groups make up 65% of NYC students.  Though we don't have the gender breakdown, in past years far fewer girls were admitted, though they tend to have higher test scores on the state exams and get better grades.  More on this in my comments to the PEP here.

 Whatever your viewpoint on testing, Pearson is not to be trusted:

In 2012, the state exam produced by Pearson featured the infamous Pineapple reading passage that made news nationwide, and years later even featured on John Oliver’s HBO Show This Week tonight.  (you can google it; we broke the story first). The Pineapple became a symbol of everything wrong about standardized testing. Worst thing was this reading passage had been included for years on lots of other Pearson state exams, sparking criticisms by students and teachers each time. The exam featured at least 30 other errors, with  faulty questions and problems with translation and scoring.

In 2013,  Pearson state exams were too long, too difficult, full of ambiguous questions that made children cry. They also featured crass, commercial product placements as well as  reading passages lifted off of Pearson textbooks that had been purchased and assigned to students elsewhere in the state but not NYC.  According to Kathleen Porter Magee of the conservative Fordham Institute, Pearson was abusing its monopoly power in way that "threatens the validity of the English Language Arts (ELA) scores for thousands of New York students and raises serious questions about the overlap between Pearson's curriculum and assessment divisions." 

Also in 2013, the Pearson Charitable Foundation paid $7.7 million fine after the State Attorney General found they had broken state laws by generating business for the company.

Also in 2013, Pearson agreed to pay $75 million in damages plus costs to settle a lawsuit over price-fixing e-books.

Also in 2013, they were found to make  mistakes in scoring the NYC Gifted and talented tests, not once but twice,

In 2015, it was discovered that Pearson was monitoring students’ social media who criticized their NJ state exams.

Here is a list of other Pearson problems through 2016.

In 2018, Pearson lax security practices led to one of the largest student data beaches in history of their AIMS web program between 2001 and 2016, and who were enrolled in 13,000 school and universities throughout the country. The breach involved probably millions of students including many  in NY, whose data should have been long deleted because they no longer had contracts for the delivery of the program.  The FBI alerted Pearson to the breach in March 2019, but they didn’t tell anyone, including the schools or the students till months later, July 31, 2019. 

In 2021, Pearson was fined $1M by the FEC for misleading investors about the AIMs breach.  [They should also have had to pay the families of the students' whose data was breached, as well as had all future contracts blocked by NY State for having violated the required timeline of reporting on the breach, but weren't.]

In 2020, Pearson was awarded a huge contract with DOE despite 34 investigations for discrimination against its employees on grounds of race, disability, gender, age, etc. and many technical issues with erroneous scoring,  online service disruptions, the above breach, and more.   

In 2021, a Pearson Middle East textbooks was pulled in the UK for bias.

In 2023, the plans of hundreds of international students to enroll in UK universities were derailed after Pearson revoked some of their online English language exam results following allegations of cheating, without giving students a  chance to appeal these decisions.

What makes DOE think that Pearson is capable of developing, scoring and administering and scoring a reliable exam -- even if you believe in the notion of high-stakes testing?

Gov. Cuomo: please call off the SHSAT, absolutely critical especially during a pandemic.

The letter below was sent to Gov. Cuomo on Monday via his webform; feel free to send your own thoughts on the matter.  

November 23, 2020

Dear Governor Cuomo, 

We, the undersigned organizations and individuals, write to request the issuance of an Executive Order to suspend the Specialized High Schools Admissions Test (SHSAT) for the specialized high schools in New York City. 

The Hecht-Calandra Act requires that admissions to the specialized high schools be solely and exclusively determined by scores on the SHSAT, which is administered by the NYC Department of Education usually in late October/ early November every year.  Nearly 30,000 students take the SHSAT for approximately 5,000 seats across 8 specialized high schools. The test is administered on campus at these schools.  Obviously this year with the pandemic and particularly now with the increasing infection rates, in-person testing is infeasible and the DOE has not announced how it plans to administer the test. 

The Mayor hinted at offering the SHSAT online at the weekly radio address last week. However, not every student has access to an adequate device or reliable internet connectivity, making the online option discriminatory. In addition to the inequitable access to the digital platform, many of our students are traumatized by the pandemic, having lost loved ones to the disease, facing a new economic reality resulting from parental job loss, or living with the anxiety of a parent who is an essential worker. These traumas disproportionately affect historically marginalized students.

Because the Mayor does not have the power to change the admissions to the specialized high schools, we call upon you to issue an Executive Order suspending the SHSAT this year and allowing the Chancellor of the NYC DOE to develop an alternative method of admissions to the specialized high schools. And given that our estimate of the costs for test administration is approximately $3 Million per year, suspending the SHSAT is also prudent in the face of the fiscal crisis.  We believe this is the only equitable path forward.  

Sincerely, 

Organizations

Alliance for Quality Education

Class Size Matters

Coalition for Asian American Children & Families (CACF)

Community Education Council District 14

Community Education Council District 16

Community Inclusion & Development Alliance

Education Council Consortium

EduColor

El Puente

Families for Real Equity in Education (FREE)

IntegrateNYC

Masa

MORE-UFT (Movement of Rank and File Educators)

NYC Kids PAC

NYC Opt Opt

S.E.E.D.S., Inc. <www.seedswork.org>

Teens Take Charge

Testimony to encourage integration and the reform of the specialized HS admissions process

Today the City Council held hearings on a raft of proposals , including bills and resolutions to encourage school integration, diversity and reforming the admissions process at the specialized high schools.  

The admissions to the specialized high schools has resulted in terribly racially disparate results and relies on one controversial high-stakes exam, the SHSAT.  Sadly, some of the proposals put forward by Council Members, including expanding gifted programs and the number of selective schools, would merely worsen segregation and stratification, as I explain in my submitted testimony below.

New evidence of an extreme gender bias in selective high school admissions demands an independent evaluation of the exam

Correction: A reader pointed out that Al Baker of the NY Times wrote an article about the gender imbalance at the NYC specialized high schools in 2013, so the Daily News is not the first media outlet to cover this important issue.

Today, Ben Chapman of the NY Daily News wrote about the disproportionate number of girls rejected from the highly-selective Specialized high schools compared to the boys. As I was quoted in the article, "You would think that the city would be taking every step they could, to ensure that girls are accepted to these high schools at, at least the same rate as boys...Girls should have the same opportunities as boys and the data suggests that this entrance exam has a gender bias that needs to be addressed.”

I have been writing about the gender imbalance at the specialized high schools since 2010.  There has been much written about the low numbers of black and Latino students admitted through the SHSAT exam, with only 10 Black students and 27 Latino students accepted into Stuyvesant high school this year.  Indeed, in 2012, a complaint was filed with the Civil Rights office of the US Department of Education about the exam's discriminatory impact.

Meanwhile, the Mayor has continued to blame the state for the problem, which passed a law years ago requiring that the exam results be the deciding factor in three high schools, Stuyvesant, Bronx Science and Brooklyn Tech; yet in the case of the other five high schools that use the exam, their method of admissions is completely under his control.  It is also important to note that  NYC is the only district in the country with selective high schools in which a single high stakes exam is the sole criterion for admissions.

Yet today's Daily News article is the first time to my knowledge that the mainstream media has reported on the sharp gender differential between the admission rates of boys vs. girls.  Here is a chart showing a large gender gap of ten percent, with only 15.4% of girls who took the exams admitted to the specialized schools compared to 20.4% of boys:

These disparate results strongly suggests the  exam exhibits gender bias, especially as among NYC public school students, girls tend to get better grades AND better test scores than boys.  Here, for example, are their comparative scores on the 2017 state 8th grade exams:


  
You can see from the above that according to the state exams, girls obtain higher scale scores, achieve higher proficiency rates and more of them score at the highest level (level 4) in both ELA and math. 

I also checked for the gender differential on the 7th grade state 2017 math tests, since many 8th graders take the Regents exams in math instead.  Girls get higher scores on these exams as well:




In past years as well, according to this paper by Sean Corcoran and Christine Baker-Smith, if state test scores, grades and attendance from 2005-2013 were used as criteria instead of the SHSAT, girls would be 9 to 13 more points more likely to be admitted to the specialized high schools: "In fact, the gender gap would shift dramatically in favor of girls with the use of grades and State tests."

That's a far greater disparity than they found for Black or Latino students (who tend to score lower on the State exams).  Yet among similar applicants with the same performance on the State exams, girls, Blacks, Latinos  and low-income students were all significantly less likely to score high enough to be admitted to these schools, and Whites and Asians significantly more likely to be accepted.

Corcoran and Baker-Smith also said, however, that over that period, girls were less likely to apply to the specialized high schools, which is no longer seems to be the case, with more girls  now taking the SHSAT than boys. 

In 2016, after much criticism of the exam and its racially disparate results, Pearson was awarded a six-year, $13.4 million contract to improve the previous SHSAT, which was also written by the company.  This is despite the fact that Pearson is not noted for its high-quality exams, to say the least.

They did eliminate the scrambled paragraph section of the exam, and the logical reasoning section, but appear to have made few other changes, other than making the exam even longer -- to 180 minutes from 150 minutes.  They also included only non-fiction passages in the ELA section (perhaps a nod to the Common Core/David Coleman personal preference for informational texts.)  

One of the most frequent criticisms in the past has been the highly unusual way in which the SHSAT was scored, to give extra weight to students who scored exceptionally high on the math or the ELA sections, rather than those who received an overall high average score.  Apparently this remained the scoring method as late as 2016. Has the methodology changed?  Is this one of the reasons for the extreme gender disparity in the results?

In any case, whether you believe that using one high-stakes exam as the sole criterion for admissions is itself unfair and highly unreliable (as I do), it is long overdue that the SHSAT be independently evaluated for gender AND racial bias.  There have been calls for this independent evaluation as far back as 2008.  Given the latest stark disparity in admissions for girls vs boys, that should be mandatory.  Or perhaps a Title 9 complaint?  Please leave your comments below.

Below are the offer of admissions by gender and  by school; you can see that the more selective the school the more unbalanced the numbers; with Stuyvesant 58% male and 41% female.