New evidence of an extreme gender bias in selective high school admissions demands an independent evaluation of the exam

Correction: A reader pointed out that Al Baker of the NY Times wrote an article about the gender imbalance at the NYC specialized high schools in 2013, so the Daily News is not the first media outlet to cover this important issue.

Today, Ben Chapman of the NY Daily News wrote about the disproportionate number of girls rejected from the highly-selective Specialized high schools compared to the boys. As I was quoted in the article, "You would think that the city would be taking every step they could, to ensure that girls are accepted to these high schools at, at least the same rate as boys...Girls should have the same opportunities as boys and the data suggests that this entrance exam has a gender bias that needs to be addressed.”

I have been writing about the gender imbalance at the specialized high schools since 2010.  There has been much written about the low numbers of black and Latino students admitted through the SHSAT exam, with only 10 Black students and 27 Latino students accepted into Stuyvesant high school this year.  Indeed, in 2012, a complaint was filed with the Civil Rights office of the US Department of Education about the exam's discriminatory impact.

Meanwhile, the Mayor has continued to blame the state for the problem, which passed a law years ago requiring that the exam results be the deciding factor in three high schools, Stuyvesant, Bronx Science and Brooklyn Tech; yet in the case of the other five high schools that use the exam, their method of admissions is completely under his control.  It is also important to note that  NYC is the only district in the country with selective high schools in which a single high stakes exam is the sole criterion for admissions.

Yet today's Daily News article is the first time to my knowledge that the mainstream media has reported on the sharp gender differential between the admission rates of boys vs. girls.  Here is a chart showing a large gender gap of ten percent, with only 15.4% of girls who took the exams admitted to the specialized schools compared to 20.4% of boys:

These disparate results strongly suggests the  exam exhibits gender bias, especially as among NYC public school students, girls tend to get better grades AND better test scores than boys.  Here, for example, are their comparative scores on the 2017 state 8th grade exams:


  
You can see from the above that according to the state exams, girls obtain higher scale scores, achieve higher proficiency rates and more of them score at the highest level (level 4) in both ELA and math. 

I also checked for the gender differential on the 7th grade state 2017 math tests, since many 8th graders take the Regents exams in math instead.  Girls get higher scores on these exams as well:




In past years as well, according to this paper by Sean Corcoran and Christine Baker-Smith, if state test scores, grades and attendance from 2005-2013 were used as criteria instead of the SHSAT, girls would be 9 to 13 more points more likely to be admitted to the specialized high schools: "In fact, the gender gap would shift dramatically in favor of girls with the use of grades and State tests."

That's a far greater disparity than they found for Black or Latino students (who tend to score lower on the State exams).  Yet among similar applicants with the same performance on the State exams, girls, Blacks, Latinos  and low-income students were all significantly less likely to score high enough to be admitted to these schools, and Whites and Asians significantly more likely to be accepted.

Corcoran and Baker-Smith also said, however, that over that period, girls were less likely to apply to the specialized high schools, which is no longer seems to be the case, with more girls  now taking the SHSAT than boys. 

In 2016, after much criticism of the exam and its racially disparate results, Pearson was awarded a six-year, $13.4 million contract to improve the previous SHSAT, which was also written by the company.  This is despite the fact that Pearson is not noted for its high-quality exams, to say the least.

They did eliminate the scrambled paragraph section of the exam, and the logical reasoning section, but appear to have made few other changes, other than making the exam even longer -- to 180 minutes from 150 minutes.  They also included only non-fiction passages in the ELA section (perhaps a nod to the Common Core/David Coleman personal preference for informational texts.)  

One of the most frequent criticisms in the past has been the highly unusual way in which the SHSAT was scored, to give extra weight to students who scored exceptionally high on the math or the ELA sections, rather than those who received an overall high average score.  Apparently this remained the scoring method as late as 2016. Has the methodology changed?  Is this one of the reasons for the extreme gender disparity in the results?

In any case, whether you believe that using one high-stakes exam as the sole criterion for admissions is itself unfair and highly unreliable (as I do), it is long overdue that the SHSAT be independently evaluated for gender AND racial bias.  There have been calls for this independent evaluation as far back as 2008.  Given the latest stark disparity in admissions for girls vs boys, that should be mandatory.  Or perhaps a Title 9 complaint?  Please leave your comments below.

Below are the offer of admissions by gender and  by school; you can see that the more selective the school the more unbalanced the numbers; with Stuyvesant 58% male and 41% female.

A letter to Tom Toch about high school choice

Hey Tom, I read your recent article in PDK lauding high school choice. You write that the example of NYC shows that high school choice can "harness the power of the marketplace to better serve students’ diverse educational interests and needs and to stimulate improvement through competition for students on a wide scale."

As a NYC parent, I can tell you that this is a highly idealized picture of what actually occurs. The high school admissions process is a nightmare for most parents (and students); even worse than the college admissions process. And your article has some egregious errors.

You write that that “By 2009, some 95% of students won places at one of their top five high schools, and city officials had to assign only 791 students to schools.”

Actually, there were about 7500 students in 2009 -- 16% of the total-- who didn’t get into any of their top twelve choices.

That’s thousands of kids who end up being forced to attend failing schools, and/or schools miles from their homes, and/or schools that specialize in areas that don’t interest them at all. Another 7,000 or so students are automatically assigned to high schools because they show up too late to apply.

As this New School report points out, many of our students are routinely assigned to vocational schools to study trades that they have no interest in pursuing, though they have to pass exams in these specialized areas to graduate.

You would also be amazed at how low the quality of many of our high schools. More than half of our students attend severely overcrowded schools, most of them sitting in classes of thirty or larger, thousands in trailers. Many students travel an hour or more each way to school. As a result, about 40 percent students who enter high school at grade level or near grade level fail to graduate after four years.

And some of the most overcrowded schools are the lowest performing, flooded with high-needs students no one wants, especially special ed and ELL students. Indeed, inequities have flourished under this system of “school choice”.

Moreover, under the current system it has become nearly impossible for students to transfer out of the high school to which they ’ve been assigned – even they identify another school where the principal is willing to take them . You have to be practically mugged first.

This is one of the reasons our dropout and discharge rates are so high . In fact, the discharge rate for students in their first year of high school has doubled under this system of high school "choice", though none of these students counted as dropouts.

You are lucky you don’t have a child who attends public high school in NYC. We would all rather live someplace where our kids could automatically attend a decent neighborhood high school.

Yours, Leonie Haimson, public school parent

The Times' enduring obsession with Gifted and talented programs, and what is left out

This morning, the NY Times has yet another article about the city's Gifted and Talented programs, and the high-stakes exams that control admissions to these programs. See today's front page story, Tips for the Admissions Test ... to Kindergarten . It all seems so familiar....and indeed it is.

By my quick count, this is at least the ninth article about G and T that the Times has run in the last seven months.

To add insult to injury, this is the second Times article about the admissions process that omits any mention of its inherently discriminatory nature - which has significantly worsened under this administration. This is due to the Chancellor's insistence that all G and T admissions should be based solely on the results of high stakes exams, which Klein claims ato be more "equitable" but which are highly inequitable in terms of results.

His policies have also directly led to the proliferation of expensive prep programs that few typical NYC families can afford. If you are going to run articles about G and T admissions, failing to cite their contribution to worsening racial and economic segregation in our schools is regrettable. In fact, many people said that these policies would have a racially discriminatory impact, including Patrick Sullivan and Debbie Meier , who both predicted this on our blog when Klein first announced the new admissions policy in the fall of 2007.

For other recent Times articles about G and T, see this one, from October 19, about a new expensive private school in Manhattan: School for the Gifted, and Only the Gifted.

Here is another, a Susan Dominus column from August 17, Connecting Anxious Parents and Educators, at $450 an Hour , about a consultant who helps get kids into private schools: "It would be her mission to democratize information for New York’s most competitive elite."

"Democratize" at $450 an hour? This is like Michael Bloomberg claiming the recent election was fair, when he outspent his opponent sixteen to one.

This was followed by yet another Dominus column on August 25: Early Testing In City Schools Called Faulty. Although she discusses the unreliability of G and T exams, in that children tested at a young age often score quite differently in later years, she fails to mention how the results are also discriminatory, given the influence of socio-economic factors. And she uncritically repeats the administration's claims that their policies are somehow equitable:

" Chancellor Joel I. Klein has tried to rejigger the testing system to be more fair, with uniform cut-offs citywide and better outreach to less-advantaged areas. But what ''Nurture Shock'' suggests, and Ms. Commitante [head of DOE's gifted and talented progrm] somewhat acknowledges, is that just means the randomness of gifted and talented placement is now more equitable."

To the contrary, see this far more informative oped in the Daily News, by James Borland, a professor of education, who points out how inherently inequitable the admissions process has become:

A one-size-fits-all approach to identifying students for the city's gifted and talented programs - which is just what the Department of Education has implemented - is neither equitable nor educationally sound. In fact, testing very young children, before the educational system exerts its admittedly limited equalizing effect, only magnifies the effects of differences in socioeconomic status. It favors children who have had the advantages of expensive preschools; of parents with time, ability and inclination to read to them; and of exposure to cultural events.

On September 7, Dominus yet wrote yet another column, about a new G and T public school in Brooklyn, Going the Distance to Get a Child to a Magnet School , in which she omits any analysis of the economic or racial composition of the school, and instead, approvingly focuses on one "highly motivated" mom, who sends her son, Benjamin, to the school, although it is miles away:

...a bus hired by a dozen families, at about $400 a month each, will pick up Benjamin and another 5-year-old before stopping at homes in Crown Heights, Carroll Gardens, Park Slope and Prospect Heights. Finally, at least an hour and a half after Benjamin has left home, he and the others will arrive at Brooklyn School of Inquiry, a brand-new citywide school for gifted and talented children at the corner of Stillwell Avenue and Avenue P, in Gravesend. Such are the lengths to which some parents — highly motivated parents — will go to take advantage of the city’s coveted magnet programs for gifted children.

Is it really only a matter of motivation? Last spring, the Times ran numerous pieces dealing with G and T programs on the Upper West Side, the epicenter of the phenomenon, including this one on the City Room blog, Are Parents Thinking Differently About Education? (June 29):

The phone keeps ringing at the Upper West Side office of Robin Aronow, an educational consultant and schools guru: anxious families suddenly rethinking whether they can afford private school, distressed parents wondering what to do if their children don’t make it into vaunted gifted and talented programs.

See also these articles from the paper: Students Must Retake Lost Gifted Tests (May 15); Gifted Tests Missing on Upper West Side (May 13), and More Children Take the Tests for Gifted Programs, and More Qualify (May 5).

In this last article, the reporter discloses that the number of students who qualified for G and T seats rose by 45 percent over the year before, but not until the sixth paragraph does the reader discover that the racial disparity in admissions remained largely unchanged.

On the upper West side, the number of children taking the tests rose by 15 percent, while the number of students making the cut off score increased by 48 percent. Though the reporter does not speculate on the cause of this phenomenon, the DOE spokesperson attributed this increase to "families’ increasing familiarity with the new admissions process." Instead, these higher scores are most likely the results of the increased amount of test prep taking place.

By continually reporting on the expensive consultants that are profiting off parents' anxieties to get their children into G and T programs, the Times is encouraging their proliferation. Indeed, the paper deserves to get a cut from these consultants, by regurgitating these articles, over and over again.

If charter schools are the obsession of the editors of the NY Post, gifted and talented programs remain the singular obsession of the Times.

Both serve a tiny proportion of NYC public school students and are far less important than other issues that affect the huge majority of our kids: the systemic and worsening crisis in overcrowding and its impact on class sizes, the lack of transparency and flawed priorities of DOE spending, including the mushrooming school bonus program and the continued growth in Tweed's accountability office, the loss of arts and enrichment programs, the obsession with closing schools rather than improving them, the increased amount of test prep that dominates classroom time and the like -- all of which have contributed to the decline of educational quality in our schools, and all of which our paper of record fails to cover adequately, or not at all.

All is not well with the admissions practices of the specialized high schools.

Josh Feinman, a NYC public school parent, recently published a study showing that the admissions practices at the specialized science high schools -- Stuyvesant, Bronx Science, and Brooklyn Tech --violate generally-accepted educational standards and practices, and may be invalid, unreliable, and unfair. While state law mandates that an exam be the sole criterion for admissions for these three high schools, Chancellor Klein extended its use to other schools across the city, including Staten Island Tech, which previously had a more holistic admissions process, as well as other new selective schools, such as the HS for Mathematics, Science and Engineering at City College, the HS of American Studies at Lehman College, the Queens HS for the Sciences at York College and Brooklyn Latin. Feinman's summary follows:

My wife and I are graduates of the specialized high schools, and our daughter currently attends one. We are supporters of public schools, and would not want to see these schools weakened. But every student – regardless of race, gender, or ethnicity -- deserves a properly vetted system for determining who is admitted to these schools. And that’s not what the NYC Department of Education currently provides.

I recently published a study -- High Stakes, but Low Validity? A Case Study of Standardized Tests and Admissions into New York City Specialized High Schools, that raises serious concerns about the admissions practices at these schools. These concerns transcend the issues of race and gender that were the focus of the recent New York Times article which revealed declining rates of admissions among minority students. In fact, the DOE’s failure to investigate whether the admissions test suffers from prediction bias along racial or gender lines is only one of many problems with current practices.

Using test results from the 2005 and 2006 specialized high schools test (SHSAT), which is the sole determinant of admissions, I found a number of glaring violations of widely-accepted educational testing standards and practices.

For example, thousands of students of all backgrounds are rejected with scores that are statistically indistinguishable from those who are admitted. And the NYC Department of Education fails to provide estimates of how well the SHSAT is able to differentiate among students who score close to the admission/rejection line, or whether other criteria could be used to reduce these uncertainties. I made several requests for this information to senior officials at the NYCDOE, to no avail.

Different test versions are used, but no details are provided about how these versions are statistically equated and how accurate that equating is (again, despite requests, and in violation of testing standards and practices). The scaled scores vary across different versions more than the chance distribution would suggest is plausible, suggesting that the equating system may not be leveling the playing field across test versions of varying difficulty – so that students who received certain versions may be more likely to gain admission than those who received other versions.

The SHSAT exhibits an unusual scoring feature that is not widely known, and may give an edge to those who have access to expensive tutors. Someone with a very high score in math and a relatively poor score in English, or vice versa, has a better chance of admission than someone with relatively strong performances in both. Alternative scoring systems would yield far different results, and no evidence is offered to support the current system.

No predictive validity studies have ever been done– not only to see if the test suffers from prediction bias across genders and ethnic groups, but to see if the test is linked to any desired outcomes. In fact, the NYCDOE has never established what specific, measurable objectives the SHSAT is supposed to achieve. Without well-specified objectives and carefully constructed validity studies, there’s no way to know if these admissions criteria are serving their purpose, or if an alternative system would be more reliable.

The SHSAT is widely assumed to produce clear-cut, valid, and equitable results. But for many students who are rejected, they might have been admitted if they’d been assigned a different test version, if the winds of random variation had blown a bit differently, if a slightly different scoring system had been used, or if they’d been made aware in advance of how the scoring was done.

Of course, no admissions criteria is “perfect.” Uncertainty and imprecision are inherent in all decisions, whether they be based on test scores, grades, portfolios, or a combination of the above. Standard psychometric practice is to choose criteria that minimize uncertainties and enable students to demonstrate the skills needed to succeed in ways other than captured on a single standardized test.

The only systematic, objective way to do this is by conducting predictive validity studies, as are regularly carried out for tests like the SAT to help refine the test, and help colleges decide how much weight to put on SAT scores, grades, and other factors in their admission decisions. Overwhelmingly, studies have found that multiple criteria, used in tandem, provide a better guide to future student performance than a single one. Indeed, it’s partly because of such validity studies that psychometric standards caution strenuously against using any single metric as the sole criterion for admission, and virtually all educational institutions use multiple criteria to determine admissions decisions.

The DOE violates accepted psychometric standards, by refusing to provide detailed information about these exams, refusing to carry out any validity studies for them, or even reveal what the tests are designed to accomplish.

We should press the DOE for answers, and more importantly, to reform the system. Formal predictive validity studies need to be carried out. Based on the results of these studies, a determination should be made as to what admissions process is most likely to achieve a specific, quantifiable admissions goal in a transparent, equitable way.

If these studies conclude that it is best to use additional criteria along with a standardized test, the New York State law—which says that admissions to these schools must be based solely on a test—would need to be changed. Whatever admissions procedures are established, all applicants should know their implications.

As parents, we should bring these issues to the attention of our elected representatives, on the City Council and in the State Legislature. I sent a copy of my study to my City Council member Jessica Lappin, my State Senator Liz Krueger, and State Senator Kenneth LaValle -- but have yet to hear back from any of them. This is unacceptable. --Josh Feinman

You can contact Josh for more information at josh.feinman@db.com

---

---