Fact-checking "Waiting for Superman": False data and fraudulent claims

In response to critical comments, I have added clarifications and corrections in bold italics to my original post. I apologize for some sloppy math in not annualizing what was a six year rate in the apparent source article for the film. After further analysis, the movie’s claims remain clearly inaccurate as well as misleading; though the source article was only partially erroneous, at least as far as I can tell. Sorry for the mistake -- and thanks to all my readers, and especially those of you who checked my figures so assiduously.

In the movie Waiting for Superman, nominated for an Oscar as the best Documentary of 2010, the following statement is made:

" ...in Illinois, 1 in 57 doctors loses his or her medical license, and 1 in 97 attorneys loses his or her law license, but only 1 teacher in 2500 has ever lost his or her credentials."

Since the movie was released, these figures have been repeated frequently. They take up five pages in the Google search engine, were cited in the NY Times review of the film, the British newspaper the Independent, as well as Brian Williams of NBC in the television program Education Nation.

But apparently not a single one of these news outlets, or the makers of Waiting for Superman, ever bothered to check them.

While looking for the source of this claim, which is repeated without citation in the movie and its companion book, I came upon a 2007 newspaper article by Scott Reeder of the Small Newspaper Group:

During the past six years, 1 in 2,500 Illinois educators have lost their teaching credentials through suspension, revocation or surrender. By comparison, during the same period 1 in 57 doctors practicing in Illinois lost their medical licenses and 1 in 97 Illinois attorneys lost their law licenses.

"Either Illinois teachers are 43 times better behaved than doctors or they are being held to a considerably lower professional standard than other professions,'' said Jeff Mays, executive director of the Illinois Business Roundtable and an advocate for educator accountability standards. ``Just like doctors and lawyers, teachers are members of an important and demanding profession. It's time that they be held to the same professional standards."

One should note that the data cited in the source article is substantially different from the claim made in the film. In the movie, the period of six years is omitted for the disbarment of physicians and/or attorneys– making indefinite the time span over which the data was collected. The film also says that only 1 in 2500 Illinois teachers have “ever” lost his or her credentials, rather than over six years.

In an effort to verify these claims, I first consulted the annual summary put out by the Federation of State Medical Boards. In reality, 121 doctors lost their licenses in Illinois in 2009, out of 43,670 physicians. That means an average of 0.3% of doctors per year lost their licenses; or 3 out 1,000 per year. Over six years, this would equal 1.8% -- substantially the same as the 1 in 57 figure cited in the source material.

I also checked the claim that 1 in 97 attorneys in Illinois lose their licenses over six years. According to data reported by the American Bar Association, 26 lawyers in Illinois were disbarred in 2009, out of a total of 58,457 - in some cases, by mutual consent.

Since 2001, the average rate of Illinois attorneys disbarred is 32 per year – with more than half of them leaving their professions “voluntarily.” This is an annual rate of about 0.05%, for a six year rate of 0 .3% -- 3 out of 1,000 – not one out of 97, as the source material claimed. As mentioned above, the movie did not specify the time frame over which this disbarment is supposed to have occurred.

The total number of lawyers disbarred in the entire country, either involuntarily or by mutual consent, is 800 per year out of 1,180,386; which is about 0.07% per year, or 7 out of 10,000. The number of those involuntarily disbarred is 441- about 0 .04% or 4 out of 10,000 per year. The six year rate for disbarment nationally would be 0.42% -- about ten times the figure cited in the film of one in 2500 Illinois teachers who “ever” lost their credentials.

I could not find any independent data verifying the number of Illinois of teachers who lose their credentials each year. According to the NY Daily News, over the past three years, 88 out of about 80,000 New York City schoolteachers have lost their jobs for "poor performance." This represents an annual rate of about 30 out of 80,000, or 0.03%, which is about the same rate as attorneys who are involuntarily disbarred each year nationally.

According to the Houston Chronicle, over the last five years, 364 Houston teachers have been fired, out of about 12,000: "Of those, 140 were ousted for performance reasons, a broad category that generally covers teachers not fulfilling their job duties."

So the rate of Houston teachers who lose their jobs due to poor performance is about 0.2% per year - higher than the rate of either doctors or attorneys in the state of Texas removed from their profession annually. For example, only 32 Texan attorneys were disbarred in 2009 out of 75,087; for an annual rate of 0.04% -- at one fifth the rate. 64 doctors per year on average lost their licenses in Texas between 2005 and 2009; out of about 60,000 physicians, at an annual rate of about 0.1 % -- about half the percentage.

Moreover, many more teachers who are untenured and/or uncertified are removed from their jobs for poor performance. Roughly 3.7% of New York City teachers were denied tenure this year, according to the NY Times.

The overall attrition rate of teachers is much higher - many of whom would probably otherwise be cited for poor performance, but who leave the profession either willingly, or "counseled" out. In New York City, the four year attrition rate is more than 40% -- a mind-boggling figure.

In reality, one of the most serious problems plaguing our urban schools, along with excessive class sizes, overcrowding, and poor support for teachers and students, is the fact that we have far too many inexperienced educators revolving through our high-needs schools each year.

Can you imagine if 40% of physicians or attorneys left their jobs after four years? A national emergency would be declared, with a commission appointed to find out how their working conditions could be improved.

Yet instead of examining this critical issue objectively, the movie Waiting for Superman cites false statistics in their effort to scapegoat teachers, unfairly blaming them for all the failures of our urban schools. The film features the views of Eric Hanushek of the Hoover Institute, a well-known conservative critic of equitable educational funding, claiming that the best way to improve our schools would be to fire 5-10% of teachers each year.

To the contrary, eliminating teacher tenure and seniority protections would likely produce an even less experienced and less effective teaching force - especially in our urban public schools, which already suffer from excessively high rates of turnover.

As a parent, I support a higher standard for teacher tenure and more rigorous teacher evaluation systems. I have seen my own children benefit from excellent teachers over the years, but also occasionally suffer as a result of poor teaching, though the latter has occurred as often in schools without union protections as those that were unionized. An improved evaluation system would take into account not only test score data, but also feedback from other teachers, administrators, students and parents.

But at this point, we simply cannot trust the corporate oligarchy currently making policies for our schools to create a fair evaluation system, including those who backed Waiting for Superman, given their proclivity to misuse and distort data, as shown by the egregiously inaccurate figures cited in the film.

Rather than a documentary, perhaps the movie should be re-categorized, with an appropriate disclaimer, as an urban myth.