The devil is in the details.
One of the most oft-cited studies on biased hiring trends hails from a study in the 1990s that allegedly proved that when women and men auditioned for orchestras behind a screen to hide their gender from those doing the hiring, women’s chances of being selected drastically improved by wide margins.
Except, as it turns out, that’s not exactly what the researchers — Claudia Goldin of Harvard and Cecilia Rouse of Princeton — found, according to new scrutiny on the famous “blind audition” study, which has been cited more than 1,500 times in academic and media articles, according to Christina Hoff Sommers in The Wall Street Journal. Her op-ed notes:
So where did Ms. Goldin and Ms. Rouse get their totemic conclusion that blind auditions dramatically improved the success of women candidates? After warning that their findings were not statistically significant, they declared them to be “economically significant.” What does that mean in this context?
“That doesn’t mean anything at all,” writes Columbia University data scientist Andrew Gelman, in a recent post about the study. “Some fine words but the punchline seems to be that the data are too noisy to form any strong conclusions.” My guess is that the authors thought they had detected something with real-world relevance despite an absence of statistical rigor. But that’s a reason to call for more research, not to declare the transformative power of screens in women’s quest for equality.
Still, isn’t it obvious that the screens at least contributed to equal hiring? No. The screens might have been a reflection of changing attitudes, and perhaps those attitudes, not the screens, helped women. After all, women didn’t need blind auditions to move ahead in law, business, medicine or the academy—or at the Cleveland Orchestra, which, according to the study, did not use them.
Yes, you read that right. A study touted by diversity experts in mandatory bias workshops across college campuses for the last two decades did not have “statistically significant” findings, only some nebulous and self-righteous “economically significant” ones.
Let this be a reminder to us all: read the fine print!
Sommers rightly concludes her piece by noting “truth matters. Overhyped claims of scientific certainty create confusion, undermine public trust and send scarce resources in the wrong direction. Most of all, they don’t solve problems.”
IMAGE: Alenavlad / Shutterstock