The College Fix

It’s even worse without ‘not sure’ option

John Villasenor, a professor of public policy at UCLA and senior fellow at the left-leaning Brookings Institution, was the subject of a smear campaign by The Guardian after he released survey results showing an alarmingly high percentage of students endorsing political violence (largely males).

His results weren’t credible, the theory goes, because he didn’t use random sampling.

Well, if he’s wrong, then so are recent surveys by organizations that reporters typically tout rather than smear.

A YouGov poll commissioned by The Economist found that adults 18-29 were the most supportive of student groups using violence to stop campus speakers “known for making offensive and hurtful statements.”

Conducted before Attorney General Jeff Sessions promised to join the legal fight against campus speech zones, the survey found 14 percent agree “strongly” or “somewhat” with the use of violence. The second-largest percentage is “not sure” at 19 percent.

A slight majority of students (55 percent) disagree “strongly” and 12 percent disagree “somewhat.” (Read the release here.)

Villasenor emailed The College Fix to note that the YouGov survey results were only a couple percentage points away from his, if the “not sure” responses were removed (keeping in mind the survey populations – undergraduates and 18-29s – were not identical):

[If] you look at the responses among people in the Yougov survey who either agreed or disagreed in that question, the agree percentage in the 18-29 age group is a bit over 17% of that subset of responses (obtained by dividing 14 by 81). So, that’s close to the 19% number I observed.

Another way to look at the YouGov survey is to note that “fully a third” of respondents did not “affirmatively” disagree with the use of violence against speech, he said.

The split is much closer on the question of “loudly and repeatedly shouting so that the audience cannot hear the speaker”: 52 percent disagree, 28 percent agree and 20 percent aren’t sure. In other words, nearly half can’t affirmatively reject shoutdowns.

MORE: Survey that shows students endorse violence is not ‘junk science’

A national survey of 800 undergraduates, conducted over several days in mid-September, found similar results as well.

Sponsored by the William F. Buckley, Jr. Program at Yale, the McLaughlin & Associates survey was “stratified” to reflect the “actual demographic composition” of undergraduates:

By a two to one margin, students do not believe that physical violence can be justified to prevent a person from using hate speech. However, 30% of students believe that violence can be justified.

Without a “not sure” option to choose, students were even more supportive of shoutdowns in this survey: 38 percent said it was “sometimes appropriate” to do so. There wasn’t even a “hate speech” or “offensive” condition in this question.

The new YouGov survey of student attitudes on speech commissioned by the Foundation for Individual Rights in Education, meanwhile, asked a different question: What would you do in response to a guest speaker whose “ideas and opinions I strongly disagree with”?

Reflecting the split in American minds between actions of Congress in general and their own member in particular, students were reluctant to say they personally might “use violent or disruptive actions” to stop the speaker (only 1 percent). The response was 2 percent on “I might” make “noise” to prevent the speaker from being heard.

MORE: Large numbers of students ambivalent about, hostile to free speech

What do all these surveys have in common? Not just similar percentages of students who don’t oppose someone using violence or shouting down controversial speakers, but their methodology.

All are based on self-selected groups who choose to participate in online surveys – exactly what Villasenor was vilified for doing.

The gold standard for polling remains probability surveys, where any given American has an equally random chance of being surveyed. They are practically impossible to accurately conduct in 2017. (This would require a “magic phone book,” as Washington Post columnist Catherine Rampell put it.)

The new surveys also include margin of error and/or confidence level, whose citation in Villasenor’s survey was also criticized. Those are supposed to be reserved for probability surveys.

McLaughlin & Associates calculated a 95 percent confidence level “in order to produce an error estimate of+/- 3.4% for the 800 respondents,” YouGov (for FIRE) “estimated margin of error of +/– 3.1 percentage points at the 95% confidence level,” and YouGove (for The Economist) came up with plus or minus three percentage points, “adjusted for weighting.”

Villasenor told The Fix:

My point is that, more broadly, if we are going to do surveys reaching large numbers of college students (and other “digital natives”) we’re going to need to be open to using online opt-in panels. And, we’re going to need to be open to the idea that if proper care is taken when applying weighting to the data to obtain the best match we can to the population of interest, it’s not unreasonable to say something about the confidence levels associated with the results.

MORE: Pollsters getting more comfortable with nonprobability surveys

IMAGE: Stacie DaPonte/Flickr

Like The College Fix on Facebook / Follow us on Twitter