Student Evals Measure Gender Bias Better Than Teaching

Generally, faculty like to use scores on student evaluations of teaching (SET) as a strong factor in hiring decisions because it is an "objective" measure while so much of tenure and promotion rests in the gray area. Did she publish enough? Maybe. Did she do enough service? Maybe. Were her teaching evals good? Yes or no.

More recently, SET has come under fire. InsideHigherEd highlights a new paper that argues that SET are not only unreliable in evaluating teaching effectiveness but SET actually gauge students' gender bias and grade expectations better than they measure teaching. 

There’s mounting evidence suggesting that student evaluations of teaching are unreliable. But are these evaluations, commonly referred to as SET, so bad that they’re actually better at gauging students’ gender bias and grade expectations than they are at measuring teaching effectiveness? A new paper argues that’s the case, and that evaluations are biased against female instructors in particular in so many ways that adjusting them for that bias is impossible.

Moreover, the paper says, gender biases about instructors -- which vary by discipline, student gender and other factors -- affect how students rate even supposedly objective practices, such as how quickly assignments are graded. And these biases can be large enough to cause more effective instructors to get lower teaching ratings than instructors who prove less effective by other measures, according to the study based on analyses of data sets from one French and one U.S. institution.

Institutions would do well to heed the warning posed by these studies of gender bias and SET, especially in hiring decisions. 

Stark (a co-author of the study) said he expected class action lawsuits against universities that rely on these evaluations for employment decisions will start this year, and that there’s evidence to support such cases.

“Our analysis would support an argument that the use of SET has adverse impact on female instructors, at least in the two settings we examined,” he said. “Replication of this kind of experiment and analysis elsewhere would strengthen the argument. Eventually, lawsuits will lead universities to do the right thing, if only to mitigate financial risks.”