USC’s David Quinn explores how rubrics could help mitigate racial bias in teachers’ evaluations of student performance with the SDP network.
Conversations about racial bias in policing have also ignited conversations about racial bias in teaching. What is it? Can it be measured? Can it be addressed?
Racial bias in teachers describes the underlying implicit and explicit stereotypes teachers can hold toward their students that may show up in the way they disproportionately behave with students of various racial identities. It’s scarcely news that both the automatic associations people absorb from society and more explicit beliefs about people of different races can affect teachers’ judgements of student aptitude, often unintentionally.
Teacher bias in the classroom matters. A range of evidence has revealed that bias can lead to discriminatory discipline and affect a student’s present and future test scores, course taking decisions—students who experience downward bias in their evaluations, for example, are less likely to enroll in more rigorous courses—and even their choice of fields of study. And while we know that teachers show bias, and we know that these biases matter for student outcomes, we know less about what we can actually do about it.
Yet a recent study by Dr. David Quinn, Assistant Professor of Education in the K-12 Education Policy concentration at USC’s Rossier School of Education, provides experimental evidence that evaluation rubrics may be a powerful tool in mitigating bias and improving fairness in the way we measure student learning.
“There’s some recent experimental and quasi-experimental work that suggests that rubrics may help mitigate bias in teachers,” offered Quinn. As he notes in his paper, clear and specific evaluation criteria may be a simple policy lever to reduce bias in scoring. Yet so far, this is only theory. “The problem is, this research has taken place outside of the U.S., a country with a specific historical precedent of Black/White bias.”
Interestingly, rubrics have gained widespread adoption in principal observation of teachers over the past 15 years or so. This tool is widely thought to help mitigate principal bias and give these leaders a more impartial method of assessing teacher performance and give actionable feedback for improvement. This method provided guardrails that protect teachers from discriminatory evaluation. Yet the widespread adoption of this tactic by principals hasn’t necessarily translated to students.
Through a web-based survey, Quinn’s study measured implicit biases--automatic associations that people acquired by living in a racist society--and explicit biases--beliefs that people hold about the aptitude of students of a particular race--in 1,549 American K-12 teachers. Respondents were randomly assigned to evaluate one of two versions of a writing sample and were told it was by a student in the fall of second grade. The writing sample, which was a response to a prompt to write about the student’s weekend, differed in the names the student listed for their brother. In one version, the student’s brother was assigned the name “Dashawn,” signaling a Black author per the Dunbar and Leavitt list of racially distinct names. In the other version, the brother’s name was Connor.
After seeing one of these two versions, teachers were asked to evaluate the sample using one of two methods: either a subjective question asking the teachers how they would rate the student’s writing on a scale ranging from far below grade level to far above grade level or a question asking the teachers to use a rubric to determine how well the writing sample recounted an event.
The results? Teachers who saw the Dashawn version were about five percentage points less likely to rate the sample as grade level or above, showing clear evidence of racial bias. The teachers who used the rubric, however, showed no evidence of bias. What’s more, White teachers showed bias by about eight percentage points, while non-White teachers showed almost no bias.
“There’s some recent suggestive evidence that rubrics may help mitigate racial bias in teachers’ evaluations of students,” offered Quinn. As he notes in his paper, clear and specific evaluation criteria may be a simple policy lever to reduce bias in scoring. “However, this has not been directly tested experimentally.”
In light of these findings, leaders would do well to ask themselves such exploratory questions about their systems for student assessment as:
-What rubrics do teachers in your education community use to assess writing?
-Does your data team look for evidence of bias in the grading of student work?
-Where can we find/introduce rubrics that might be useful to both educators and students?
To view the full discussion of this study and its findings, along with a discussion of another study about achievement gap discourse, check out David Quinn’s full webinar.