Global exam grading algorithm under fire for suspected bias

Global exam grading algorithm under fire for suspected bias
Students and experts say the formula the International Baccalaureate program used to generate grades may be discriminatory.
By Avi Asher-Schapiro Tuesday, 21 July 2020 15:38 GMT
NEW YORK, July 21 (Thomson Reuters Foundation) - When Colorado high school student Isabel Castaneda checked her final grades for the International Baccalaureate program in July, she was shocked.
Despite being one of the top-ranking students in her public school, she failed a number of courses — including high-level Spanish, her native language.
The International Baccalaureate (IB) program - a global standard of educational testing that also allows U.S. high-school students to obtain college credit - cancelled its exams in May, due to the coronavirus pandemic.
Instead of sitting final exams, which usually account for the majority of students' scores, students were assigned their marks based on a mathematical "awarding model", as described by the IB program.
"I come from a low-income family - and my entire last two years were driven by the goal of getting as many college credits as I could to save money on school," Castaneda said in a phone interview. "When I saw those scores, my heart sank."
The COVID-19 pandemic has disrupted exams all over the world, and educational institutions have adapted in a range of ways, from moving tests online to asking students to wear protective gear during testing.
Relying on an algorithm to help determine results comes with its own specific risks, researchers warn.
Depending on the kinds of data the model considers, and how it makes predictions, it has the potential to reproduce - or even exacerbate - existing patterns of inequality for low-income and minority students, they say.
About 160,000 students take IB courses every year, including nearly 90,000 in the United States - and almost 60% of public schools that offer IB in the U.S. are "Title I" schools, with significant low-income student populations, according to the program.
"The choice to use a statistical model in place of a traditional examination warrants several concerns," said Esther Rolf, a PhD candidate at the University of California-Berkeley, who studies algorithmic fairness.
"Using historical records ... often leads to bias against individuals from historical underprivileged groups."
IB spokesman Dan Rene shared with the Thomson Reuters Foundation an explanation of the model which relied on three main components.
They were coursework, predictions teachers made about how students would perform on the exam, and the "school context", which included historical data on predicted results, and performance on past coursework for each subject.
"This process was subjected to rigorous testing by educational statistic specialists," the spokesman said in an emailed statement.
IB also released a statistical May bulletin showing that average scores in 2020 were in line with previous years, and said it had a process to "review extraordinary cases".
In previous years, students' grades have been generated by combining final exams graded by IB and coursework marked by their teachers - which the IB spot-checked, according to its website.
Teachers also make predictions about their students' final grades, which students can use to secure provisional college admissions before taking their final exams.
"[A] school's own record was built into the model" by using "historical data to model predicted grade accuracy, as well as the record of the school to do better or worse on examinations compared with coursework," the IB's statement noted.
Although the IB insists its model is not an algorithm, experts say it is.


Popular posts from this blog

Report: World’s 1st remote brain surgery via 5G network performed in China

Visualizing The Power Of The World's Supercomputers

BMW traps alleged thief by remotely locking him in car