AI programs exhibit racial and gender biases, research reveals
AI programs exhibit racial and gender biases, research
reveals
Thursday 13 April 2017 14.00 EDT Last modified on
Thursday 13 April 2017 17.00 EDT
An artificial intelligence tool that has revolutionised
the ability of computers to interpret everyday language has been shown to
exhibit striking gender and racial biases.
The findings raise the spectre of existing social
inequalities and prejudices being reinforced in new and unpredictable ways as
an increasing number of decisions affecting our everyday lives are ceded to
automatons.
In the past few years, the ability of programs such as
Google Translate to interpret language has improved dramatically. These gains
have been thanks to new machine learning techniques and the availability of
vast amounts of online text data, on which the algorithms can be trained.
However, as machines are getting closer to acquiring
human-like language abilities, they are also absorbing the deeply ingrained
biases concealed within the patterns of language use, the latest research
reveals.
Joanna Bryson, a computer scientist at the University of
Bath and a co-author, said: “A lot of people are saying this is showing that AI
is prejudiced. No. This is showing we’re prejudiced and that AI is learning
it.”
But Bryson warned that AI has the potential to reinforce
existing biases because, unlike humans, algorithms may be unequipped to
consciously counteract learned biases. “A danger would be if you had an AI
system that didn’t have an explicit part that was driven by moral ideas, that
would be bad,” she said.
The research, published in the journal Science, focuses
on a machine learning tool known as “word embedding”, which is already
transforming the way computers interpret speech and text. Some argue that the
natural next step for the technology may involve machines developing human-like
abilities such as common sense and logic.
“A major reason we chose to study word embeddings is that
they have been spectacularly successful in the last few years in helping
computers make sense of language,” said Arvind Narayanan, a computer scientist
at Princeton University and the paper’s senior author.
The approach, which is already used in web search and
machine translation, works by building up a mathematical representation of
language, in which the meaning of a word is distilled into a series of numbers
(known as a word vector) based on which other words most frequently appear
alongside it. Perhaps surprisingly, this purely statistical approach appears to
capture the rich cultural and social context of what a word means in the way
that a dictionary definition would be incapable of.
For instance, in the mathematical “language space”, words
for flowers are clustered closer to words linked to pleasantness, while words
for insects are closer to words linked to unpleasantness, reflecting common
views on the relative merits of insects versus flowers.
The latest paper shows that some more troubling implicit
biases seen in human psychology experiments are also readily acquired by
algorithms. The words “female” and “woman” were more closely associated with
arts and humanities occupations and with the home, while “male” and “man” were
closer to math and engineering professions.
And the AI system was more likely to associate European
American names with pleasant words such as “gift” or “happy”, while African
American names were more commonly associated with unpleasant words.
The findings suggest that algorithms have acquired the
same biases that lead people (in the UK and US, at least) to match pleasant
words and white faces in implicit association tests.
These biases can have a profound impact on human
behaviour. One previous study showed that an identical CV is 50% more likely to
result in an interview invitation if the candidate’s name is European American
than if it is African American. The latest results suggest that algorithms,
unless explicitly programmed to address this, will be riddled with the same
social prejudices.
“If you didn’t believe that there was racism associated
with people’s names, this shows it’s there,” said Bryson.
The machine learning tool used in the study was trained
on a dataset known as the “common crawl” corpus – a list of 840bn words that
have been taken as they appear from material published online. Similar results
were found when the same tools were trained on data from Google News.
Sandra Wachter, a researcher in data ethics and
algorithms at the University of Oxford, said: “The world is biased, the
historical data is biased, hence it is not surprising that we receive biased
results.”
Rather than algorithms representing a threat, they could
present an opportunity to address bias and counteract it where appropriate, she
added.
“At least with algorithms, we can potentially know when
the algorithm is biased,” she said. “Humans, for example, could lie about the
reasons they did not hire someone. In contrast, we do not expect algorithms to
lie or deceive us.”
However, Wachter said the question of how to eliminate
inappropriate bias from algorithms designed to understand language, without
stripping away their powers of interpretation, would be challenging.
“We can, in principle, build systems that detect biased
decision-making, and then act on it,” said Wachter, who along with others has
called for an AI watchdog to be established.
“This is a very complicated task, but it is a
responsibility that we as society should not shy away from.”
Comments
Post a Comment