The tyranny of algorithms is part of our lives: soon they could rate everything we do
The tyranny of algorithms is part of our lives: soon they
could rate everything we do
Credit scores already control our finances. With personal
data being increasingly trawled, our politics and our friendships will be next
By John Harris Mon 5 Mar 2018 01.00 EST Last modified on
Mon 5 Mar 2018 03.42 EST
For the past couple of years a big story about the future
of China has been the focus of both fascination and horror. It is all about
what the authorities in Beijing call “social credit”, and the kind of
surveillance that is now within governments’ grasp. The official rhetoric is
poetic. According to the documents, what is being developed will “allow the
trustworthy to roam everywhere under heaven while making it hard for the discredited
to take a single step”.
As China moves into the newly solidified President Xi
Jinping era, the basic plan is intended to be in place by 2020. Some of it will
apply to businesses and officials, so as to address corruption and tackle such
high-profile issues as poor food hygiene. But other elements will be focused on
ordinary individuals, so that transgressions such as dodging transport fares
and not caring sufficiently for your parents will mean penalties, while living
the life of a good citizen will bring benefits and opportunities.
Online behaviour will inevitably be a big part of what is
monitored, and algorithms will be key to everything, though there remain doubts
about whether something so ambitious will ever come to full fruition. One of
the scheme’s basic aims is to use a vast amount of data to create individual
ratings, which will decide people’s access – or lack of it – to everything from
travel to jobs.
The Chinese notion of credit – or xinyong – has a
cultural meaning that relates to moral ideas of honesty and trust. There are up
to 30 local social credit pilots run by local authorities, in huge cities such
as Shanghai and Hangzhou and much smaller towns. Meanwhile, eight ostensibly
private companies have been trialling a different set of rating systems, which
seem to chime with the government’s controlling objectives.
The most high-profile system is Sesame Credit – created
by Ant Financial, an offshoot of the Chinese online retail giant Alibaba.
Superficially, it reflects the western definition of credit, and looks like a
version of the credit scores used all over the world, invented to belatedly
allow Chinese consumers the pleasures of buying things on tick, and manage the
transition to an economy in which huge numbers of people pay via smartphones.
But its reach runs wider.
Using a secret algorithm, Sesame credit constantly scores
people from 350 to 950, and its ratings are based on factors including
considerations of “interpersonal relationships” and consumer habits. Bluntly
put, being friends with low-rated people is bad news. Buying video games, for
example, gets you marked down. Participation is voluntary but easily secured,
thanks to an array of enticements. High scores unlock privileges such as being
able to rent a car without a deposit, and fast-tracked European visa
applications. There are also more romantic benefits: the online dating service
Baihe gives people with good scores prominence on its platforms.
Exactly how all this will relate to the version of social
credit eventually implemented is unclear: licences that might have enabled the
systems to be rolled out further ran out last year. There again, Ant Financial
has stated that it wants to “help build a social integrity system” – and the
existing public and private pilots have a similar sense of social control, and
look set to feed the same social divisions. If you are mouldering away towards
the bottom of the hierarchies, life will clearly be unpleasant. But if you
manage to be a high-flyer, the pleasures of fast-tracking and open doors will
be all yours, though even the most fleeting human interaction will give off the
crackle of status anxiety.
It would be easy to assume none of this could happen here
in the west. But the 21st century is not going to work like that. These days
credit reports and scores – put together by agencies whose reach into our lives
is mind-boggling – are used to judge job applications, thereby threatening to
lock people into financial problems. And in the midst of the great deluge of
personal data that comes from our online lives, there is every sign of these
methods being massively extended.
Three years ago Facebook patented a system of credit
rating that would consider the financial histories of people’s friends. Opaque
innovations known as e-scores are used by increasing numbers of companies to
target their marketing, while such outfits as the already infamous Cambridge Analytica
trawl people’s online activities so as to precisely target political messaging.
The tyranny of algorithms is now an inbuilt part of our lives.
These systems are sprawling, often randomly connected,
and often beyond logic. But viewed from another angle, they are also the
potential constituent parts of comprehensive social credit systems, awaiting
the moment at which they will be glued together. That point may yet come,
thanks to the ever-expanding reach of the internet. If our phones and debit cards
already leave a huge trail of data, the so-called internet of things is now
increasing our informational footprint at speed.
In the short term, the biggest consequences will arrive
in the field of insurance, where the collective pooling of risk is set to be
supplanted by models that focus tightly on individuals. Thanks to connected
devices, insurers could soon know how much television you watch, whether you always
obey traffic signals, and how well your household plumbing works. Already, car
insurance schemes offer lower premiums if people install tracking devices that
monitor their driving habits; and health insurance companies such as the
British firm Vitality offer deals based on access to data from fitness
trackers. In the near future, as with Sesame Credit, people will presumably
sign up for surveillance-based insurance in their droves because of such simple
incentives, and those squeamish about privacy may simply have to pay more. Many
people, of course, will simply be deemed impossible to protect.
Personal data and its endless uses form one of the most
fundamental issues of our time, which boils down to the relationship between
the individual and power, whether exercised by government or private
organisations. It speaks volumes that in Whitehall responsibility for such
things falls uncertainly between the culture secretary, Matt Hancock, whose
“digital” brief includes what the official blurb limply calls “telecommunications
and online”, the Treasury and an under-secretary of state in the business
department, Andrew Griffiths, whose portfolio takes in “consumers”.
That is absurd, and it may yet play its part in our rapid
passage into a future that could materialise in both east and west, in which we
do what we’re told, avoid the company of undesirables – and endlessly panic
about how the algorithms will rate us tomorrow.
Comments
Post a Comment