The tyranny of algorithms is part of our lives: soon they could rate everything we do

The tyranny of algorithms is part of our lives: soon they could rate everything we do

Credit scores already control our finances. With personal data being increasingly trawled, our politics and our friendships will be next

By John Harris Mon 5 Mar 2018 01.00 EST Last modified on Mon 5 Mar 2018 03.42 EST

For the past couple of years a big story about the future of China has been the focus of both fascination and horror. It is all about what the authorities in Beijing call “social credit”, and the kind of surveillance that is now within governments’ grasp. The official rhetoric is poetic. According to the documents, what is being developed will “allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step”.

As China moves into the newly solidified President Xi Jinping era, the basic plan is intended to be in place by 2020. Some of it will apply to businesses and officials, so as to address corruption and tackle such high-profile issues as poor food hygiene. But other elements will be focused on ordinary individuals, so that transgressions such as dodging transport fares and not caring sufficiently for your parents will mean penalties, while living the life of a good citizen will bring benefits and opportunities.

Online behaviour will inevitably be a big part of what is monitored, and algorithms will be key to everything, though there remain doubts about whether something so ambitious will ever come to full fruition. One of the scheme’s basic aims is to use a vast amount of data to create individual ratings, which will decide people’s access – or lack of it – to everything from travel to jobs.

The Chinese notion of credit – or xinyong – has a cultural meaning that relates to moral ideas of honesty and trust. There are up to 30 local social credit pilots run by local authorities, in huge cities such as Shanghai and Hangzhou and much smaller towns. Meanwhile, eight ostensibly private companies have been trialling a different set of rating systems, which seem to chime with the government’s controlling objectives.

The most high-profile system is Sesame Credit – created by Ant Financial, an offshoot of the Chinese online retail giant Alibaba. Superficially, it reflects the western definition of credit, and looks like a version of the credit scores used all over the world, invented to belatedly allow Chinese consumers the pleasures of buying things on tick, and manage the transition to an economy in which huge numbers of people pay via smartphones. But its reach runs wider.

Using a secret algorithm, Sesame credit constantly scores people from 350 to 950, and its ratings are based on factors including considerations of “interpersonal relationships” and consumer habits. Bluntly put, being friends with low-rated people is bad news. Buying video games, for example, gets you marked down. Participation is voluntary but easily secured, thanks to an array of enticements. High scores unlock privileges such as being able to rent a car without a deposit, and fast-tracked European visa applications. There are also more romantic benefits: the online dating service Baihe gives people with good scores prominence on its platforms.

Exactly how all this will relate to the version of social credit eventually implemented is unclear: licences that might have enabled the systems to be rolled out further ran out last year. There again, Ant Financial has stated that it wants to “help build a social integrity system” – and the existing public and private pilots have a similar sense of social control, and look set to feed the same social divisions. If you are mouldering away towards the bottom of the hierarchies, life will clearly be unpleasant. But if you manage to be a high-flyer, the pleasures of fast-tracking and open doors will be all yours, though even the most fleeting human interaction will give off the crackle of status anxiety.

It would be easy to assume none of this could happen here in the west. But the 21st century is not going to work like that. These days credit reports and scores – put together by agencies whose reach into our lives is mind-boggling – are used to judge job applications, thereby threatening to lock people into financial problems. And in the midst of the great deluge of personal data that comes from our online lives, there is every sign of these methods being massively extended.

Three years ago Facebook patented a system of credit rating that would consider the financial histories of people’s friends. Opaque innovations known as e-scores are used by increasing numbers of companies to target their marketing, while such outfits as the already infamous Cambridge Analytica trawl people’s online activities so as to precisely target political messaging. The tyranny of algorithms is now an inbuilt part of our lives.

These systems are sprawling, often randomly connected, and often beyond logic. But viewed from another angle, they are also the potential constituent parts of comprehensive social credit systems, awaiting the moment at which they will be glued together. That point may yet come, thanks to the ever-expanding reach of the internet. If our phones and debit cards already leave a huge trail of data, the so-called internet of things is now increasing our informational footprint at speed.

In the short term, the biggest consequences will arrive in the field of insurance, where the collective pooling of risk is set to be supplanted by models that focus tightly on individuals. Thanks to connected devices, insurers could soon know how much television you watch, whether you always obey traffic signals, and how well your household plumbing works. Already, car insurance schemes offer lower premiums if people install tracking devices that monitor their driving habits; and health insurance companies such as the British firm Vitality offer deals based on access to data from fitness trackers. In the near future, as with Sesame Credit, people will presumably sign up for surveillance-based insurance in their droves because of such simple incentives, and those squeamish about privacy may simply have to pay more. Many people, of course, will simply be deemed impossible to protect.

Personal data and its endless uses form one of the most fundamental issues of our time, which boils down to the relationship between the individual and power, whether exercised by government or private organisations. It speaks volumes that in Whitehall responsibility for such things falls uncertainly between the culture secretary, Matt Hancock, whose “digital” brief includes what the official blurb limply calls “telecommunications and online”, the Treasury and an under-secretary of state in the business department, Andrew Griffiths, whose portfolio takes in “consumers”.

That is absurd, and it may yet play its part in our rapid passage into a future that could materialise in both east and west, in which we do what we’re told, avoid the company of undesirables – and endlessly panic about how the algorithms will rate us tomorrow.


Popular posts from this blog

Report: World’s 1st remote brain surgery via 5G network performed in China

Visualizing The Power Of The World's Supercomputers

BMW traps alleged thief by remotely locking him in car