Uh-oh: Silicon Valley is building a Chinese-style social credit system
Uh-oh: Silicon Valley is building a Chinese-style social
credit system
In China, scoring citizens’ behavior is official government
policy. U.S. companies are increasingly doing something similar, outside the
law.
In place since 2014, the social credit system is a
work in progress that could evolve by next year into a single, nationwide point
system for all Chinese citizens, akin to a financial credit score. It aims to
punish for transgressions that can include membership in or support for the
Falun Gong or Tibetan Buddhism, failure to pay debts, excessive video gaming,
criticizing the government, late payments, failing to sweep the sidewalk in
front of your store or house, smoking or playing loud music on trains,
jaywalking, and other actions deemed illegal or unacceptable by the Chinese
government.
It can also award points for charitable donations or
even taking one’s own parents to the doctor.
Punishments can be harsh, including bans on leaving
the country, using public transportation, checking into hotels, hiring for
high-visibility jobs, or acceptance of children to private schools. It can also
result in slower internet connections and social stigmatization in the form of
registration on a public blacklist.
China’s social credit system has been characterized in
one pithy tweet as “authoritarianism, gamified.”
In China, Your
Credit Score Is Now Affected By Your Political Opinions - And Your Friends'
Politi...
At present, some
parts of the social credit system are in force nationwide and others are local
and limited (there are 40 or so pilot projects operated by local governments
and at least six run by tech giants like Alibaba and Tencent).
Beijing maintains two nationwide lists, called the
blacklist and the red list—the former consisting of people who have
transgressed, and the latter people who have stayed out of trouble (a “red
list” is the Communist version of a white list.) These lists are publicly
searchable on a government website called China Credit.
The Chinese government also shares lists with
technology platforms. So, for example, if someone criticizes the government on
Weibo, their kids might be ineligible for acceptance to an elite school.
Public shaming is also part of China’s social credit
system. Pictures of blacklisted people in one city were shown between videos on
TikTok in a trial, and the addresses of blacklisted citizens were shown on a
map on WeChat.
Some Western press reports imply that the Chinese
populace is suffocating in a nationwide Skinner box of oppressive behavioral
modification. But some Chinese are unaware that it even exists. And many others
actually like the idea. One survey found
that 80% of Chinese citizens surveyed either somewhat or strongly approve of
social credit system.
IT CAN HAPPEN HERE
Many Westerners are disturbed by what they read about
China’s social credit system. But such systems, it turns out, are not unique to
China. A parallel system is developing in the United States, in part as the
result of Silicon Valley and technology-industry user policies, and in part by
surveillance of social media activity by private companies.
Here are some of the elements of America’s growing
social credit system.
INSURANCE COMPANIES
The New York State Department of Financial Services announced
earlier this year that life insurance companies can base
premiums on what they find in your social media posts. That Instagram pic
showing you teasing a grizzly bear at Yellowstone with a martini in one hand, a
bucket of cheese fries in the other, and a cigarette in your mouth, could cost
you. On the other hand, a Facebook post showing you doing yoga might save you
money. (Insurance companies have to demonstrate that social media evidence
points to risk, and not be based on discrimination of any kind—they can’t use
social posts to alter premiums based on race or disability, for example.)
The use of social media is an extension of the
lifestyle questions typically asked when applying for life insurance, such as
questions about whether you engage in rock climbing or other adventure sports.
Saying “no,” but then posting pictures of yourself free-soloing El Capitan,
could count as a “yes.”
PATRONSCAN
A company called PatronScan sells three products—kiosk,
desktop, and handheld systems—designed to help bar and restaurant owners manage
customers. PatronScan is a subsidiary of the Canadian software company Servall
Biometrics, and its products are now on sale in the United States, Canada,
Australia, and the United Kingdom.
PatronScan helps spot fake IDs—and troublemakers. When
customers arrive at a PatronScan-using bar, their ID is scanned. The company
maintains a list of objectionable customers designed to protect venues from
people previously removed for “fighting, sexual assault, drugs, theft, and
other bad behavior,” according to its website. A “public” list is shared among
all PatronScan customers. So someone who’s banned by one bar in the U.S. is
potentially banned by all the bars in the U.S., the U.K., and Canada that use
the PatronScan system for up to a year. (PatronScan Australia keeps a separate
system.)
Judgment about what kind of behavior qualifies for inclusion
on a PatronScan list is up to the bar owners and managers. Individual bar
owners can ignore the ban, if they like. Data on non-offending customers is
deleted in 90 days or less. Also: PatronScan enables bars to keep a “private”
list that is not shared with other bars, but on which bad customers can be kept
for up to five years.
PatronScan does have an “appeals” process, but it’s up
to the company to grant or deny those appeals.
UBER AND AIRBNB
Thanks to the sharing economy, the options for travel
have been extended far beyond taxis and hotels. Uber and Airbnb are leaders in
providing transportation and accommodation for travelers. But there are many
similar ride-sharing and peer-to-peer accommodations companies providing
similar services.
Airbnb—a major provider of travel accommodation and
tourist activities—bragged in March that it now has more than 6 million
listings in its system. That’s why a ban from Airbnb can limit travel options.
Airbnb can disable your account for life for any
reason it chooses, and it reserves the right to not tell you the reason. The
company’s canned message includes the assertion that “This decision is
irreversible and will affect any duplicated or future accounts. Please
understand that we are not obligated to provide an explanation for the action
taken against your account.” The ban can be based on something the host
privately tells Airbnb about something they believe you did while staying at
their property. Airbnb’s competitors have similar policies.
It’s now easy to get banned by Uber, too. Whenever you
get out of the car after an Uber ride, the app invites you to rate the driver.
What many passengers don’t know is that the driver now also gets an invitation
to rate you. Under a new policy announced
in May: If your average rating is “significantly below average,”
Uber will ban you from the service.
WHATSAPP
You can be banned from communications apps, too. For
example, you can be banned on WhatsApp if too many other users block you. You
can also get banned for sending spam, threatening messages, trying to hack or
reverse-engineer the WhatsApp app, or using the service with an unauthorized
app.
WhatsApp is small potatoes in the United States. But
in much of the world, it’s the main form of electronic communication. Not being
allowed to use WhatsApp in some countries is as punishing as not being allowed
to use the telephone system in America.
WHAT’S WRONG WITH SOCIAL CREDIT, ANYWAY?
Nobody likes antisocial, violent, rude, unhealthy,
reckless, selfish, or deadbeat behavior. What’s wrong with using new technology
to encourage everyone to behave?
The most disturbing attribute of a social credit
system is not that it’s invasive, but that it’s extralegal. Crimes are punished
outside the legal system, which means no presumption of innocence, no legal
representation, no judge, no jury, and often no appeal. In other words, it’s an
alternative legal system where the accused have fewer rights.
Social credit systems are an end-run around the pesky
complications of the legal system. Unlike China’s government policy, the social
credit system emerging in the U.S. is enforced by private companies. If the
public objects to how these laws are enforced, it can’t elect new rule-makers.
An increasing number of societal “privileges” related
to transportation, accommodations, communications, and the rates we pay for
services (like insurance) are either controlled by technology companies or
affected by how we use technology services. And Silicon Valley’s rules for
being allowed to use their services are getting stricter.
If current trends hold, it’s possible that in the
future a majority of misdemeanors and even some felonies will be punished not
by Washington, D.C., but by Silicon Valley. It’s a slippery slope away from
democracy and toward corporatocracy.
In other words, in the future, law enforcement may be
determined less by the Constitution and legal code, and more by end-user
license agreements.
Comments
Post a Comment