Data isn't just being collected from your phone. It's being used to score you
Data isn't just being
collected from your phone. It's being used to score you.
Harvey Rosenfield and Laura
Antonini, The Washington Post Published 2:44 pm CDT, Friday, July 31,
2020
Operating in the shadows of
the online marketplace, specialized tech companies you've likely never heard of
are tapping vast troves of our personal data to generate secret
"surveillance scores" - digital mug shots of millions of Americans -
that supposedly predict our future behavior. The firms sell their scoring
services to major businesses across the U.S. economy.
People with low scores can
suffer harsh consequences.
CoreLogic and TransUnion say
that scores they peddle to landlords can predict whether a potential tenant
will pay the rent on time, be able to "absorb rent increases," or
break a lease. Large employers use HireVue, a firm that generates an
"employability" score about candidates by analyzing "tens of
thousands of factors," including a person's facial expressions and voice
intonations. Other employers use Cornerstone's score, which considers where a
job prospect lives and which web browser they use to judge how successful they
will be at a job.
Brand-name retailers purchase
"risk scores" from Retail Equation to help make judgments about
whether consumers commit fraud when they return goods for refunds. Players in
the gig economy use outside firms such as Sift to score consumers'
"overall trustworthiness." Wireless customers predicted to be less
profitable are sometimes forced to endure longer customer service hold times.
Auto insurers raise premiums
based on scores calculated using information from smartphone apps that track
driving styles. Large analytics firms monitor whether we are likely to take our
medication based on our propensity to refill our prescriptions; pharmaceutical
companies, health-care providers and insurance companies can use those scores
to, among other things, "match the right patient investment level to the
right patients."
Surveillance scoring is the
product of two trends. First is the rampant (and mostly unregulated) collection
of every intimate detail about our lives, amassed by the nanosecond from
smartphones to cars, toasters to toys. This fire hose of data - most of which
we surrender voluntarily - includes our demographics, income, facial
characteristics, the sound of our voice, our precise location, shopping
history, medical conditions, genetic information, what we search for on the
Internet, the websites we visit, when we read an email, what apps we use and
how long we use them, and how often we sleep, exercise and the like.
The second trend driving
these scores is the arrival of technologies able to instantaneously crunch this
data: exponentially more powerful computers and high-speed communications
systems such as 5G, which lead to the scoring algorithms that use artificial
intelligence to rate all of us in some way.
The result: automated
decisions, based on each consumer's unique score, that are, as a practical
matter, irreversible.
That's because the entire
process - the scores themselves, as well as the data upon which they are based
- is concealed from us. It is mostly impossible to know when one has become the
casualty of a score, let alone whether a score is inaccurate, outdated or the product
of biased or discriminatory code programmed by a faceless software engineer.
There is no appeal.
Surveillance scoring bears a
faint resemblance to credit scoring in the 1960s. In that pre-computer era,
private investigators working for banks, retailers and insurance companies
tailed consumers and scoured newspapers for information about arrests,
promotions, sexual orientation, drinking habits and cleanliness to decide a
consumer's creditworthiness -- until Congress established rules in the 1970s giving
consumers the right to review and question their credit scores.
Today's data snoops obtain
infinitely more information about their targets, and in real time. And the
impact of surveillance scoring is far more pernicious.
The tech industry insists
that its every advance improves our lives. But that's a myth. Surveillance
scoring enables companies to cloak old-school discrimination in an aura of
technological infallibility and wonder.
Secret surveillance scores
create micro-markets in which some consumers are no longer welcome. They divide
Americans into "haves" and "have-nots," with the losers
relegated to the status of second-class citizens.
Consumers need a 21st-century
solution to this emergent threat. Congress, awash in tech money, is mired in an
outdated legal paradigm: "disclosure" of privacy policies and
"consent" via a click. No one pretends that these industrial age
contract law concepts will do anything to curb data larceny, let alone regulate
or bar secret surveillance scores.
We petitioned the Federal
Trade Commission to investigate and regulate surveillance scoring. The
commission's response? A blog post urging the firms that develop and apply
scores to regulate themselves.
This is only the latest
example of Washington's capitulation to the tech industry, whose continuous
loop of privacy violations, abject apologies and payment of inconsequential
penalties confirms that consumers cannot rely on the federal government for
protection.
Secret surveillance scoring
places us at the precipice of the "singularity," a dystopian turning
point after which machines will make judgments about humans that will determine
our fate. We either seize control of our future, or risk losing it.
Comments
Post a Comment