How Facebook’s Master Algorithm Powers the Social Network
How Facebook’s Master Algorithm Powers the Social Network
The algorithm behind Facebook’s News Feed, a “modular
layered cake,” extracts meaning from every post and photo
By Christopher Mims Oct. 22, 2017 8:00 a.m. ET
Instagram engineers faced a Herculean task in early 2016.
Fearing that people would miss the most important posts, Instagram’s leadership
asked the engineers to transform the chronological photo feed into a curated
list of posts based on users’ individual preferences.
Development of a similar algorithm for Facebook ’s News
Feed, which determines what 2 billion Facebook users see, required an enormous
investment of time by some of the world’s most highly compensated engineers.
At Instagram, three or four engineers got the job done in
less than five weeks, says Joaquin Candela, Facebook’s head of applied machine
learning. The team was able to clone the existing News Feed algorithm, then
tweak it to suit Instagram.
However much Instagram’s engineers tweaked it, the fact
that most of what powers Instagram came straight from Facebook’s News Feed
shows the dominance and success of this basic engine of social media. Think of
it—and the endless, modular chunks of AI that go into it—as Facebook’s master
algorithm (my words, not Facebook’s).
Mr. Candela recently explained how his team works with or
creates tools for almost every other engineering team at the company—and how AI
has become core to most things Facebook does.
If telling us what to look at next is Facebook’s raison
d’être, then the AI that enables that endless spoon-feeding of content is the
company’s most important, and sometimes most controversial, intellectual
property. A sorted, curated feed tuned for engagement is the product of a
device that may someday be viewed by historians as a milestone on par with the
steam engine.
Only this engine, built to capture human attention, has
shown itself to be exploitable by bad actors and possibly detrimental to our
democracy, even when it is functioning as advertised. This has prompted
congressional hearings for Facebook and other tech companies, scheduled for
November. Facebook has been a vessel for Russian influence and the spread of
fake news, and a potential cause for envy and unhappiness. The personalization
of content that Facebook’s master algorithm allows, and the hyperpartisan news
sites that have risen to feed it, have created, for many users, personalized
“filter bubbles” of what is essentially nonoverlapping reality.
At the same time, the company’s announcement that it is
hiring more humans to screen ads and filter content shows there is so much
essential to Facebook’s functionality that AI alone can’t accomplish.
AI algorithms are inherently black boxes whose workings
can be next to impossible to understand—even by many Facebook engineers. “If
you look at all the engineers at Facebook, more than one in four are users of
our AI platform,” says Mr. Candela. “But more than 70% [of those] aren’t
experts.”
How so many Facebook engineers can use its AI algorithms
without necessarily knowing how to build them, Mr. Candela says, is that the
system is “a very modular layered cake where you can plug in at any level you
want.” He adds, “The power of this is just hard to describe.” Pieces of that
platform are performing all kinds of “domain-specific” tasks across Facebook’s
properties, from translation to speech recognition.
Information Butler, or Time Vampire?
Every time one of Facebook’s two billion monthly users
opens the Facebook app, a personalization algorithm sorts through all the posts
that a person could theoretically see, and dishes up the fraction it thinks she
or he would like to see first. The system weighs hundreds of frequently updated
signals, says Mr. Candela. Without AI, many of these signals would be
impossible to analyze.
An example of updated signals would be Facebook’s recent
fight against clickbait—links to stories that are “misleading, sensational or
spammy.” Training the algorithm takes human labor: A team analyzed hundreds of
thousands of posts in 10 languages, flagging offending headlines that either
withheld information (“Here’s the one thing…”) or exaggerated (“…will blow your
mind”). The resulting system autonomously scans links, suppressing the ones
that match what it learned from the human-generated data.
Facebook’s master algorithm now also can extract
additional meaning from our posts and photos, Mr. Candela says.
The “recommendations” feature, for instance, allows a
person to ask what to check out on a trip to Barcelona. Because Facebook’s AI
actually “knows” what La Sagrada Familia is and where it is located, anyone who
recommends it in a comment will see it pop up on a map above the post.
These capabilities are versatile enough that Facebook
users have repurposed them in unexpected ways, Mr. Candela says. When Hurricane
Irma hit Florida, people used Facebook to build a map of stores with bottled
water for sale. The person who created the post activated the “recommendations”
feature; others added to it simply by commenting on the post with retailers’
addresses.
Mr. Candela says teams add new features to Facebook’s
master algorithm to “add value to social interactions.” Chief Executive Mark Zuckerberg
recently said the company’s goal was to “bring the world closer together.”
However it is phrased, it is measured in the way people
engage with Facebook’s apps and networks, whether that is increasing the number
of posts they like or comment on, or how useful they find machine-translated
posts, or how often they use M, Facebook’s Messenger-based smart assistant, Mr.
Candela says.
Time spent on Facebook’s various properties correlates
with the company’s revenue, and that number was going up at last report: In
April 2016, Facebook said it was capturing on average 50 minutes of every
American’s day, up from 40 minutes in July 2014.
The unstated assumption behind the work of Facebook’s
more than 20,000 employees is that getting people to use Facebook more is a
good thing. It is certainly hard to imagine a world without it, given how it
has become central to the way we connect, find news and keep up with friends
and family.
But given what we have learned over the past year, it is
worth asking whether the intentions of the hugely powerful Oz that is
Facebook’s master algorithm are ultimately benevolent or malign.
Appeared in the October 23, 2017, print edition as 'The
Algorithm Driving Facebook.'
Comments
Post a Comment