The big Facebook
outage offers a behind-the-scenes look at how the social network's AI 'sees'
your photos and interprets them for blind users
Jul. 3, 2019, 6:00 PM·
·
·
The tags sparked anger and
amusement on Twitter, as people shared how their photos were interpreted — or
misinterpreted.
·
·
On Twitter, people shared
some of their tags, which included the mundane ("one person, beard")
and the disconcerting ("people standing, hoes, and indoor").
·
·
An image outage across
Facebook, Instagram, and WhatsApp gives a behind-the-scenes
look at how Facebook's AI sees your photos.
The outage, which started on Wednesday and
affected Facebook's 1.5 billion-plus daily active users and rendered Instagram
all but unusable, stopped social-media images from loading and left in their
place descriptions like: "image may contain: table, plant, flower, and
outdoor" and "image may contain: tree, plant, sky."
(Or maybe that's just my feed.)
This
is the kind of thing a lot of people saw on Facebook, Instagram, and WhatsApp
instead of their actual photos.Screenshot/Matt Weinberger
These descriptions, or
tags, show how Facebook's artificial intelligence interprets images.
On Twitter, people shared screenshots of how
their photos were tagged. "To be fair, 'one person, beard' is pretty much
a spot-on description of me," Zack Whittaker, an editor at TechCrunch,
wrote.
Oh yeah! I forgot Facebook uses machine learning to tag
our photos with what it sees in the picture.
To be fair, "one person, beard" is pretty much a spot-on description of me.
To be fair, "one person, beard" is pretty much a spot-on description of me.
So, what's going on?
Facebook automatically scans all photos on the
social network with facial- and image-recognition software powered by AI to
detect who or what is being pictured.
This is then used in the company's accessibility efforts to
describe photos to people who are blind or otherwise visually impaired, and who
are accessing the site via a screen reader.
In short, Facebook uses machine learning to
automatically interpret photos, then reads this interpretation aloud to blind
users. The photo outage meant that we got an in-your-face look at those
interpretations, too.
At times, it appeared the AI incorrectly tagged
the photo, as in the case of the
Fortune reporter Danielle Abril's where the photo description
read: "5 people, including Danielle Abril, people smiling, people
standing, hoes and indoor." Hopefully there was a gardening tool in the
photo — we've reached out to Abril but did not immediately hear back.
So while there could be a
benign explanation here, it's still a stark reminder of just how much data
Facebook is gathering at all times, even when we don't realize it's happening.
Thanks to its increasingly sophisticated AI technology, Facebook can even
gather information from something as innocuous as a vacation photo.
"Once something is legible, of course, it
becomes easy to store, analyze, and extract data from. It's only when the
system breaks down, like today, that we realize it's happening at all,"
James Vincent of The Verge,
wrote.
Comments
Post a Comment