N.S.A. Collecting Millions of Faces From Web Images
N.S.A. Collecting Millions of Faces From Web Images
By JAMES RISEN and LAURA POITRAS MAY 31, 2014
The National Security Agency is harvesting huge numbers
of images of people from communications that it intercepts through its global
surveillance operations for use in sophisticated facial recognition programs,
according to top-secret documents.
The spy agency’s reliance on facial recognition
technology has grown significantly over the last four years as the agency has
turned to new software to exploit the flood of images included in emails, text
messages, social media, videoconferences and other communications, the N.S.A.
documents reveal. Agency officials believe that technological advances could
revolutionize the way that the N.S.A. finds intelligence targets around the
world, the documents show. The agency’s ambitions for this highly sensitive
ability and the scale of its effort have not previously been disclosed.
The agency intercepts “millions of images per day” —
including about 55,000 “facial recognition quality images” — which translate
into “tremendous untapped potential,” according to 2011 documents obtained from
the former agency contractor Edward J. Snowden.
While once focused on written and oral communications,
the N.S.A. now considers facial images, fingerprints and other identifiers just
as important to its mission of tracking suspected terrorists and other
intelligence targets, the documents show.
“It’s not just the traditional communications we’re
after: It’s taking a full-arsenal approach that digitally exploits the clues a
target leaves behind in their regular activities on the net to compile
biographic and biometric information” that can help “implement precision
targeting,” noted a 2010 document.
One N.S.A. PowerPoint presentation from 2011, for
example, displays several photographs of an unidentified man — sometimes
bearded, other times clean-shaven — in different settings, along with more than
two dozen data points about him. These include whether he was on the
Transportation Security Administration no-fly list, his passport and visa
status, known associates or suspected terrorist ties, and comments made about
him by informants to American intelligence agencies.
It is not clear how many people around the world, and how
many Americans, might have been caught up in the effort. Neither federal
privacy laws nor the nation’s surveillance laws provide specific protections
for facial images. Given the N.S.A.’s foreign intelligence mission, much of the
imagery would involve people overseas whose data was scooped up through cable
taps, Internet hubs and satellite transmissions.
Because the agency considers images a form of
communications content, the N.S.A. would be required to get court approval for
imagery of Americans collected through its surveillance programs, just as it
must to read their emails or eavesdrop on their phone conversations, according
to an N.S.A. spokeswoman. Cross-border communications in which an American
might be emailing or texting an image to someone targeted by the agency overseas
could be excepted.
Civil-liberties advocates and other critics are concerned
that the power of the improving technology, used by government and industry,
could erode privacy. “Facial recognition can be very invasive,” said Alessandro
Acquisti, a researcher on facial recognition technology at Carnegie Mellon
University. “There are still technical limitations on it, but the computational
power keeps growing, and the databases keep growing, and the algorithms keep
improving.”
State and local law enforcement agencies are relying on a
wide range of databases of facial imagery, including driver’s licenses and
Facebook, to identify suspects. The F.B.I. is developing what it calls its
“next generation identification” project to combine its automated fingerprint
identification system with facial imagery and other biometric data.
The State Department has what several outside experts say
could be the largest facial imagery database in the federal government, storing
hundreds of millions of photographs of American passport holders and foreign
visa applicants. And the Department of Homeland Security is funding pilot
projects at police departments around the country to match suspects against
faces in a crowd.
The N.S.A., though, is unique in its ability to match
images with huge troves of private communications.
“We would not be doing our job if we didn’t seek ways to
continuously improve the precision of signals intelligence activities — aiming
to counteract the efforts of valid foreign intelligence targets to disguise
themselves or conceal plans to harm the United States and its allies,” said
Vanee M. Vines, the agency spokeswoman.
She added that the N.S.A. did not have access to
photographs in state databases of driver’s licenses or to passport photos of
Americans, while declining to say whether the agency had access to the State
Department database of photos of foreign visa applicants. She also declined to
say whether the N.S.A. collected facial imagery of Americans from Facebook and
other social media through means other than communications intercepts.
“The government and the private sector are both investing
billions of dollars into face recognition” research and development, said
Jennifer Lynch, a lawyer and expert on facial recognition and privacy at the
Electronic Frontier Foundation in San Francisco. “The government leads the way
in developing huge face recognition databases, while the private sector leads
in accurately identifying people under challenging conditions.”
Ms. Lynch said a handful of recent court decisions could
lead to new constitutional protections for the privacy of sensitive face
recognition data. But she added that the law was still unclear and that
Washington was operating largely in a legal vacuum.
Laura Donohue, the director of the Center on National
Security and the Law at Georgetown Law School, agreed. “There are very few
limits on this,” she said.
An excerpt of a document obtained by Edward J. Snowden, a
former contractor with the National Security Agency, referring to the agency’s
use of images in intelligence gathering.
Congress has largely ignored the issue. “Unfortunately,
our privacy laws provide no express protections for facial recognition data,”
said Senator Al Franken, Democrat of Minnesota, in a letter in December to the
head of the National Telecommunications and Information Administration, which
is now studying possible standards for commercial, but not governmental, use.
Facial recognition technology can still be a clumsy tool.
It has difficulty matching low-resolution images, and photographs of people’s
faces taken from the side or angles can be impossible to match against mug
shots or other head-on photographs.
Dalila B. Megherbi, an expert on facial recognition
technology at the University of Massachusetts at Lowell, explained that “when
pictures come in different angles, different resolutions, that all affects the
facial recognition algorithms in the software.”
That can lead to errors, the documents show. A 2011
PowerPoint showed one example when Tundra Freeze, the N.S.A.’s main in-house
facial recognition program, was asked to identify photos matching the image of
a bearded young man with dark hair. The document says the program returned 42
results, and displays several that were obviously false hits, including one of
a middle-age man.
Similarly, another 2011 N.S.A. document reported that a
facial recognition system was queried with a photograph of Osama bin Laden.
Among the search results were photos of four other bearded men with only slight
resemblances to Bin Laden.
But the technology is powerful. One 2011 PowerPoint
showed how the software matched a bald young man, shown posing with another man
in front of a water park, with another photo where he has a full head of hair,
wears different clothes and is at a different location.
It is not clear how many images the agency has acquired.
The N.S.A. does not collect facial imagery through its bulk metadata collection
programs, including that involving Americans’ domestic phone records,
authorized under Section 215 of the Patriot Act, according to Ms. Vines.
The N.S.A. has accelerated its use of facial recognition
technology under the Obama administration, the documents show, intensifying its
efforts after two intended attacks on Americans that jarred the White House.
The first was the case of the so-called underwear bomber, in which Umar Farouk
Abdulmutallab, a Nigerian, tried to trigger a bomb hidden in his underwear
while flying to Detroit on Christmas in 2009. Just a few months later, in May
2010, Faisal Shahzad, a Pakistani-American, attempted a car bombing in Times
Square.
The agency’s use of facial recognition technology goes
far beyond one program previously reported by The Guardian, which disclosed
that the N.S.A. and its British counterpart, General Communications
Headquarters, have jointly intercepted webcam images, including sexually
explicit material, from Yahoo users.
The N.S.A. achieved a technical breakthrough in 2010 when
analysts first matched images collected separately in two databases — one in a
huge N.S.A. database code-named Pinwale, and another in the government’s main
terrorist watch list database, known as Tide — according to N.S.A. documents.
That ability to cross-reference images has led to an explosion of analytical
uses inside the agency. The agency has created teams of “identity intelligence”
analysts who work to combine the facial images with other records about
individuals to develop comprehensive portraits of intelligence targets.
The agency has developed sophisticated ways to integrate
facial recognition programs with a wide range of other databases. It intercepts
video teleconferences to obtain facial imagery, gathers airline passenger data
and collects photographs from national identity card databases created by
foreign countries, the documents show. They also note that the N.S.A. was
attempting to gain access to such databases in Pakistan, Saudi Arabia and Iran.
The documents suggest that the agency has considered
getting access to iris scans through its phone and email surveillance programs.
But asked whether the agency is now doing so, officials declined to comment.
The documents also indicate that the N.S.A. collects iris scans of foreigners
through other means.
In addition, the agency was working with the C.I.A. and
the State Department on a program called Pisces, collecting biometric data on
border crossings from a wide range of countries.
One of the N.S.A.’s broadest efforts to obtain facial
images is a program called Wellspring, which strips out images from emails and
other communications, and displays those that might contain passport images. In
addition to in-house programs, the N.S.A. relies in part on commercially
available facial recognition technology, including from PittPatt, a small
company owned by Google, the documents show.
The N.S.A. can now compare spy satellite photographs with
intercepted personal photographs taken outdoors to determine the location. One
document shows what appear to be vacation photographs of several men standing
near a small waterfront dock in 2011. It matches their surroundings to a spy
satellite image of the same dock taken about the same time, located at what the
document describes as a militant training facility in Pakistan.
A version of this article appears in print on June 1,
2014, on page A1 of the New York edition with the headline: N.S.A. Collecting
Millions of Faces From Web Images.
Comments
Post a Comment