Swarms of A.I. Drones May Soon Patrol European Borders...
SWARMS
OF DRONES, PILOTED BY ARTIFICIAL INTELLIGENCE, MAY SOON PATROL EUROPE’S BORDERS
IMAGINE YOU’RE
HIKING through the woods near a border.
Suddenly, you hear a mechanical buzzing, like a gigantic bee. Two quadcopters
have spotted you and swoop in for a closer look. Antennae on both drones and on
a nearby autonomous ground vehicle pick up the radio frequencies coming from
the cell phone in your pocket. They send the signals to a central server, which
triangulates your exact location and feeds it back to the drones. The robots
close in.
Cameras and other sensors on the machines
recognize you as human and try to ascertain your intentions. Are you a threat?
Are you illegally crossing a border? Do you have a gun? Are you engaging in
acts of terrorism or organized crime? The machines send video feeds to their
human operator, a border guard in an office miles away, who checks the videos
and decides that you are not a risk. The border guard pushes a button, and the
robots disengage and continue on their patrol.
This is not science fiction. The European
Union is financing a project to develop drones piloted by artificial intelligence
and designed to autonomously patrol Europe’s borders. The drones will operate
in swarms, coordinating and corroborating information among fleets of
quadcopters, small fixed-wing airplanes, ground vehicles, submarines, and
boats. Developers of the project, known as Roborder, say the robots will be
able to identify humans and independently decide whether they represent a
threat. If they determine that you may have committed a crime, they will notify
border police.
President Donald Trump has used the specter
of criminals crossing the southern border to stir nationalist political
sentiment and energize his base. In Europe, two years after the height of the
migration crisis that brought more than a million people to the continent,
mostly from the Middle East and Africa, immigration remains a hot-button issue,
even as the number of new arrivals has dropped. Political parties across the
European Union are winning elections on anti-immigrant platforms and enacting
increasingly restrictive border policies. Tech ethicists and privacy
advocates worry that Roborder and projects like it outsource too much law
enforcement work to nonhuman actors and could easily be weaponized against
people in border areas.
“The development of these systems is a dark
step into morally dangerous territory,” said Noel Sharkey, emeritus professor
of robotics and artificial intelligence at Sheffield University in the U.K. and
one of the founders of the International Committee for Robot Arms Control, a
nonprofit that advocates against the military use of robotics. Sharkey lists
examples of weaponized drones currently on the market: flying robots equipped
with Tasers, pepper spray, rubber bullets, and other weapons. He warns of the
implications of combining that technology with AI-based decision-making and
using it in politically-charged border zones. “It’s only a matter of time
before a drone will be able to take action to stop people,” Sharkey told The
Intercept.
Roborder’s developers also may be violating
the terms of their funding, according to documents about the project obtained
via European Union transparency regulations. The initiative is mostly financed
by an €8 million EU research and innovation grant designed for projects that
are exclusively nonmilitary, but Roborder’s developers acknowledge that parts
of their proposed system involve military technology or could easily be
converted for military use.
Much of the development of Roborder is
classified, but The Intercept obtained internal reports related to ethical
considerations and concerns about the program. That documentation was
improperly redacted and inadvertently released in full.
In one of the reports, Roborder’s developers
sought to address ethical criteria that are tied to their EU funding.
Developers considered whether their work could be modified or enhanced to harm
humans and what could happen if the technology or knowledge developed in the
project “ended up in the wrong hands.” These ethical issues are raised, wrote
the developers, when “research makes use of classified information, materials
or techniques; dangerous or restricted materials[;] and if specific results of
the research could present a danger to participants or to society as a whole.”
Roborder’s developers argued that these
ethical concerns did not apply to their work, stating that their only goal was
to develop and test the new technology, and that it would not be sold or
transferred outside of the European Union during the life cycle of the project.
But in interviews with The Intercept, project developers acknowledged that
their technology could be repurposed and sold, even outside of Europe, after
the European project cycle has finished, which is expected to happen next year.
Beyond the Roborder project, the ethics
reports filed with the European Commission suggest a larger question: When it
comes to new technology with the potential to be used against vulnerable people
in places with few human rights protections, who decides what we should and
should not develop?
ROBORDER WON ITS funding grant in 2017 and has set out to develop a marketable
prototype — “a swarm of robotics to support border monitoring” — by mid-2020.
Its developers hope to build and equip a collection of air, sea, and land
drones that can be combined and sent out on border patrol missions, scanning
for “threats” autonomously based on information provided by human operators,
said Stefanos Vrochidis, Roborder’s project manager.
The drones will employ optical, infrared,
and thermal cameras; radar; and radio frequency sensors to determine threats
along the border. Cell phone frequencies will be used to triangulate the
location of people suspected of criminal activity, and cameras will identify
humans, guns, vehicles, and other objects. “The main objective is to have as
many sensors in the field as possible to assist patrol personnel,” said Kostas
Ioannidis, Roborder’s technical manager.
The end product will be tested by border
police in several European countries, including Portugal, Hungary, and Greece,
but the project has also generated considerable interest in the private sector.
“Eventually, we have companies that would certainly like to exploit this
commercially,” Vrochidis told The Intercept. “They might exploit the whole
outcome or part of the outcome, depending. They can exploit this in Europe but
also outside of Europe.”
In their grant agreement, Roborder’s
developers told the European Commission that they did not foresee any exports
of their technology outside of the EU. In interviews, however, developers told
The Intercept that the companies involved would be open to selling their
technology beyond Europe. According to a spokesperson from the grant program
funding Roborder, Horizon 2020, there is nothing Roborder’s EU
backers can do to control where or how the technology they bankrolled is
eventually used.
The documents obtained by The Intercept show
Roborder responding to some ethical concerns about the project but not about
the technology itself. In their grant application, Roborder’s developers
conceded that their research “may be exploited by criminal organizations and
individual criminals when planning to perpetrate acts of serious crime or
terrorism” but wrote that the consortium of public and private companies developing
the technology would work to keep their data safe. That group includes drone
manufacturing companies, several national police departments, two national
guards, a defense ministry, a port authority, a cyberdefense company, a company
that specializes in developing equipment for electronic warfare, and another
that provides “predictive analytics” for European police forces.
As for the technology’s possible
modification for future clients, the answers were less clear. The developers
would not comment on the potential for military sales after the project cycle
ends. Developers added that their work is delayed because one of Roborder’s key
consortium partners, Portuguese drone manufacturer Tekever, has left the
project. Spokespeople for Roborder, Tekever, and Horizon 2020 would not explain
the rationale for Tekever’s departure.
Horizon 2020 supports many security-oriented projects
but maintains that “only research that has an
exclusive focus on civil applications is eligible for funding.” The grant
program previously funded a project that uses artificial intelligence to detect whether travelers are lying as they pass
through automated border crossings.
Yet the documents obtained by The Intercept
highlight inconsistent statements about Roborder’s potential military uses.
According to one report, the project has no potential for “dual use,” or both
military and civil deployment. Ten pages later, asked whether their work
involved items that could be considered dual-use by European standards,
Roborder’s developers wrote that it did.
Roborder hired a consultant, Reinhard
Hutter, as an external ethics adviser to the project, according to another
document from the Horizon 2020 program that was inadvertently released in full
by the European Commission. In his report, Hutter wrote that “Roborder involves
technology with military potential,” and that “the results of this project have
the potential to be used back in the defense sector.” The technology involved,
Hutter wrote, had “some dual-use potential but no dual-use activity on the
project.” In other words, it could be used for military purposes but wouldn’t
be used that way within the scope of Roborder.
Hutter declined to speak to The Intercept.
This blurring of lines between military and
civilian development by the EU funding program might be deliberate. In
a 2014 guidebook on European funding for
dual-use projects, the European Commission notes that the regulation
establishing Horizon 2020 mandates that all funded projects have “an exclusive
focus” on civilian development, but the document also says that
“substantial parts of the research funded is of relevance for defense and can
lead to technologies that will be used by defense actors.”
The authors of a 2016 study commissioned by
the security and defense sub-committee of the European
Parliament went further, arguing that Horizon 2020’s clause on
exclusive civilian development should be reinterpreted to include defense
research. In order to compete with U.S. technological
development, the study’s authors advocated for the creation of a European
equivalent of DARPA, the U.S. Defense Advanced Research Projects
Agency, whose work contributed to the development of the internet, GPS, and
other technologies. In a 2017 speech, French President Emmanuel Macron echoed that, calling for, “a European
agency for disruptive innovation in the same vein as” DARPA.
The 2016 report does not represent the views
of the European Parliament or its security and defense sub-committee, and was
not used to develop any specific legislation, a spokesperson for the European
Parliament said. A spokesperson for Horizon 2020 rejected the idea that there
was any ambiguity in what kind of projects the European Union would fund.
“The European Commission does not fund
research intended for military use,” she said.
THE DRONES
ROBORDER plans to deploy are common technology.
What would be groundbreaking for the companies involved is a functional system
that allows swarms of drones to operate autonomously and in unison to reliably
identify targets. AI threat detection is often inaccurate, according to
robotics researchers, so any system that could correctly and consistently
identify people, cars, and weapons, among other things, would be a substantial
and lucrative advancement.
Drone cameras will not use facial
recognition technology within the scope of the project, explained Ioannidis,
Roborder’s technical manager, nor will they be able to determine any human
characteristics, such as height, weight, age, skin color, or perceived gender.
“The system will only identify that ‘this object is human,’” he added, “nothing
more.”
Still, Ioannidis admitted that adding facial
recognition to the Roborder system later on would be “technologically
possible.” What about weaponizing the Roborder system to act against humans?
“No,” he said, firmly. “The robots don’t have any authority to take any action
against anyone. It’s just monitoring and giving alerts.”
But Sharkey, the U.K. robotics professor,
argues that there is a thin line between using robots to monitor a border and
using them to enforce one. Weaponizing a drone is relatively easy, he said,
citing the 2015 case of the Connecticut teenager who equipped a drone with a handgun and a
flamethrower. Sharkey worries about the implications of developing
autonomous systems to patrol borders, including how the system could be used by
a country coping with a large influx of people.
“The question is, where is this going?”
Sharkey asked. “The current project does not propose to weaponize them, but it
would just be too tempting for countries if a tipping point were to happen with
mass migration.”
Hannah Couchman, a researcher at the U.K.
human rights organization Liberty, agrees. “There are deep human rights and
civil liberty concerns with this technology,” she told The Intercept. “Once
this tech is developed, it’s seen as a solution, as a response to austerity,
and a way to do a job efficiently with a lower cost, all rolled out without
proper consultation and legislative scrutiny.”
“It’s not just about mitigating the human
rights risk,” Couchman said. “It’s about whether we should use the technology
in the first place.”
Comments
Post a Comment