Google DeepMind Gives Computer ‘Dreams’ to Improve Learning
Google DeepMind Gives Computer ‘Dreams’ to Improve
Learning
By Jeremy Kahn November 17, 2016 — 1:53 PM EST
Androids may not, as science fiction writer Philip Dick
once posited, dream of electric sheep. But the newest artificial intelligence
system from Google’s DeepMind division does indeed dream, metaphorically at
least, about finding apples in a maze.
Researchers at DeepMind wrote in a paper published online
Thursday that they had achieved a leap in the speed and performance of a
machine learning system. It was accomplished by, among other things, imbuing
technology with attributes that function in a way similar to how animals are
thought to dream.
The paper explains how DeepMind’s new system -- named
Unsupervised Reinforcement and Auxiliary Learning agent, or Unreal -- learned
to master a three-dimensional maze game called Labyrinth 10 times faster than
the existing best AI software. It can now play the game at 87 percent the
performance of expert human players, the DeepMind researchers said.
Artificial Intelligence
"Our agent is far quicker to train, and requires a
lot less experience from the world to train, making it much more data
efficient," DeepMind researchers Max Jaderberg and Volodymyr Mnih jointly
wrote via e-mail. They said Unreal would allow DeepMind’s researchers to
experiment with new ideas much faster because of the reduced time it takes to
train the system. DeepMind has already seen its AI products achieve highly
respected results teaching itself to play video games, notably the retro Atari
title Breakout.
Apple Maze
Labyrinth is a game environment that DeepMind developed,
loosely based on the design style used by the popular video game series Quake.
It involves a machine having to navigate routes through a maze, scoring points
by collecting apples.
This style of game is an important area for artificial
intelligence research because the chance to score points in the game, and thus
reinforce "positive" behaviors, occurs less frequently than in some
other games. Additionally, the software has only partial knowledge of the
maze’s layout at any one time.
One way the researchers achieved their results was by
having Unreal replay its own past attempts at the game, focusing especially on
situations in which it had scored points before. The researchers equated this
in their paper to the way "animals dream about positively or negatively
rewarding events more frequently."
The researchers also helped the system learn faster by
asking it to maximize several different criteria at once, not simply its
overall score in the game. One of these criterion had to do with how much it
could make its visual environment change by performing various actions.
"The emphasis is on learning how your actions affect what you will
see," Jaderberg and Mnih said. They said this was also similar to the way
newborn babies learn to control their environment to gain rewards -- like
increased exposure to visual stimuli, such as a shiny or colorful object, they
find pleasurable or interesting.
Jaderberg and Mnih, who are among seven scientists who
worked on the paper, said it was "too early to talk about real-world
applications" of Unreal or similar systems.
Gaming Champions
Mastering games, from Chess to trivia contests like the
U.S. gameshow Jeopardy!, have long served as important milestones in artificial
intelligence research. DeepMind achieved what is considered a major
breakthrough in the field earlier this year when its AlphaGo software beat one
of the world’s reigning champions in the ancient strategy game Go.
Earlier this month DeepMind announced the creation of an
interface that will open Blizzard Entertainment Inc’s science fiction video
game Starcraft II to machine learning software. Starcraft is considered one of
the next milestones for AI researchers to conquer because many aspects of the
game approximate "the messiness of the real world," according to
DeepMind researcher Oriol Vinyals. Unreal is expected to help DeepMind master
the mechanics of that game.
Improved Performance
DeepMind’s Unreal system also mastered 57 vintage Atari
games, such as Breakout, much faster -- and achieved higher scores -- than the
company’s existing software. The researchers said Unreal could play these games
on average 880 percent better than top human players, compared to 853 percent
for DeepMind’s older AI agent.
But on the most complex Atari games, such as Montezuma’s
Revenge, Jaderberg and Mnih said the new system made bigger leaps in
performance. On this game, they said, the prior AI system scored zero points,
while Unreal achieved 3,000 -- greater than 50 percent of an expert human’s
best effort.
Comments
Post a Comment