Robots need 'tough love' to learn best...
If You
Want a Robot to Learn Better, Be a Jerk to It
When
humans give robots “tough love” by trying to knock objects out of their hands,
it actually helps them find the best ways to hold things.
In what will go down as one of the
greatest robotics experiments ever, a few years
back researchers in Japan let a robot loose in a mall and watched how kids reacted.
Far from the sense of wonder you might expect from children, the mood soured
into a sense of concern for the next generation, as the kids proceeded to kick
and punch the robot and call it names.
Call it unconstructive criticism. But maybe the kids were
on to something—maybe we should be challenging the robots,
albeit in more constructive ways, instead of always holding their hands as they
learn to navigate our world. To that end, researchers at the University of
Southern California have shown that when working in a simulation, you can give
robots “tough love” by trying to knock objects out of their hands, and it’ll
actually help them better learn to grasp objects.
The experiment took place entirely in simulation,
as so much robot training does these days. In a
digital environment, a robot undergoes a supercharged form of trial and error
called reinforcement learning. The environment simulates variables like
friction, and a robotic arm tries to grasp an object over and over using
different grips. If it stumbles on a good grip, the system tallies that as a
victory—if it does something stupid, the system counts that as a defeat. Over
many attempts, the robot learns what constitutes a robust grasp.
But in comes a so-called adversarial human
actor, a sort of additional signal. If the robot finds a good grasp, the human
uses a graphical interface to click on the object it’s gripping and apply a
force in a certain direction. That disturbance basically tests how good the
grasp really is, and it helps the robot rule out the less effective ones.
“The robot learned to grasp objects much more robustly
using this additional signal that the human was providing, but also learned to
generalize to new objects much better,” says USC roboticist Stefanos
Nikolaidis, coauthor on a new paper describing the work. To put a number on it,
when a human was giving the robot tough love, the machine had a 52 percent
success rate at grasping, compared to 26.5 percent without the tough love.
After
about 20 minutes of training with the human adversary, the robot succeeds more
frequently in grasping objects, plus its grasps are more resilient to
disturbances. Notice how it maintains a grip when the object moves.
Now,
some critical caveats here. First of all, a simulation is a necessarily
imperfect model of the real world—there’s no way to fully replicate all the
physics and uncertainty of meatspace (or metalspace, in this case). So porting
what a robot learns in simulation into a physical robotic arm is still very
difficult, a challenge known as the reality gap. And
two, this wasn’t willy-nilly tough love, as the human participants were working
with certain rules and constraints.
All that said, the experiment shows there’s
merit in challenging robots instead of constantly coddling them. This is
particularly important with a problem as complex as grasping, which has little
margin for error. “If we want robots to be out there helping people with
different types of motor impairments, we don't want them to break things 10
percent of the time,” says Nikolaidis. Imagine unloading your dishwasher and
dropping 10 percent of the dishes, and how mad you would be at yourself. Now
imagine how mad you would be if a robot in your house did the same.
I mean, just look at what the children are capable of,
and that robot was just standing there minding its business.
Comments
Post a Comment