Back in Mountain View, Kuffner’s former employer (Google) has made great progress on its robotic cloud under its new leader Sergey Levine of the Google Brain team, which announced this past Monday that it completed a series of three tests that proved how one can utilize the cloud for “general-purpose skill learning across multiple robots.”
The first test involved robots learning motor skills directly from trial-and-error practice. Each robot started with a copy of a neural net as it attempted to open a door over and over. At regular intervals, the robots sent data about their performances to a central server, which used the data to build a new neural network that better captured how action and success were related. The server then sent the updated neural net back to the robots.
In the second scenario, the researchers wanted robots to learn how to interact with objects not only through trial-and-error but also by creating internal models of the objects, the environment, and their behaviors. Just as with the door opening task, each robot started with its own copy of a neural network as it “played” with a variety of household objects. The robots then shared their experiences with each other and together built what the researchers describe as a “single predictive model” that gives them an implicit understanding of the physics involved in interacting with the objects.
The final trial involved robots learning skills with help from humans. The idea is that people have a lot of intuition about their interactions with objects and the world, and that by assisting robots with manipulation skills we could transfer some of this intuition to robots to let them learn those skills faster. In the experiment, a researcher helped a group of robots open different doors while a single neural network on a central server encoded their experiences. Next, the robots performed a series of trial-and-error repetitions that were gradually more difficult, helping to improve the network.
According to the Brain Team’s blog post, all three of the experiments proved the robots’ ability to communicate and exchange their experiences enabling them to learn more quickly and effectively. This becomes particularly important when we combine robotic learning with deep learning, as is the case in all of the experiments discussed above. We’ve seen before that deep learning works best when provided with ample training data. For example, the popular ImageNet benchmark uses over 1.5 million labeled examples. While such a quantity of data is not impossible for a single robot to gather over a few years, it is much more efficient to gather the same volume of experience from multiple robots over the course of a few weeks. Besides faster learning times, this approach might benefit from the greater diversity of experience: a real-world deployment might involve multiple robots in different places and different settings, sharing heterogeneous, varied experiences to build a single, highly generalizable representation.
According to the researchers, “given that this updated network is a bit better at estimating the true value of actions in the world, the robots will produce better behavior… This cycle can then be repeated to continue improving on the task.”
In other SkyNet news, TRI and Google have inspired startups to pursue the robotic cloud as well. Just last month, Rapyuta Robotics, an ETH Zurich spin-off, received $10 million in Series A funding from Japanese-based SBI investments Co. According to the website, Rapyuta Robotics’ mission is to empower lives with cloud-connected mobile autonomous machines. An open-source version of its robotic cloud platform is expected to be released next year. I suppose someone should alert John Connor…