Google DeepMind’s New AI Model: Teaching Robots to Learn

Google DeepMind has developed a groundbreaking AI model called RT-2 that can direct robots to perform tasks it was never trained for. Unlike its predecessor, RT-1, RT-2 learns from both web and robotics data, allowing it to provide simple instructions to machines.

In tests, RT-2 was given commands that were not present in the robotic data, such as placing oranges in a matching bowl. To follow these instructions, the model had to translate knowledge from web-based data. DeepMind reported that RT-2 achieved a 62% success rate for these operations, which is double the success rate of RT-1.

“Just like language models are trained on text from the web to learn general ideas and concepts, RT-2 transfers knowledge from web data to inform robot behavior,” explained Vincent Vanhoucke, head of robotics at DeepMind. “In other words, RT-2 can speak robot.”

RT-2 demonstrated impressive generalization capabilities and an improved understanding of robotic data that it had not encountered before. It can also use rudimentary reasoning to follow new user commands and even perform multi-stage semantic reasoning. For example, when instructed to find an object that could be used as a hammer, RT-2 correctly identified a rock as the best option.

In one test, RT-2 successfully figured out that a rock would be the most suitable object to use as an improvised hammer. In another evaluation, the model was able to push a bottle of ketchup towards a blue cube, even though the only item in the training dataset was the cube.

DeepMind considers RT-2 to be a significant breakthrough in artificial intelligence. The London-based lab believes that this model brings us closer to a future where robots can be truly helpful.

“Not only does RT-2 show how advances in AI are rapidly impacting robotics, but it also holds enormous promise for more general-purpose robots,” said Vanhoucke. “While there is still much work to be done to enable helpful robots in human-centered environments, RT-2 shows us an exciting future for robotics that is within reach.”