Google has created the computer equivalent of a teenager: an artificial-intelligence system that spends all of its time playing—and mastering—video games. The company introduced the new development in machine-learning technology on Wednesday, describing it as "the first significant rung of the ladder" to building intelligent AI that can figure out how to do things on its own.
The research project, built by a London startup called DeepMind Technologies that Google acquired last year, exposed computers running general AI software to retro Atari games. The machines were shown 49 games on the Atari 2600, the home console beloved by all ’80s babies, and were told to play them, without any direction about how to do so.
When the computers passed a level or racked up a high score, they were automatically rewarded with the digital equivalent of a dog treat. Google's AI system surpassed the performance of expert humans in 29 games, and outperformed the best-known algorithmic methods for completing games in 43 instances. Some games, like Ms. Pac-Man, can't be easily beaten with a mathematical formula. In others, like Video Pinball, the AI crushed human players with a system that was more than 20 times better than a professional human game tester.
For a window into how a computer learns to play Breakout, this was the machine's strategy when it first got its hands on the game:
And here it is after a few hundred lives:
The goal of the experiment wasn't to try to find a better way to cheat at video games. The principles of being given a task and finding the best solution can be applied to real-life scenarios in the future. The system, at its base, should be able to look at the world, navigate around it, and take actions accordingly. One day, Google's self-driving cars could learn how to drive based on experience, rather than needing to be taught, says Demis Hassabis, a co-founder of DeepMind and vice president of engineering at Google. This research marks the "first time anyone has built a single learning system that can learn directly from experience and manage a wide range of challenging tasks," he says.
Games are close to Hassabis's heart. He was a teenage chess and programming prodigy, who helped write the clever game logic behind popular titles such as Theme Park, before earning a degree in computer science from Cambridge University and a Ph.D. in cognitive neuroscience from University College London. Though beating a game may seem a far cry from the humanoid brilliance we see in sci-fi movies, it turns out simple games could be the perfect training wheels to shape a more elaborate digital brain. As the researchers write in a letter to be published in the science periodical Nature, machines can use games to "derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations."
Though an important step, Google is a long way away from an AI tech that can truly think independently, Hassabis says. "It's mastering and understanding the structure of these games, but we wouldn't say yet it's building conceptual knowledge or abstract knowledge," he says. "The ultimate goal here is to build smart, general-purpose machines, but we're many decades off from doing that."
Google's system pairs well-understood AI tech with a memory and reward system inspired by what's found in human and animal brains. By fusing these together, the team was able to create a system that can learn from its environment, refer back to previous actions, and adjust its behavior. This represents a big advancement over previous AI efforts. Prominent image-recognition systems developed by Microsoft, IBM, Clarifai, and MetaMind typically require supervision and annotated pictures to learn how to recognize things. As Google develops more sophisticated techniques, it will continue to draw from biology to add systems for long-term memory and strategic planning, Hassabis says.
The next steps for the game-playing machines are to develop and train systems to navigate complicated three-dimensional worlds contained in games that came out in the 1990s, Hassabis says. (Robots are going to love Tomb Raider.) By exposing the system to more complicated games, the team hopes to eventually be able to take it into real life. "If this can drive the car in a racing game, then potentially, with a few real tweaks, it should be able to drive a real car," Hassabis says. "That's the ultimate aim."