Does DeepMind represent the initial step in software development becoming a major assist or replacement for controlling bodily functions in individual with Traumatic Brain and/or Spinal Cord injuries? Imagine a person suffering from one of those afflictions utilizing software to control the muscle functions of their body as they go about their normal everyday activities.
DeepMind, the Google-owned artificial-intelligence company, has revealed how it created a single computer algorithm that can learn how to play 49 different arcade games, including the 1970’s classics Pong and Space Invaders. In more than half of those games, the computer became skilled enough to beat a professional human player.
The algorithm — which has generated a buzz since publication of a preliminary version in 2013 (V. Mnih et al. Preprint at http://arxiv.org/abs/1312.5602; 2013) — is the first artificial-intelligence (AI) system that can learn a variety of tasks from scratch given only the same, minimal starting information. “The fact that you have one system that can learn several games, without any tweaking from game to game, is surprising and pretty impressive,” says Nathan Sprague, a machine-learning scientist at James Madison University in Harrisonburg, Virginia.
DeepMind, which is based in London, says that the brain-inspired system could also provide insights into human intelligence. “Neuroscientists are studying intelligence and decision-making, and here’s a very clean test bed for those ideas,” says Demis Hassabis, co-founder of DeepMind. He and his colleagues describe the gaming algorithm in a paper published this week
Games are to AI researchers what fruit flies are to biology — a stripped-back system in which to test theories, says Richard Sutton, a computer scientist who studies reinforcement learning at the University of Alberta in Edmonton, Canada. “Understanding the mind is an incredibly difficult problem, but games allow you to break it down into parts that you can study,” he says. But so far, most human-beating computers — such as IBM’s Deep Blue, which beat chess world champion Garry Kasparov in 1997, and the recently unveiled algorithm that plays Texas Hold ’Em poker essentially perfectly.
DeepMind’s versatility comes from joining two types of machine learning— an achievement that Sutton calls “a big deal”. The first, called deep learning, uses a brain-inspired architecture in which connections between layers of simulated neurons are strengthened on the basis of experience. Deep-learning systems can then draw complex information from reams of unstructured data (seeNature 505, 146–148; 2014). Google, of Mountain View, California, uses such algorithms to automatically classify photographs and aims to use them for machine translation.
The second is reinforcement learning, a decision-making system inspired by the neurotransmitter dopamine reward system in the animal brain. Using only the screen’s pixels and game score as input, the algorithm learned by trial and error which actions — such as go left, go right or fire — to take at any given time to bring the greatest rewards. After spending several hours on each game, it mastered a range of arcade classics, including car racing, boxing and Space Invaders.
This rapid growth of data sets means that machine learning can now use complex model classes and tackle highly non-trivial inference problems. Our human brain constantly solves non-trivial problems as we conduct our daily activities. Interpreting high-dimensional sensory data to determine how best to control all of the muscles in our body, including the functioning of our internal organs. The development of cutting edge software, like DeepMind, could assist and resolve many of the issues confronting Traumatic Brain and Spinal Cord Injury patients, beginning with recovering the ability to move their arms and legs.