Google’s DeepMind AI lab has been hard at work creating programs to take on human players across a variety of games these last few years. In 2016 its AlphaGo beat the best Go player in the world. Earlier this year, its Alpha Star defeated two middle-tier pros at StarCraft II. Now it’s learning to win at multiplayer games like Quake III.
“Artificially intelligent agents are getting better and better at two-player games, but most real-world endeavors require teamwork,” DeepMind’s researchers wrote in a paper published in Science today. To test DeepMind’s capacity to excel in multiplayer games, the team created a program to play against humans in a modified version of Quake III Arena’s Capture the Flag mode. The program, nicknamed For the Win, was fed thousands of hours of game data to learn from. Weeks later it was able to take on human players. “Computer agents competed successfully against humans even when their reaction times were slowed to match those of humans,” the researchers wrote.
Games took place on randomly generated maps with teams’ bases on opposite sides. In Capture the Flag, players score points by picking up enemy flags and returning them to the home-team base. They can also shoot one another with lasers, causing any player who might be holding a flag to drop it and respawn back at their base. By giving the computer agents 450,000 rounds of Capture the Flag to analyze, they were able to learn the game well enough to deploy strategies capable of defeating a team of humans.
According to the paper, the AI agents analyzed situations for data like “‘Do I have the flag?,’ ‘Did I see my teammate recently?,’ and ‘Will I be in the opponent’s base soon?’” By checking the answers to questions like these against how many points were being scored, the AI agents were able to come up with proactive tactics like having one player race toward the enemy base while another retreated with the enemy flag, knowing that as soon as that teammate dumped it at their base, the flag would respawn and immediately be recapturable at the other end.
In the end, the DeepMind team found that its agents were, on average, able to beat teams of two human players by a margin of 16 captures. “In a separate study, we probed the exploitability of the FTW [For The Win] agent by allowing a team of two professional games testers with full communication to play continuously against a fixed pair of FTW agents,” the researchers wrote. “Even after 12 hours of practice, the human game testers were only able to win 25% (6.3% draw rate) of games against the agent team.”
The only time humans were able to beat the AI agents was when they teamed up together. A team consisting of one human and one AI agent had a 5 percent greater win probability than a team of just AI agents. It’s one of the more interesting findings, since it suggests that the AI program is able to adapt to playing with non-AI teammates and that the two players potentially make up for one another’s weaknesses. Human players tended to be better at long-distance shooting, while AI agents were better at navigating the environment to capture the flag.
These findings don’t mean that AI players are likely to defeat the world’s best Call of Duty or Halo players any time soon, but as the researchers point out, the work is progressing faster than many had previously thought possible.