Games took place on randomly generated maps with teams’ bases on opposite sides. In Capture the Flag, players score points by picking up enemy flags and returning them to the home-team base. They can also shoot one another with lasers, causing any player who might be holding a flag to drop it and respawn back at their base. By giving the computer agents 450,000 rounds of Capture the Flag to analyze, they were able to learn the game well enough to deploy strategies capable of defeating a team of humans.

Advertisement

According to the paper, the AI agents analyzed situations for data like “‘Do I have the flag?,’ ‘Did I see my teammate recently?,’ and ‘Will I be in the opponent’s base soon?’” By checking the answers to questions like these against how many points were being scored, the AI agents were able to come up with proactive tactics like having one player race toward the enemy base while another retreated with the enemy flag, knowing that as soon as that teammate dumped it at their base, the flag would respawn and immediately be recapturable at the other end.

Advertisement
Advertisement

In the end, the DeepMind team found that its agents were, on average, able to beat teams of two human players by a margin of 16 captures. “In a separate study, we probed the exploitability of the FTW [For The Win] agent by allowing a team of two professional games testers with full communication to play continuously against a fixed pair of FTW agents,” the researchers wrote. “Even after 12 hours of practice, the human game testers were only able to win 25% (6.3% draw rate) of games against the agent team.”

The only time humans were able to beat the AI agents was when they teamed up together. A team consisting of one human and one AI agent had a 5 percent greater win probability than a team of just AI agents. It’s one of the more interesting findings, since it suggests that the AI program is able to adapt to playing with non-AI teammates and that the two players potentially make up for one another’s weaknesses. Human players tended to be better at long-distance shooting, while AI agents were better at navigating the environment to capture the flag.

Advertisement

These findings don’t mean that AI players are likely to defeat the world’s best Call of Duty or Halo players any time soon, but as the researchers point out, the work is progressing faster than many had previously thought possible.