While studying abroad at the NYU Tandon School of Engineering (NYU Tandon) last semester, NYU Shanghai senior Wang Che (Watcher) received the opportunity to get involved with research in artificial intelligence (AI). The research paper, titled “Portfolio Online Evolution for Real-Time Strategy Games,” has been accepted for an oral presentation at the Twelfth Annual AAAI conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-16), which will be held in Burlingame, California, from Oct. 8-12, 2016. On Century Avenue got the opportunity to find out more about Watcher’s research and his pursuits. The research proposes Portfolio Online Evolution (POE), a new method to control multiple units in real-time strategy (RTS) games. In a typical RTS unit battle, two players each control a group of army units to fight with the aim of eliminating the other player. When using POE, the player has a set of strategies to choose from and each unit will be assigned a strategy; the collection of all the strategies used for all the units is called a genome. When making a move, the player starts with a few genomes that all use random strategies. The player must evolve the genome, change some of the strategies, and then evaluate the new genome to be able to select the best ones and discard the rest. This process is repeated until the set time limit is reached; the AI agent then uses the strategies in the best genome for the next move. According to Watcher, the POE method is essentially an evolutionary algorithm, hence is similar to the concept of natural selection for species. The performance of this new algorithm was tested in a combat simulator for the game StarCraft by Blizzard Entertainment, and results showed that this method outperforms previous state of the art methods in most combat experiments. This is particularly exciting because creating AI in RTS games is a challenging problem, as it needs to be able to control multiple units and each of its commands needs to be issued within 40 milliseconds in order to guarantee a 24-frames per second gameplay. The huge branching factor and the harsh time limit thus often render traditional search algorithms almost useless. Research in the AI field has been Watcher’s childhood dream, which is why he decided to pursue this topic: “I love science fiction and games, and ever since I read Isaac Asimov’s “The Galactic Empire,” I have been crazy about AI,” Watcher said. “The thought of being able to create AI agents that control units in one of my favorite games, Starcraft, is just so exciting. So I didn’t hesitate when the chance came.” He became involved in the research through his enrollment in the NYU Tandon class “AI for Games”, an advanced AI course taught by AI research and game innovation expert Professor Julian Togelius. “Prof. Togelius directed me to resources and papers that helped me start the research, and two other graduate students, Pan Chen and Yuanda Li, from NYU Tandon were interested and joined me. We also got help from Christoffer Holmgård, a PhD, Postdoctoral Associate at the Game Innovation Lab at NYU Tandon, who gave us feedback on our research and along with Prof. Togelius, revised our paper. Most of the research was conducted by students, but the two served as our advisors,” added Watcher.A majority of the research was conducted while Watcher was studying abroad in New York: “We first thought to apply a novice method called Online Evolution to the RTS scenario, proposed by Niels Justesen, Tobias Mahlmann, and Prof. Togelius in March. We contacted researchers who worked on StarCraft combat simulators, namely David Churchill from the University of Alberta and Bálint Tillman from IT University of Copenhagen. They helped us with the simulator and we then started coding and testing the algorithm, but it didn’t work so well. We thus decided to combine the idea of Online Evolution with another method called Portfolio Greedy Search, proposed by David Churchill and Michael Buro, and came up with the algorithm proposed in our paper. After testing, data collection, data visualization, and other necessary tasks, we were happy to confirm that this method outperforms the previous state of the art algorithms.” Now back in Shanghai, Watcher is working on final refinements of the paper as well as preparing for its oral presentation in California this October. Watcher believes that the experience of conducting research in AI provided a lot to learn: “In addition to all the new knowledge and skills I acquired from the actual AI research, I learnt that research in this field is really a community thing. There are many open-source projects on the Internet and people love to help each other and share their results. It is amazing to see how artificial intelligence is being advanced by people around the world cooperating together.”

This article was written by Lathika Chandra Mouli. Please send an email to [email protected] to get in touch. Photo Credit: Watcher Wang