Article Preview
TopIntroduction
Video games provide a good environment for research in autonomous agent learning. This research is motivated by two goals. One is to increase the entertainment value of the game by creating non-player agents, both cooperative and competitive, that increase the challenge of the game for a human player. The other is to test learning methods to create control programs that can be used in other agents such as autonomous robots. In this research, we use a learning method to develop a control program for a video game agent. Creating competitive non-player agents can be difficult, especially if a diversity of controller or agents that can adapt are desired. One method of learning that can create distinct competitors and has the potential for adapting to different players is Evolutionary Computation (EC). Video games can make viable test beds for researching different methods for evolving control programs for autonomous agents including actual robots. The video game used for this research, Xpilot, is an excellent environment due to its low computational requirements and the Xpilot-AI add-on that allows a user to create an agent to control ships during the game (Parker, 2010).
Many researchers choose to use video games to test various artificial intelligence techniques because they offer a large dynamic environment. These environments make the learning that is required for an evolved agent to compete seriously in these games much more challenging. The resultant evolved solutions for video game agents are also often more versatile, because they are not hard-coded, and may develop strategies a human programmer would never consider.
Research with learning control for video game agents has a long history. In the past, researchers have evolved agents for the video game Quake3, a 3-dimensional multiplayer game (Priesterjahn, 2006). They used genetic algorithms to evolve an agent that preformed better than the bot provided with the game, and then went on to co-evolve opponents to use instead of the provided bot. In other work, the game Pac-Man was used to consider a predator-prey scenario (Yannakakis, 2004). In this case, the connection weights for a neural network were evolved by a GA to serve as the control for the predators’ team behavior. Another example is the development of an agent to play a real-time strategy game known as DEF-CON through a combination of artificial intelligence techniques such as simulated annealing, decision tree learning, and case-based reasoning (Baumgarten, 2009).
The goal of the research reported in this paper is to create a competitive agent for Xpilot, a 2D combat-based video game. Many experiments to evolve agents for the Xpilot environment have been completed in the past, although all have concentrated on the development of agents for a combat role. The parameters for a combat expert agent were successfully evolved (Parker, 2007a) using a standard GA. Other artificial intelligence strategies, such as multi-layer neural networks (Parker, 2007b) and reinforcement learning (Allen, 2010), have also been used. These approaches have all been successful strategies to make a better Xpilot combat agent. In most of these works, the agent developed was far superior to those provided by the game.