by The field of human-computer gaming has a rich history and has served as a significant platform for testing and advancing artificial intelligence (AI) technologies. It all started with the famous Turing test, proposed in 1950, which aimed to determine whether a machine can exhibit human-like intelligence. This test inspired researchers to develop AI systems capable of challenging and defeating professional human players in various games. Notable examples include Chinook, an AI checker program that defeated the world champion in 1994, and Deep Blue from IBM, which defeated chess grandmaster Garry Kasparov in 1997, marking a major milestone in human-computer gaming.
In recent years, there has been a rapid advancement in human-computer gaming AIs. From the DQN agent and AlphaGo to Libratus and OpenAI Five, these AIs have demonstrated the ability to outperform professional human players in specific games, showcasing significant breakthroughs in decision-making intelligence.
For instance, AlphaGo Zero, utilizing techniques such as Monte Carlo tree search, self-play, and deep learning, surpassed numerous professional go players, setting new benchmarks in large state perfect information games. OpenAI Five, employing techniques like self-play, deep reinforcement learning, and continual transfer via surgery, became the first AI system to defeat world champions in an eSports game, proving its mettle in complex imperfect information games.
The successes of AlphaStar and OpenAI Five in games like StarCraft and Dota2 have demonstrated that current techniques can effectively address highly complex games. Furthermore, recent breakthroughs in games such as Honor of Kings and Mahjong, which follow similar frameworks to AlphaStar and OpenAI Five, suggest the universality of these techniques.
However, despite these achievements, several challenges remain in the field of human-computer gaming AI and future trends need to be explored. A new research paper published in Machine Intelligence Research aims to review recent successful human-computer gaming AIs and address these challenges through a comprehensive analysis of current techniques.
The paper encompasses four categories of games, including board games like Go, card games such as heads-up no-limit Texas hold’em (HUNL), DouDiZhu, and Mahjong, first-person shooting games (FPS) like Quake III Arena in capture the flag (CTF), and real-time strategy games (RTS) like StarCraft, Dota2, and Honor of Kings. The corresponding AIs discussed in the paper include AlphaGo, AlphaGo Zero, AlphaZero, Libratus, DeepStack, DouZero, Suphx, FTW, AlphaStar, OpenAI Five, JueWu, and Commander.
The paper’s subsequent sections delve into each category, describing the games and their corresponding AIs. It emphasizes the key factors challenging intelligent decision-making, such as imperfect information, long time horizons, intransitive game dynamics, and multi-agent cooperation. The techniques employed by the various AIs are thoroughly analyzed and compared.
Section 3 focuses on board game AIs, highlighting the Monte Carlo tree search (MCTS) algorithm used by the AlphaGo series. The paper also discusses the evolution of AlphaGo, leading to the development of AlphaGo Zero and AlphaZero. Section 4 delves into card game AIs, featuring DeepStack and Libratus, which utilize counterfactual regret minimization (CFR) to defeat professional poker players in HUNL. The emergence of AI systems like Suphx and DouZero in Mahjong and DouDiZhu, respectively, is also explored.
Section 5 presents first-person shooting game AIs, focusing on the CTF game mode in Quake III Arena. The learned agent FTW, based solely on pixels and game points, exhibits human-level performance. Section 6 delves into RTS games, discussing the accomplishments of AlphaStar, Commander, and OpenAI Five in games like StarCraft, Dota2, and Honor of Kings.
In Section 7, researchers summarize and compare the different techniques utilized in current human-computer gaming AIs, categorizing them into tree search (TS) with self-play (SP) and distributed deep reinforcement learning (DDRL) with self-play or population play (PP). They discuss the challenges of reaching Nash equilibrium and achieving generalizability in these techniques.
The paper’s conclusion, presented in Section 8, highlights the challenges posed by current game AIs, such as the lack of versatility across different game maps, the significant computational resources required, and the limited evaluation against professional human players. Future research is suggested to address these limitations, paving the way for further advancements in the field.
This comprehensive survey of AI techniques in human-computer gaming serves as a valuable resource for beginners seeking to familiarize themselves with this exciting field. It also provides inspiration for researchers venturing into deeper studies. The potential opportunities and challenges discussed in the paper will shape the future directions of AI in human-computer gaming, unlocking new possibilities for intelligent decision-making and game-playing systems.
*Note:
1. Source: Coherent Market Insights, Public sources, Desk research
2. We have leveraged AI tools to mine information and compile it
