Human Strategy Learning-Based Multi-Agent Deep Reinforcement Learning for Online Team Sports Game

Abstract

In 3 vs. 3 online basketball games, finite state machine (FSM)-based Game artificial intelligence (AI) has traditionally been employed. However, limitations such as repetitive behavior patterns and challenges in maintaining systems during redesigns have led to increased research into reinforcement learning-based AI. This shift aims to address the shortcomings of FSM-based approaches. Nevertheless, applying multi-agent reinforcement learning-based AI in commercial online basketball games presents significant challenges, particularly in ensuring real-time processing, which requires efficient methods for managing observational data. In addition, the stochastic nature of action selection in reinforcement learning complicates the accurate learning of behaviors through explicit decision data. Moreover, reinforcement learning, which self-optimizes through exploration and develops its own rules, struggles to mimic human-like behavior patterns that follow predefined strategies in Sports Game. This study introduces a human strategy-based reinforcement learning method designed to address these challenges and replicate human gameplay that adheres to human-defined strategies. The learning of human strategies is enhanced using Ray for the real-time processing of observational data and a multi-phase reward system that distinctly defines rewards based on specific objectives. Furthermore, the proposed method enables real-time, strategy-based action guidance through a Human Strategy AI trained on human-defined strategies. Experimental results demonstrate that in a stochastic basketball game environment, this approach enabled the determination of precise actions and achieved human-like gameplay through the Human Strategy AI.