Enhancing Multi-Agent Reinforcement Learning via Knowledge-Embedded Modular Framework for Online Basketball Games

Abstract

High sample complexity presents a major challenge in applying multi-agent reinforcement learning (MARL) to dynamic, high-dimensional sports such as basketball. To address this problem, we proposed the knowledge-embedded modular framework (KEMF), which partitions the environment into offense, defense, and loose-ball modules. Each module employs specialized policies and a knowledge-based observation layer enriched with basketball-specific metrics such as shooting success and defensive accuracy. These metrics are also incorporated into a dynamic and dense reward scheme that offers more direct and situation-specific feedback than sparse win/loss signals. We integrated these components into a multi-agent proximal policy optimization (MAPPO) algorithm to enhance training speed and improve sample efficiency. Evaluations using the commercial basketball game Freestyle indicate that KEMF outperformed previous methods in terms of the average points, winning rate, and overall training efficiency. An ablation study confirmed the synergistic effects of modularity, knowledge-embedded observations, and dense rewards. Moreover, a real-world deployment in 1457 live matches demonstrated the robustness of the framework, with trained agents achieving a 52.43% win rate against experienced human players. These results underscore the promise of the KEMF to enable efficient, adaptive, and strategically coherent MARL solutions in complex sporting environments. © 2026 by the authors.