Enhanced Random Ensemble Mixture: Weight Referring and Merging

Abstract

Featured Application The proposed DQN-based Enhanced Random Ensemble Mixture with Weight Referring and Merging was evaluated on the Catch Game environment. In this experimental setting, it showed more stable learning behavior than a standard DQN, suggesting that the method may be useful for reinforcement learning tasks where stable training is important.Abstract Reinforcement learning (RL) is widely used to learn sequential decision-making policies in complex environments. Deep Q-network (DQN) extends Q-Learning using deep neural networks, enabling learning in high-dimensional state spaces. However, conventional DQN-based approaches can exhibit variability in learning stability and convergence speed even under similar training conditions. Random Ensemble Mixture (REM) has been introduced to improve stability by combining multiple Q-value estimates, but it typically requires running multiple models simultaneously, which increases computational cost. This paper proposes an enhanced DQN method that integrates REM with a Weight Referring and Merging (WRM) mechanism to improve training stability and efficiency. The proposed approach updates a single primary agent using standard DQN learning while maintaining diversity among auxiliary agents by selectively referring to and partially merging weights from the primary network. Q-values from the primary and auxiliary agents are then combined through REM to produce the final value estimate for action selection. Experiments in the Catch Game environment indicate that the proposed method reaches stable performance earlier than a baseline DQN and reduces training time under the tested configuration (approximately 78%). While the results are encouraging in this environment, further evaluation on additional benchmarks is required to assess broader applicability.