Abstract

Multi-agent reinforcement learning (MARL) has gained significant attention due to its applications in complex, interactive environments. Traditional MARL approaches often struggle with scalability and non-stationarity as the number of agents increases. Mean Field Reinforcement Learning (MFRL) provides a scalable alternative by approximating interactions using aggregated statistics. However, existing MFRL models fail to capture causal relationships between agent interactions, leading to suboptimal decision-making. In this work, we introduce Causal Mean Field Multi-Agent Reinforcement Learning (Causal-MFRL), which integrates causal inference techniques into the mean field framework. By leveraging causal graphs and counterfactual reasoning, Causal-MFRL improves policy learning and enhances the interpretability of agent behaviors. We evaluate our approach on standard MARL benchmarks, demonstrating superior performance in efficiency, robustness, and generalization.