AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-21
Non-Stationarity Breaks Permutation Surrogates in Multi-Agent Reinforcement Learning: Diagnosis and Remedies
In two multi-agent reinforcement learning games, a social dilemma and a coordination race where directed influence between selected agent pairs is zero by construction, the authors test a reporting guardrail for information-theoretic measures against ground truth over 100 seeds. Omitting exclusion of the non-stationary training transient yields false-positive rates of 100.00% and 99.95%, and excluding the transient alone still leaves 11.8% in the coordination game. They instead recommend permuting the source within blocks of training time, which reaches 5.25% and 5.50% and is the only one of the tested nulls at or below the size of the test in both games, and code and data are released.