AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-21
AURA: Angular Update Rate Adaptation for training complex-valued neural networks
Complex-valued neural networks are often trained with first-order optimizers inherited from the real-valued case, whose step-size rules ignore the angular information available in the complex plane. AURA is a per-parameter step-size adaptation that can be added on top of any first-order optimizer: it measures the agreement between consecutive updates of each complex parameter in length, alignment, and sense of rotation, enlarging the step when they are consistent and reducing it when they are not, with no additional gradient evaluations. Combined with Adam and Muon and compared on four test cases of increasing complexity, it improved the convergence of its base optimizer in most cases with a small per-step overhead, and the conditions under which it fails were identified.