AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-20
Dynamical Batching: How Class-Cycling Training Shapes the Internal Structure of Neural Networks
The authors introduce Dynamical Batching (DB), which periodically emphasizes each class in training mini-batches while keeping the loss and optimizer unchanged, and compare it with uniform class batching. On a three-arm spiral, a capacity-limited MLP that plateaus under uniform batching fits the training data under DB. On CIFAR-10, DB modestly improved test accuracy for a channel-narrowed ResNet-10 and a second capacity-limited convolutional network.