AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-26
Neural networks become centroid classifiers when trained on heavily corrupted inputs
The authors study attribute noise, where inputs are corrupted while labels stay intact, and find that MLPs keep well-above-chance accuracy even when more than 90% of the inputs are corrupted. Analyzing infinite-width networks with a mean-field-inspired approach, they show the network acts as a nearest-class-mean classifier that assigns each test point to the class whose training average it most resembles, a rule that holds across depths, activation functions and noise distributions. The same centroid mechanism matches finite-width behavior and explains why learning can succeed even when individual training examples carry almost no signal.