AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-22
The role of locality and weight sharing in convolutional neural networks
Researchers compared convolutional and locally connected students in a teacher-student model to separate the effects of locality, weight sharing, and spatial correlations on learning. Weight sharing accelerated learning for uncorrelated input positions, while matching the teacher's kernel size reduced sample requirements; spatial correlations could weaken the advantage of weight sharing. With learning rates tuned to the training horizon, locally connected students performed better early and convolutional students performed better later.