AI and the World's Hard Problems · Mathematics · arXiv · 2026-10-19
Optimal approximation of smooth functions by ReLU neural networks
This paper studies uniform approximation of Hölder balls by fully connected ReLU networks. Except for the one-dimensional Lipschitz case, existing constructions fell short of the information-theoretic efficiency limit. The authors show that networks with width O(N), depth O(L) and parameter magnitude O(1) reach error of order (N^2 L^2 log N)^(-s/D) for every s>0 and D≥1, and that empirical risk minimizers over the corresponding clipped network classes attain the minimax regression rate n^(-2s/(2s+D)) without extra logarithmic factors.