AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-20
Formally Verified Autoresearch for Analyzing the Expressivity of Multi-Head Attention
The study examines how many attention heads a one-layer, attention-only transformer needs to represent a Boolean function, defined as its head complexity. The authors derive an exact normal form, prove a threshold-degree lower bound, and show that for symmetric functions the head complexity equals the number of value changes across Hamming-weight levels. A frontier model generated the conjectures and proofs, which were machine-verified in Lean, with human researchers asking the questions and reviewing the results.