AI and the World's Hard Problems · Theoretical computer science · Anthropic · 2026-09-28
A mathematical framework for Transformer Circuits
The researchers reverse-engineered small attention-only transformers to study how induction heads form. The work shows that they form through head composition.