AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-26
Testing Transformer Learnability on the Arithmetic Sequence of Rooted Trees
The study asks whether a transformer can learn the deterministic sequence of rooted trees produced by iterated prime factorization of the natural numbers. A GPT-2-style model trained from scratch on the first 10^11 elements reached a word accuracy of about 0.4 on next- and masked-token prediction, above a Hidden Markov Model baseline, and stayed stable on test blocks at 10^13 to 10^15. The model's likelihood also separated the arithmetic text from controls that keep token frequencies but not their order, and from sequences with more than three consecutive square-free integers, which arithmetic forbids.