AI and the World's Hard Problems · Linguistics · arXiv · 2026-10-22
Word-Class and Construction-Like Structure Emerges in Neural Successor Representations Trained on Natural Language
Researchers trained a residual network on WikiText-103 to predict successor representations at three horizons without part-of-speech supervision, testing whether predictive representations recover linguistic structure. At the shortest horizon, clustering reliably recovered nouns, verbs, and adjectives; finer analysis also revealed semantic-functional groups crossing word-class boundaries and relations tracing candidate date, measurement, and title-name constructions.