AI and the World's Hard Problems · Chemistry & materials · Nature Computational Science · 2026-10-12
Less can be more for predicting properties with large language models
Predicting properties from coordinate-category data, such as formation energies or elastic moduli from crystal structures, is central to computational science, and large language models (LLMs) have been applied to it. Using synthetic datasets and the MatText benchmark, the authors find that LLMs consistently fail to capture coordinate information while handling category patterns well. This holds across model sizes up to 70 billion parameters, datasets up to 2 × 10^6 structures and different text representations, and for property prediction dominated by structural effects, specialized geometric architectures outperform LLMs.