AI and the World's Hard Problems · Earth & climate science · arXiv · 2026-10-22
SciExam for ENSO: Can AI Agents Build Climate Models?
SciExam for ENSO is a benchmark in which language-model agents build low-order stochastic models of ENSO from real observations. Within a six-hour budget, agents process the observations, freeze their own written diagnostics, and develop a model from those alone, while hidden graders test whether it reproduces ENSO's statistics, recovers unobserved variables, and forecasts held-out years. Of twelve agent systems, six produced models scoring higher than a published model, and the stronger models' simplified forms each align with one of the two competing explanations of ENSO's warm-cold asymmetry.