AI and the World's Hard Problems · Theoretical computer science · Anthropic · 2026-09-28
LLM generalization with influence functions
This work scales influence functions, which estimate how training examples affect a model's behavior, to LLMs with up to 52 billion parameters using the EK-FAC approximation. It achieves accuracy similar to traditional estimators while making the inverse-Hessian-vector product computation orders of magnitude faster. The authors use it to study generalization patterns and find that influence drops to near zero when the order of key phrases is flipped.