AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-26
A Mathematical Theory of the Jacobian Lens
The Jacobian lens (J-lens), an interpretability tool from Anthropic, detected working-memory-like directions in language models, but why these features were detectable was unclear. The authors present a mathematical theory showing that J-lens preferentially identifies coherent features, whose perturbations amplify across layers and stay consistent across token positions and contexts. They validated this on a Bayes filter and a toy transformer predicting tokens from a hidden Markov model, where changing parameters that alter feature coherence changed the J-lens response as predicted.