AI and the World's Hard Problems · Medicine & drug discovery · arXiv · 2026-10-20
Same path, different walk: a mechanistic comparison of looped and stacked transformer encoders on 12-lead ECG
Recurrent Transformers that reuse their weights are increasingly adopted for parameter efficiency, but the representational and dynamical differences between looped and stacked architectures have remained uncharacterized. The authors train bViT, which applies one weight-tied block L times, and a standard ViT on 12-lead ECG classification from the PTB-XL dataset under identical training protocols, finding that bViT achieves accuracy parity with ViT despite an 8.9x parameter reduction. Both build comparable latent representations in an equivalent canonical order, but their dynamics differ: bViT shows smaller step sizes and inter-patient sensitivity with near-neutral behavior off the data manifold, while ViT exhibits collapsing dimensionality of representations and out-of-distribution feature expansion.