AI Reports · Amazon · 2026-10-20 · Blog / commentary
Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch
i14 trained and served Laplace 3B, a linear attention model, on AWS Trainium2 using native PyTorch through TorchNeuron Native. The model's recurrent state stays fixed at 29.5 MB per sequence regardless of context length, avoiding the growing key-value cache of transformers and keeping serving memory predictable. On one trn2.48xlarge instance, Laplace 3B trained at about 120,000 tokens per second and served 11,544 output tokens per second across 1,280 concurrent sequences.