AI and the World's Hard Problems · Life sciences · Nature (machine learning) · 2026-10-26
Efficient structural information integration into protein language models with implicit mixture-of-experts ProBEiT
Protein language models can predict function and mutation effects from sequence, but few incorporate the structural information that determines function. The authors propose ProBEiT, which jointly encodes amino-acid sequences and tokenized structures using a parameter-efficient implicit mixture-of-experts scheme with deterministic routing. Trained on 1.7 million AlphaFoldDB proteins, the 400-million-parameter model outperforms larger baselines on six fitness and functional annotation tasks and correctly identifies optimized variants with positive epistatic effects in a thermostability engineering task on glycerol-3-phosphate oxidase.