AI and the World's Hard Problems · Life sciences · arXiv · 2026-10-21
VGL-stGen: Learning a shared multimodal model for lung histology and spatial transcriptomics
VGL-stGen is a multimodal large language model adapted from a pretrained medical vision-language model using paired lung H&E and spatial transcriptomics with single-cell references. Gene-conditioned pseudo-labels transfer transcriptomic cell-type supervision to histology images, and one deployed checkpoint supports thirteen evaluated tasks spanning tissue and cell-type classification, gene prediction, and conditional expression completion. Image-permutation experiments indicate that correctly paired histology adds information beyond observed expression, though external classification partly relies on computational references with shared annotation lineage.