AI and the World's Hard Problems · Theoretical computer science · arXiv · 2026-10-19
Information Estimation with Discrete Diffusion
Estimating information measures such as mutual information on discrete data is difficult, and existing methods typically embed the data into a continuous space first. InfoSEDD is a discrete diffusion-based approach, grounded in continuous-time Markov chain theory, that links information estimation with generative modeling to compute KL divergences. It was applied to motif discovery in genetic promoter data, semantic-aware model selection in text summarization, and entropy estimation in Ising models, and in experiments it outperformed alternatives that rely on the embedding trick.