AI and the World's Hard Problems · Medicine & drug discovery · Nature (computational biology) · 2026-10-24
An adaptive transformer-based model with entropy-driven knowledge distillation for single-channel EEG REM sleep detection
Detecting REM sleep from a single EEG channel is important for neurophysiological monitoring and early identification of sleep-related disorders. The authors propose a lightweight Transformer trained with an entropy-driven adaptive knowledge distillation method that adjusts temperature scaling and distillation weighting based on teacher–student entropy. On three public datasets under subject-independent cross-validation, it outperformed CNN, LSTM and standard Transformer baselines, with 93.2% average accuracy and a 92.8% F1 score, while the distilled student model cut training time and memory use by about 30%.