Multimodal Transformer Framework for Integrating EEG, fMRI, and Structural Features toward Accurate Prediction of Resting Motor Threshold

Accurately predicting the transcranial magnetic stimulation (TMS) resting motor threshold (RMT) is essential for personalized and safe neuromodulation, yet the factors governing cortical excitability remain incompletely characterized. Building upon our previous hybrid model that combined fMRI and structural–electromagnetic features [1], we propose to develop an advanced multimodal transformer framework that integrates resting-state electroencephalography (EEG), functional MRI, and anatomical–field metrics to achieve more precise and interpretable RMT estimation in healthy individuals. Resting-state EEG signals were collected at 100 Hz for 10 minutes under a fixed stimulation intensity, alongside gray-matter volume (GMV), coil-to-cortex distance (CCD), and simulated cortical electric-field (E-field) distributions. The minimally preprocessed EEG and fMRI time-series will be encoded through a self-supervised multimodal transformer to learn shared spatiotemporal representations of cortical activity. These representations will then be fused with structural and E-field features through a transformer-based regression head to estimate subject-specific RMT. We anticipate that integrating EEG dynamics with imaging-derived predictors will substantially improve model accuracy and interpretability, establishing a unified and data-driven approach for advancing RMT prediction.

[1] Y. R. Saxena, C. J. Lewis, M. Sabbir Alam, J. Atulasimha, U. M. Mehta and R. L. Hadimani, "A Hybrid Machine Learning Algorithm for Predicting Resting Motor Thresholds in Patients With Schizophrenia and Healthy Individuals Undergoing Transcranial Magnetic Stimulation," in IEEE Transactions on Magnetics, vol. 61, no. 9, pp. 1-6, Sept. 2025, Art no. 5800306, doi: 10.1109/TMAG.2025.3554122.