fMRI from EEG is only Deep Learning away: the use of interpretable DL to unravel EEG-fMRI relationships

Working papers by Cornell University. Series cond-mat.soft "arxiv.org" (. 2022. Article 4650840.

Ossadtchi A., Mikheev I., Ковалев А. В.

The access to activity of subcortical structures offers unique opportunity for building intention dependent brain-computer interfaces, renders abundant options for exploring a broad range of cognitive phenomena in the realm of affective neuroscience including complex decision making processes and the eternal free-will dilemma and facilitates diagnostics of a range of neurological deceases. So far this was possible only using bulky, expensive and immobile fMRI equipment. Here we present an interpretable domain grounded solution to recover the activity of several subcortical regions from the multichannel EEG data and demonstrate up to 60 % correlation between the actual subcortical blood oxygenation level dependent (sBOLD) signal and its EEG-derived twin. Then, using the novel and theoretically justified weight interpretation methodology we recover individual spatial and time-frequency patterns of scalp EEG predictive of the hemodynamic signal in the subcortical nuclei. The described results not only pave the road towards wearable subcortical activity scanners but also showcase an automatic knowledge discovery process facilitated by deep learning technology in combination with an interpretable domain constrained architecture and the appropriate downstream task.

Keywords: фМРТ, fMRI, deep learning, глубинное обучение, ЭЭГ/МЭГ, EEG/MEG

Publication based on the results of: Двунаправленные электрокортикографические интерфейсы мозг-компьютер для управления, стимуляции и коммуникации (2020)