Reconstruction of Words, Syllables, and Phonemes of Internal Speech by EEG Activity

Advances in Intelligent Systems and Computing. 2021. Vol. 1358. P. 319–328.

Suyuncheva A., Saada D., Gavrilenko Y., Andrey Schevchenko, Vartanov A., Ilyushin E.

In this article, the process of internal pronunciation (covert speech) is associated with the internal speech through an intellectual process such as silent reading. The objective of the research is to compare the EP of visual and auditory perception and internal pronunciation of phonemes and syllables; to classify phonemes, words and syllables from covert speech, according to the EEG-data. Electrical activity was measured in tasks: for visual and auditory perception and pronunciation; for perception and pronunciation of conditioned stimuli. Electrophysiological experiment registrated using 19-channel EEG. Seven phonemes (A, B, F, G, M, R, U) and ten syllables composed of these phonemes (BA, FA, GA, MA, RA, BU, RU, MU, FU, GU) were selected for the experiment. Japanese words constructed using current phonemes were used as conditional stimuli (a trigger for the stimulus's pronunciation). The obtained data analysis was carried out using the statistical programming language R. Results, based on the ANOVA have significant differences for all experimental stages. Pronunciation as a reaction to a conditioned stimulus in the form of Japanese words were compared with two types of of covert speech initialization. The averaged reconstruction was between 63–67% (group averaging). In pairwise classifications of Japanese phonemes, stimuli reconstruction was between 76% and 83%. There is also a significant difference between the pronunciation and initialization type, regardless of the stimulus type (p < 0.001), but changes across channels are significant only for phonemes (p = 0.03). Main significant differences (p < 0.05) were found on C3 and F3 (for audio feed the differences are also seen on channel F7) in the early EP latencies (temporal zone).

Keywords: EEG, neologisms, covert speech, SVM, Japanese, phonemes, words, internal pronunciation, EP, Syllables, conditioned stimuli, Non-Existent Words