Weight Perturbations for Simulating Virtual Lesions in a Convolutional Neural Network

P. 221–234.

W. Joseph MacInnes, Zhozhikashvili N., Feurra M.

Convolutional Neural Networks (CNNs) match human performance in many visual tasks like the classification of images, however they may not simulate the underlying biological processes. We implemented a CNN to try replicate results from an object inversion experiment with Transcranial Magnetic Stimulation (TMS). After training on upright faces, the CNN model went through three stages of testing: checking (1) for overall accuracy, (2) for the presence of the Face Inversion Effect (FIE) and (3) for an FIE reduction after weight perturbations. Results of the model were compared with human performance in an analogous experiment, where disruption of the extrastriate cortex (the Occipital Face Area (OFA) and the control Occipital Place Area (OPA)) was performed using TMS. The resulting model (1) showed a level of accuracy similar to humans, but (2) did not show the FIE, but rather showed a general object inverted effect. Disruption with TMS (3) led to a reduction in the FIE, however disruption of model layers only led to reduction of the general object inverted effect. Thus, CNNs were observed to successfully simulate some results of objects recognition in general, but are unable to simulate the specific mechanisms of modularity and face processing. CNNs are certainly a useful metaphor for human visual processing, but it’s important to understand the limits of that metaphor if they are to be used as models in medicine and neuroscience.

Keywords: visual perception, зрительное восприятие, deep learning, TMS, сверточные сети, транскраниальная магнитная стимуляция (ТМС), virtual lesions

In book

First International Conference, AIiH 2024, Swansea, UK, September 4–6, 2024, Proceedings, Part II. Artificial Intelligence in Healthcare. LNCS, volume 14976

Vol. 14976. , Springer, 2024.