Measuring Respondents’ Cognitive Load Related to Making Factual and Normative Judgments

Monitoring Obshchestvennogo Mneniya: Ekonomichekie i Sotsial'nye Peremeny. 2022. No. 6. P. 291–308.

Deviatko I. F., Byzov A.

The use of various methods for measuring cognitive load and mental effort in recent years has become increasingly popular in various fields of social and affective neuroscience, in applied research on the comparative effectiveness of teaching methods and training platforms, in the study of the distribution of attention in solving various problems and using informational tips in decision making, etc. In this broader context, the specific request for a multimodal assessment of the cognitive load of interviewers and respondents and of its impact on the quality of the survey data, including the use of paradata and webcams for this purpose, has been also growing recently. We conducted a within-subject methodological experiment (N = 50) aiming at comparative measurement of task-evoked cognitive load of respondents related to two tasks of making factual and normative judgments. The first task implied making causal and blame judgments for two institutional domains (medical, work dress-code) using the similar factorial vignettes, while the second task presupposed making lay factual and normative-deontic judgments concerning migrant rights to free health care. We used in parallel two measures of task-evoked cognitive load – pupillometry (Pupil Lab glasses), and the Paas scale of mental effort. The results provide limited evidence in support of the difference that exists between ordinary judgments of cause, blame, and severity of harm in terms of their propensity to evoke psychosensory pupillary response and subjectively perceived mental effort, both reflecting the variability in the cognitive load imposed on survey respondent when performing a pertinent survey task. We also briefly discuss the evidence obtained in support of the task-specific difference in sensitivity and validity of neurophysiological and self-report-based measures of survey-related cognitive load.

Keywords: cognitive load, когнитивная нагрузка, mental effort, factual judgements, normative judgements, vignette experiment, умственное усилие, фактические суждения, нормативные суждения, факторный эксперимент с виньетками