?
Toward Cumulative Cognitive Science: A Comparison of Meta-Analysis, Mega-Analysis, and Hybrid Approaches
There is increasing interest in cumulative approaches to science, in which instead of
analyzing the results of individual papers separately, we integrate information
qualitatively or quantitatively. One such approach is meta-analysis, which has over 50
years of literature supporting its usefulness, and is becoming more common in
cognitive science. However, changes in technical possibilities by the widespread use of
Python and R make it easier to fit more complex models, and even simulate missing
data. Here we recommend the use of mega-analyses (based on the aggregation of
data sets collected by independent researchers) and hybrid meta-mega-analytic
approaches, for cases where raw data is available for some studies. We illustrate the
three approaches using a rich test-retest data set of infants’ speech processing as well
as synthetic data. We discuss advantages and disadvantages of the three approaches
from the viewpoint of a cognitive scientists contemplating their use, and limitations of
this paper, to be addressed in future work.