Insights into the accuracy of social scientists’ forecasts of societal change

Nature Human Behaviour. 2023. Vol. 7. No. 4. P. 484–501.

Grossmann I., Rotella A., Hutcherson C., Sharpinskyi K., Varnum M. ., Achter S., Dhami M., Guo X. E., Kara-Yakoubian M., Mandel D., Raes L., Tay L., Vie A., Wagner L., Adamkovic M., Arami A., Arriaga P., Bandara K., Baník G., Bartoš F., Baskin E., Bergmeir C., Białek M., Børsting C., Browne D., Caruso E., Chen R., Chie B., Chopik W., Collins R., Cong C. W., Conway L., Davis M., Day M., Dhaliwal N., Durham J., Dziekan M., Elbaek C., Shuman E., Fabrykant M., Firat M., Fong G., Frimer J., Gallegos J., Goldberg S., Gollwitzer A., Goyal J., Graf-Vlachy L., Gronlund S., Hafenbrädl S., Hartanto A., Hirshberg M., Hornsey M., Howe P. ., Izadi A., Jaeger B., Kačmár P., Kim Y. J., Krenzler R., Lannin D., Lin H., Lou N. M., Lua V. ., Lukaszewski A., Ly A., Madan C., Maier M., Majeed N., March D., Marsh A., Misiak M., Myrseth K., Napan J., Nicholas J., Nikolopoulos K., O J., Otterbring T., Paruzel-Czachura M., Pauer S., Protzko J., Raffaelli Q., Ropovik I., Ross R., Roth Y., Røysamb E., Schnabel L., Schütz A., Seifert M., Sevincer A. T., Sherman G., Simonsson O., Sung M., Tai C., Talhelm T., Teachman B., Tetlock P., Thomakos D., Tse D. ., Twardus O., Tybur J., Ungar L., Vandermeulen D., Williams L. V., Vosgerichian H., Wang Q., Wang K., Whiting M., Wollbrant C., Yang T., Yogeeswaran K., Yoon S., Alves V., Andrews-Hanna J., Bloom P., Boyles A., Charis L., Choi M., Darling-Hammond S., Ferguson Z. E., Kaiser C., Karg S., López Ortega A., Mahoney L., Marsh M., Martinie M. ., Michaels E., Millroth P., Naqvi J., Ng W., Rutledge R., Slattery P., Smiley A., Strijbis O., Sznycer D., Tsukayama E., van Loon A., Voelkel J., Wienk M. ., Wilkening T.

How well can social scientists predict societal change, and what processes
underlie their predictions? To answer these questions, we ran two
forecasting tournaments testing the accuracy of predictions of societal
change in domains commonly studied in the social sciences: ideological
preferences, political polarization, life satisfaction, sentiment on social
media, and gender–career and racial bias. After we provided them with
historical trend data on the relevant domain, social scientists submitted
pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams
and 359 forecasts), with an opportunity to update forecasts on the basis of
new data six months later (Tournament 2; N = 120 teams and 546 forecasts).
Benchmarking forecasting accuracy revealed that social scientists’ forecasts
were on average no more accurate than those of simple statistical models
(historical means, random walks or linear regressions) or the aggregate
forecasts of a sample from the general public (N = 802). However, scientists
were more accurate if they had scientifc expertise in a prediction domain,
were interdisciplinary, used simpler models and based predictions on
prior data.

Keywords: forecasting, прогнозирование, социальные изменения, social change, предсказание, prediction

Publication based on the results of: Basic human values and subjective evaluations of living conditions: their evolution in cross-country perspective (2023)