In this study, nine methods for measuring indirect importance are compared on the basis of their discriminatory power and stability. To the best knowledge of the authors, the stability of results obtained with different methods is assessed for the first time. The deficiencies of an existing criterion for assessing diagnosticity are pointed out and a modified version suggested. The empirical comparison is based on two real-world datasets from the ecommerce industry. Even though none of the methods appeared to be the best according to both criteria simultaneously, there seem to be grounds for recommending the theoretically sound Shapley value decomposition of R-square if stability and discrimination are about equally important for a decision maker, while negative contributions are undesirable.
The paper measures a gamification effect in longitudinal web surveys among children and adolescents 7-15 years old. Two waves of the study were conducted using a volunteer online access panel in Russia among 737 children. Respondents were randomly assigned to one of the three conditions in the first wave without changing the treatment in the second wave: (1) a text-only survey, (2) a visual survey and (3) a gamified survey. Though in the first wave of the study respondents found it more enjoyable and easier to complete the gamified survey, no differences in participation rates were found between the conditions in the second wave. Contrary to expectations, a higher breakoff was found in the gamified condition. Moreover, it produced lower test-retest reliability correlations than the text-only and visual conditions in all survey questions. The promising gamification effect found in the first wave of the study faded in the second wave. It seems that implementing gamified elements in longitudinal web surveys might differ from the implementation of gamified elements in cross-sectional surveys.
Previous studies have not found effective ways of encouraging participants to use smartphones to complete web surveys. We hypothesize that conditional differential incentives (the amount depending on the device the respondent uses to complete the web survey) can increase overall participation rates and the proportion of respondents who use a particular device in web surveys. We conducted an experiment using a volunteer online access panel in Russia with 5,474 invitations sent to regular mobile Internet users. We varied the invitation mode (SMS vs. e-mail) and encouragement to use a particular device for completing the survey – mobile phone or personal computer (PC). SMS increased the proportion of mobile web respondents, while e-mail increased the proportion of PC web respondents. As expected, differential incentives increased the overall participation rates by 8-10 percentage points if higher incentives were offered for completing the survey on a mobile phone. Contrary to expectations, offering higher incentives to PC web respondents did not produce higher participation rates compared to the control condition. Both encouraging the use of a mobile phone and offering higher incentives were effective at increasing the proportion of respondents using mobile devices. In terms of both participation rates and the proportion of respondents using mobile devices, offering incentives 50% higher was as efficient as offering incentives 100% higher for mobile web respondents. Offering higher incentives to mobile web respondents also had an effect on sample composition. Significantly higher participation rates were found among females and those with higher education.