Descriptive, but not Injunctive, Normative Appeals Increase Response Rates In Web-based Surveys
Main Article Content
Abstract
Background: In an earlier experiment Misra, Stokols, & Marino (2012) found that participants who received a descriptive normative prompt in the message requesting them to complete an online survey were more likely to comply with the request compared to participants who did not receive any normative prompts.
Purpose: Building on that earlier study, the present field experiment compared the separate and additive effects of descriptive and injunctive norm- based persuasive messages on response rates of online surveys. We also investigate the influence of email reminders on response rates.
Intervention: Participants in an interdisciplinary conference were assigned to one of four groups. The three experimental groups received one of the following messages asking them to complete an online survey that highlighted: (1) a descriptive social norm indicating typical response rates among attendees of prior similar conferences; (2) an injunctive norm appealing them to join fellow participants in completing the survey; or (3) both social norms. The control group received a generic request to complete the online survey without any norm-based appeals.
Research Design: This study used an experimental design which afforded a within subjects replication of participants’ assignment to each of two treatment groups and the control prompts for two successive surveys.
Data Collection and Analysis: One hundred and twenty nine conference participants were requested to complete an online questionnaire about their experiences of an interdisciplinary conference in Fall 2011, immediately following the conference (Time 1) and then three months later (Time 2). At both time points, participants were given three weeks to complete the online surveys. For both time points, participants who had not completed the survey one week after they were sent the initial request to complete the survey received an e-mail reminder. One week after the first email reminder, participants who had still not answered the survey were sent a second email reminder. Once the online surveys were closed at both time points, response rates were calculated. To evaluate the differences in response rates among the various treatment and control groups at both time points, we conducted one-tailed z-tests for proportions to ascertain the z-scores.
Findings: Participants receiving a message highlighting the descriptive social norm when asked to complete an online survey were more likely to comply with the request compared to all the other groups. Additionally, one and two email reminders were found to be effective in improving response rates of online surveys.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Misra, S., Stokols, D., & Marino, A. H. (2012). Using norm-based appeals to increase response rates in evaluation research: A field experiment. American Journal of Evaluation, 33, 88-98. doi: 10.1177/1098214011414862 https://doi.org/10.1177/1098214011414862 DOI: https://doi.org/10.1177/1098214011414862
Bator, R. J., & Cialdini, R. B. (2000). The application of persuasion theory to the development of effective pro-environmental public service announcements. Journal of Social Issues, 56, 527-541. https://doi.org/10.1111/0022-4537.00182 DOI: https://doi.org/10.1111/0022-4537.00182
Cialdini, R. B., Reno, R. R., & Kallgren, C. A. (1990). A focus theory of normative conduct: Recycling the concept of norms to reduce littering in public places. Journal of Personality and Social Psychology, 58, 1015-1026. https://doi.org/10.1037/0022-3514.58.6.1015 DOI: https://doi.org/10.1037//0022-3514.58.6.1015
Galen, M. & Grodzicki, D (2011). Utilizing emerging technology in program evaluation. In S. Mathison (Ed.). Really New Directions On Evaluation: Young Evaluators' Perspectives. New Directions for Evaluation, 131, 123-128. https://doi.org/10.1002/ev.389 DOI: https://doi.org/10.1002/ev.389
Goldstein, N., Cialdini, R. B., & Griskevicius, V. (2008). A room with a viewpoint: Using norm-based appeals to motivate conservation behaviors in a hotel setting. Journal of Consumer Research, 35, 472-482. https://doi.org/10.1086/586910 DOI: https://doi.org/10.1086/586910
Greenlaw, C., & Brown-Welty, S. (2009). A comparison of web-based and paper-based survey methods: Testing assumptions of survey mode and response costs. Evaluation Review, 33, 464-480. https://doi.org/10.1177/0193841X09340214 DOI: https://doi.org/10.1177/0193841X09340214
Henry, G. T., & Mark, M. M. (2003). Toward an agenda for research on evaluation. New Directions for Evaluation, 97, 69-80. https://doi.org/10.1002/ev.77 DOI: https://doi.org/10.1002/ev.77
Jamieson, V., & Azzam, T. (2012). Use of technology in evaluation practice. Journal of Multidisciplinary Evaluation, 8(18), 1-15. Retrieved from http://survey.ate.wmich.edu/jmde/index.php/jmde_1/article/view/340/345. https://doi.org/10.56645/jmde.v8i18.340 DOI: https://doi.org/10.56645/jmde.v8i18.340
Kallgren, C. A., Reno, R. R., & Cialdini, R. B. (2000). A focus theory of normative conduct: When norms do and do not affect behavior. Personality and Social Psychology Bulletin, 26, 1002-1012. https://doi.org/10.1177/01461672002610009 DOI: https://doi.org/10.1177/01461672002610009
Reno, R. R., Cialdini, R. B., & Kallgren, C. A. (1993). The transsituational influence of social norms. Journal of Personality and Social Psychology, 64, 104-112. https://doi.org/10.1037/0022-3514.64.1.104 DOI: https://doi.org/10.1037//0022-3514.64.1.104
Russell, C. W., Boggs, D. A., Palmer, J. R., & Rosenberg, L. (2010). Use of a Web-based Questionnaire in the Black Women's Health Study. American Journal of Epidemiology, 172(11), 1286-1291. https://doi.org/10.1093/aje/kwq310 DOI: https://doi.org/10.1093/aje/kwq310
Schultz, P. W. (1999). Changing behavior with normative feedback interventions: A field experiment on curbside recycling. Basic and Applied Social Psychology, 21, 25-38. https://doi.org/10.1207/s15324834basp2101_3 DOI: https://doi.org/10.1207/s15324834basp2101_3
Shadish, W. R., Jr., Cook, T. D., & Leviton, L.C.L. (1991). Foundations of Program Evaluation. Thousand Oaks, CA: Sage Publications.
Szanyi, M., Azzam, T., & Galen, M. (2013). Research on evaluation: A needs assessment. Canadian Journal of Program Evaluation, 27(1), 39-64. https://doi.org/10.3138/cjpe.027.002 DOI: https://doi.org/10.3138/cjpe.027.002