Empowerment Evaluation of Programs Involving Youth Evaluators’ Perceptions
Article Sidebar
Main Article Content
Abstract
Background: Participatory and collaborative evaluation approaches, including Empowerment Evaluation (EE), are useful for evaluating programs involving youth. Empowerment evaluation involves stakeholders in the evaluation process through a set of structured steps. It is primarily concerned with empowering, illuminating, and building program beneficiaries’ self-determination. Given the emphasis that EE places on inclusivity of stakeholders, it appears to be a good fit for evaluating programs that involve youth.
Purpose: To explore the extent to which evaluators use EE to evaluate programs involving youth as well as what factor(s) facilitate and hinder their use of EE in these programs.
Setting: The study involved evaluators associated with the Collaborative, Participatory and Empowerment Evaluation and Youth-Focused Evaluation Targeted Interest Groups (TIGs) of the American Evaluation Association (AEA) who are involved in evaluating programs targeted at youth.
Intervention: Not applicable.
Research Design: We used a two-phase sequential mixed-methods research design. In Phase 1, we surveyed evaluators. In Phase 2, we interviewed a sample of evaluators from Phase 1.
Findings: In Phase 1, 41 (53.9%) respondents indicated not using EE to evaluate programs involving youth, 30 (39.5%) had used EE and 5 (6.6%) were unsure. Of those who used EE, they used it to teach youth program stakeholders about evaluation (n=8, 24.2%), produce more authentic results by engaging youth as experts of their lived experience (n=7, 21.2%) or produce more useful results for stakeholders to use (n=6, 18.2%), as well as other less popular reasons. In Phase 2, 12 interviewees raised five factors that facilitate or hinder the use of EE to evaluate programs involving youth including, evaluator perceptions, type of evaluation experience, evaluator knowledge and professional training, guidelines from organizations and funders, and stakeholders and time. Factors that some interviewees viewed as facilitators others viewed as hinderances.
Keywords: empowerment evaluation, program evaluation, youth-focused evaluation
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
American Evaluation Association. (2019). Topical interest groups. https://comm.eval.org/communities/allcommunities
Barrington, G. V. (1999). Empowerment goes large scale: The Canada prenatal nutrition experience. The Canadian Journal of Program Evaluation, 14, 179–192.
Beresford, P. (2000). Service users’ knowledges and social work theory: Conflict or collaboration? British Journal of Social Work, 30(4), 489–503. https://doi.org/10.1093/bjsw/30.4.489
Berg, B. (2008). Qualitative research methods for the social sciences (7th ed.). Pearson Education.
Bulanda, J. J., Szarzynski, K., Siler, D., & McCrea, K. T. (2013). “Keeping it real”: An evaluation audit of five years of youth-led program evaluation. Smith College Studies in Social Work, 83(2–3), 279–302. https://doi.org/10.1080/00377317.2013.802936
Checkoway, B., & Richards‐Schuster, K. (2003). Young people as competent citizens. Community Development Journal, 38(4), 298–309. https://doi.org/10.1093/cdj/38.4.298
Chen, P., Weiss, F. L., & Nicholson, H. J. (2010). Girls study Girls Inc.: Engaging girls in evaluation through participatory action research. American Journal of Community Psychology, 46(1–2), 228–237. https://doi.org/10.1007/s10464-010-9328-7
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues for field settings. Rand McNally College Publishing Company.
Cousins, J. B. (2005). Will the real empowerment evaluation please stand up? A critical friend perspective. In D. M. Fetterman & A. Wandersman (Eds.), Empowerment evaluation principles in practice (pp. 183–208). Guilford Press.
Cousins, J. B., Donohue, J. J., & Bloom, G. A. (1995). Collaborative evaluation in North America: Evaluators’ self-reported opinions, practices and consequences. Evaluation Practice, 17(3), 207–226. https://doi.org/10.1177/109821409601700302
Cousins, J. B., & Whitmore, E. (1998). Framing participatory evaluation. New Directions for Evaluation, 1998(80), 5–24. https://doi.org/10.1002/ev.1114
Carley-Baxter, L.R., C.A. Hill, D.J. Roe, S.E. Twiddy, R.K. Baxter & Ruppenkamp, J. (2009). Does response rate matter? Journal editors use of survey quality measures in manuscript publication decisions. Survey Practice. 2 (7), 1-7.
Creswell, J. W. (2014). A concise introduction to mixed methods research. SAGE Publications, Inc.
Creswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Sage Publications.
Dillman, D. A. (2011). Mail and internet surveys: The tailored design method (2nd ed.). John Wiley & Sons.
Dold, C. J., & Chapman, R. A. (2012). Hearing a voice: Results of a participatory action research study. Journal of Child and Family Studies, 21(3), 512–519. https://doi.org/10.1007/s10826-011-9505-9
Doyle, L., Brady, A. M., & Byrne, G. (2009). An overview of mixed methods research. Journal of Research in Nursing, 14(2), 175–185. https://doi.org/10.1177/1744987108093962
Fetterman, D. M. (1994). Empowerment evaluation. Evaluation Practice, 15(1), 1–15. https://doi.org/10.1177/109821409401500101
Fetterman, D. M. (2001). Foundations of empowerment evaluation. Sage Publications.
Fetterman, D. M., Rodríguez-Campos, L., & Zukoski, A. (2018). Collaborative, participatory, and empowerment evaluation. Guilford Press.
Fetterman, D. M., & Wandersman, A. (Eds.). (2005). Empowerment evaluation principles in practice. Guilford Press.
Flores, K. S. (2007). Youth participatory evaluation: Strategies for engaging young people. Jossey-Bass.
Fox, J., & Cater, M. (2011). Participatory evaluation: Factors to consider when involving youth. Journal of Extension, 49(2).
Kaplowitz, M. D., Hadlock, T. D. & Levine, R. (2004). A comparison of web and mail survey response rates. Public Opinion Quarterly, 68(1), 94-101.
Lancaster, G. A., Dodd, S., & Williamson, P. R. (2004). Design and analysis of pilot studies: Recommendations for good practice. Journal of Evaluation in Clinical Practice, 10(2), 307–312. https://doi.org/10.1111/j..2002.384.doc.x
Langhout, R. D., & Fernandez, J. S. (2015). Empowerment evaluation conducted by fourth- and fifth-grade students. In D. M. Fetterman, S. J. Kaftarian, & A. Wandersman (Eds.), Empowerment evaluation: Knowledge and tools for self-assessment, evaluation capacity building, and accountability (2nd ed., pp. 193–232). SAGE Publications, Inc. https://dx.doi.org/10.4135/9781483387079
Lee, S. (2008). Noncoverage. In P. J. Lavrakas (Ed.), Encyclopedia of survey research methods (pp. 520–521). SAGE Publications, Inc. http://dx.doi.org/10.4135/9781412963947.n333
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. SAGE Publications, Inc.
Mark, M. M. (2008). Building a better evidence base for evaluation theory: Beyond general calls to a framework of types of research on evaluation. In N. L. Smith & P. R. Brandon (Eds.), Fundamental issues in evaluation (pp. 111–134). Guilford Press.
Miles, M. B., Huberman, A. M., & Saldana, J. (2020). Qualitative data analysis: A method sourcebook (4th ed.). SAGE Publications, Inc.
Miller, R. L., & Campbell, R. (2006). Taking stock of empowerment evaluation: An empirical review. American Journal of Evaluation, 27(3), 296–319. https://doi.org/10.1177/109821400602700303
Moreau, K. A., & Cousins, J. B. (2012). Program evaluation in family-centred pediatric rehabilitation settings: A review of evaluation studies and the potential use of participatory and collaborative evaluation approaches. Evaluation Journal of Australasia,11(2), 3–13. https://doi.org/10.1177/1035719X1101100202
Patton, M. Q. (1997). Utilization-focused evaluation: The new century text. SAGE Publications, Inc.
Patton, M. Q. (2005). Utilization-focused evaluation. In S. Mathison (Ed.), Encyclopaedia of evaluation (pp. 429–432). SAGE Publications, Inc. https://dx.doi.org/10.4135/9781412950558
Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice (4th ed.). SAGE Publications, Inc.
Samuelson, B. L., Smith, R., Stevenson, E., & Ryan, C. (2013). A case study of youth participatory evaluation in co-curricular service learning. Journal of the Scholarship of Teaching and Learning, 13(3), 63–81.
Seidman, I. (2013). Interviewing as qualitative research: A guide for researchers in education and the social sciences (4th ed.). Teachers College Press.
Sheldon, J. F. (2016). Evaluation as social intervention: An empirical study of empowerment evaluation practice and principle effects on psychological empowerment and self-determination outcomes [Unpublished doctoral dissertation]. The Claremont Graduate University.
Smith, N. L. (1993). Improving evaluation theory through the empirical study of evaluation practice. Evaluation Practice, 14(3), 237–242. https://doi.org/10.1177/109821409301400302
Upshur, C. C., & Barreto-Cortez, E. (1995). What is participatory evaluation (PE)? What are its roots. The Evaluation Exchange, 1(3/4), 1–7.
Varpio, L., Ajjawi, R., Monrouxe, L. V., O’Brien, B. C., & Rees, C. E. (2016). Shedding the cobra effect: Problematising thematic emergence, triangulation, saturation and member checking. Medical education, 51(1), 40–50. https://doi.org/10.1111/medu.13124
Wandersman, A., Alia, K. A., Cook, B., & Ramaswamy, R. (2015). Integrating empowerment evaluation and quality improvement to achieve healthcare improvement outcomes. British Medical Journal, 24(10), 645–652. https://doi.org/10.1136/bmjqs-2014-003525
Wandersman, A., Imm, P., Chinman, M., & Kaftarian, S. (2000). Getting to outcomes: A results-based approach to accountability. Evaluation and Program Planning, 23(3), 389–395.
Wanzer, D. & Wisner, D. (2020, April 1). Research on evaluation: It takes a village (the problem). Dana Wanzer. https://danawanzer.com/roe-theproblem/
Zeller-Berkman, S., Muñoz-Proto, C., & Torre, M. E. (2015). A youth development approach to evaluation: Critical participatory action research. Afterschool Matters, 22, 24–31. https://files.eric.ed.gov/fulltext/EJ1083955.pdf