Forms and Functions of Participatory Evaluation in International Development: A Review of the Empirical and Theoretical Literature
Main Article Content
Abstract
Background: Since the late 1970s participatory approaches have been widely promoted to evaluate international development programs. However, there is no universal agreement of what is meant by participatory evaluation. For some evaluators, participatory evaluations involve the extensive participation of all stakeholder groups (from donor to non-recipients) in every phase of the evaluation (from design to dissemination). For others, the participation of donors in the design constitutes a participatory evaluation approach. Participatory evaluation approaches are best considered on a continuum. In other words, there are many gradations to participation and evaluations should be classified accordingly.
Purpose: The lack of shared meaning of participatory evaluation approaches also impedes serious discussion on their use including their merits and demerits, suggestions for their improvement, and their overall effectiveness. The purpose of this article is to present an examination of the literature on participatory evaluation approaches to highlight commonalities and differences.
Setting: Not applicable
Intervention: Not applicable.
Research Design: Not applicable.
Data Collection and Analysis: Desk review.
Findings: This article clearly demonstrates how broadly participatory evaluation is conceptualized and practiced and underscores the clear need for specification and precision when discussing what is meant by participatory evaluation. Recommendations for how evaluators should describe participatory evaluations are provided.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Bamberger, M., Rugh, J., & Mabry, L. (2006). Real world evaluation: Working under budget, time, data, and political constraints. Thousand Oaks, CA: Sage.
Brisolara, S. (1998). The history of participatory evaluation and current debates in the field. New Directions for Evaluation, 80, 25-41. https://doi.org/10.1002/ev.1115 DOI: https://doi.org/10.1002/ev.1115
Brunner, I., & Guzman, A. (1989). Participatory evaluation: A tool to assess projects and empower people. New Directions for Evaluation, 42, 9- 18. https://doi.org/10.1002/ev.1509 DOI: https://doi.org/10.1002/ev.1509
Bryk, A. S. (Ed.). (1983). Stakeholder- based evaluation. [Special issue]. New Directions for Evaluation, 17. https://doi.org/10.1002/ev.1321 DOI: https://doi.org/10.1002/ev.1321
Chambers, R. (1992). Rapid but relaxed and participatory rural appraisal: Towards applications in health and nutrition. In N. S. Scrimshaw and G. R. Gleason (Eds.), Rapid Assessment Procedures: Qualitative methodologies for planning and evaluation of health related programmes (Section III, Chap. 24). Boston, MA: International Nutrition Foundation for Developing Countries.
Chambers, R. (1994). The origins and practice of participatory rural appraisal. World Development, 22, 953-969. https://doi.org/10.1016/0305-750X(94)90141-4 DOI: https://doi.org/10.1016/0305-750X(94)90141-4
Chambers, R. (2007). From PRA to PLA to Pluralism (IDS Working Paper, No. 286). Sussex, UK: IDS.
Chung, K. (2000). Issues and approaches in the use if integrated methods. In M. Bamberger (Ed.), Integrating quantitative and qualitative research in development projects (pp. 37-46). Washington, DC: The World Bank.
Coryn, C. L. S., Noakes, L. A., Westine, C. D., & Schröter, D. C., (2011). A systematic review of theory-driven evaluation practice from 1990 to 2009. American Journal of Evaluation, 32(2), 199-226. https://doi.org/10.1177/1098214010389321 DOI: https://doi.org/10.1177/1098214010389321
Cousins, J. B. (1996). Consequences of researcher involvement in participatory evaluation. Studies in Educational Evaluation, 22, 3-27. https://doi.org/10.1016/0191-491X(96)00001-6 DOI: https://doi.org/10.1016/0191-491X(96)00001-6
Cousins, J. B. (2001). Do evaluator and program practitioner perspectives converge in collaborative evaluation? Canadian Journal of Program Evaluation, 16, 113-133. https://doi.org/10.3138/cjpe.16.008 DOI: https://doi.org/10.3138/cjpe.16.008
Cousins, J. B. (2003). Utilization efforts of participatory evaluation. In T. Kellaghan & D. L. Stufflebeam (Eds.), International handbook of educational evaluation(pp.245-266). Dordrecht, Netherlands: Kluwer. https://doi.org/10.1007/978-94-010-0309-4_16 DOI: https://doi.org/10.1007/978-94-010-0309-4_16
Cousins, J. B. (2004). Commentary: Minimizing evaluation misuse as principled practice. American Journal of Evaluation, 25, 391-397. https://doi.org/10.1177/109821400402500311 DOI: https://doi.org/10.1177/109821400402500311
Cousins, J. B., Donohue, J. J., & Bloom, G. A. (1996). Collaborative evaluation in North America: Evaluators' self- reported opinions, practices and consequences. Evaluation Practice, 17, 207-226. https://doi.org/10.1177/109821409601700302 DOI: https://doi.org/10.1016/S0886-1633(96)90002-1
Cousins, J. B., & Earl, L. M. (1992). The case for participatory evaluation. Educational Evaluation and Policy Analysis, 14, 397-418. https://doi.org/10.3102/01623737014004397 DOI: https://doi.org/10.3102/01623737014004397
Cousins, J. B., & Earl, L. M. (1999). When the boat gets missed: Response to M.F. Smith. American Journal of Evaluation, 20, 309-317. https://doi.org/10.1177/109821409902000212 DOI: https://doi.org/10.1016/S1098-2140(99)00015-6
Cousins, J. B., & Whitmore, E. (1998). Framing participatory evaluation. New Directions for Evaluation, 80, 5-23. https://doi.org/10.1002/ev.1114 DOI: https://doi.org/10.1002/ev.1114
Cullen, A. (2009). The politics and consequences of stakeholder participation in international development evaluations. Unpublished doctoral dissertation, Western Michigan University, Kalamazoo.
Cullen, A. E., Coryn, C. L. S., & Rugh, J. (2011). The politics and consequences of including stakeholders in international development evaluation. American Journal of Evaluation, 32(3), 345-361. https://doi.org/10.1177/1098214010396076 DOI: https://doi.org/10.1177/1098214010396076
Dart. (2005). Rapid rural appraisal. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 357-358). Thousand Oaks, CA: Sage.
Donaldson, S. I., Patton, M. Q., Fetterman, D. M., & Scriven, M. (2010). The 2009 Claremont debates: The promise and pitfalls of utilization- focused and empowerment evaluation. Journal of MultiDisciplinary Evaluation, 6, 15-57. https://doi.org/10.56645/jmde.v6i13.260 DOI: https://doi.org/10.56645/jmde.v6i13.260
Estrella, M. & Gaventa, J. (1998). Who counts reality? Participatory monitoring and evaluation: A literature review (IDS Working Paper, No. 70). Sussex, UK: IDS.
Estrella, M., Blauert, J., Campilan, D., Gaventa, J., Gonsalves, J., Guijt, I., Johnson, D. & Ricafort, R. (Eds.). (2000). Learning from change: issues and experiences in participatory monitoring and evaluation. London, UK: IT Books/IDRC. https://doi.org/10.3362/9781780441214 DOI: https://doi.org/10.3362/9781780441214.000
Fetterman, D. M. (1994). Empowerment evaluation. Evaluation Practice, 1, 1- 15. https://doi.org/10.1177/109821409401500101 DOI: https://doi.org/10.1016/0886-1633(94)90055-8
Fetterman, D. M. (2001). Foundations of empowerment evaluation. Thousand Oaks, CA: Sage.
Fetterman, D. M. (2005). Empowerment evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 125- 129). Thousand Oaks, CA: Sage.
Fetterman, D. M. (2005). In response to Drs. Patton and Scriven. American Journal of Evaluation, 26, 418-420. https://doi.org/10.1177/1098214005278769 DOI: https://doi.org/10.1177/1098214005278769
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2004). Program evaluation: Alternative approaches and practical guidelines (3rd edition). Boston, MA: Pearson.
Francis, P. (2001). Participatory development at The World Bank: The primacy of process. In B. Cooke & U. Kothari (Eds.), Participation: The new tyranny? London, UK: Zed Books Ltd.
Freire, P. (1970). Pedagogy of the oppressed. New York, NY: Continuum. Hall, B. L. (1981). Participatory research, popular knowledge and power: A personal reflection. Convergence: An International Journal of Adult Education, 14, 6-19.
Hall, B. L. (1992). From margins to center? The development and purpose of participatory research. The American Sociologist, 23, 15-28. https://doi.org/10.1007/BF02691928 DOI: https://doi.org/10.1007/BF02691928
Holland, J. H., & Blackburn, J. (1998). Whose voice? Participatory research and policy change. London, UK: Intermediate Technology Publications. https://doi.org/10.3362/9781780446431 DOI: https://doi.org/10.3362/9781780446431
House, E. R. (2005) Deliberative democratic evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 104-108). Thousand Oaks, CA: Sage.
House, E. R., & Howe K. R. (2000). Deliberative democratic evaluation checklist. Retrieved February 24, 2009, from http://www.wmich.edu/evalctr/checklists/dd_checklist.PDF https://doi.org/10.1002/ev.1157 DOI: https://doi.org/10.1002/ev.1157
King, J. A. (2005). Participatory evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 291- 294). Thousand Oaks, CA: Sage.
MacDonald, B., & Kushner, S. (2005). Democratic evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 109-113). Thousand Oaks, CA: Sage.
Mark, M. M., & Shotland, R. L. (1985). Stakeholder-based evaluation and value judgments. Evaluation Review, 9, 605-626. https://doi.org/10.1177/0193841X8500900504 DOI: https://doi.org/10.1177/0193841X8500900504
Mathison, S. (Ed.). (2005). Encyclopedia of evaluation. Thousand Oaks, CA: Sage. Merriam-Webster's collegiate dictionary (11th ed.). (2003). Springfield, MA: Merriam-Webster, Inc. https://doi.org/10.4135/9781412950558 DOI: https://doi.org/10.4135/9781412950558
Miller, R. L. (2010). Developing standards for empirical examinations of evaluation theory. American Journal of Evaluation, 31, 390-399. https://doi.org/10.1177/1098214010371819 DOI: https://doi.org/10.1177/1098214010371819
O'Sullivan, R. G., & D'Agostino, A. (2002). Promoting evaluation through collaboration: Findings from community-based programs for young children and their families. Evaluation, 8, 372-387. https://doi.org/10.1177/135638902401462466 DOI: https://doi.org/10.1177/135638902401462466
Park, P. (1992). The discovery of participatory research as a new scientific paradigm: Personal and intellectual accounts. The American Sociologist, 23, 29-42. https://doi.org/10.1007/BF02691929 DOI: https://doi.org/10.1007/BF02691929
Patton, M. Q. (2005a). Developmental evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (p. 116). Thousand Oaks, CA: Sage.
Patton, M. Q. (2005b). Utilization-focused evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 429- 432). Thousand Oaks, CA: Sage.
Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Thousand Oaks, CA: Sage.
Rodriguez-Campos, L. (2005). Collaborative evaluations: A step-by- step model for the evaluator. Tarmac, FL: Lumina Press.
Rugh, J. (1986). Self-evaluation: Ideas for participatory evaluation of rural community development projects. Oklahoma City, OK: World Neighbors.
Ryan, K., Green, K., Lincoln, Y., Mathison, S., & Mertens, D. M. (1998). Advantages and challenges of using inclusive evaluation approaches in evaluation practice. American Journal of Evaluation, 19, 101-122. https://doi.org/10.1177/109821409801900111 DOI: https://doi.org/10.1177/109821409801900111
Ryan, K. (2004). Serving public interests in educational accountability: Alternative approaches to democratic evaluation. American Journal of Evaluation, 25, 443-461. https://doi.org/10.1177/109821400402500403 DOI: https://doi.org/10.1016/j.ameval.2004.09.005
Ryan, K. (2005). Democratic evaluation approaches for equity and inclusion. Evaluation Exchange, 11, 2-3.
Salmen, L. F., & Kane, E. (2006). Bridging diversity: Participatory learning for responsive development. Washington, DC: The World Bank. https://doi.org/10.1596/978-0-8213-6336-2 DOI: https://doi.org/10.1596/978-0-8213-6336-2
Scrimshaw, N. S., & Gleason, G. R. (Eds.). (1992). Rapid assessment procedures: Qualitative methodologies for planning and evaluation of health related programmes. Boston, MA: International Nutrition Foundation for Developing Countries.
Scriven, M. (1997). Empowerment evaluation examined. Evaluation Practice, 18, 165-175. https://doi.org/10.1177/109821409701800115 DOI: https://doi.org/10.1177/109821409701800207
Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Newbury Park, CA: Sage.
Smits,P.A.,&Champagne,F.(2008).An assessment of the theoretical underpinnings of practical participatory evaluation. American Journal of Evaluation, 29, 427-442. https://doi.org/10.1177/1098214008325023 DOI: https://doi.org/10.1177/1098214008325023
Stake, R. E. (2004). Standards-based and responsive evaluation. Thousand Oaks, CA: Sage. https://doi.org/10.4135/9781412985932 DOI: https://doi.org/10.4135/9781412985932
Stake, R. E., & Abma, T. (2005). Responsive evaluation. In S. Mathison (Ed.), Encyclopedia of evaluation (pp. 376-379). Thousand Oaks, CA: Sage.
Stoecker, R., & Bonacich, E. (1992). Why participatory research? Guest editors' introduction. The American Sociologist, 23, 5-14. https://doi.org/10.1007/BF02691927 DOI: https://doi.org/10.1007/BF02691927
Stufflebeam, D. L. (1994). Empowerment evaluation, objectivist evaluation, and evaluation standards: Where the future of evaluation should not go and where it needs to go. Evaluation Practice, 15, 321-338. https://doi.org/10.1177/109821409401500313 DOI: https://doi.org/10.1177/109821409401500313
Stufflebeam, D. L. (2001). Evaluation models. New Directions in Evaluation, 89. San Francisco, CA: Jossey-Bass. https://doi.org/10.1002/ev.1 DOI: https://doi.org/10.1002/ev.3
Stufflebeam, D. L., & Shinkfield, A. J. (2007). Evaluation theory, models, and applications. Somerset, NJ: John Wiley & Sons.
Townsley, P. (1996). The history of RRA and PRA [Appendix 1]. In Rapid rural appraisal, participatory rural appraisal and aquaculture (FAO Fisheries Technical Paper, No. 358). Rome, Italy: Fisheries and Aquaculture Department.
Wandersman, A., & Snell-Johns, J. (2005). Empowerment evaluation: Clarity, dialogue, and growth. American Journal of Evaluation, 26, 421-428. https://doi.org/10.1177/1098214005278774 DOI: https://doi.org/10.1177/1098214005278774
Weaver, L., & Cousins, J. B. (2004). Unpacking the participatory process. Journal of MultiDisciplinary Evaluation, 1, 19-40. https://doi.org/10.56645/jmde.v1i1.144 DOI: https://doi.org/10.56645/jmde.v1i1.144
Weiss, C. H. (1983). The stakeholder approach to evaluation: Origins and promise. New Directions for Program Evaluation, 17, 3-14. https://doi.org/10.1002/ev.1322 DOI: https://doi.org/10.1002/ev.1322