Culturally Competent Evaluation for Aboriginal Communities: A Review of the Empirical Literature
Main Article Content
Abstract
Much of program evaluation is concerned with understanding and improving social programs so that they are ultimately more responsive and more reflective of program participant needs. At the same time, these programs exist and are embedded within specific social, cultural and historical contexts which impact program development, implementation, and eventual outcomes. Evaluations that attempt to address responsiveness to contextual and cultural specificity are often referred to as culturally competent, culturally responsive, inclusive, multicultural, or cross-cultural, among other terms. While there are no agreed upon terminologies, definitions, or even methodologies, what these approaches all share is the recognition that culture and context matter, and that there are no universally agreed upon rules or abstractions that can be applicable in all contexts (Guba & Lincoln, 2005). The recognition of culture and context thus becomes “an explicit criterion rather than an unspoken expectation” (SenGupta, Hopson, & Thompson-Robinson, 2004, p. 15) in evaluations of this type.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
Funding data
References
American Evaluation Association, Diversity Committee. (2004). Final Report: A cultural reading of the program evaluation standards (2nd edition).
Budak, M., & Taylor, D. (2007, June). Photovoice: An innovative method for community involvement in evaluation. Paper presented at the Canadian Evaluation Society, Winnipeg.
Caldwell, J. Y., Davis, J. D., Du Bois, B., Echo- Hawk, H., Erickson, J. S., Goins, R. T., Hill, C., Hillabrant, W., Johnson, S. R., Kendall, E., Keemer, K., Manson, S. P., Marshall, C. A., Running Wolf, P., Santiago, R. L., Schacht, R., & Stone, J. B. (2005). Culturally competent research with American Indians and Alaska Natives: Findings and recommendations of the first symposium of the work group on American Indian research and program evaluation methodology. The Journal of the National Centre, 12 (1), 1-21. DOI: https://doi.org/10.5820/aian.1201.2005.1
https://doi.org/10.5820/aian.1201.2005.1 DOI: https://doi.org/10.5820/aian.1201.2005.1
Christensen, R. A. (2003). Cultural context and evaluation: A balance of form and function, National Science Foundation, Arlington, VA.
Cousins, J. B., & Whitmore, E. (1998). Framing participatory evaluation. In E. Whitmore (ed.), Understanding and practicing participatory evaluation. New Directions for Evaluation, no.80. (pp. 3-23) San Francisco: Jossey-Bass. DOI: https://doi.org/10.1002/ev.1114
https://doi.org/10.1002/ev.1114 DOI: https://doi.org/10.1002/ev.1114
Cousins, J. B. (2003). Utilization effects of participatory evaluation. In T. Kellaghan, D. L. Stufflebeam & L. A. Wingate (Eds.), International handbook of educational evaluation (pp. 245-265). Boston: Kluwer. DOI: https://doi.org/10.1007/978-94-010-0309-4_16
https://doi.org/10.1007/978-94-010-0309-4_16 DOI: https://doi.org/10.1007/978-94-010-0309-4_16
Cousins, J. B. (2005). Will the real empowerment evaluation please stand up? A critical friend perspective (Chapter 8), In D.M. Fetterman, A. Wandersman (Eds.), Empowerment evaluation: Principles in practice (pp. 183-208). NY: The Guilford Press.
Cousins, J. B., Goh, S., & Clark, S.. (2005). Data use leads to data valuing: Evaluative inquiry for school decision making. Leadership and Policy in Schools, 4, 155-176. DOI: https://doi.org/10.1080/15700760500365468
https://doi.org/10.1080/15700760500365468 DOI: https://doi.org/10.1080/15700760500365468
Endo, T., Joh, T., & Cao Yu, H. (2003). Voices from the field: Health and evaluation leaders on multicultural evaluation. Social Policy Research Associates, The California Endowment.
Fisher, P. A., & Ball, T. J. (2002). The Indian family wellness project: An application of the tribal participatory research model. Prevention Science, 3(3), 235-240. DOI: https://doi.org/10.1023/A:1019950818048
https://doi.org/10.1023/A:1019950818048 DOI: https://doi.org/10.1023/A:1019950818048
Fisher, P. A., & Ball, T. J. (2005). Balancing empiricism and local cultural knowledge in the design of prevention research. Journal of Urban Health: Bulletin of the new York Academy of Medicine, 82(2), Supplement 3, iii44-iii55. DOI: https://doi.org/10.1093/jurban/jti063
https://doi.org/10.1093/jurban/jti063 DOI: https://doi.org/10.1093/jurban/jti063
Foucault, M. (1980). The history of sexuality (Volume 1). NY: Vintage.
Gordon, E. W., Miller, F., Rollock, D. (1990). Coping with communicentric bias in knowledge production in the social sciences, Educational Researcher, 19(2), 14-19. DOI: https://doi.org/10.3102/0013189X019003014
https://doi.org/10.3102/0013189X019003014 DOI: https://doi.org/10.3102/0013189X019003014
Greene, J. C., Lisey, M. W., Schwandt, T. A., Smith, N. L., Tharp, R. G. (2007). Method choices: Five discussant commentaries. In G. Julnes, D.J. Rog (eds.), Informing federal policies on evaluation methodologies: Building the evidence base for method choice in government sponsored evaluation. New Directions for Evaluation, no. 113. (pp. 111-127) San Francisco: Jossey-Bass. DOI: https://doi.org/10.1002/ev.218
https://doi.org/10.1002/ev.218 DOI: https://doi.org/10.1002/ev.218
Gregory, A. (2000). Problematizing participation: A critical review of approaches to participation in evaluation theory. Evaluation, 6(2), 179-199. DOI: https://doi.org/10.1177/13563890022209208
https://doi.org/10.1177/13563890022209208 DOI: https://doi.org/10.1177/13563890022209208
Guba, E. G. & Lincoln, Y. S. (2005). Paradigmatic controversies, contradictions, and emerging confluences (Chapter 8). In N.K. Denzin, and Y.S.Lincoln (eds.), SAGE Handbook of Qualitative Research. Thousand Oaks, CA: Sage.
Hall, S. (1999). Cultural studies and its theoretical legacies. In S.During (ed.). The cultural studies reader (pp 97-109). Second edition. NY: Routledge.
Hopson, R. (2001). Global and local conversations on culture, diversity, and social justice in evaluation: Issues to consider in a 9/11 era. American Journal of Evaluation, 22(3), 375-380. DOI: https://doi.org/10.1016/S1098-2140(01)00156-4
https://doi.org/10.1177/109821400102200314 DOI: https://doi.org/10.1177/109821400102200314
Hopson, R. (2003). Overview of multicultural and culturally competent program evaluation: Issues, challenges and opportunities, Social Policy Research Associates, The California Endowment, Oakland, CA.
Hughes, D., Seidman, E., & Williams, N. (1993). Cultural phenomena and the research enterprise: Toward a culturally anchored methodology. American Journal of Community Psychology, 21(6), 687-703. DOI: https://doi.org/10.1007/BF00942243
https://doi.org/10.1007/BF00942243 DOI: https://doi.org/10.1007/BF00942243
Israel, B. A., Schilz, A. J., Parker, E. A., Becker, A. B., Allen, A. J., & Guzman, R. (2003). Critical issues in developing and following community based participatory research principles (Chapter 3), In M. Minkler, N. Wallerstein (Eds.), Community-Based Participatory Research for Health (53-76). San Francisco, CA: John Wiley & Sons, Inc.
Johnston, A. (2005, Oct). Using methods that work with Aboriginal communities. Paper presented at the Joint meeting of the Canadian Evaluation Society and the American Evaluation Association, Toronto.
Johnston, A. (2007, June). Using technology to enhance Aboriginal evaluations II. Paper presented at the Canadian Evaluation Society, Winnipeg.
Jolly, E. J. (2003). One the quest for cultural context in evaluation: All things are not equal-They never were. Proceedings on the Cultural Context of Educational Evaluation (03- 032), National Science Foundation, Arlington, VA.
Kirkhart, K. E. (1995). Seeking multicultural validity: A postcard from the road. Evaluation Practice, 16(1), 1-12. DOI: https://doi.org/10.1016/0886-1633(95)90002-0
https://doi.org/10.1016/0886-1633(95)90002-0 DOI: https://doi.org/10.1016/0886-1633(95)90002-0
Kirkhart, K. E. (2005). Through a cultural lens: Reflections on validity and theory in evaluation (Chapter 3), In S. Hood, R. Hopson, H. Frierson (Eds.), The role of culture and cultural context: A mandate for inclusion, the discovery of truth, and understanding in evaluative theory and practice (pp. 21-39). Greenwich, Connecticut: Information Age Publishing. DOI: https://doi.org/10.1108/978-1-60752-783-120251004
Kumanyika, S. K. (2003). Commentary: Cultural appropriateness: Working our way toward a practicable future. Health Education & Behavior, 30(2), 147-150. DOI: https://doi.org/10.1177/1090198102251022
https://doi.org/10.1177/1090198102251022 DOI: https://doi.org/10.1177/1090198102251022
LaFrance, J. (2004). Culturally competent evaluation in Indian country. In M. Thompson-Robinson, R.Hopson, S. SenGupta (eds.), In search of cultural competency in evaluation. New Directions for Evaluation, no.102. (pp. 39-50) San Francisco: Jossey-Bass. DOI: https://doi.org/10.1002/ev.114
https://doi.org/10.1002/ev.114 DOI: https://doi.org/10.1002/ev.114
Lather, P. (1991). Getting smart: Feminist research and pedagogy with/in the postmodern, NY: Routledge, Chapman and Hall, Inc.
Lather, P. (1993). Fertile Obsession: Validity after poststructuralism. The Sociological Quarterly, 34(4), 673-693. DOI: https://doi.org/10.1111/j.1533-8525.1993.tb00112.x
https://doi.org/10.1111/j.1533-8525.1993.tb00112.x DOI: https://doi.org/10.1111/j.1533-8525.1993.tb00112.x
Letiecq, B. L., & Bailey, S. J. (2004). Evaluating from the outside: Conducting cross-cultural evaluation research on an American Indian reservation. Evaluation Review, 28(4), 342- 357. DOI: https://doi.org/10.1177/0193841X04265185
https://doi.org/10.1177/0193841X04265185 DOI: https://doi.org/10.1177/0193841X04265185
Levin-Rozalis, M. (2003). Evaluation and research: Differences and similarities. The Canadian Journal of Program Evaluation, 18(2), 1-31. DOI: https://doi.org/10.3138/cjpe.18.001
https://doi.org/10.3138/cjpe.18.001 DOI: https://doi.org/10.3138/cjpe.18.001
Lum, D. (2003). Culturally competent practice: A framework for understanding diverse groups and justice issues, in D. Lum, Ed., Pacific Grover, CA: Thomas Learning.
Madison, A-M. (1992). Primary inclusion of culturally diverse minority program participants in the evaluation process. In A.M. Mdison (ed.), Minority issues in program evaluation. New Directions for Evaluation, no. 53. (pp. 35-43) San Francisco: Jossey-Bass. DOI: https://doi.org/10.1002/ev.1599
https://doi.org/10.1002/ev.1599 DOI: https://doi.org/10.1002/ev.1599
Mathie, A., & Greene, J. C. (1997). Stakeholder participation in evaluation: How important is diversity? Evaluation and Program Planning, 20, 279-285. DOI: https://doi.org/10.1016/S0149-7189(97)00006-2
https://doi.org/10.1016/S0149-7189(97)00006-2 DOI: https://doi.org/10.1016/S0149-7189(97)00006-2
McKenzie, B. (1997). Developing First Nations child welfare standards: Using evaluation research within a participatory framework. The Canadian Journal of Program Evaluation, 12(1), 133-148. DOI: https://doi.org/10.3138/cjpe.012.008
https://doi.org/10.3138/cjpe.012.008 DOI: https://doi.org/10.3138/cjpe.012.008
Nelson-Barber, S., LaFrance, J., Trumbull, E., & Aburto, S. (2005) Promoting culturally reliable and valid evaluation practice (Chapter 5), In S. Hood, R. Hopson, H. Frierson (Eds.), The role of culture and cultural context: A mandate for inclusion, the discovery of truth, and understanding in evaluative theory and practice (pp. 61-85.). Greenwich, Connecticut: Information Age Publishing. DOI: https://doi.org/10.1108/978-1-60752-783-120251006
Novins, D. K., King, M., & Son Stone, L. (2004). Developing a plan for measuring outcomes in model systems of care for American Indian and Alaska Native children and youth, American Indian and Alaska Native Mental Health Research, The Journal of the National Centre, 11(2), 88-98. DOI: https://doi.org/10.5820/aian.1102.2004.88
https://doi.org/10.5820/aian.1102.2004.88 DOI: https://doi.org/10.5820/aian.1102.2004.88
Peter, L., Christie, E., Cochran, M., Dunn, D., Elk, L., Fields, E., Fields, J., Hirata-Edds, T., Huckaby, A., Raymond, M., Shade, D. C. H., Sly, G., Wickliffe, G., & Yamamoto, A. (2003). Assessing the impact of total immersion on Cherokee language revitalization: A culturally responsive, participatory approach. In J. Reyhner, O. Trujillo, R.L. Carrasco, L. Lockard (eds) Nurturing Native languages. Flagstaff, AZ: Northern Arizona University.
Potvin, L., Cargo, M., McComber, A. M., Delormier, T., & Macaulay, A.C. (2003). Implementing participatory intervention and research in communities: Lessons from the Kahnawake schools diabetes prevention project in Canada, Social Science & Medicine, 56, 1295-1305. DOI: https://doi.org/10.1016/S0277-9536(02)00129-6
https://doi.org/10.1016/S0277-9536(02)00129-6 DOI: https://doi.org/10.1016/S0277-9536(02)00129-6
Reagan, T. (1996). Non-western educational traditions: Alternative approaches to educational thought and practice. Mahwah, NJ: Lawrence Erlbaum Associates.
Robertson, P., Jorgensen, M., & Garrow, C. (2004). Indigenizing evaluation research. American Indian Quarterly, 28(3/4), 499-526. DOI: https://doi.org/10.1353/aiq.2004.0106
https://doi.org/10.1353/aiq.2004.0106 DOI: https://doi.org/10.1353/aiq.2004.0106
Rowe, W. E. (1997). Changing ATOD norms and behaviors: A native American community commitment to wellness. Evaluation and Program Planning, 20(3), 323- 333. DOI: https://doi.org/10.1016/S0149-7189(97)00012-8
https://doi.org/10.1016/S0149-7189(97)00012-8 DOI: https://doi.org/10.1016/S0149-7189(97)00012-8
Running Wolf, P., Soler, R., Manteuffel, B., Sondheimer, D., Santiago, R. L., & Erickson, J. S. (2002). Cultural competence approaches to evaluation in tribal communities. Symposium on Research and Evaluation Method: Lifespan Issues related to American Indians/Alaska Natives with Disabilities, AIRPEM Working Group on American Indian Research and Program Evaluation Methodology. Washington, DC.
Scheurich, J. J., Young, M. D. (2002). Coloring epistemology: Are our research epistemologies racially biased? in J.J. Scheurich (ed.). Anti-racist scholarship: An advocacy, Albany, NY: State University of New York Press.
Senese, G. (2005). The PENAL project: program evaluation and Native American liability (Chapter 8), In S. Hood, R. Hopson, H. Frierson (Eds.), The role of culture and cultural context: A mandate for inclusion, the discovery of truth, and understanding in evaluative theory and practice (pp. 129-147). Greenwich, Connecticut: Information Age Publishing. DOI: https://doi.org/10.1108/978-1-60752-783-120251009
SenGupta, S., Hopson, R., & Thompson- Robinson, M. (2004). Cultural competence in evaluation: An overview. In M. Thompson-Robinson,R. Hopson, S. SenGupta (eds.). In search of cultural competence in evaluation: Towards principles and practices. New Directions for Evaluation, , no. 102, (pp. 5- 19) San Francisco: Jossey-Bass). DOI: https://doi.org/10.1002/ev.112
https://doi.org/10.1002/ev.112 DOI: https://doi.org/10.1002/ev.112
Shulha, L. M., & Cousins, J. B. (1997). Evaluation use: Theory, research, and practice since 1986, Evaluation Practice, 18(3), 195-209. DOI: https://doi.org/10.1016/S0886-1633(97)90027-1
https://doi.org/10.1016/S0886-1633(97)90027-1 DOI: https://doi.org/10.1016/S0886-1633(97)90027-1
Smylie, J., Martin, C. M., Kaplan-Myrth, N., Steele, L., Tait, C., & Hogg, W. (2003). Knowledge translation and indigenous knowledge. Circumpolar Health, 63 Supplement (2), 139-143. DOI: https://doi.org/10.3402/ijch.v63i0.17877
https://doi.org/10.3402/ijch.v63i0.17877 DOI: https://doi.org/10.3402/ijch.v63i0.17877
Stanfield, J. H. (1999). Slipping through the front door: Relevant social scientific evaluation in the people of color century. American Journal of Evaluation, 20(3), 415-431. DOI: https://doi.org/10.1016/S1098-2140(99)00029-6
https://doi.org/10.1177/109821409902000301 DOI: https://doi.org/10.1177/109821409902000301
Sue, D. W., & Sue, D. (1999). Counselling the culturally different: Theory and practice (3rd Edition). NY, NY: John Wiley & Sons, Inc.
Symonette, H. (2004). Walking pathways toward becoming a culturally competent evaluator: Boundaries, borderlands, and border crossings. In M. Thompson-Robinson, R. Hopson, S. SenGupta (eds.), In search of cultural competence in evaluation: Towards principles and practices. New Directions for Evaluation, no. 102, (pp. 95-109) San Francisco: Jossey-Bass. DOI: https://doi.org/10.1002/ev.118
https://doi.org/10.1002/ev.118 DOI: https://doi.org/10.1002/ev.118
Thomas, W., & Bellefeuille, G. (2006). An evidence-based formative evaluation of a cross-cultural Aboriginal mental health program in Canada, Australian e-Journal for the Advancement of Mental Health, 5(3), 1-14. DOI: https://doi.org/10.5172/jamh.5.3.202
https://doi.org/10.5172/jamh.5.3.202 DOI: https://doi.org/10.5172/jamh.5.3.202
Thompson-Robinson, M., Hopson, R., & SeGupta, S. (2004). Editors notes. In M. Thompson-Robinson, R. Hopson, S. SenGupta (eds.), In search of cultural competence in evaluation: Towards principles and practices. New Directions for Evaluation, no. 102. (pp. 1- 4) San Francisco: Jossey-Bass). DOI: https://doi.org/10.1002/ev.111
https://doi.org/10.1002/ev.111 DOI: https://doi.org/10.1002/ev.111
Weaver, H. N. (1999). Assessing the needs of Native American communities: A Northeastern example. Evaluation and Program Planning, 22, 155-161. DOI: https://doi.org/10.1016/S0149-7189(99)00010-5
https://doi.org/10.1016/S0149-7189(99)00010-5 DOI: https://doi.org/10.1016/S0149-7189(99)00010-5
Weaver, L., & Cousins, J. B. (2004). Unpacking the participatory process. Journal of Multidisciplinary Evaluation, 1, 19-40. DOI: https://doi.org/10.56645/jmde.v1i1.144
https://doi.org/10.56645/jmde.v1i1.144 DOI: https://doi.org/10.56645/jmde.v1i1.144
White, C. J., & Hermes, M. (2005). Learning to play scholarly jazz: A culturally responsive evaluation of the Hopi teachers for Hopi schools project (Chapter 7), In S. Hood, R. Hopson, H. Frierson (Eds.), The role of culture and cultural context: A mandate for inclusion, the discovery of truth, and understanding in evaluative theory and practice (pp. 105-128). Greenwich, Connecticut: Information Age Publishing. DOI: https://doi.org/10.1108/978-1-60752-783-120251008
Willging, C. E., Helitzer, D., & Thompson, J. (2006). 'Sharing wisdom': Lessons learned during the development of a diabetes prevention program for urban American Indian women. Evaluation and Program Planning, 29, 130-140. DOI: https://doi.org/10.1016/j.evalprogplan.2006.01.005
https://doi.org/10.1016/j.evalprogplan.2006.01.005 DOI: https://doi.org/10.1016/j.evalprogplan.2006.01.005