Thinking and Feeling Matter: Utilizing Impact Assessment and Photovoice to Evaluate a Community Program
Article Sidebar
Main Article Content
Abstract
Background The benefits of adopting mixed methods have been widely recognized in evaluation studies. However, such an methodological approach is also criticized for its lack of rigorous epistemological concerns, or simply as a-paradigmic. Therefore, in using mixed methods, how knowledge’s truth value can possibly contribute to program betterment remains uncertain.
Purpose This paper aims to provide an example of how the use of a mixed methods approach in a community program can lead to reflections on its epistemological values and possibly result in program improvements.
Setting The evaluation took place in the longest cultural walk trail in urban Hong Kong.
Intervention The evaluation method employed a combination of impact assessment, participant’s photovoice, and multiple meetings with the program managers.
Research design A mixed methods design.
Data collection & analysis Relevant statistical and thematic analyses were conducted.
Findings The data obtained from the mixed methods approach, along with its underlying epistemological foundations, revealed a unique experience for the participants, encompassing both cognitive evaluation and emotional appreciation of the program. Based on these findings, relevant suggestions were made to enhance the program.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Bamberger, M., Rao, V., & Woolcock, M. (2010). Using mixed methods in monitoring and evaluation: Experiences from international development (World Bank policy research working paper 5245). https://doi.org/10.4135/9781506335193.n24
Brandon, P. R. (1998). Stakeholder participation for the purpose of helping ensure evaluation validity: Bridging the gap between collaborative and non-collaborative evaluations. American Journal of Evaluation, 19, 325-337. https://doi.org/10.1177/109821409801900305
Coakes, S., & Anagnostaras, J. (2024). Social research methods for project social impact assessment. In F. Vanclay & A. M. Esteves (Eds.), Handbook of social impact assessment and management (pp. 460-474). Edward Elgar. https://doi.org/10.4337/9781802208870.00040
Coryn, C. L., Schröter, D. C., Cullen, A., Semen, L., & McLaughlin, J. (2012). Assessing implementation integrity of a national nutrition education program: A case study of Share Our Strength's Operation Frontline. Journal of MultiDisciplinary Evaluation, 8(19), 15-25. https://doi.org/10.56645/jmde.v8i19.366
Cullen, A. E. (2009). The politics and consequences of participation in international development evaluation [Unpublished doctoral dissertation]. Western Michigan University, Kalamazoo.
Cullen, A. E., Coryn, C. L. S., & Rugh, J. (2011). The politics and consequences of including stakeholders in international development evaluation. American Journal of Evaluation, 32(3), 345-361. https://doi.org/10.1177/1098214010396076
de Oliveira, A. R. (2024). Visual tools in social impact assessment: Issues of perception, communication, and ethical dilemmas. In F. Vanclay & A. M. Esteves (Eds.), Handbook of social impact assessment and management (pp. 500-516). Edward Elgar. https://doi.org/10.4337/9781802208870.00043
Denscombe, M. (2008). Communities of practice: A research paradigm for the mixed methods approach. Journal of Mixed Methods Research, 2(3), 270-283. https://doi.org/10.1177/1558689808316807
Denzin, N. (2012). Triangulation 2.0. Journal of Mixed Methods Research, 6(2), 80-88. https://doi.org/10.1177/1558689812437186
Development Bureau. (2011). Urban renewal strategy. Hong Kong Special Administrative Region Government.
Garbarino, S. & Holland, J. (2009). Quantitative and qualitative methods in impact evaluation and measuring results [Discussion paper]. University of Birmingham.
Greene, J. C. (2007). Mixed methods in social inquiry. Wiley.
Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255-274. https://doi.org/10.3102/01623737011003255
Greene, J. C., Kreider, H., & Mayer, E. (2005). Combining qualitative and quantitative methods in social inquiry. In B. Somekh & C. Lewin (Eds.), Research methods in the social sciences (pp. 275-282). Sage Publications.
Hall, J. N. (2013). Pragmatism, evidence, and mixed methods evaluation. New Directions for Evaluation, 138, 15-26. https://doi.org/10.1002/ev.20054
Hergenrather, K. C., Rhodes, S. D., Cowan, C. A., Bardhoshi, G., & Pula, S. (2009). Photovoice as community-based participatory research: A qualitative review. American Journal of Health Behavior, 33(6), 686-698. https://doi.org/10.5993/AJHB.33.6.6
Ho, W. C. (2014). Urban renewal plan for Kowloon City: Social impact assessment (executive summary). DURF. https://ssweb.cityu.edu.hk/system/files/2024-09/SIA-ES.pdf
Ho, W. C. (2022) Consultancy to conduct study on the outcomes and social impacts of the Urban Renewal Fund: Final report [Unpublished report submitted to the Urban Renewal Fund].
Ho, W. C. (2023). The myth of programmatic intervention to tackle intergenerational poverty in post-industrial societies: A longitudinal study in Hong Kong as a vantage point. Journal of Sociology and Social Welfare, 51(3),116-152. https://doi.org/10.15453/0191-5096.4699
Hunter, O., Leeburg, E., & Harnar, M. (2020). Using photovoice as an evaluation method. Journal of MultiDisciplinary Evaluation, 16(34), 14-20. https://doi.org/10.56645/jmde.v16i34.603
Lambert, V. A., & Loiselle, C. G. (2008). Combining individual interviews and focus groups to enhance data richness. Journal of Advanced Nursing, 62(2), 228-237. https://doi.org/10.1111/j.1365-2648.2007.04559.x
Lemos, A. R., Sandars, J. E., Alves, P., & Costa, M. J. (2014). The evaluation of student- centredness of teaching and learning: A new mixed-methods approach. International Journal of Medical Education, 4(5), 157-164. https://doi.org/10.5116/ijme.53cb.8f87
Lipscomb, M. (2008). Mixed method nursing studies: A critical realist critique. Nursing Philosophy, 9(1), 32-45. https://doi.org/10.1111/j.1466-769X.2007.00325.x
Lo, H. H. M., Liu, K. H. K., Ho, W. C., Lau, E. N. S., Poon, M. F., Lo, C. S. L., & Tam, H. S. W. (2022). Using photovoice in a mindfulness-based program to understand the experiences of caregivers of young adults with psychosis. International Journal of Environmental Research and Public Health, 19(23), 15461. https://doi.org/10.3390/ijerph192315461
Nicholson, D. J. (2012). The concept of mechanism in biology. Studies in History and Philosophy of Biological and Biomedical Sciences, 43(1), 152-163 https://doi.org/10.1016/j.shpsc.2011.05.014
Petrosino, A., Pace, J., Fronius, T., Goold, C., & Turner, L. (2018). Systematic reviews and meta- analyses of criminal justice interventions. Journal of Experimental Criminology, 14(1), 1-26. https://doi.org/10.1093/acrefore/9780190264079.013.388
Tashakkori, A., & Teddlie, C. (2010). Sage handbook of mixed methods in social & behavioral research (2nd ed.). Sage. https://doi.org/10.4135/9781506335193
Vanclay, F. (2003). International principles for social impact assessment. Impact Assessment and Project Appraisal, 21(1), 5-12. https://doi.org/10.3152/147154603781766491
White, H. (2008). Of probits and participation: The use of mixed methods in quantitative impact evaluation. IDS Bulletin, 39(1), 98-109. https://doi.org/10.1111/j.1759-5436.2008.tb00436.x
Yung, E. H. K., Zhang, Q., & Chan, E. H. W. (2017). Underlying social factors for evaluating heritage conservation in urban renewal districts. Habitat International, 66, 135-148. https://doi.org/10.1016/j.habitatint.2017.06.004