Translating Project Achievements into Strategic Plans: A Case Study in Utilization-Focused Evaluation
Main Article Content
Abstract
Background: Utilization-focused evaluation (UFE) is a decision-making framework intended to design and implement evaluations that get utilized. In this case study, the UFE approach was applied to the evaluation of a youth training and employment program in Kenya.
Purpose: Analyze a case study based on empirical experience with the lens of evaluation use and influence.
Setting: Nairobi and other urban and peri-urban settings, Kenya.
Intervention: The evaluation of a youth training and employment program that provided direct training for marginalized youth as well as capacity building for employment and adapted a Basic Employability Skills Training (BEST) model from India.
Research design: Analysis of a case study to describe: how the evaluation approach provided enabling factors for funders and grantees to turn evaluation into a learning intervention; the benefit of clarifying a project’s theory of change; and learning how to combine summative and developmental evaluation.
Data collection and analysis: A case study based on an evaluation consultancy. The evaluation included site visits, extensive documentation review, qualitative and quantitative data collection.
Findings: The ‘facilitation of use’ (a step in UFE) provided a bridge between a summative and a developmental evaluation. The evaluators and program partners developed a learning relationship wherein the evidence influenced the subsequent project design and strategy.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
CAP YEI. (2015). Youth learning, earning and savings initiative in Sub-Saharan Africa: The story of CAP YEI and the BEST model. PowerPoint presentation shown to the evaluation team during the inception trip, July 2015. Nairobi: CAP YEI
Chambers, R. (2007). Overview. Immersions: something is happening. Participatory Learning and Action, 57, 9-14.
Earl, S., Carden, F., & Smutylo, T. (2003). Outcome mapping: Building learning and reflection into development programs. Ottawa: IDRC.
Eyben, R. (2013). Uncovering the politics of 'evidence' and 'results'. A framing paper for development practitioners. Institute of Development Studies.
Gamble, J. 2008. A developmental evaluation primer. Montreal: The J.W. McConnell Family Foundation.
Hall, J., Freeman, M., & Roulston, K. (2014). Right timing in formative program evaluation. Evaluation and Program Planning, 45, 151-156. https://doi.org/10.1016/j.evalprogplan.2014.04.007 DOI: https://doi.org/10.1016/j.evalprogplan.2014.04.007
Hearn, S. & Buffardi, A. (2016). What is impact? ODI Methods Lab. London: ODI.
Henry, G.T. & Mark, M.M. (2003). Beyond use: Understanding evaluation's influence on attitudes and actions. American Journal of Evaluation, 24(3), 293-314. https://doi.org/10.1177/109821400302400302 DOI: https://doi.org/10.1177/109821400302400302
Jones, N., Jones, H., Steer, L. & Datta, A. (2009). Improving impact evaluation production and use. ODI Working Paper 300. London: ODI.
Kaane, H.L. (2014). Kenya country report for the 2014 Ministerial Conference on Youth Employment. Abidjan, Côte d'Ivoire, July 2014, 21-23.
Kenya National Bureau of Statistics, (2015). Economic survey 2015. Nairobi: KNBS.
Kirkhart, K.E. (2000). Reconceptualizing evaluation use: An integrated theory of influence. New Directions in Evaluation 88, 5-23. https://doi.org/10.1002/ev.1188 DOI: https://doi.org/10.1002/ev.1188
Liket, K. C., Rey-Garcia, M., & Maas, K. E. H. (2014). Why aren't evaluations working and what to do about it: A framework for negotiating meaningful evaluation in nonprofits. American Journal of Evaluation, 35(2), 171-188. https://doi.org/10.1177/1098214013517736 DOI: https://doi.org/10.1177/1098214013517736
Ling, T. (2012). Evaluating complex and unfolding interventions in real time. Evaluation, 18(1), 79-91. https://doi.org/10.1177/1356389011429629 DOI: https://doi.org/10.1177/1356389011429629
Marope, P.T.M., Chekroun, B. & Homes, K.P. (2015). Unleashing the potential: Transforming technical and vocational education and training. Paris: Unesco
MasterCard Foundation, (n.d.) Youth livelihoods program.
MasterCard Foundation, (2015). Final evaluation terms of reference - CAP Youth Empowerment Institute. Toronto: The MasterCard Foundation.
MasterCard Foundation (2015). Research and evaluation policy. Toronto: The MasterCard Foundation.
Mayne, J. (2015). Useful theory of change models. Canadian Journal of Program Evaluation, 30(2), 119-142. https://doi.org/10.3138/cjpe.230 DOI: https://doi.org/10.3138/cjpe.230
Patton, M. Q. (2008). Utilization-focused evaluation, 4th ed. Los Angeles, London, New Delhi, Singapore: Sage Publications.
Patton, M.Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York and London: The Guildford Press.
Patton, M.Q., McKegg, K. & Wehipeihana, N. (eds). (2016). Developmental evaluation exemplars: Principles in practice. New York and London: The Guildford Press.
Ramírez, R. & Brodhead, D.P. (2013). Utilization focused evaluation: A primer for evaluators. Penang: Southbound.
Ramírez, R., Kora, G., Nyangaga, J. & Brodhead, D.P. (2015). CAP Youth Empowerment Institute final evaluation report. Ottawa: New Economy Development Group.
Ramírez, R., Kora, G., & Shephard, D. (2015). Utilization focused developmental evaluation: Learning through practice. Journal of Multidisciplinary Evaluation, 11(24), 37-53. https://doi.org/10.56645/jmde.v11i24.422 DOI: https://doi.org/10.56645/jmde.v11i24.422
Republic of Kenya. (2013). The technical and vocational education and training act. Kenya Gazette Supplement. No. 44 (Acts No. 29). Nairobi.
Republic of Kenya. (2007). Kenya vision 2030.
Visser, I., Kusters, C., Guijt, I., Roefs, M. & Buizer, N. (2014). Improving the use of monitoring and evaluation processes and findings: Conference report. Wageningen, the Netherlands: Centre for Development Innovation.
Vogel, I. (2012a). Review of the use of theory of change in international development: Review report. London: Isabel Vogel and the UK Department of International Development.
Vogel, I. (2012b). ESPA guide to working with theory of change for research projects. London: Ecosystem Services for Poverty Alleviation.