Building Spaces for Dialogues to Rethink Evaluator Competencies: Lessons from the Webinars Organized by the Evaluation Centre for Complex Health Interventions
Article Sidebar
Main Article Content
Abstract
Background: There is a need to rethink evaluator competencies given the harsh and paralyzing realities of COVID. The pandemic was a time where there was a need to balance diverse perspectives given the limited scientific evidence that existed when faced with a genuinely unprecedented time. In the Fall of 2021 (September to October), the Evaluation Centre for Complex Health Interventions in partnership with the Asia Pacific Evaluation Association organized a three-part webinar series in response to the multiple issues that surfaced during COVID-19, and specifically, the implications of the pandemic for rethinking evaluator competencies and evaluator training. The presenters were from multiple countries including India, Canada, USA, UK, and South Africa.
Purpose: The presenters pushed for more responsive evaluation approaches to address inequities and sustainability and for a decolonized approach to knowledge building. The webinar raised a number of themes that have potential implications for future discussions on evaluator competencies including: enhancing evaluation contributions to the Sustainable Development Goals (SDGs), the need to rethink evaluation criteria, the need to embrace and address varieties of uncertainties, focus on diversity and heterogeneity; understanding the role of contexts in complex programs and policies; the need to reconceptualize sustainability; being more explicit about inequities and vulnerabilities; and the need to pay attention to systems and system dynamics.
Setting: The webinars were organized by the Evaluation Centre and the Asia Pacific Evaluation Association on a Zoom platform.
Intervention: Not applicable.
Research Design: Not applicable.
Data Collection and Analysis: Not applicable.
Findings: Not applicable.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Anser, M. K., Godil, D. I., Khan, M. A., Nassani, A. A., Askar, S. E., Zaman, K., Salamun, H., Sasmoko, Indrianti, Y., & Abro, M. M. Q. (2022). Nonlinearity in the relationship between COVID-19 cases and carbon damages: controlling financial development, green energy, and R&D expenditures for shared prosperity. Environmental Science and Pollution Research, 29(4), 5648–5660. https://doi.org/10.1007/s11356-021-15978-w DOI: https://doi.org/10.1007/s11356-021-15978-w
Cayley, D., (2020). The Prognosis. Retrieved from https://reviewcanada.ca/magazine/2020/10/the-prognosis/
Cook, T. D. (2000). The false choice between theory-based evaluation and experimentation. New Directions in Evaluation, 87, 27–34. https://doi.org/10.1002/ev.1179 DOI: https://doi.org/10.1002/ev.1179
Heider, C. (2017). Rethinking Evaluation – Have we had enough of R/E/E/I/S? https://ieg.worldbankgroup.org/blog/rethinking-evaluation
Leviton, L. C., & Hughes, E. F. X. (1981). Research on the utilization of evaluations: A review and synthesis. Evaluation Review, 5(4), 525–548. https://doi.org/10.1177/01 93841X8100500405 DOI: https://doi.org/10.1177/0193841X8100500405
Markauskaite, L., & Goodyear, P. (2017). Introduction. In Professional and Practice-based Learning (Vol. 14). https://doi.org/10.1007/978-94-007-4369-4_1 DOI: https://doi.org/10.1007/978-94-007-4369-4_1
OECD. (2019). Better criteria for better evaluation: Revised evaluation criteria definitions and principles for use. OECD/DAC Network on Development Evaluation. www.oecd.org/dac/evaluation https://doi.org/10.1787/15a9c26b-en DOI: https://doi.org/10.1787/15a9c26b-en
Patton, MQ (2010). Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. Guilford Press.
Patton, M.Q. (2017). Principles-Focused Evaluation: The GUIDE. Guilford Press
Patton, M. Q. (2021). Evaluation Criteria for Evaluating Transformation: Implications for the Coronavirus Pandemic and the Global Climate Emergency. American Journal of Evaluation, 42(1), 53–89. https://doi.org/10.1177/1098214020933689 DOI: https://doi.org/10.1177/1098214020933689
Pawson, R. (2013). The Science of Evaluation. Sage. https://doi.org/10.4135/9781473913820 DOI: https://doi.org/10.4135/9781473913820
Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2004). Realist synthesis: an introduction. ESRC research methods programme: RMP methods paper series. University of Manchester.
Pawson, R., & Tilley, N. (1997). An introduction to scientific realist evaluations. In E. Chelimsky & W. R. Shadish (Eds.), Evaluation for the 21st century: A handbook (pp. 405–418). Sage. https://doi.org/10.4135/9781483348896.n29 DOI: https://doi.org/10.4135/9781483348896.n29
Pritchett, L., & Woolcock, M. (2004). Solutions when the solution is the problem: Arraying the disarray in development. World Development, 32(2), 191–212. https://doi.org/10.1016/j.worlddev.2003.08.009 DOI: https://doi.org/10.1016/j.worlddev.2003.08.009
Schwandt, T. A. (2019). Post-normal evaluation? Evaluation, 25(3), 317–329. https://doi.org/10.1177/1356389019855501 DOI: https://doi.org/10.1177/1356389019855501
Shah, N., Mathew, S., Pereira, A., Nakaima, A., & Sridharan, S. (2021). The role of evaluation in iterative learning and implementation of quality of care interventions. Global Health Action, 14(1). https://doi.org/10.1080/16549716.2021.1882182 DOI: https://doi.org/10.1080/16549716.2021.1882182
Sridharan, S. (2020, May 5). Adaptations and nimbleness at the time of crisis: Some questions for evaluators. Evaluating Complexity Blog. Retrieved 17 July 2023. https://torontoevaluation.ca/blog/index.php/2020/05/05/adaptations-and-nimbleness-at-the-time-of-crisis-some-questions-for-evaluators/
Sridharan, S., & Gillespie, D. (2004). Sustaining collaborative problem solving capacity. Criminology and Public Policy, 3(2), 601–631. https://doi.org/10.1111/j.1745-9133.2004.tb00038.x DOI: https://doi.org/10.1111/j.1745-9133.2004.tb00038.x
Sridharan, S., & Nakaima, A. (2019). Till time (and poor planning) do us part: Programs as dynamic systems—Incorporating planning of sustainability into theories of change. Canadian Journal of Program Evaluation, 33(3), 375–394. https://doi.org/10.3138/cjpe.53055 DOI: https://doi.org/10.3138/cjpe.53055
Sridharan, S., Zhao, K., & Nakaima, A. (2017). Editors’ Notes. New Directions for Evaluation, 2017(154), 9–16. https://doi.org/10.1002/ev.20242 DOI: https://doi.org/10.1002/ev.20242
Uitto, J. I. (2019). Sustainable Development Evaluation: Understanding the Nexus of Natural and Human Systems. New Directions for Evaluation, 2019(162), 49–67. https://doi.org/10.1002/ev.20364 DOI: https://doi.org/10.1002/ev.20364
Uitto, J. I. (2021). Confronting storms, fires and pestilence: Meaningful evaluation for a hazardous world. Evaluation Matters—He Take Tō Te Aromatawai, 7, 96–126. https://doi.org/10.18296/em.0068 DOI: https://doi.org/10.18296/em.0068
UN (2023). Progress towards the Sustainable Development Goals: Towards a rescue plan for people and planet report. Report of the Secretary-General (Special Edition). Seventh-eighth session, Item 19 of the preliminary list, Sustainable development. https://hlpf.un.org/sites/default/files/2023-04/SDG%20Progress%20Report%20Special%20Edition.pdf
Woolcock, M. (2009). Toward a plurality of methods in project evaluation: a contextualised approach to understanding impact trajectories and efficacy. Journal of Development Effectiveness, 1(1), 1–14. https://doi.org/10.1080/19439340902727719 DOI: https://doi.org/10.1080/19439340902727719
Zhao, K., Sridharan, S., Ingabire, M. G., Yu, M., Nakaima, A., Li, X., Xiao, Y., & Chen, E. (2017). An Experiment on Building Evaluation Capacity to Address Health Inequities in China. New Directions for Evaluation, 2017(154), 17–28. https://doi.org/10.1002/ev.20239 DOI: https://doi.org/10.1002/ev.20239