Reorienting Evaluator Competencies: Learnings from Evaluation Practice During the COVID-19 Pandemic
Article Sidebar
Main Article Content
Abstract
Background: The Covid-19 Pandemic has emphasized the need for evaluators to reorient their skills based on ongoing learnings from evaluations. As a result, evaluators must possess a variety of competencies to meet the challenges created by such unprecedented circumstances.
Purpose: Using existing literature about evaluators' competencies and the experiences of conducting evaluations during pandemic, this paper proposes a set of competencies that enable an evaluator to collaborate meaningfully with grassroots organizations and co-design evaluations with communities that empower evaluators to think beyond the boundaries of a project intervention towards achieving a larger goal.
Setting: Covid 19 Pandemic and evaluation practice in India
Intervention: NA
Research Design: Case study with interpretive approach
Data Collection and analysis: NA
Findings: The paper suggests that acquiring 'strategic thinking' and 'emotional intelligence and resilience' strengthens evaluators' competencies to be flexible and innovative in evolving evaluation methods.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Acharyya, K. (2020, April 23). NGOs, volunteers emerge as only hope for Delhi’s undocumented people as absence of Aadhaar keeps free ration, hunger relief centres out of reach-Health News. Firstpost. https://www.firstpost.com/health/ngos-volunteers-emerge-as-only-hope-for-delhis-undocumented-people-as-absence-of-aadhaar-keeps-free-ration-hunger-relief-centres-out-of-reach-8284131.html
Agrawal, R., Ramarao, D., VLN Rao, B., & Kalugampitiya, A. (2021). Impact of Covid-19 on Behaviours and Well Being. DPS Publishing House.
Ajmal, A. (2020, March 28). Citizens, NGOs reach out to the needy with food, rations and Cash. Times of India. https://timesofindia.indiatimes.com/india/citizens-ngos-reach-out-to-the-needy-with-food-rations-cash/articleshow/74855828.cms
Ali, Z., del Valle, L. A., Fletcher, E. K., Josephat, J., & Salisbury, T. (2020, August 27). Challenges of remote surveying during COVID-19. Results for Development. https://r4d.org/blog/remote-surveying-during-covid-19-challenges/
American Evaluation Association. (2018). The 2018 AEA evaluator competencies. https://www.eval.org/Portals/0/Docs/AEA%20Evaluator%20Competencies.pdf. Retrieved August 15, 20 from www.eval.org23
Arriagada Bruneau, G., Gilthorpe, M., & Müller, V. C. (2020). The ethical imperatives of the COVID-19 pandemic: A review from data ethics. Veritas, (46), 13-35. https://dx.doi.org/10.4067/S0718-92732020000200013 DOI: https://doi.org/10.4067/S0718-92732020000200013
Aston, T., Roche, C., Schaaf, M., & Cant, S. (2022). Monitoring and evaluation for thinking and working politically. Evaluation, 28(1), 36–57. https://doi.org/10.1177/13563890211053028 DOI: https://doi.org/10.1177/13563890211053028
Bansal, P. The Ravaged Psyche: Impact of the COVID-19 Pandemic on the Human Mind. Hu Arenas 5, 694–706 (2022). https://doi.org/10.1007/s42087-021-00190-6 DOI: https://doi.org/10.1007/s42087-021-00190-6
Barron, P., Cord, L., Cuesta, J., Espinoza, S. A., Larson, G., & Woolcock, M. (2023). Social Sustainability in Development: Meeting the Challenges of the 21st Century. New Frontiers of Social Policy. World Bank. https://doi.org/10.1596/978-1-4648-1946-9 DOI: https://doi.org/10.1596/978-1-4648-1946-9
Bhagat, R. B., R.S., R., Sahoo, H., Roy, A. K., & Govil, D. (2020). The COVID-19, Migration and Livelihood in India: Challenges and Policy Issues: Challenges and Policy Issues. Migration Letters, 17(5), 705–718. https://doi.org/10.33182/ml.v17i5.1048 DOI: https://doi.org/10.33182/ml.v17i5.1048
Chakraborty, S. (2021). Emerging from the Lockdown: Insights from Women Informal Workers’ Lives in Delhi. Institute of Social Studies Trust. http://dx.doi.org/10.2139/ssrn.3949202 DOI: https://doi.org/10.2139/ssrn.3958289
Diaz, J., Chaudhary, A. K., Jayaratne, K. S. U., & Assan, E. (2020). Expanding evaluator competency research: Exploring competencies for program evaluation using the context of non-formal education. Evaluation and Program Planning, 79, 101790. https://doi.org/10.1016/j.evalprogplan.2020.101790 DOI: https://doi.org/10.1016/j.evalprogplan.2020.101790
Collins, C. S., & Cooper, J. E. (2014). Emotional Intelligence and the Qualitative Researcher. International Journal of Qualitative Methods, 13(1), 88–103. https://doi.org/10.1177/160940691401300134 DOI: https://doi.org/10.1177/160940691401300134
Froncek, B., Mazziotta, A., Piper, V., & Rohmann, A. (2018). Evaluator competencies in the context of diversity training: The practitioners’ point of view. Evaluation and Program Planning, 67, 122–128. https://doi.org/10.1016/j.evalprogplan.2018.01.004 DOI: https://doi.org/10.1016/j.evalprogplan.2018.01.004
Ghere, G., King, J. A., Stevahn, L., & Minnema, J. (2006). A professional development unit for reflecting on program evaluator competencies. American Journal of Evaluation, 27(1), 108–123. https://doi.org/10.1177/1098214005284974 DOI: https://doi.org/10.1177/1098214005284974
Gokhroo, S. (2021, February 2). 80% of India's informal workers lost jobs during COVID lockdown, 63% survived on two meals a day, shows data. FirstPost. https://www.firstpost.com/india/80-of-indias-informal-workers-lost-jobs-during-covid-lockdown-63-survived-on-two-meals-a-day-shows-data-9264141.html
Goldstein, M., & Kondylis, F. (2020, March 26). Impact evaluations in the time of Covid-19, Part 1. World Bank Blogs. https://blogs.worldbank.org/impactevaluations/impact-evaluations-time-covid-19-part-1
Goleman, D. (1998). Working with emotional intelligence. New York, Bantam Books. DOI: https://doi.org/10.1002/ltl.40619981008
Greene, J. C. (2007). Mixed methods in social inquiry. John Wiley & Sons.
ILO. (2020). Protecting migrant workers during the COVID-19 pandemic Recommendations for Policy-makers and Constituents. [Policy Brief] https://www.ilo.org/global/topics/labour-migration/publications/WCMS_743268/lang--en/index.htm
ISST. (2020). Impact of Covid 19 National Lockdown on Women Informal Workers in Delhi. http://dx.doi.org/10.2139/ssrn.3993514
Islam, N., & Winkel, J. (2017). Climate Change and Social Inequality. UN Department of Economic and Social Affairs [Working Papers, No. 152], UN https://doi.org/10.18356/2c62335d-en
Jesline, J., Romate, J., Rajkumar, E., et al. (2021). The plight of migrants during COVID-19 and the impact of circular migration in India: a systematic review. Humanit Soc Sci Commun, 8, 231. https://doi.org/10.1057/s41599-021-00915-6 DOI: https://doi.org/10.1057/s41599-021-00915-6
King, J. A. (2020). The American Evaluation Association’s program evaluator competencies [Special issue]. New Directions for Evaluation, 168. https://doi.org/10.1002/ev.20358 DOI: https://doi.org/10.1002/ev.20435
Kirkhart, K. (1981). Defining evaluator competencies: New light on an old issue. American Journal of Evaluation, 2(2), 188–192. DOI: https://doi.org/10.1177/109821408100200225
Kvale, S., & Brinkmann, S. (2009). Interviews: Learning the craft of qualitative research
interviewing. Thousand Oaks, CA: Sage.
Mertens, D. M. (2011). Transformative research and evaluation. Guilford Press.
Mertens, D.M. (1994), Training evaluators: Unique skills and knowledge. New Directions for Program Evaluation, 1994: 17-27. https://doi.org/10.1002/ev.1673 DOI: https://doi.org/10.1002/ev.1673
OECD. (2022). First lessons from government evaluations of COVID-19 responses: A synthesis. https://www.oecd.org/coronavirus/policy-responses/first-lessons-from-government-evaluations-of-covid-19-responses-a-synthesis-483507d6/
Ofir, Z., & Rugg, D. (2021). American Journal of Evaluation Section on International Developments in Evaluation: Transforming Evaluation for Times of Global Transformation. American Journal of Evaluation, 42(1), 47–52. https://doi.org/10.1177/1098214020979070 DOI: https://doi.org/10.1177/1098214020979070
Patton, M. Q. (2018). Facilitating evaluation. SAGE Publications, Inc.
Patton, M. Q. (1990). The challenge of being a profession. Evaluation Practice, 11(1), 45–51. https://doi.org/10.1016/0886-1633(90)90008-2 DOI: https://doi.org/10.1177/109821409001100107
Peshkin, A. (1991). The color of strangers, the color of friends. Chicago, IL: University of
Chicago Press.
Podems, D. (2014). Research and evaluation in education and psychology: Integrating diversity with quantitative, qualitative, and mixed methods. SAGE Publications.
Preskill, H., & Boyle, S. (2008). A Multidisciplinary Model of Evaluation Capacity Building. American Journal of Evaluation, 29(4), 443–459. https://doi.org/10.1177/1098214008324182 DOI: https://doi.org/10.1177/1098214008324182
Scriven, M. (1996). Types of evaluation and types of evaluator. Evaluation Practice, 17(2), 151–161. https://doi.org/10.1016/S0886-1633(96)90020-3 DOI: https://doi.org/10.1177/109821409601700207
Somma, S., & Kilroy, G. (2020, July 14). During the pandemic, monitoring and evaluation matter more than ever. Asian Development Blog. https://blogs.adb.org/blog/during-pandemic-monitoring-and-evaluation-matter-more-than-ever
Sridharan, S. (2021, June 3). Rethinking Evaluation Competencies in 2021: Questions to Focus Our Efforts. Toronto Evaluation Platform. https://torontoevaluation.ca/blog/index.php/2021/06/03/rethinking-evaluation-competencies-in-2021-questions-to-focus-our-efforts/
Srivastava, A. (2022). Challenges for evaluation practices and innovative approaches: Lessons during COVID-19 pandemic. Evaluation and Program Planning, 92, 102095. https://doi.org/10.1016/j.evalprogplan.2022.102095 DOI: https://doi.org/10.1016/j.evalprogplan.2022.102095
Stevahn, L., King, J. A., Ghere, G., & Minnema, J. (2006). Evaluator competencies in university-based evaluation training programs. Canadian Journal of Program Evaluation, 20(2), 101–123. https://doi.org/10.3138/cjpe.20.006 DOI: https://doi.org/10.3138/cjpe.20.006
Stevahn, L., King, J., Ghere, G., & Minnema, J. (2005). Establishing Essential Competencies for Program Evaluators. American Journal of Evaluation, 26(1), 43–59. https://doi.org/10.1177/1098214004273195 DOI: https://doi.org/10.1177/1098214004273180
Sumalatha, B. S., Bhat, L. D., & Chitra, K. P. (2021). Impact of Covid-19 on Informal Sector: A Study of Women Domestic Workers in India. The Indian Economic Journal, 69(3), 441–461. https://doi.org/10.1177/00194662211023845 DOI: https://doi.org/10.1177/00194662211023845
Tedlock, B. (1991). From participant observation to the observation of participation: The DOI: https://doi.org/10.1086/jar.47.1.3630581
emergence of the narrative ethnography. Journal of Anthropological Research, 47, 69–
UNEG. (2016). Evaluation Competency Framework. United Nations Evaluation Group. http://www.unevaluation.org/document/detail/1915
UN News. (2021, April 29). First Person: Panic as COVID-19 ‘tsunami’ hits India. https://news.un.org/en/story/2021/04/1090882
Weiss, C. H. (1998). Have We Learned Anything New About the Use of Evaluation? American Journal of Evaluation, 19(1), 21–33. https://doi.org/10.1177/109821409801900103 DOI: https://doi.org/10.1016/S1098-2140(99)80178-7
Wilcox, Y., & King, J. A. (2014). A professional grounding and history of the development and formal use of evaluator competencies. Canadian Journal of Program Evaluation, 28(3), 1–28. https://doi.org/10.3138/cjpe.0028.005 DOI: https://doi.org/10.3138/cjpe.0028.005