Using Competency-Informed Tasks to Guide Evaluation Capacity Strengthening
Article Sidebar
Main Article Content
Abstract
Background: EvaluATE, a federally funded evaluation hub, offers an array of resources and activities to strengthen evaluation capacity. However, there were some gaps across the full range of tasks that evaluators and evaluation users need to know how to do. The team created a framework based on evaluation tasks to guide their future evaluation capacity-strengthening work, with the aim of offering resources for all essential evaluation tasks.
Purpose: The article explains the need for a context-specific, task-based evaluation capacity strengthening framework, the framework’s relationship to the American Evaluation Association’s Evaluator Competencies, how the framework is being used, and lessons learned.
Setting: The Advanced Technological Education (ATE) program funded by the U.S. National Science Foundation
Intervention: EvaluATE, the evaluation hub for the ATE program.
Research Design: Not applicable.
Data Collection and Analysis: The evaluation task framework was validated through a task-tracking study, Delphi study, and expert review. Insights for this article are based on the authors’ experience.
Findings: The main lessons learned for developing task-based frameworks to guide evaluation capacity strengthening include: (1) Be clear about the scope and purpose of the task list (for both internal and external audiences). (2) In addition to stating what must be done for a given task, say why. (3) Verify the appropriateness of the tasks with typical practitioners and experts within the programmatic context.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
Funding data
-
National Science Foundation
Grant numbers 1841783
References
American Evaluation Association. (2018). The 2018 AEA evaluator competencies. https://www.eval.org/Portals/0/Docs/AEA%20Evaluator%20Competencies.pdf
ATE Central. (2023). ATE projects and centers. https://atecentral.net/
BetterEvaluation. (n.d.). Strengthen evaluation capacity. https://www.betterevaluation.org/en/rainbow_framework/manage/strengthen_evaluation_capacity
Galport, N., & Azzam, T. (2017). Evaluator training needs and competencies: A gap analysis. American Journal of Evaluation, 38(1), 80-100. https://doi.org/10.1177/1098214016643183
Ghere, G., King, J. A., Stevahn, L., & Minnema, J. (2006). A professional development unit for reflecting on program evaluator competencies. American Journal of Evaluation, 27(1), 108-123. https://doi.org/10.1177/1098214005284974
Knowles, M. (1984a). The adult learner: A neglected species (3rd ed.). Gulf Publishing.
Knowles, M. (1984b). Andragogy in action. Jossey-Bass.
Knowles, M. S., Holton, E. F., Swanson, R. A., & Robinson, P. A. (2020). The adult learning: The definitive classic in adult education and human resource development (9th ed.). Routledge.https://doi.org/10.4324/9780429299612
Labin, S. N. (2014). Developing common measures in evaluation capacity building: An Iterative science and practice process. American Journal of Evaluation, 35(1), 107-115. https://doi.org/10.1177/1098214013499965
LaVelle, J. & Galport, N. (2020). Using the 2018 AEA evaluator competencies for evaluator education and professional development. New Directions for Evaluation, 168, 99-116. https://doi.org/10.1002/ev.20437
Lindeman, E. C. (1926). The meaning of adult education. New Republic.
McGuire, M. & Zorzi, R. (2005). Evaluator competencies and performance development. The Canadian Journal of Program Evaluation, 20(2), 73-99. https://doi.org/10.3138/cjpe.20.005
Murerwa, T. (2017). Tasks analysis and its importance in organization. International Journal of Advanced Research in Management and Social Sciences, 6(6), 36-47.
Norton, S., Milat, A., Edwards, B., Giffin, M. (2016). Narrative review of strategies by organizations for building evaluation capacity. Evaluation and program planning, 58, 1-19. https://doi.org/10.1016/j.evalprogplan.2016.04.004
Robertson, K. N., & Wingate, L. A. (2022). Essential ATE evaluation tasks. EvaluATE. https://evalu-ate.org/essential-ate-evaluation-tasks/
Schwarzman, J., Bauman, A., Gabbe, B. J., Rissel, C., Shilton, T., & Smith, B. J. (2019). Understanding the factors that influence health promotion evaluation: The development and validation of the evaluation practice analysis survey. Evaluation and Program Planning, 74, 76-83. https://doi.org/10.1016/j.evalprogplan.2019.03.002
Stevahn, L., Berger, D. E., Tucker, S. A., & Rodell, A. (2020). Using the 2018 AEA Evaluator Competencies for effective program evaluation practice. New Directions for Evaluation, 168, 75-97. https://doi.org/10.1002/ev.20434
Stevahn, L., King, J. A., Ghere, G., & Minnema, J. (2005a). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26(1), 43-59. https://doi.org/10.1177/1098214004273180
Stevahn, L., King, J. A., Ghere, G., & Minnema, J. (2005b). Evaluator competencies in university-based evaluation training programs. The Canadian Journal of Program Evaluation, 20(2), 101-123. https://doi.org/10.3138/cjpe.20.006
Stockdill, S., Baizerman, M., & Compton, D. W. (2002). Toward a definition of the ECB process: A conversation with the ECB literature. New Directions for Evaluation, 93, 7-26. https://doi.org/10.1002/ev.39
Tarsilla, M. (2014). Evaluation capacity development in Africa: Current landscape of international partners' initiatives, lessons learned and the way forward. African Evaluation Journal, 2(1), 1-13. https://doi.org/10.4102/aej.v2i1.89
Taylor-Powell, E., & Boyd, H. H. (2008). Evaluation capacity building in complex organizations. In M. T. Braverman, M. Engle, M. E. Arnold, & R. A. Rennekamp (Eds.), Program evaluation in a complex organizational system: Lessons from cooperative extension. New Directions for Evaluation, 120, 55-69. https://doi.org/10.1002/ev.276
Tucker, S. (2021). AEA's professionalization journey: Presentation to the AEA business meeting. https://www.eval.org/Portals/0/AEA%202021%20Bus%20Mtg%20about%20P%26CWG%20v3.pptx
Tucker, S. A., & King, J. A. (2020). Next steps for AEA and the newly created AEA professionalization and competencies working group. New Directions for Evaluation, 168, 149-162. https://doi.org/10.1002/ev.20436
Wilcox, Y., & King, J. A. (2014). A professional grounding and history of the development and formal use of evaluator competencies. The Canadian Journal of Program Evaluation, 28(3), 1-28. https://doi.org/10.3138/cjpe.0028.005