Evaluation Champions: A Literature Review
Main Article Content
Abstract
Background: Numerous studies call for the identification and engagement of evaluation champions in the implementation of an evaluation initiative. However, no agreed definition of an evaluation champion exists in the extant literature. Published studies on evaluation champions are limited and motivations of evaluation champions are not well understood. Understanding of how evaluation champions interact with their colleagues to generate momentum for change is lacking.
Purpose: This article explores champions in organizational settings and highlights the need for an increased understanding of evaluation champions. The research question posed is, ‘What does the literature tell us about evaluation champions in organizational settings?’
Setting: Not Applicable.
Intervention: Not Applicable.
Research Design: Relevant articles were identified through systematic searches of selected databases and reference reviews of retrieved articles from the evaluation and organizational development bodies of evidence. Theories that could assist with understanding the role of evaluation champions were also drawn upon.
Data Collection and Analysis: Not Applicable.
Findings: The analysis indicates evaluators and organizations value champions because they work to bring evaluative thinking into their practice and positively promote evaluation among their colleagues. A list of activities that may be indicative of the behaviours of champions has been compiled to contribute to the evidence base.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Alaimo, S. (2008). Nonprofits and evaluation: Managing expectations from the leader's perspective. New Directions for Evaluation, 119(Fall), 73-92. https://doi.org/10.1002/ev.269 DOI: https://doi.org/10.1002/ev.269
Arnold, M. E. (2006). Developing evaluation capacity in extension 4-H field faculty: A framework for success. American Journal of Evaluation, 27, 257-269. https://doi.org/10.1177/1098214006287989 DOI: https://doi.org/10.1177/1098214006287989
Baron, M. (2011). Designing internal evaluation for a small organization with limited resources. New Directions for Evaluation, 132(Winter), 87-99. https://doi.org/10.1002/ev.398 DOI: https://doi.org/10.1002/ev.398
Bourgeois, I., Whynot, J., & Thériault, T. (2015). Application of an organizational evaluation capacity self-assessment instrument to different organizations: Similarities and lessons learned. Evaluation and Program Planning, 50, 47-55. https://doi.org/10.1016/j.evalprogplan.2015.01.004 DOI: https://doi.org/10.1016/j.evalprogplan.2015.01.004
Boyd, H. (2009). Practical tips for evaluators and administrators to work together in building evaluation capacity. Journal of Extension, 47(2), 1-3.
Brandon, P. R., Smith, N. L., & Hwalek, M. (2011). Aspects of successful evaluation practice at an established private evaluation firm. American Journal of Evaluation, 32(2), 295-307. https://doi.org/10.1177/1098214011401369 DOI: https://doi.org/10.1177/1098214011401369
Bryson, J. M., & Patton, M. Q. (2010). Analyzing and engaging stakeholders. In J. S. Wholey, K. E. Newcomer, & H. P. Hatry (Eds.), Handbook of practical program evaluation (3rd ed., pp. 30 - 54). San Francisco, CA: John Wiley & Sons.
Buckley, J., Archibald, T., Hargraves, M., & Trochim, W. M. (2015). Defining and teaching evaluative thinking: Insights from research on critical thinking. American Journal of Evaluation, 36(3), 375-388. https://doi.org/10.1177/1098214015581706 DOI: https://doi.org/10.1177/1098214015581706
Cerra, C., Drake, D., Sick, B., & King, J. (2015). Building the foundation for culture change through the design, implementation, and assessment of an interprofessional education intervention. Journal of Nursing. 5(1), 46-57. https://doi.org/10.5430/jnep.v5n1p46 DOI: https://doi.org/10.5430/jnep.v5n1p46
Choong, M. K., Galgani, F., Dunn, A. G., & Tsafnat, G. (2014). Automatic evidence retrieval for systematic reviews. Journal Of Medical Internet Research, 16(10), e223-e223. https://doi.org/10.2196/jmir.3369 DOI: https://doi.org/10.2196/jmir.3369
Christie, C. A., & Vo, A. T. (2011). Promoting diversity in the field of evaluation: Reflections on the first year of the Robert Wood Johnson Foundation Evaluation Fellowship Program. https://doi.org/10.1177/1098214011399644 DOI: https://doi.org/10.1177/1098214011399644
American Journal of Evaluation, 32(4), 547- 564.
Coakes, E., & Smith, P. (2007). Developing communities of innovation by identifying innovation champions. The Learning Organization, 14(1), 74-85. https://doi.org/10.1108/09696470710718366 DOI: https://doi.org/10.1108/09696470710718366
Cousins, J. B., & Bourgeois, I. (2014). Cross-case analysis and implications for research, theory, and practice. New Directions for Evaluation, 141, 101-119. https://doi.org/10.1002/ev.20078 DOI: https://doi.org/10.1002/ev.20078
Cousins, J. B., Goh, S. C., Elliott, C. J., & Bourgeois, I. (2014). Framing the capacity to do and use evaluation. New Directions for Evaluation, 141, 7-23. https://doi.org/10.1002/ev.20076 DOI: https://doi.org/10.1002/ev.20076
Dickinson, P., & Adams, J. (2012). Building evaluation capability in the public health workforce: Are evaluation trining workshops effective and what else is needed? Evaluation Journal of Australasia, 12(2), 28-40. https://doi.org/10.1177/1035719X1201200204 DOI: https://doi.org/10.1177/1035719X1201200204
Fujimoto, Y., Rentschler, R., Le, H., Edwards, D., & Härtel, C. E. J. (2014). Lessons learned from community organizations: Inclusion of people with disabilities and others. British Journal of Management, 25(3), 518-537. https://doi.org/10.1111/1467-8551.12034 DOI: https://doi.org/10.1111/1467-8551.12034
Gattiker, T. F., & Carter, C. R. (2010). Understanding project champions' ability to gain intra-organizational commitment for environmental projects. Journal of Operations Management, 28(1), 72-85. https://doi.org/10.1016/j.jom.2009.09.001 DOI: https://doi.org/10.1016/j.jom.2009.09.001
Gay, B., & Weaver, S. (2011). Theory building and paradigms: A primer on the nuances of theory construction. American International Journal of Contemporary Research, 1(2), 24-32.
Gill, S. J. (2010). Developing a learning culture in nonprofit organizations. Thousand Oaks, CA: SAGE Publications, Inc. https://doi.org/10.4135/9781452272030 DOI: https://doi.org/10.4135/9781452272030
Goodman, R., & Steckler, A. (1989). A model for the institutionalization of health promotion programs. Family & Community Health, 11(4), 63-78. https://doi.org/10.1097/00003727-198902000-00009 DOI: https://doi.org/10.1097/00003727-198902000-00009
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82(4), 581-629. https://doi.org/10.1111/j.0887-378X.2004.00325.x DOI: https://doi.org/10.1111/j.0887-378X.2004.00325.x
Guerra-López, I., & Hicks, K. (2015). The participatory design of a performance oriented monitoring and evaluation system in an international development environment. Evaluation and Program Planning, 48, 21-30. https://doi.org/10.1016/j.evalprogplan.2014.09.003 DOI: https://doi.org/10.1016/j.evalprogplan.2014.09.003
Hanwright, J., & Makinson, S. (2008). Promoting evaluation culture: The development and implementation of an evaluation strategy in the Queensland Department of Education, Training and the Arts. Evaluation Journal of Australasia, 8, 20-25. https://doi.org/10.1177/1035719X0800800104 DOI: https://doi.org/10.1177/1035719X0800800104
Henry, G. T., & Mark, M. M. (2003). Beyond use: Understanding evaluation's influence on attitudes and actions. American Journal of Evaluation, 24, 293-314. https://doi.org/10.1177/109821400302400302 DOI: https://doi.org/10.1177/109821400302400302
Howell, J. M. (2005). The right stuff: Identifying and developing effective champions of innovation. The Academy of Management Executive, 19(2), 108-119. https://doi.org/10.5465/ame.2005.16965104 DOI: https://doi.org/10.5465/ame.2005.16965104
Howell, J. M., & Higgins, C. A. (1990). Champions of change: Identifying, understanding, and supporting champions of technological innovations. Organizational Dynamics, 19(1), 40-55. https://doi.org/10.1016/0090-2616(90)90047-S DOI: https://doi.org/10.1016/0090-2616(90)90047-S
Howell, J. M., Shea, C. M., & Higgins, C. A. (2005). Champions of product innovations: Defining, developing, and validating a measure of champion behavior. Journal of Business Venturing, 20(5), 641-661. https://doi.org/10.1016/j.jbusvent.2004.06.001 DOI: https://doi.org/10.1016/j.jbusvent.2004.06.001
Johnson, D. W. (2003). Social interdependence: Interrelationships among theory, research, and practice. American Psychologist, 58(11), 934945. https://doi.org/10.1037/0003-066X.58.11.934 DOI: https://doi.org/10.1037/0003-066X.58.11.934
Johnson, D. W., & Johnson, R. T. (2002). Cooperative learning and social interdependence. In R. S. Tindale, L. Heath, J. Edwards, E. J. Posavac, F. B. Bryant, Y. SuarezBalcazar, … J. Myers, (Eds.) Theory and Research on Small Groups. New York: Plenum Press.
Johnson, D. W., & Johnson, R. T. (2003). Training for cooperative group work. In M. A. West, D. Tjosvold, & K. G. Smith (Eds.), International handbook of organizational teamwork and cooperative working (pp. 167-184). West Sussex, England: John Wiley & Sons. https://doi.org/10.1002/9780470696712.ch9 DOI: https://doi.org/10.1002/9780470696712.ch9
Johnson, D., & Johnson, R. (2005). New developments in social interdependence theory. Genetic, Social, and General Psychology Monographs 131(4), 285-358. https://doi.org/10.3200/MONO.131.4.285-358 DOI: https://doi.org/10.3200/MONO.131.4.285-358
Johnson, D. W., Johnson, R. T., & Smith, K. A. (1998). Cooperative learning returns to college: What evidence is there that it works? Change, 30(4), 26-35. https://doi.org/10.1080/00091389809602629 DOI: https://doi.org/10.1080/00091389809602629
King, J. A. (2002). Building the evaluation capacity of a school district. New Directions for Evaluation, 93, 63-80. https://doi.org/10.1002/ev.42 DOI: https://doi.org/10.1002/ev.42
King, J. A. (2004). How viable is evaluation capacity building in schools that give standarized tests? Keynote address prepared for the Virginia Association of test directors. Richmond, VA. Retrieved from www.vatd.org/EvaluationCapacityBuildingNov2004.doc
King, J. A. (2005). A proposal to build evaluation capacity at the Bunche-Da Vinci Learning Partnership Academy. New Directions for Evaluation, 106, 85-97. https://doi.org/10.1002/ev.153 DOI: https://doi.org/10.1002/ev.153
King, J. A. (2007). Developing evaluation capacity through process use. New Directions for Evaluation, 116, 45-59. https://doi.org/10.1002/ev.242 DOI: https://doi.org/10.1002/ev.242
King, J. A. (2016). Taking what action for change? Dare evaluation build a new social order? Visitor Studies, 19(1), 3-11. https://doi.org/10.1080/10645578.2016.1144022 DOI: https://doi.org/10.1080/10645578.2016.1144022
King, J. A., & Stevahn, L. (2013). Interactive evaluation practice; mastering the interpersonal dynamics of program evaluation. [Kindle book] Thousand Oaks, CA: Sage Publications. https://doi.org/10.4135/9781452269979 DOI: https://doi.org/10.4135/9781452269979
King, J. A., & Volkov, B. B. (2005). A framework for building evaluation capacity based on the experiences of three organizations. CURA Reporter, 3(35), 10-16.
Kirkhart, K. E. (2000). Reconceptualizing evaluation use: An integrated theory of influence. New Directions for Evaluation, 88, 5-23 https://doi.org/10.1002/ev.1188 DOI: https://doi.org/10.1002/ev.1188
Labin, S. (2014). Developing common measures in evaluation capacity building: An iterative science and practice process. American Journal of Evaluation, 35, 107-115. https://doi.org/10.1177/1098214013499965 DOI: https://doi.org/10.1177/1098214013499965
Mackay, K. (2002). The World Bank's ECB experience. New Directions for Evaluation, 93, 81-99. https://doi.org/10.1002/ev.43 DOI: https://doi.org/10.1002/ev.43
Mark, M., Donaldson, S., & Campbell, B. (2011). Social psychology and evaluation. New York: The Guilford Press.
Maxwell, N., Rotz, D., & Garcia, C. (2016). Data and decision making: Same organization, different perceptions; Different organizations, different perceptions. American Journal of Evaluation, 37(4) 1-23. https://doi.org/10.1177/1098214015623634 DOI: https://doi.org/10.1177/1098214015623634
Mayne, J. (2010). Building an evaluative culture: The key to effective evaluation and results management. The Canadian Journal of Program Evaluation, 24, 1-30. https://doi.org/10.3138/cjpe.24.001 DOI: https://doi.org/10.3138/cjpe.24.001
McDonald, B., Rogers, P., & Kefford, B. (2003). Teaching people to fish? Building the evaluation capability of public sector organizations. Evaluation, 9(1), 9-29. https://doi.org/10.1177/1356389003009001002 DOI: https://doi.org/10.1177/1356389003009001002
Mertens, D., & Wilson, A. (2012). Program evaluation theory and practice: A comprehensive guide. New York: The Guilford Press
Milstein, B., Chapel, T. J., Wetterhall, S. F., & Cotton, D. A. (2002). Building capacity for program evaluation at the Centers for Disease Control and Prevention. New Directions for Evaluation, 93, 27-46. https://doi.org/10.1002/ev.40 DOI: https://doi.org/10.1002/ev.40
Muttitt, S., Vigneault, R., & Loewen, L. (2004). Integrating telehealth into Aboriginal healthcare: The Canadian experience. International Journal of Circumpolar Health, 63(4), 27 - 46. https://doi.org/10.3402/ijch.v63i4.17757 DOI: https://doi.org/10.3402/ijch.v63i4.17757
Nielsen, S., Lemire, A., & Skov, M. (2011). Measuring evaluation capacity: Results and implications of a Danish study. American Journal of Evaluation, 32(3), 324-344. https://doi.org/10.1177/1098214010396075 DOI: https://doi.org/10.1177/1098214010396075
Nu'Man, J., King, W., Bhalakia, A., & Criss, S. (2007). A framework for building organizational capacity integrating planning, monitoring, and evaluation. Journal of Public Health Management and Practice, January 2007, S24-S32. https://doi.org/10.1097/00124784-200701001-00006 DOI: https://doi.org/10.1097/00124784-200701001-00006
NVivo. (2012). NVivo qualitative data analysis software. QSR International Pty Ltd.
Nylen, K., & Sridharan, S. (2017). Experiments in evaluation capacity building: Enhancing brain disorders research impact in Ontario. Evaluation and Program Planning, 2017.
Owen, J. (2003). Evaluation culture: a definition and analysis of its development within organisations. Evaluation Journal of Australasia, 3(1), 43-47. https://doi.org/10.1177/1035719X0300300107 DOI: https://doi.org/10.1177/1035719X0300300107
Patton, M. (2008). Utilization-focused evaluation. Thousand Oaks, CA: Sage Publications.
Patton, M. Q. (2012). Essentials of utilizationfocused evaluation. Thousand Oaks, CA: Sage Publications.
Pejsa, L. J. (2011). Improving evaluation in the nonprofit sector: The promise of evaluation capacity building for nonprofit social service organizations in an age of accountability. University of Michigan, Ann Arbor. Retrieved from http://internal-pdf//PejsaKing-0197138433/PejsaKing.pdf
Perry, J. A. (2014). Changing schools of education through grassroots faculty-led change. Innovative Higher Education, 39(2), 155-168. https://doi.org/10.1007/s10755-013-9267-y DOI: https://doi.org/10.1007/s10755-013-9267-y
Preskill, H. (2007). Building an organization's evaluation system: A case example of using appreciative inquiry. Online Submission. Retrieved from http://internal-pdf//Preskill 2007(2)-0905711621/Preskill 2007(2).pdf
Preskill, H., & Boyle, S. (2008a). A multidisciplinary model of evaluation capacity building. American Journal of Evaluation, 29(4), 443-459. https://doi.org/10.1177/1098214008324182 DOI: https://doi.org/10.1177/1098214008324182
Preskill, H., & Boyle, S. (2008b). Insights into evaluation capacity building: motivations, strategies, outcomes, and lessons learned. The Canadian Journal of Program Evaluation, 23(3), 147 - 174. https://doi.org/10.3138/cjpe.0023.008 DOI: https://doi.org/10.3138/cjpe.0023.008
Preskill, H., & Torres, R. (1999). Evaluative inquiry for learning in organizations. Thousand Oaks, CA: Sage Publications. https://doi.org/10.4135/9781452231488 DOI: https://doi.org/10.4135/9781452231488
Rogers, E. M. (2002). Diffusion of preventive innovations. Addictive Behaviors, 27(6), 989- 993. https://doi.org/10.1016/S0306-4603(02)00300-3 DOI: https://doi.org/10.1016/S0306-4603(02)00300-3
Rogers, E. M. (2003). The diffusion of innovation (5th ed.). [Kindle Book] New York: Free Press.
Rogers, P., & Williams, B. (2006). Evaluation for practice improvement and organizational learning. In I. Shaw, J. Greene, & M. Mark (Eds.), The Sage handbook of evaluation, (77- 98). Thousand Oaks, CA: Sage Publications. https://doi.org/10.4135/9781848608078.n3 DOI: https://doi.org/10.4135/9781848608078.n3
Rogers, A., Ferguson, M., Ritchie, J., Van Den Boogaard, C., & Brimblecombe, J. (2016). Strengthening food systems with remote indigenous Australians: Stakeholders' perspectives. Health Promotion International, July, 1-11. https://doi.org/10.1093/heapro/daw047 DOI: https://doi.org/10.1093/heapro/daw047
Schon, D. (1963). Champions for radical new inventions. Harvard Business Review, 41(2), 77-86.
Scheirer, M. (2005). Is sustainability possible? A review and commentary on empirical studies of program sustainability. American Journal of Evaluation, 26(3), 320-347. https://doi.org/10.1177/1098214005278752 DOI: https://doi.org/10.1177/1098214005278752
Silliman, B., Crinion, P., & Archibald, T. (2016a). Evaluation champions: What they do, why they do it, and why it matters to organizations. Journal of Human Sciences and Extension, 4(3). https://doi.org/10.54718/NLUB7394 DOI: https://doi.org/10.54718/NLUB7394
Silliman, B., Crinion, P., & Archibald, T. (2016b). Evaluation champions: What they need and where they fit in organizational learning. Journal of Human Sciences and Extension, 4(3), 1-21. https://doi.org/10.54718/WOLP9257 DOI: https://doi.org/10.54718/WOLP9257
Shea, C., & Belden, C. (2016). What is the extent of research on the characteristics, behaviors, and impacts of health information technology champions? A scoping review. BMC Medical Informatics and Decision, 16(2), 1-17. https://doi.org/10.1186/s12911-016-0240-4 DOI: https://doi.org/10.1186/s12911-016-0240-4
Smith, A. G. (2012). Internet search tactics. Online Information Review, 36(1), 7-20. https://doi.org/10.1108/14684521211219481 DOI: https://doi.org/10.1108/14684521211219481
Solitander, N., Fougère, M., Sobczak, A., & Herlin, H. (2012). We are the champions: Organizational learning and change for responsible management education. Journal of Management Education, 36(3), 337-363. https://doi.org/10.1177/1052562911431554 DOI: https://doi.org/10.1177/1052562911431554
Sonnichsen, R. C. (2000). High impact internal evaluation. Thousand Oaks, CA: Sage Publications.
Stevahn, L. (2013). Integrating co-operative community- based research (CBR) into doctoral leadership studies. Journal of CoOperative Studies, 42(2), 35-45.
Stevahn, L., & King, J. A. (2005). Managing conflict constructively in program evaluation. Evaluation, 11, 415-427. https://doi.org/10.1177/1356389005059384 DOI: https://doi.org/10.1177/1356389005059384
Stevahn, L., & King, J. A. (2016). Facilitating interactive evaluation practice: Engaging stakeholders constructively. New Directions for Evaluation, 149, 67-80. https://doi.org/10.1002/ev.20180 DOI: https://doi.org/10.1002/ev.20180
Stockdill, S., Baizerman, M., & Compton, D. (2002). Toward a definition of the ECB process: A conversation with the ECB literature. New Directions for Evaluation, 93, 7-26. https://doi.org/10.1002/ev.39 DOI: https://doi.org/10.1002/ev.39
Stockmann, R., & Meyer, W. (2016). The future of evaluation: Global trends, new challenges and shared perspectives. UK: Palgrave MacMillan. https://doi.org/10.1057/9781137376374_1 DOI: https://doi.org/10.1057/9781137376374_1
Tarricone, P., & Luca, J. (2002). Employees, teamwork and social interdependence: A formula for successful business? Team Performance Management, 8(3/4), 54-59. https://doi.org/10.1108/13527590210433348 DOI: https://doi.org/10.1108/13527590210433348
Taylor, A., Cocklin, C., & Brown, R. (2012). Fostering environmental champions: A process to build their capacity to drive change. Journal of Environmental Management, 98, 84-97. https://doi.org/10.1016/j.jenvman.2011.12.001 DOI: https://doi.org/10.1016/j.jenvman.2011.12.001
Taylor, A., Cocklin, C., Brown, R., & Wilson-Evered, E. (2011). An investigation of champion-driven leadership processes. The Leadership Quarterly, 22(2), 412-433. https://doi.org/10.1016/j.leaqua.2011.02.014 DOI: https://doi.org/10.1016/j.leaqua.2011.02.014
Taylor-Powell, E., & Boyd, H. H. (2008). Evaluation capacity building in complex organisations. New Directions for Evaluation, 120, 55-69. https://doi.org/10.1002/ev.276 DOI: https://doi.org/10.1002/ev.276
Tindale, R. S., & Anderson, E. M. (2002). Small group research and applied social psychology. In R. S. Tindale, L. Heath, J. Edwards, E. Posavac, F. Bryant, Y. Suarez-Balcazar, … J. Myers (Eds.), Theory and research on small groups (1-8). Boston: Kluwer Academic Publishers. https://doi.org/10.1007/b108066 DOI: https://doi.org/10.1007/b108066
Tomioka, M., & Braun, K. L. (2014). Examining sustainability factors for organizations that adopted Stanford's Chronic Disease SelfManagement Program. Frontiers in Public Health, 2, 140. https://doi.org/10.3389/fpubh.2014.00140 DOI: https://doi.org/10.3389/fpubh.2014.00140
Vo, A. (2015). Foreword. In C. Christie & A. Vo (Eds.), Evaluation use and decision making in society: A tribute to Marvin C. Alkin (vii-xviii). Charlotte, NC: Information Age Publishing. DOI: https://doi.org/10.1108/978-1-68123-006-120251011
Volkov, B. B. (2011). Beyond being an evaluator: The multiplicity of roles of the internal evaluator. New Directions for Evaluation, 132, 25-42. https://doi.org/10.1002/ev.394 DOI: https://doi.org/10.1002/ev.394
Volkov, B. B & King, J. A. (2007). A checklist for building organizational evaluation capacity. The Evaluation Center: Western Michigan University. Retrieved from http://www.wmich.edu/evalctr/checklists/ecb.pdf
Wandersman, A. (2014). Getting to outcomes: An evaluation capacity building example of rationale, science, and practice. American Journal of Evaluation, 35(1), 100-106. https://doi.org/10.1177/1098214013500705 DOI: https://doi.org/10.1177/1098214013500705
Weiss, C. (2000). What we have learned from 25 years of knowledge utilization. In Social science and governance: The linkage between social science research and social policy. In Henke, R. Final Report of the International Conference on Social Science and Governance. Zeist, The Netherlands (20-21).