Reframing Evaluation: Complexity, Systems, and Transformation

Article Sidebar

cover page
PDF
Published: Oct 9, 2026
Keywords:
systems, systems thinking, Evaluation, Complexity, Transformational change, indigenous evaluation, portfolios, learning-oriented evaluation

Main Article Content

Hayat Askar
Independent Consultant
https://orcid.org/0009-0002-7956-1144
Barbara Schmidt-Abbey
Independent Consultant
https://orcid.org/0000-0003-3125-1163
Liliana Olivia Lucaciu
Lideea Development Actions
https://orcid.org/0000-0002-2699-6040
Anna Maria Augustyn
Independent Consultant
https://orcid.org/0000-0001-5847-1970
Scott Chaplowe
International Evaluation Academy
https://orcid.org/0009-0008-5735-3145
Neha Dhingra
Center for Research in Schemes and Policies (CRISP)
https://orcid.org/0009-0001-9059-9584
Ian Goldman
International Evaluation Academy
https://orcid.org/0000-0002-6671-1118
Amrita Gupta
Independent
Jonathan (Jonny) Morell
4.669 Evaluation and Planning
https://orcid.org/0009-0000-9252-9216
Nan Wehipeihana
Weaving Insights
Anna Matheson
Victoria University of Wellington

Abstract

The world is increasingly facing rapid degeneration of social and ecological systems. This requires evaluation to focus on the underlying systems rather than isolated interventions, prompting an interest in systems-informed evaluation, and evaluation that focuses on systems. This article proposes a practical conceptual framework that links widely recognized systems principles with concrete evaluation choices across the policy and evaluation lifecycle. The framework is grounded in current systems thinking and complexity literature and informed by multiple traditions in evaluation, including theory-based, participatory, and learning-oriented approaches. The article draws on literature, practitioner insights from the Three Horizons study and three real-world examples: humanitarian energy systems (RE4R), Indigenous community health systems (Healthy Families NZ), and portfolio-based municipal development (UNDP). It examines how systems principles are enacted through evaluation design, governance, methodology, analysis, validation, follow-up and use. The cross-case analysis highlights recurring features: shared ownership, attention to context and adaptive systems, and approaches that strengthen evaluative thinking. We conclude that systems-informed evaluation is less a specialist method than a practice stance, requiring intentional choices that hold complexity, share power amongst multiple stakeholders, and support ongoing learning. An agenda emerges that may be motivating for interested partners to collaborate on taking forward.

Downloads

Download data is not yet available.

Article Details

How to Cite
Askar, H., Schmidt-Abbey, B., Lucaciu, L. O., Augustyn, A. M., Chaplowe, S., Dhingra, N., … Matheson, A. (2026). Reframing Evaluation: Complexity, Systems, and Transformation. Journal of MultiDisciplinary Evaluation, 22(53), 85–107. Retrieved from https://jmde.journals.publicknowledgeproject.org/index.php/jmde_1/article/view/1333
Issue
Vol. 22 No. 53 (2026): IEAc Special Issue on Evaluation and the Transformational Imperative
Section
IEAc Special Issue on Evaluation and the Transformational Imperative
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright and Permissions

Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org

 

Author Biographies

Hayat Askar, Independent Consultant

Hayat is an independent evaluation and public policy consultant with over 15 years of experience. She holds a Graduate Diploma in Public Policy and Program Evaluation from Carleton University. Her career spans diverse sectors and contexts, combining full-time roles, consulting assignments, and pro bono work at local and global levels. Hayat has contributed to the design of M&E frameworks, organizational assessments, program evaluation, global reports and indices, and institutional capacity strengthening. She brings a systems-thinking and behavioral lens to her work, with expertise in results-based management, gender-sensitive evaluation, and evaluation capacity development, and is actively engaged in global evaluation networks.

Barbara Schmidt-Abbey, Independent Consultant

Barbara Schmidt-Abbey is a practitioner-scholar at the intersection of systems thinking, evaluation, and public policy. Her doctoral research in Systems Thinking in Practice examines how knowledge and evidence are produced and mobilised in complex knowledge-for-policy environments, focusing on evaluation as a systemic practice at the research–policy interface. She has over three decades of professional experience in European public policy, working in EU agencies supporting research, monitoring, and evaluation. 

She is a co-founder / co-leader of the European Evaluation Society’s Thematic Working Group on Systems Approaches in Evaluation, and a member of the EES Board.

Liliana Olivia Lucaciu, Lideea Development Actions

Liliana is an evaluation specialist in adaptive policies, systems, and learning, with over 25 years of experience working across the European Union and neighbouring regions. She holds a PhD in Cybernetics and Statistics and has led and coordinated complex evaluation, monitoring, and strategic planning assignments for European institutions, international organisations, and national governments. Her experience is grounded in EU policies (cohesion, neighbourhood, international partnership and development) with a focus on regional development, large infrastructure programmes, education, social inclusion, migration, and institutional reform. She is co-leader of the European Evaluation Society’s Thematic Working Group on Systems Approaches in Evaluation.

Anna Maria Augustyn, Independent Consultant

Anna Maria is a senior evaluation consultant with over two decades of experience at international, national, and local levels. She has worked on evaluation assignments in climate change, research and innovation, agriculture, and digital transformation, collaborating with the European Commission, the World Bank, and UN entities. She is an author and contributor to several evaluation guidelines and handbooks, including for the European Commission, the Adaptation Fund, and the European Evaluation Helpdesk for Rural Development. Her work spans standard and innovative evaluation methods, including systems approaches, with strong expertise in environment, climate, access to climate finance, particularly in Europe and Asia.

Scott Chaplowe, International Evaluation Academy

Scott Chaplowe is an evaluation, strategy, and capacity development specialist with over 30 years’ international experience across development, humanitarian, and climate sectors. His work emphasizes complex systems analysis, participatory approaches, and capacity development to support evaluation’s transformational potential in addressing today’s interconnected global challenges. He has worked extensively with multilateral and global institutions, civil society organizations, and philanthropy, and is actively engaged in evaluation communities of practice, including as a founding member of the International Evaluation Academy. His work includes multiple publications, practitioner guides, and contributions to evaluation policies, standards, and frameworks for complex contexts.

Neha Dhingra, Center for Research in Schemes and Policies (CRISP)

Neha Dhingra is a development professional specialising in gender equity, evaluation, and policy. With 15+ years' experience, she leads multi-state program evaluation at CRISP, examining intersections of poverty, inclusive livelihoods and gender. Core member of Gender & Equity Network South Asia, she has worked extensively on violence against women, gender-responsive programming, and evidence translation for policy. Her expertise spans qualitative research, participatory and equity-focused evaluations. She has worked extensively with International organisations and engaged stakeholders, including policymakers, donors, and grassroots organisations. She has published on gender and young women's agency, especially from marginalised backgrounds.

Ian Goldman, International Evaluation Academy

Ian is focused using monitoring, evaluation and learning to address the polycrisis. He has been President of the International Evaluation Academy since 2022. Ian was Deputy Director General in the South African Department of Planning, Monitoring and Evaluation where he established the National Evaluation System. He works with the CLEAR Evaluation Centres for Pakistan/Central Asia and Anglophone Africa, the latter at the University of Witwatersrand. He is a Professor at the University of Cape Town and an editor and author of the book Evidence Use in Policy and Practice: lessons from Africa.

Amrita Gupta, Independent

Amrita has 20 years of experience in mixed-methods research, monitoring and evaluation, and policy review, with a strong focus on advancing gender equality, women’s health, and economic opportunities. She holds a Master’s in Public Health, Health Policy, Economics and Finance, and a Master’s in Sociology. Currently an evaluation consultant, she leads national and global evaluations on care policy, education, youth, and women’s leadership and skills. She is a core team member of the Gender & Equity Network South Asia (GENSA), a member of the Evaluation Community of India and APEA’s Partnerships Thematic Group, and an IPDET 2024 alumna.

Jonathan (Jonny) Morell, 4.669 Evaluation and Planning

Jonathan A. Morell PhD (Jonny) believes that evaluation requires a systems-oriented approach because interventions produce complex outcomes, but that evaluations should be as simple and straightforward as possible. Drawing from his practical experience, Jonny has made theoretical contributions to Evaluation that focus on several related topics: unintended consequences, Complexity Science, Ecology and Evolutionary Biology, program schedules as logic models, and assumptions as complex systems. Jonny is a recipient of the American Evaluation Association’s Lazarsfeld Evaluation Theory Award. He is President of 4.669… Evaluation and Planning. Details of his work can be found at www.evaluationuncertainty.com.

Nan Wehipeihana, Weaving Insights

Nan Wehipeihana is an independent evaluator based in Wellington, New Zealand. In 1999, she established Research Evaluation Consultancy Limited and is a member of the Kinnect Group. Her evaluation work has a strong focus on Māori, the Indigenous people of Aotearoa New Zealand, and on culturally grounded, equity-focused approaches. She is an international leader in Developmental Evaluation and systems-informed practice. Her recent work focuses on large-scale system change and community-led wellbeing initiatives, including long-term evaluations such as Healthy Families NZ. Her interests include Indigenous evaluation, power-sharing, and how evaluation can support learning and adaptation in complex systems.

References

Aston, P., & Apgar, M. (2022). Building confidence in evaluative judgements: Methodological bricolage and contribution analysis in complex contexts. Evaluation, 28(4), 481–499.

Beach, D., & Pedersen, R. B. (2013). Process-tracing methods: Foundations and guidelines. University of Michigan Press.

Befani, B., & Mayne, J. (2014). Process tracing and contribution analysis: A combined approach to generative causal inference for impact evaluation. IDS Bulletin, 45(6), 17–36. https://doi.org/10.1111/1759-5436.12110

Befani, B. (2016). Testing contribution claims with Bayesian updating. CECAN Evaluation and Policy Practice Note No. 2.1. Centre for the Evaluation of Complexity Across the Nexus. https://www.cecan.ac.uk/sites/default/files/2018-01/BARBARA%20v2.5.pdf

Bellamy, J. A., Walker, D. H., McDonald, G. T., & Syme, G. J. (2001). A systems approach to the evaluation of natural resource management initiatives. Journal of Environmental Management, 63(4), 407–423. https://doi.org/10.1006/jema.2001.0493

Binder, C. R., Hinkel, J., Bots, P. W. G., & Pahl-Wostl, C. (2013). Comparison of frameworks for analyzing social-ecological systems. Ecology and Society, 18(4). https://doi.org/10.5751/es-05551-180426

Buckton, S. J., Fazey, I., Ball, P., Ofir, Z., Colvin, J., Darby, M., Hejnowicz, A. P., Leicester, G., Newman, R., Page, G. G., Parsons, K., & van Mierlo, B. (2025). Twelve principles for transformation-focused evaluation. PLOS Sustainability and Transformation, 4(4), e0000164. https://doi.org/10.1371/journal.pstr.0000164

Chaplowe, S., & Gupta, T. (2022). Final report: Independent evaluation (systems impact assessment) of the Renewable Energy for Refugees project. Practical Action and UNHCR.

Climate Investment Funds (CIF). (2021). Transformational change concepts: Working definition, dimensions, questions and signals. Retrieved from https://www.cif.org/knowledge-documents/transformational-change-concepts-working-definition-dimensions-questions-and

Coffman, J. (2007). A framework for evaluating systems initiatives. Harvard Family Research Project.

Darnhofer, I., Gibbon, D., & Dedieu, B. (2017). Towards a complexity-aware theory of change for participatory research programs. Agricultural Systems, 153, 36–45.

Department for Environment, Food and Rural Affairs (Defra). (2020). Complexity evaluation framework: Recognising complexity and key considerations for complexity-appropriate evaluation (Version 2.0, updated by Steer Economic Development). https://www.betterevaluation.org/tools-resources/complexity-evaluation-framework-recognising-complexity-key-considerations-for-complexity-appropriate

Douthwaite, B., & Hoffecker, E. (2017). Towards a complexity-aware theory of change for participatory research programs working within agricultural innovation systems. Agricultural Systems, 155, 88–98. https://doi.org/10.1016/j.agsy.2017.04.002

European Commission / UNDP. (n.d.). Mayors for Economic Growth (M4EG). https://eum4eg.com

Garcia, J. R., & Uitto, J. I. (2025). Using a systems lens in evaluations at the nexus of environment and development. Evaluation and Program Planning, 112, 102560. https://doi.org/10.1016/j.evalprogplan.2025.102560

Geels, F. W., & Schot, J. (2007). Typology of sociotechnical transition pathways. Research Policy, 36(3), 399-417.

Goldman, I., Cairney, P., Carden, F., Dias, T., Dillon, B., Ferrari, R., & Morgan, G. (2026). Futures, Feedback, and Flexibility: Reimagining Public Sector Management and Evaluation in the Time of Polycrisis. Journal of MultiDisciplinary Evaluation, 22(53), 233–254.

Goldman, I., & Pabari, M. (2020). Lessons for using evidence in policy and practice. In I. Goldman & M. Pabari (Eds.), Using evidence in policy and practice: Lessons from Africa. Routledge. https://www.taylorfrancis.com/books/e/9781003007043

Ison, R., Blackmore, C., & Collins, L. (2021). Fostering complexity thinking in action research for change in social-ecological systems. Systemic Practice and Action Research, 34(1), 1–19.

Latham, N. (2014). Pathways to system change: The evaluation of complex systems. Evaluation, 20(1), 60–77.

Leeuw, F. L., & Furubo, J.-E. (2008). Evaluation systems: What are they and why study them? Evaluation, 14(2), 157–169. https://doi.org/10.1177/1356389007087537

Matheson, A., Gray, R., Wehipeihana, N., et al. (2020). Evaluation of Healthy Families NZ: Interim report 2020. Te Herenga Waka – Victoria University of Wellington, Wellington.

Mingers, J., & Gill, A. (Eds.). (1997). Multimethodology: Theory and practice of combining management science methodologies. John Wiley & Sons.

Morell, J. A. (2010). Evaluation in the face of uncertainty: Anticipating surprise and responding to the inevitable. Guilford Press.

Morell, J. A. (2025). Characteristics of sustainability: A complexity-based view to inform evaluation. Evaluation and Program Planning, 111, 102561.

Morell, J. A., Hilscher, R., Magura, S., & Ford, J. (2010). Integrating evaluation and agent-based modeling: Rationale and an example for adopting evidence-based practices. Journal of MultiDisciplinary Evaluation, 6(14), 32–57. https://doi.org/10.56645/jmde.v6i14.275

Ndhlovu, L., Smith, L., & Narsoo, S. (2017). The search for a method to unpack urban complexity: Case study of the City of Johannesburg. African Evaluation Journal, 5(1), 1–11. https://doi.org/10.4102/aej.v5i1.195

Ofir, Z. (2021). Evaluation in transition: The promise and challenge of South-South cooperation. Canadian Journal of Program Evaluation, 36(1), 18–38. https://doi.org/10.3138/cjpe.71630

Ofir, Z., Kendrick, I., Goldman, I., Ferretti, S., Norins, J., Sithole, S., Belzer, A., Chaplowe, S., Diwakar, Y., Kalugampitiya, A., Lareza, A. E., Verveld-Nguyen, L., Zaveri, S. (2026). Where Next for Evaluation? Navigating the Polycrisis Through Three Horizons. Journal of MultiDisciplinary Evaluation, 22(53), 12–40.

Patton, M. Q. (2010). Developmental evaluation applying complexity concepts to enhance innovation and use. Guildford Press. http://www.guilford.com/cgi-bin/cartscript.cgi?page=pr/patton.htm&dir=research/res_eval&cart_id=519053.25995

Patton, M. Q. (2011). A complexity-based plan for evaluating transformation. Evaluation, 17(4), 373–389.

Pawson, R., & Tilley, N. (1997). Realistic evaluation. Sage Publications.

Reynolds, M., Gates, E., Hummelbrunner, R., Marra, M., & Williams, B. (2016). Towards systemic evaluation. Systems Research and Behavioral Science, 33(5), 662–673. https://doi.org/10.1002/sres.2423

Schweitzer, F., Fagiolo, G., Sornette, D., Vega-Redondo, F., White, H., & Helbing, D. (2009). Economic networks: The new challenges. Science, 325(5939), 422–425.

Snowden, D. J., & Boone, M. E. (2007). A leader’s framework for decision making. Harvard Business Review, 85(11), 68–149.

Systems in Evaluation Topical Interest Group (SETIG). (2018). Principles for effective use of systems thinking in evaluation practice. American Evaluation Association. https://www.betterevaluation.org/sites/default/files/SETIG-Principles-FINAL-DRAFT-2018-9-9.pdf

Shipurkar, T. J., Narayanan, P., Manjutha, C., Ponnuchamy, M., Khan, N., & Joseph, A. R. (2026). Measuring Collectivization Initiatives: Creating a Knowledge Base from Collective Voices. Journal of MultiDisciplinary Evaluation, 22(53), 255–279.

UNDP. (2021). UNDP evaluation guidelines. https://erc.undp.org/pdf/UNDP_Evaluation_Guidelines.pdf

UNDP. (2022). System change: A guidebook for adopting portfolio approaches. https://www.undp.org/publications/system-change-guidebook-adopting-portfolio-approaches

UNDP. (2024). Mayors for Economic Growth: Final evaluation report. https://erc.undp.org/evaluation/documents/download/23754

UNDP. (n.d.). Systems portfolio resource hub. https://www.undp.org/systems-portfolio

van den Berg, R. D., & McCollum, D. (2021). Meta-analysis of climate mitigation evaluations. Evaluation and Program Planning, 85, 101910.

Williams, B., & Hummelbrunner, R. (2010). Systems concepts in action: A practitioner’s toolkit. Stanford University Press.

Wilson-Grau, R., & Britt, H. (2012). Outcome harvesting. Ford Foundation. https://www.betterevaluation.org/tools-resources/outcome-harvesting-0

Zaveri, S., Goldman, I., Norins, J., Ghiano, C., Kotschy, K., & Soal, S. (in press). How can evaluations support a socially just transition to planetary health? In G. Batra, J. I. Uitto, & M. A. Spearman (Eds.), Integrating for sustainability: Evaluation across environmental and socioeconomic domains (Chapter 11). Springer.

Most read articles by the same author(s)