Complexity-Aware Monitoring and Evaluation
Main Article Content
Abstract
Background: Addressing today’s sustainability challenges requires adopting a systemic approach where social and ecological systems are treated as integrated social-ecological systems. Such systems are complex, and the international development sector increasingly recognises the need to account for the complexity of the systems that they seek to transform.
Purpose: This paper sketches out the elements of a complexity-aware monitoring and evaluation (M&E) system for international development programmes in the area of sustainable development.
Setting: Not applicable.
Data Collection and Analysis: The authors draw on existing literature on complexity and evaluation and on their own experience from working in the field of M&E.
Findings: An M&E system should not be seen simply as a tool to track compliance against a pre-determined theory of change. Instead, it is most useful as a real-time approach, constantly defining and re-defining narratives for change that help push systems along trajectories of interest. Dealing with complexity involves embracing uncertainty; and this challenges established notions of accountability—something which funders and implementers must begin to redefine together.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
Funding data
-
European Research Council
Grant numbers 682472 — MUSES
References
Bamberger, M., Vaessen, J., & Raimondo, E. (2016). Dealing with complexity in development evaluation: A practical approach. SAGE Publications, Inc. https://doi.org/10.4135/9781483399935 DOI: https://doi.org/10.4135/9781483399935
Biggs, R. (Ed.). (2021). The Routledge handbook of research methods for social-ecological systems. Routledge. https://doi.org/10.4324/9781003021339 DOI: https://doi.org/10.4324/9781003021339
Biggs, R., Schlüter, M., Biggs, D., Bohensky, E. L., BurnSilver, S., Cundill, G., Dakos, V., Daw, T. M., Evans, L. S., Kotschy, K., Leitch, A. M., Meek, C., Quinlan, A., Raudsepp-Hearne, C., Robards, M. D., Schoon, M. L., Schultz, L., & West, P. C. (2012). Toward Principles for Enhancing the Resilience of Ecosystem Services. Annual Review of Environment and Resources, 37(1), 421-448. https://doi.org/10.1146/annurev-environ-051211-123836 https://doi.org/10.1146/annurev-environ-051211-123836 DOI: https://doi.org/10.1146/annurev-environ-051211-123836
Boulton, J. G., Allen, P. M., & Bowman, C. (2015). Embracing complexity: Strategic perspectives for an age of turbulence (First edition). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199565252.001.0001 DOI: https://doi.org/10.1093/acprof:oso/9780199565252.001.0001
Box, G. E. P., & Draper, N. R. (1987). Empirical model-building and response surfaces. Wiley.
Burns, D., & Worsley, S. (2015). Navigating complexity in international development: Facilitating sustainable change at scale. https://doi.org/10.3362/9781780448510.000 DOI: https://doi.org/10.3362/9781780448510.000
Cadena, M. de la. (2015). Earth beings: Ecologies of practice across Andean worlds. Duke University Press.
Deleuze, G., & Guattari, F. (1987). A thousand plateaus: Capitalism and schizophrenia. University of Minnesota Press.
Folke, C. (2016). Resilience (Republished). Ecology and Society, 21(4), art44. https://doi.org/10.5751/ES-09088-210444 https://doi.org/10.5751/ES-09088-210444 DOI: https://doi.org/10.5751/ES-09088-210444
Folke, C., Carpenter, S., Walker, B., Scheffer, M., Elmqvist, T., Gunderson, L., & Holling, C. S. (2004). Regime Shifts, Resilience, and Biodiversity in Ecosystem Management. Annual Review of Ecology, Evolution, and Systematics, 35(1), 557-581. https://doi.org/10.1146/annurev.ecolsys.35.021103.105711 https://doi.org/10.1146/annurev.ecolsys.35.021103.105711 DOI: https://doi.org/10.1146/annurev.ecolsys.35.021103.105711
Hertz, T., Mancilla Garcia, M., & Schlüter, M. (2020). From nouns to verbs: How process ontologies enhance our understanding of social-ecological systems understood as complex adaptive systems. People and Nature, 2(2), 328-338. https://doi.org/10.1002/pan3.10079 https://doi.org/10.1002/pan3.10079 DOI: https://doi.org/10.1002/pan3.10079
Holland, J. H. (2014). Complexity: A very short introduction (First edition). Oxford University Press. https://doi.org/10.1093/actrade/9780199662548.001.0001 DOI: https://doi.org/10.1093/actrade/9780199662548.001.0001
Kurtz, C. F., & Snowden, D. J. (2003). The new dynamics of strategy: Sense-making in a complex and complicated world. IBM Systems Journal, 42(3), 462-483. https://doi.org/10.1147/sj.423.0462 https://doi.org/10.1147/sj.423.0462 DOI: https://doi.org/10.1147/sj.423.0462
Lakoff, G., & Johnson, M. (2003). Metaphors we live by. University of Chicago Press. https://doi.org/10.7208/chicago/9780226470993.001.0001 DOI: https://doi.org/10.7208/chicago/9780226470993.001.0001
Lansing, J. S. (1987). Balinese "Water Temples" and the Management of Irrigation. American Anthropologist, 89(2), 326-341. https://doi.org/10.1525/aa.1987.89.2.02a00030 https://doi.org/10.1525/aa.1987.89.2.02a00030 DOI: https://doi.org/10.1525/aa.1987.89.2.02a00030
Leichenko, R. M., & O'Brien, K. L. (2019). Climate and society: Transforming the future. Polity.
Levin, S. A. (1998). Ecosystems and the Biosphere as Complex Adaptive Systems. Ecosystems, 1(5), 431-436. https://doi.org/10.1007/s100219900037 https://doi.org/10.1007/s100219900037 DOI: https://doi.org/10.1007/s100219900037
Levin, S., Xepapadeas, T., Crépin, A.-S., Norberg, J., de Zeeuw, A., Folke, C., Hughes, T., Arrow, K., Barrett, S., Daily, G., Ehrlich, P., Kautsky, N., Mäler, K.-G., Polasky, S., Troell, M., Vincent, J. R., & Walker, B. (2013). Social-ecological systems as complex adaptive systems: Modeling and policy implications. Environment and Development Economics, 18(2), 111-132. https://doi.org/10.1017/S1355770X12000460 https://doi.org/10.1017/S1355770X12000460 DOI: https://doi.org/10.1017/S1355770X12000460
Mancilla Garcia, M., Hertz, T., & Schlüter, M. (2020). Towards a Process Epistemology for the Analysis of Social-Ecological System. Environmental Values, 29(2), 221-239. https://doi.org/10.3197/096327119X15579936382608 https://doi.org/10.3197/096327119X15579936382608 DOI: https://doi.org/10.3197/096327119X15579936382608
Manning, E. (2012). Always More Than One: Individuation's Dance. Duke University Press. https://doi.org/10.1215/9780822395829 https://doi.org/10.1215/9780822395829 DOI: https://doi.org/10.1215/9780822395829
Moore, M.-L., Olsson, P., Nilsson, W., Rose, L., & Westley, F. R. (2018). Navigating emergence and system reflexivity as key transformative capacities: Experiences from a Global Fellowship program. Ecology and Society, 23(2), art38. https://doi.org/10.5751/ES-10166-230238 https://doi.org/10.5751/ES-10166-230238 DOI: https://doi.org/10.5751/ES-10166-230238
Natsios, A. (2010). The Clash of the Counter-bureaucracy and Development. Center for Global Development. https://www.cgdev.org/publication/clash-counter-bureaucracy-and-development
Page, S. E. (2015). What Sociologists Should Know About Complexity. Annual Review of Sociology, 41(1), 21-41. https://doi.org/10.1146/annurev-soc-073014-112230 https://doi.org/10.1146/annurev-soc-073014-112230 DOI: https://doi.org/10.1146/annurev-soc-073014-112230
Pascual, U., Palomo, I., Adams, W. M., Chan, K. M. A., Daw, T. M., Garmendia, E., Gómez-Baggethun, E., de Groot, R. S., Mace, G. M., Martín-López, B., & Phelps, J. (2017). Off-stage ecosystem service burdens: A blind spot for global sustainability. Environmental Research Letters, 12(7), 075001. https://doi.org/10.1088/1748-9326/aa7392 https://doi.org/10.1088/1748-9326/aa7392 DOI: https://doi.org/10.1088/1748-9326/aa7392
Patsalides, M., & Britt, H. (2014). Complexity-Aware Monitoring at USAID. https://usaidlearninglab.org/library/complexity-aware-monitoring-usaid-brief
Patton, M. Q. (2018). Principles-focused evaluation: The guide. Guilford Press.
Patton, M. Q. (2020). Blue marble evaluation: Premises and principles. The Guilford Press. https://doi.org/10.31244/zfe.2020.02.09 DOI: https://doi.org/10.31244/zfe.2020.02.09
Preiser, R., Biggs, R., De Vos, A., & Folke, C. (2018). Social-ecological systems as complex adaptive systems: Organizing principles for advancing research methods and approaches. Ecology and Society, 23(4), art46. https://doi.org/10.5751/ES-10558-230446 https://doi.org/10.5751/ES-10558-230446 DOI: https://doi.org/10.5751/ES-10558-230446
Preiser, R., Pereira, L., Biggs, O., Hamann, M., Maciejewski, K., & Cloete, D. (2017). Guidance for Resilience in the Anthropocene: Investments for Development (GRAID)-Annual Report 2016 (Centre for Complex Systems in Transition CSTStellenbosch University, South Africa). https://www0.sun.ac.za/cst/wp-content/uploads/2017/03/CST-GRAID-2016-Report-FINAL-1.pdf
Raworth, K. (2017). Doughnut economics: Seven ways to think like a 21st century economist. Chelsea Green Publishing.
Rittel, H., & Webber, M. (1973). Dilemmas in a General Theory of Planning. Policy Sciences, 4(2). https://doi.org/10.1007/BF01405730 DOI: https://doi.org/10.1007/BF01405730
Srnicek, Nick. (2007). Assemblage Theory, Complexity and Contentious Politics. University of Western Ontario.
Steffen, W., Richardson, K., Rockstrom, J., Cornell, S. E., Fetzer, I., Bennett, E. M., Biggs, R., Carpenter, S. R., de Vries, W., de Wit, C. A., Folke, C., Gerten, D., Heinke, J., Mace, G. M., Persson, L. M., Ramanathan, V., Reyers, B., & Sorlin, S. (2015). Planetary boundaries: Guiding human development on a changing planet. Science, 347(6223), 1259855-1259855. https://doi.org/10.1126/science.1259855 https://doi.org/10.1126/science.1259855 DOI: https://doi.org/10.1126/science.1259855
Tengö, M., Brondizio, E. S., Elmqvist, T., Malmer, P., & Spierenburg, M. (2014). Connecting Diverse Knowledge Systems for Enhanced Ecosystem Governance: The Multiple Evidence Base Approach. AMBIO, 43(5), 579-591. https://doi.org/10.1007/s13280-014-0501-3 https://doi.org/10.1007/s13280-014-0501-3 DOI: https://doi.org/10.1007/s13280-014-0501-3
Urry, J. (2005). The Complexity Turn. Theory, Culture & Society, 22(5), 1-14. https://doi.org/10.1177/0263276405057188 https://doi.org/10.1177/0263276405057188 DOI: https://doi.org/10.1177/0263276405057188
USAID Office of Learning Evaluation and Research (LER). (n.d.). Complexity-Aware Monitoring Discussion Note [Discussion Note]. https://usaidlearninglab.org/library/complexity-aware-monitoring-discussion-note-brief
Westley, F., & Antadze, N. (2010). Making a Difference: Strategies for Scaling Social Innovation for Greater Impact. The Innovation Journal: The Public Sector Innovation Journal, 15(2).
Williams, B. (2016). Systemic Evaluation Design. A Workbook. https://gumroad.com/l/evaldesign.
Williams, B. (2021). Systemic Evaluation Design. https://www.youtube.com/watch?v=VUTqG5j21mU
Zazueta, A. E., Le, T. T., & Bahramalian, N. (2020). Development Trajectories and Complex Systems-Informed Theories of Change. American Journal of Evaluation, 109821402094778. https://doi.org/10.1177/1098214020947782 https://doi.org/10.1177/1098214020947782 DOI: https://doi.org/10.1177/1098214020947782