Towards a Complexity Framework for Transformative Evaluation
Main Article Content
Abstract
Background: Complexity ideas originating in mathematics and the natural sciences have begun to inform evaluation practice. A new wave in evaluation history is about to break. A new mindset, new methods, and new evaluation processes are being summoned to explore and address the challenges of global pandemics, growing inequities, and existential environmental risks. This is part of a broader paradigm shift underway in science where interdisciplinarity has become the norm rather than the exception.
Purpose: This article explores the utility of a complexity framework for a more effective evaluation function. It unearths the antecedents of complexity thinking; explores its relevance to evolving knowledge paradigms; provides a bird’s eye view of complexity concepts; uses the logic of complex adaptive systems to unpack the role of evaluation in society; and draws the implications of contemporary social challenges for evaluation policy directions.
Setting: Not applicable.
Intervention: Not applicable.
Research design: Not applicable.
Findings: The evaluation complexity challenge coincides with an urgent imperative: social transformation. The on-going pandemic has brought to light the disproportionate effects of health emergencies on disadvantaged groups and emphasized the urgency of improving the interface between humans and nature. It has also demonstrated the importance of modelling for policy making – as well as its limitations. Evaluation, a complex adaptive system, should be transformed to serve society.
Keywords: complexity; computers; disciplines; emergence; modelling; paradigm, systems
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Apgar, J.M, Argumedo, A., & Allen, W. (2009). Building Transdisciplinarity for Managing Complexity: Lessons from Indigenous Practice. International Journal of Interdisciplinary Social Sciences, Volume 4, Number 5, pp.255-270 https://doi.org/10.18848/1833-1882/CGP/v04i05/52925 DOI: https://doi.org/10.18848/1833-1882/CGP/v04i05/52925
Arrow, K.J. (1974). The limits of organization. W.W. Norton, New York, NY and London, United Kingdom
Box, G. E. P. (1976), "Science and statistics" (PDF), Journal of the American Statistical Association, 71 (356): pp. 791-799 https://doi.org/10.1080/01621459.1976.10480949 DOI: https://doi.org/10.1080/01621459.1976.10480949
Case, A. and Deaton, A. (2020). The Epidemic of Despair. Foreign Affairs, March/April.
Castellani, B. (2014). Brian Castellani on the Complexity Sciences. Theory Culture and Society Website. October. https://www.theoryculturesociety.org/brian-castellani-on-the-complexity-sciences/
Cecere, G., Corrocher, N., Gossart, C., & Ozman. M. (2014) Lock-in and path dependence: an evolutionary approach to eco-innovations, Journal of Evolutionary Economics, 24, pp. 1037-1065 https://doi.org/10.1007/s00191-014-0381-5 DOI: https://doi.org/10.1007/s00191-014-0381-5
Cilliers, P. (1998) Complexity and Postmodernism Routledge London United Kingdom
Cohen, E.B. and Lloyd S. J. (2014). Disciplinary Evolution and the Rise of the Transdiscipline. Informing Science: The International Journal of an Emerging Transdiscipline. University of Rhode Island, Kingston, Rhode Island. 17 189-215 https://doi.org/10.28945/2045 DOI: https://doi.org/10.28945/2045
Denzin, N. K. (2010). Moments, Mixed Methods, and Paradigm Dialogs. Qualitative Inquiry. 16 (6) pp. 419-427 https://doi.org/10.1177/1077800410364608 DOI: https://doi.org/10.1177/1077800410364608
Dobusch, L. and Schubler, E. (2013) Theorizing Path Dependence: A Review of Positive Feedback Mechanisms in Technology Markets, Regional Clusters, and Organizations. Industrial and Corporate Change, 22 (3), pp. 617-647 https://doi.org/10.1093/icc/dts029 DOI: https://doi.org/10.1093/icc/dts029
Feinstein, O, (2020). Development and radical uncertainty. Development in Practice. Routledge. Taylor and Francis. January 6 (online) https://doi.org/10.1080/09614524.2020.1763258 DOI: https://doi.org/10.1080/09614524.2020.1763258
Forss, K., Marra, M., Schwartz, R. Eds. (2011), Evaluating the Complex: Attribution, Contribution and Beyond, Transaction Publishers, New Brunswick, NJ
Foucault, M. (1977). Discipline and Punish: the Birth of the Prison, Random House, New York, NY, p.27
Gerrits, L., and Veweij, S. (2015). Taking stock of complexity in evaluation: a discussion of three recent publications. Evaluation. Sage Publications, Thousand Oaks, California, 21 (4) 481-491 https://doi.org/10.1177/1356389015605204 DOI: https://doi.org/10.1177/1356389015605204
Gladwell, M. (1963). The Tipping Point : How Little Things Can Make a Big Difference. Back Bay Books. Boston, Massachussetts
Granovetter, M. (1983). The Strength of Weak Ties: A Network Theory Revisited. Sociological Theory. Vol 1, pp. 201-233 https://doi.org/10.2307/202051 DOI: https://doi.org/10.2307/202051
Hirschman, A.O. (1977) The Passions and the Interests: Political Arguments for Capitalism before its Triumph Princeton University Press, Princeton, New Jersey https://doi.org/10.1515/9781400848515 DOI: https://doi.org/10.1515/9781400848515
Hirschman, A.O. (1970) Exit, Voice and Loyalty: Responses to Decline in Firms, Organizations and States. Harvard University Press, Cambridge, Massachusetts and London, United Kingdom
Hogan, J. (1995) From Complexity to Perplexity. Scientific American. June. Pp. 104-109 https://doi.org/10.1038/scientificamerican0695-104 DOI: https://doi.org/10.1038/scientificamerican0695-104
Holland, J. H. (1975) Adaptation in Natural and Artificial Systems. U. of Michigan Press
Holland, J. H. (2014) Complexity: A Very Short Introduction, Oxford University Press, Oxford, UK, p.90 https://doi.org/10.1093/actrade/9780199662548.001.0001 DOI: https://doi.org/10.1093/actrade/9780199662548.001.0001
Homer-Dixon, T., Maynard, J. L., Midlenberger, M., Milkoreit, M., Mock, S.J.. Quilley, S.. Schroder, T., Thagard, P. (2013) A Complex Systems Approach to the Study of Ideology: Cogenitive-Affective Structures and the Dynamics of Belief Systems, Journal of Social and Political Psychology, 1 (1) pp. 337-363 https://doi.org/10.5964/jspp.v1i1.36 DOI: https://doi.org/10.5964/jspp.v1i1.36
Hoy, D.C. (2004) Critical Resistance: From Poststructuralism to Post-Critique. The MIT Press, Cambridge, Massachusetts and London, United Kingdom https://doi.org/10.7551/mitpress/2217.001.0001 DOI: https://doi.org/10.7551/mitpress/2217.001.0001
Inglehart, R. and Baker, W.E. (2000). Modernization, Cultural Change and the Persistence of Traditional Values, American Sociological Review. February, 65 pp. 19-51 https://doi.org/10.1177/000312240006500103 DOI: https://doi.org/10.1177/000312240006500103
Kuhn, T.S. (1962), The Structure of Scientific Revolutions. Chicago: University of Chicago Press
Latour, B. (1991) We have never been modern Harvard University Press, Cambridge, Massachusetts
Latour, B., (2005) Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford University Press, Oxford. United Kingdom https://doi.org/10.1093/oso/9780199256044.001.0001 DOI: https://doi.org/10.1093/oso/9780199256044.001.0001
Maskin. E. and Sen, A. (2014) The Arrow Impossibility Theorem, Columbia University Press, New York https://doi.org/10.7312/mask15328 DOI: https://doi.org/10.7312/mask15328
Mathison, S. ed., (2005) Encyclopedia of Evaluation, Sage Publications, pp 139-140 https://doi.org/10.4135/9781412950558 DOI: https://doi.org/10.4135/9781412950558
McKibben, B. (2020). Racism, Police Violence and the Climate are not Separate Issues. The New Yorker. June 4. New York, NY
Merton, R.K., (1996). On Social Structure and Science, edited by Piort Sztompka. University of Chicago Press, Chicago, IL
Morrell, J, (2021). A Complexity-based Metatheory of Action for Transformation to a Green Energy Future. In Transformational Evaluation for the Global Crises of our Times, to be published by the International Development Evaluation Association (IDEAS), Exeter, UK
Mikulecky, D.C., (2001) The emergence of complexity: science coming of age or science growing old? Computers and Chemistry, Elsevier, 25. pp. 341-348 https://doi.org/10.1016/S0097-8485(01)00070-5 DOI: https://doi.org/10.1016/S0097-8485(01)00070-5
Mingers, J. (2004). Paradigm wars: Ceasefire announced who will set up the new administration. Journal of Information Technology. August. Palgrave. pp.165-171 https://doi.org/10.1057/palgrave.jit.2000021 DOI: https://doi.org/10.1057/palgrave.jit.2000021
Mishra, P. (2017). The Age of Anger: A History of the Present. New York: Farrar, Straus, and Giroux.
Mitchell,M. (2009). Complexity: A Guided Tour. Oxford University Press, Oxford, UK https://doi.org/10.1093/oso/9780195124415.001.0001 DOI: https://doi.org/10.1093/oso/9780195124415.001.0001
Oreskes, N., Shrader-Frechette, K., Belitz, K., (1994) Verification, Validation, and Confirmation of Numerical Models in the Earth Sciences, Science, New Series, Vol. 263, No. 5147 (February). Pp. 641-646 https://doi.org/10.1126/science.263.5147.641 DOI: https://doi.org/10.1126/science.263.5147.641
Pawson, R. (2013). The science of evaluation: a realist manifesto, London, SAGE Publications. https://doi.org/10.4135/9781473913820 DOI: https://doi.org/10.4135/9781473913820
Patton, M.Q. (2008) Utilization-focused evaluation (4th Ed). Sage Publications. Thousand Oaks. CA
Patton, M.Q. (2010) Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. Guilford Press, New York
Pearl, J. & Mackenzie, D. (2018). The book of why: The new science of cause and effect. New York. Basic Books
Picciotto, R. (2017). Evaluation: Discursive practice or communicative action? Evaluation, July Issue 23 (3) pp. 312-322 https://doi.org/10.1177/1356389017714384 DOI: https://doi.org/10.1177/1356389017714384
Picciotto, R. (2019) Is Adversary Evaluation Worth a Second Look? American Journal of Evaluation, 40 (1) Sage Publications, pp. 92-103 https://doi.org/10.1177/1098214018783068 DOI: https://doi.org/10.1177/1098214018783068
Popper, K. (1959) The Logic of Scientific Discovery, translation of Logik der Forschung, Hutchinson, London, UK
Ramalingam, B. and Jones H. (2008) Exploring the science of complexity: Ideas and implications for development and humanitarian efforts. Second Edition. Working Paper 285. Overseas Development Institute, London, UK. https://www.odi.org/sites/odi.org.uk/files/odi-assets/publications-opinion-files/833.pdf
Rajan, R. (2019) The Third Pillar: How Markets and the State Leave the Community Behind. Penguin Press, New York
Rupert M., Rattrout, A., Hassas, S. (2008). The web from a complex adaptive systems perspective, Journal of Computer and System Sciences. Elsevier, 74, pp. 133-145 https://doi.org/10.1016/j.jcss.2007.04.001 DOI: https://doi.org/10.1016/j.jcss.2007.04.001
Scriven, M. (1991) Evaluation Thesaurus. Fourth Edition. Sage Publications. Newbury Park, London UK, New Delhi. p. 364
Scriven, M. (2013) Key Evaluation Checklist (KEC). July 24. p. 3 (footnote 8) http://michaelscriven.info/images/KEC_7.25.2013.pdf
Shubik, M. (1998) Game Theory, Complexity, and Simplicity Part 1: A Tutorial. Working Paper 04-027. Santa Fe Institute. Santa Fe, New Mexico https://sfi-edu.s3.amazonaws.com/sfi-edu/production/uploads/sfi-com/dev/uploads/filer/da/53/da53eb6a-a1e6-41dc-898f-f27bda9f7b97/98-04-027.pdf
Smith, A. (1759). The Theory of Moral Sentiments, D.D. Raphael and A.L. Macfie (eds.), Oxford: Oxford University Press, 1976. https://doi.org/10.1093/oseo/instance.00042831 DOI: https://doi.org/10.1093/oseo/instance.00042831
Smith, A. (1776). An Inquiry into the Nature and Causes of The Wealth of Nations, Representative Selections, edited by Bruce Mazlish, Bobbs Merrill, Indianapolis, Indiana (1961) https://doi.org/10.1093/oseo/instance.00043218 DOI: https://doi.org/10.1093/oseo/instance.00043218
Stame, M. (2010) What Doesn't Work? Three Failures, Many Answers, Evaluation, Sage Publications, 16(4) pp. 371-387 https://doi.org/10.1177/1356389010381914 DOI: https://doi.org/10.1177/1356389010381914
Taleb, N. N. (2007) The Black Swan: The Impact of the Highly Improbable. Random House, Penguin Books, London. UK
Vedung, E. (2010) Four Waves of Evaluation Diffusion, Evaluation, Sage Publications, 16: 263 pp. 263-277 https://doi.org/10.1177/1356389010372452 DOI: https://doi.org/10.1177/1356389010372452
Williams, B. & Imam I., Eds. (2007). Systems concepts in evaluation: An expert anthology. Edge Press of Inverness. Point Reyes CA
Woermann M. (2013) The Ethics of Complexity and the Complexity of Ethics. In: On the (Im)Possibility of Business Ethics. Issues in Business Ethics, Springer, Dordrecht, vol 37. p. 31 https://doi.org/10.1007/978-94-007-5131-6_2 DOI: https://doi.org/10.1007/978-94-007-5131-6
Wolf-Branigin, M. (2013). Using Complexity Theory for Research and Program Evaluation. Oxford University Press, Oxford, UK https://doi.org/10.1093/acprof:oso/9780199829460.001.0001 DOI: https://doi.org/10.1093/acprof:oso/9780199829460.001.0001
Wood, J.C. and Wood, M.C. (2005). Peter F. Drucker, Critical Evaluations in Business and Management. Vol.1. Routledge, London UK and New York, NY