Social-Centric Discovery Evaluation: A Theory-Based Approach to Evaluating Complex Public Health Programs
Article Sidebar
Main Article Content
Abstract
There is strong evidence that social conditions drive health outcomes and disparities. Consequently, there is scientific and policy interest in understanding the effectiveness of interventions designed to ameliorate such social conditions, with particular attention on place-based interventions and reducing inequalities. However, neither the science nor the policy interest is well served by conventional evaluation frameworks and, furthermore, few evaluations show that impacts have consistently been achieved. In short, current approaches are ill-suited to either proving what has worked or improving what might work. Transferable lessons are often limited. In this context, recent public health literature is helpful in relation to approaches to evaluation framework-building and theory-building. We describe an alternative approach, in contrast to contribution, attribution or developmental approaches, as a ‘social-centric discovery’’ approach. This explores the dynamic relationship between social context and an evolving intervention viewed as a sequence of events within a complex social system.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Cabrera, D., Colosi, L., & Lobdell, C. (2008). Systems thinking. Evaluation and Program Planning, 31(3), 299–310 DOI: https://doi.org/10.1016/j.evalprogplan.2007.12.001
Campbell, M., Fitzpatrick, R., Haines, A., Kinmonth, A. L., Sandercock, P., Spiegelhalter, D., & Tyrer, P. (2000). Framework for design and evaluation of complex interventions to improve health. BMJ, 321(7262), 694–696. https://doi.org/10.1136/bmj.321.7262.694 DOI: https://doi.org/10.1136/bmj.321.7262.694
CORE Group. (2021). Complexity-aware monitoring, evaluation & learning tools for social and behavior change interventions. https://coregroup.org/resource-library/complexity-aware-monitoring-evaluation-learning-for-social-and-behavior-change-interventions/
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008). Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ, 337. https://doi.org/10.1136/bmj.a1655 DOI: https://doi.org/10.1136/bmj.a1655
Dalkin, S. M., Greenhalgh, J., Jones, D., Cunningham, B., & Lhussier, M. (2015). What’s in a mechanism? Development of a key concept in realist evaluation. Implementation Science, 10, 1–7. https://doi.org/10.1186/s13012-015-0237-x DOI: https://doi.org/10.1186/s13012-015-0237-x
Datta, J., & Petticrew, M. (2013). Challenges to evaluating complex interventions: A content analysis of published papers. BMC Public Health, 13(568). https://doi.org/10.1186/1471-2458-13-568 DOI: https://doi.org/10.1186/1471-2458-13-568
DFID. (2013). Planning evaluability assessments: A synthesis of the literature with recommendations. https://www.gov.uk/research-for-development-outputs/planning-evaluability-assessments-a-synthesis-of-the-literature-with-recommendations-dfid-working-paper-40
Gates, E. F. (2016). Making sense of the emerging conversation in evaluation about systems thinking and complexity science. Evaluation and Program Planning, 59, 62–73. DOI: https://doi.org/10.1016/j.evalprogplan.2016.08.004
Gibbert, W. S., Keating, S. M., Jacobs, J. A., Dodson, E., Baker, E., Diem, G., Giles, W., Gillespie, K.N., Grabauskus, V., Shatchkute, A, & Brownson, R. C. (2013). Training the workforce in evidence-based public health: an evaluation of impact among US and international practitioners. Preventing chronic disease, 10, E148. DOI: https://doi.org/10.5888/pcd10.130120
Hawe, P., Shiell, A., & Riley, T. (2009). Theorising interventions as events in systems. American Journal of Community Psychology, 43, 267–276. https://doi.org/10.1007/s10464-009-9229-9 DOI: https://doi.org/10.1007/s10464-009-9229-9
Hughes, G., Shaw, S. E., & Greenhalgh, T. (2022). Why doesn't integrated care work? Using Strong Structuration Theory to explain the limitations of an English case. Sociology of Health and Illness, 44(1), 113–129. https://doi.org/10.1111/1467-9566.13398 DOI: https://doi.org/10.1111/1467-9566.13398
Hummelbrunner, R., & Reynolds, M. (2013). Systems thinking, learning and values in evaluation. Evaluation Connections (EES Newsletter), 9–10.
Mayne, J. (2017). Theory of change analysis: Building robust theories of change. Canadian Journal of Program Evaluation, 32(2), 155–173. DOI: https://doi.org/10.3138/cjpe.31122
Mickwitz, P., Neij, L., Johansson, M., Benner, M., & Sandin, S. (2021). A theory-based approach to evaluations intended to inform transitions toward sustainability. Evaluation, 27(3), 281–306. DOI: https://doi.org/10.1177/1356389021997855
Neubauer, B. E., Witkop, C. T., & Varpio, L. (2019). How phenomenology can help us learn from the experiences of others. Perspectives on medical education, 8, 90–97. https://doi.org/10.1007/s40037-019-0509-2 DOI: https://doi.org/10.1007/S40037-019-0509-2
Ornstein, J. T., Hammond, R. A., Padek, M., Mazzucca, S., & Brownson, R. C. (2020). Rugged landscapes: Complexity and implementation science. Implementation Science, 15, 1–9. https://doi.org/10.1186/s13012-020-01028-5 DOI: https://doi.org/10.1186/s13012-020-01028-5
Patton, M. Q. (2010). Developmental evaluation: Applying complexity concepts to enhance innovation and use. Guilford Press.
Prasad, V., & Ioannidis, J. P. (2014). Evidence-based de-implementation for contradicted, unproven, and aspiring healthcare practices. Implementation Science, 9, 1–5. https://doi.org/10.1186/1748-5908-9-1 DOI: https://doi.org/10.1186/1748-5908-9-1
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4(1), 1–32. https://doi.org/10.1146/annurev.clinpsy.3.022806.091415 DOI: https://doi.org/10.1146/annurev.clinpsy.3.022806.091415
Skivington, K., Matthews, L., Simpson, S. A., Craig, P., Baird, J., Blazeby, J. M., Boyd, K. A., Craig, N., French, D. P., McIntosh, E., Petticrew, M., Rycroft-Malone, J., White, M., & Moore, L. (2021). A new framework for developing and evaluating complex interventions: Update of Medical Research Council guidance. BMJ, 374. https://doi.org/10.1136/bmj.n2061 DOI: https://doi.org/10.1136/bmj.n2061
Stame, N. (2022). Program, complexity, and system when evaluating sustainable development. Evaluation, 28(1), 58–71. https://doi.org/10.1177/13563890211065488. DOI: https://doi.org/10.1177/13563890211065488
University of Cambridge. (2024). Creating cross-cultural inclusive deliberative spaces. Department of Land Economy and Centre for Resilience and Sustainable Development. https://www.landecon.cam.ac.uk/centre-for-resilience-and-sustainable-development/page/creating-cross-cultural-inclusive#:~:text=CRSD%2C%20in%20partnership%20with%20the,challenges%20of%20the%2021st%20Century.
WHO Commission on Social Determinants of Health. (2008). Closing the gap in a generation: Health equity through action on the social determinants of health [Final report]. Commission on Social Determinants of Health.