Learning How to Learn in Sustainability Transitions Projects: The Potential Contribution of Developmental Evaluation
Main Article Content
Abstract
Background: Community-based sustainability transitions projects are increasingly being considered for their potential as policy delivery vehicles for the UK government Climate Change Act commitments. At the same time, project funders seek reassurances that their investments are relevant in helping communities mitigate, and adapt to, the effects of climate change. Despite this increased pressure, recent research suggests that evaluations of such sustainability transitions projects have, on the one hand variable impacts, or impacts that are of an inadequate scale, duration, or type, or on the other, that project staff lack the capacity or resources to undertake monitoring and evaluation to the degree of rigour expected by policy makers and funders.
Purpose: This article reports on an extended case study of a fully-funded five year community-based sustainability transitions project in Leicestershire, England. In particular, it reviews the deployment of developmental evaluation (DE) methods in an attempt to capture the project team’s learning about doing community-based sustainability work.
Setting: A funded community-based sustainability transitions project in a south Leicestershire market town.
Intervention: Developmental evaluation methods were used to capture project-based learning as a resource for project innovation and adaptation.
Research Design: Phronetic case study.
Data Collection and Analysis: Participant-observation, action research, focus and special issue group facilitation.
Findings: Use of a developmental evaluation method identified key learning points for the project actors; these focused on how the project had adapted to the complexities of the operating environment through innovations in second-order learning or learning how to learn. The paper makes recommendations for the design and funding arrangements of community-based sustainability transitions initiatives and developmental evaluation is endorsed as a viable and promising adjunct to more traditional impact, economic, and process evaluation methodologies.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Abrahamse, W., Steg, L., Vlek, C. & Rothengatter, T. (2005). A review of intervention studies aimed at household energy conservation. Journal of Environmental Psychology, 25(3), 273-291. https://doi.org/10.1016/j.jenvp.2005.08.002.
Aiken, G. T. (2015). (Local-) community for global challenges: carbon conversations, transition towns and governmental elisions. Local Environment, 20(17), 764-781. https://doi.org/10.1080/13549839.2013.870142.
Bamberg, S., & Möser, G. (2007) Twenty years after Hines, Hungerford, and Tomera: A new meta-analysis of psycho-social determinants of pro-environmental behavior. Journal of Environmental Psychology, 27(1), 14-25. https://doi.org/10.1016/j.jenvp.2006.12.002.
Beers, P. J., Hermans, F., Veldkamp, T., & Hinssen, J. (2014), Social learning inside and outside transition projects: Playing free jazz for a heavy metal audience. NJAS - Wageningen Journal of Life Sciences, 69, 5-13. https://doi.org/10.1016/j.njas.2013.10.001.
Benford, R. D., & Snow, D. (2000). Framing processes and social movements: An overview and assessment. Annual Review Sociologic, 26, 611-639. https://doi.org/10.1146/annurev.soc.26.1.611.
Blamey, A., & Mackenzie, M. (2007) Theories of change and realistic evaluation: Peas in a pod or apples and oranges? Evaluation, 13(4), 439–-55. https://doi.org/10.1177/1356389007082129.
Bolderdijk, J. W., Steg, L., Geller, E. S., Lehman, P. K., & Postmes, T. (2013). Comparing the effectiveness of monetary versus moral motives in environmental campaigning. Nature Climate Change, 3(4), 413-416. https://doi.org/10.1038/nclimate1767.
Bradley, Q. (2014). Bringing democracy back home: Community localism and the domestication of political space. Environment & Planning D: Society & Space, 32(4), 642-657. https://doi.org/10.1068/d17312.
Burns, D. (2010). Systematic action research: A strategy for whole system change. Bristol, UK: The Policy Press.
Burns, D., & Worsley, S. (2015). Navigating complexity in international development: Facilitating sustainable change at scale. Rugby: Practical Action Publishing Ltd. https://doi.org/10.3362/9781780448510.000
Checkland, P., & Holwell, S. (2007). Action research: Its nature and validity. Information Systems Action Research: An Applied View of Emerging Concepts and Methods, 11(1), 3-17. https://doi.org/10.1023/A:1022908820784.
Connell, J. P., & Kubisch, A. C. (1998). Applying a theory of change approach to the evaluation of comprehensive community initiatives: Progress, prospects, and problems. In K. Fulbright-Anderson, A. C. Kubisch, & J. P. Connell, J. P. (Eds.), New approaches to evaluating community initiatives. Volume 2: Theory, measurement, and analysis (pp. 15-44). Washington, DC: Aspen Institute.
Cress, D. M., & Snow, D. (2000). The outcomes of homeless mobilization: The influence of organization, disruption, political mediation and framing. American Journal of Sociology, 105(4), 1063-1104. https://doi.org/10.1086/210399.
Dahl, A. L. (2012). Achievements and gaps in indicators for sustainability. Ecological Indicators, 17, 14-19. https://doi.org/10.1016/j.ecolind.2011.04.032.
DECC. (2014) Community energy strategy: full report.
Dunkley, R. A., & Franklin, A. (2017). Failing better: The stochastic art of evaluating community-led environmental action programs. Evaluation and Program Planning, 60, 112-122. https://doi.org/10.1016/j.evalprogplan.2016.11.005.
Gooding, K. (2016). What do we mean by evidence-based advocacy? Ideas from NGOs in Malawi'. In R. Hayman, S. King, T. Kontinen, & L. Narayanaswamy (Eds.), Negotiating knowledge: Evidence and experience in development NGOs (pp. 15-28). Rugby: Practical Action Publishing Ltd. https://doi.org/10.3362/9781780449258.002
Gupta, R., Eyre, N., Darby, S., Lucas, K., Barnfield, L., Hamilton, J., Mayne, R., Gregg, M., Fratter, C., & Irving, B. (2015). Evaluating the impacts, effectiveness and success of low carbon communities on localised energy behaviours (EVALOC) Final Report. Oxford Brookes University and University of Oxford, Oxford. Available at: http://media.wix.com/ugd/caf2de_c6393816adeb44e8862c11f1dabd276b.pdf.
H. M. Government. (2011). Localism Act 2011 (c.20). UK.
Hamilton, J. (2013). Community monitoring and evaluation methods survey: Report and analysis. Evaluating low carbon communities: Working papers and reports. Available at: http://media.wix.com/ugd/caf2de_fd61a89150c88692722b9b0ea329ec98.pdf (Accessed: 12 December 2014).
Hargreaves, T., Hielscher, S., Seyfang, G., & Smith, A. (2013). Grassroots innovations in community energy: The role of intermediaries in niche development. Global Environmental Change, 23(5), 868-880. https://doi.org/10.1016/j.gloenvcha.2013.02.008.
Hobson, K., Hamilton, J., & Mayne, R. (2014). Monitoring and evaluation in UK low-carbon community groups : benefits, barriers and the politics of the local. Local Environment : The International Journal of Justice and Sustainability, 21(1), 37-41. https://doi.org/10.1080/13549839.2014.928814.
Hobson, K., Mayne, R., & Hamilton, J. (2016). Monitoring and evaluating eco-localisation: Lessons from UK low carbon community groups. Environment and Planning A, 48(7): 1393-1410. https://doi.org/10.1177/0308518X16640531.
Hoffman, S. M., & High-Pippert, A. (2010). From private lives to collective action: Recruitment and participation incentives for a community energy program. Energy Policy, 38(12), 7567-7574. https://doi.org/10.1016/j.enpol.2009.06.054.
Huang, R. (2014). RQDA: R-based qualitative data analysis. Available at http://rqda.r-forge.r-project.org/.
Ison, R., Blackmore, C., Collins, K., & Furniss, P. (2007). Systemic environmental decision making: designing learning systems. Kybernetes, 36(9/10), 1340-1361. https://doi.org/10.1108/03684920710827346.
Kern, F., & Smith, A. (2008). Restructuring energy systems for sustainability? Energy transition policy in the Netherlands. Energy Policy, 36(11), 4093-4103. https://doi.org/10.1016/j.enpol.2008.06.018.
Letcher, R., Roberts, S., & Redgrove, Z. (2007). Mobilising individual behavioural change through community initiatives: Lessons for tackling climate change, the energy review study. Available at https://www.cse.org.uk/downloads/reports-and-publications/community-energy/behaviour-change/mobilising-individual-behavioural-change-through-community-summary.pdf.
McDonald, H. (2016). Developmental evaluation : A tool to support innovation. Evaluation Matters—He Take Tō Te Aromatawai, 2, 79-97. https://doi.org/10.18296/em.0012.
Milligan, S., Coulton, C., York, P., & Register, R. (1998). Implementing a theory of change evaluation in the Cleveland community-building initiative: A case study. In K. Fulbright-Anderson, A. C. Kubisch, & J. P. Connell (Eds.), New approaches to evaluating community initiatives. Volume 2: Theory, measurement, and analysis (pp. 45-85). Washington, DC: Aspen Institute.
Mitchell, A. S. (2019). Second-order learning in developmental evaluation: New methods for complex conditions. Cham, Switzerland: Palgrave Macmillan. https://doi.org/10.1007/978-3-319-99371-3
Mitchell, A. S., Lemon, M., & Fletcher, G. (2019). What's being tested and what's being learnt? A contribution to lessons learned evaluation methods for community-based sustainability initiatives. Central European Review of Economics and Management, 3(3), 129-168. https://doi.org/10.29015/cerem.596
Mowles, C. (2014). Complex, but not quite complex enough: The turn to the complexity sciences in evaluation scholarship. Evaluation, 20(2), 60-75. https://doi.org/10.1177/1356389014527885.
Patton, M. Q. (2001). Evaluation, knowledge management, best practices, and high quality lessons learned. American Journal of Evaluation, 22, 329-336. https://doi.org/10.1177/109821400102200307.
Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York, NY: Guildford.
Petersen, J. E., Shunturov, V., Janda, K., Platt, G., & Weinberger, K. (2007). Dormitory residents reduce electricity consumption when exposed to real-time visual feedback and incentives. International Journal of Sustainability in Higher Education, 8(1), 16-33. https://doi.org/10.1108/14676370710717562.
Seyfang, G. (2010). Community action for sustainable housing: Building a low-carbon future. Energy Policy, 38, 7624-7633. https://doi.org/10.1016/j.enpol.2009.10.027.
Seyfang, G., Hielscher, S., Hargreaves, T., Martiskainen, M., & Smith, A. (2014). Environmental innovation and societal transitions: A grassroots sustainable energy niche? Reflections on community energy in the UK. Environmental Innovation and Societal Transitions. https://doi.org/10.1016/j.eist.2014.04.004.
Steg, L., & Vlek, C. (2009). Encouraging pro-environmental behaviour: An integrative review and research agenda. Journal of Environmental Psychology, 29(3), 309-317. https://doi.org/10.1016/j.jenvp.2008.10.004.
Stephenson, J., Barton, B., Carrington, G., Gnoth, D., Lawson, R., & Thorsnes, P. (2010) Energy cultures: A framework for understanding energy behaviours. Energy Policy, 38, 6120-6129. https://doi.org/10.1016/j.enpol.2010.05.069.
TESS. (2016). Success factors of community-based sustainability initiatives—Results from TESS. Available at: http://www.tess-transition.eu/wp-content/uploads/2016/10/Tess_draft_Policy-Brief_success_1-1.pdf.
Turcu, C. (2013). Re-thinking sustainability indicators: Local perspectives of urban sustainability. Journal of Environmental Planning and Management, 56(5), 695-719. https://doi.org/10.1080/09640568.2012.698984.
UK Parliament. (2008). Climate Change Act. https://doi.org/10.1136/bmj.39469.569815.47.
Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. Connell, A. Kubisch, L. Schorr, & C. Weiss (Eds.), New Approaches to Evaluating Community Initiatives (pp. 65-92). Washington, DC: Aspen Institute.
Weiss, C. H. (1997). How can theory-based evaluation make greater headway? Evaluation Review, 21(4), 501-524. https://doi.org/10.1177/0193841X9702100405.