Concepts and Contexts of Creative Evaluation Approaches
Main Article Content
Abstract
Background: Creative Evaluation (CE) is an unformed constellation of evaluation approaches that is based on varied understandings of creativity. Although creativity in evaluation has been consistently valued and needed by evaluation practitioners and researchers, the use of the term CE is currently limited in its applications as it has been developed inconsistently and often in isolation to the broader evaluation practice. The authors review examples where the term CE is being used, to present this constellation of approaches and group them under concepts and contexts, aiming at fostering an exploration into CE.
Purpose: This paper examines the use of term Creative Evaluation by reviewing a set of examples from varied settings like educational programmes, cultural industries and social work and to present the reader with a better understanding about the contexts and concepts associated with CE. The authors discuss the transdiciplinary nature of evaluation along with a review of historical developments in evaluation theory and practice that help identify the emergence of responsive, flexible and problem-solving driven approaches to evaluation that can showcase a growing connection between creativity and evaluation.
Setting: Not applicable.
Intervention: Not applicable.
Research Design: Desk review was utilised for the preparation of this paper.
Findings: There are different conceptualisations of the term CE applied in a variety of disciplinary and practice contexts. The term CE is used to describe both the process of applying creative thinking and the process of employing creative methods in the evaluation process. Creative thinking indicates a broader application of creativity that shapes the evaluation process and often results in the development of novel methods while creative methods indicate a more targeted use of methods that employ creative practices to achieve a specific evaluation goal like increasing engagement in the evaluation process.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Aish, R., & Hanna, S. (2017). Comparative evaluation of parametric design systems for teaching design computation. Design Studies, 52, 144-172. https://doi.org/10.1016/j.destud.2017.05.002 DOI: https://doi.org/10.1016/j.destud.2017.05.002
Albaek, E. (1998). Knowledge, interests and the many meanings of evaluation: a developmental perspective. Scandinavian Journal of Social Welfare, 7(2), 94-98. https://doi.org/10.1111/j.1468-2397.1998.tb00208.x DOI: https://doi.org/10.1111/j.1468-2397.1998.tb00208.x
Alkin, M. C., & King, J. A. (2016). The Historical Development of Evaluation Use. American Journal of Evaluation, 37(4), 568-579. https://doi.org/10.1177/1098214016665164 DOI: https://doi.org/10.1177/1098214016665164
Alvik, T. (1995). School-based evaluation: A close-up. Studies in educational evaluation, 21(3), 311-343. https://doi.org/10.1016/0191-491X(95)00018-P DOI: https://doi.org/10.1016/0191-491X(95)00018-P
Barbot, B., Besançon, M., & I Lubart, T. (2011). Assessing creativity in the classroom. The Open Education Journal, 4(1). https://doi.org/10.2174/1874920801104010058 DOI: https://doi.org/10.2174/1874920801104010058
Basadur, Min, Graen, George B, & Green, Stephen G. (1982). Training in creative problem solving: Effects on ideation and problem finding and solving in an industrial research organization. Organizational Behavior and Human Performance, 30(1), 41-70. https://doi.org/10.1016/0030-5073(82)90233-1 DOI: https://doi.org/10.1016/0030-5073(82)90233-1
Baur, V. E., Abma, T. A., & Widdershoven, G. A. (2010). Participation of marginalized groups in evaluation: Mission impossible?. Evaluation and program planning, 33(3), 238-245. https://doi.org/10.1016/j.evalprogplan.2009.09.002 DOI: https://doi.org/10.1016/j.evalprogplan.2009.09.002
Boleman, C. T., Rollins, D., & Pierce Jr, L. W. (2009). Creative Approach to Evaluating: The Tri-Fold Display Example. Journal of extension. 47(3).
Buckland, H., & Murillo, D. (Eds.), 2013. Antenna for Social Innovation: Pathways to Systemic Change: Inspiring Stories and a New Set of Variables for Understanding Social Innovation. Sheffield: Greenleaf. https://doi.org/10.9774/GLEAF.978-1-78353-054-0_3 DOI: https://doi.org/10.9774/GLEAF.978-1-78353-054-0_3
Cajaiba-Santana, G. (2014). "Social Innovation: Moving the Field Forward. A Conceptual Framework." Technological Forecasting & Social Change 82: 42-51. https://doi.org/10.1016/j.techfore.2013.05.008 DOI: https://doi.org/10.1016/j.techfore.2013.05.008
Campbell, B., & Mark, M. M. (2006). Toward More Effective Stakeholder Dialogue: Applying Theories of Negotiation to Policy and Program Evaluation 1. Journal of Applied Social Psychology, 36(12), 2834-2863. https://doi.org/10.1111/j.0021-9029.2006.00131.x DOI: https://doi.org/10.1111/j.0021-9029.2006.00131.x
Candy L. (2013) Evaluating Creativity. In J. Carroll (Ed.), Creativity and Rationale. Human-Computer Interaction Series, vol 20. Springer, London. https://doi.org/10.1007/978-1-4471-4111-2_4 https://doi.org/10.1007/978-1-4471-4111-2_4 DOI: https://doi.org/10.1007/978-1-4471-4111-2_4
Chelimsky, E. (1997). The Coming Transformations in Evaluation. In E. Chelimsky & W. R. Shadish (Eds.), Evaluation for the 21st Century: A Handbook. SAGE Publications, Inc. https://doi.org/10.4135/9781483348896 DOI: https://doi.org/10.4135/9781483348896
Chelimsky, E. (2006) The Purposes of Evaluation in a Democratic Society. In I. F. Shaw, J. C. Greene & M. M. Mark (Eds.), The Sage Handbook of Evaluation. Sage Publications, Inc. https://doi.org/10.4135/9781848608078.n1 DOI: https://doi.org/10.4135/9781848608078.n1
Chouinard, J. A., & Milley, P. (2018). Uncovering the mysteries of inclusion: Empirical and methodological possibilities in participatory evaluation in an international context. Evaluation and Program Planning, 67, 70-78. https://doi.org/10.1016/j.evalprogplan.2017.12.001 DOI: https://doi.org/10.1016/j.evalprogplan.2017.12.001
Chouinard, J. A., & Cram, F. (2020). Culturally Responsive Approaches to Evaluation: Empirical Implications for Theory and Practice. SAGE Publications. https://doi.org/10.4135/9781506368559 DOI: https://doi.org/10.4135/9781506368559
Christensen, L., Nielsen, J., Rogers, C., & Volkov, B. (2005). 'Creative data collection in nonformal settings', New Directions for Evaluation, non. 108, pp. 73-79. https://doi.org/10.1002/ev.172 DOI: https://doi.org/10.1002/ev.172
Christensen, B. T., & Ball, L. J. (2016). Dimensions of creative evaluation: Distinct design and reasoning strategies for aesthetic, functional and originality judgments. Design Studies, 45, 116-136. https://doi.org/10.1016/j.destud.2015.12.005 DOI: https://doi.org/10.1016/j.destud.2015.12.005
Coryn, C. L. S., & Hattie, J. A. (2007). 'The transdisciplinary model of evaluation' Journal of MultiDisciplinary Evaluation, 3(4), 107-114. https://doi.org/10.56645/jmde.v3i4.81 DOI: https://doi.org/10.56645/jmde.v3i4.81
Coughlan, P., & Mashman, R. (1999). Once is not enough: repeated exposure to and aesthetic evaluation of an automobile design prototype. Design Studies, 20(6), 553-563. https://doi.org/10.1016/S0142-694X(99)00007-1 DOI: https://doi.org/10.1016/S0142-694X(99)00007-1
Cousins, J. B., & Earl, L. M. (1992). The case for participatory evaluation. Educational evaluation and policy analysis, 14(4), 397-418. https://doi.org/10.3102/01623737014004397 DOI: https://doi.org/10.3102/01623737014004397
Cousins, J. B., & Whitmore, E. (1998). Framing participatory evaluation. New directions for evaluation, 1998(80), 5-23. https://doi.org/10.1002/ev.1114 DOI: https://doi.org/10.1002/ev.1114
Cousins, J. B., Whitmore, E., & Shulha, L. (2013). Arguments for a common set of principles for collaborative inquiry in evaluation. American Journal of Evaluation, 34(1), 7-22. https://doi.org/10.1177/1098214012464037 DOI: https://doi.org/10.1177/1098214012464037
Cousins, J.B., Shulha, L., Whitmore, E., Hudib, H.A., and Gilbert, N. (2020). Situating Evidence-Based Principles to Guide Practice in Collaborative Approaches to Evaluation (CAE). In J.B. Cousins (Ed.), Collaborative approaches to evaluation: principles in use. Thousand Oaks: SAGE Publications. https://doi.org/10.4135/9781544344669.n1 DOI: https://doi.org/10.4135/9781544344669.n1
Dahler-Larsen, P. (2006). Evaluation After Disenchantment? Five Issues Shaping the Role of Evaluation in Society. In I. Shaw, J. C. Greene, & M. M. Mark (Eds.), The Sage Handbook of Evaluation. SAGE Publications. https://doi.org/10.4135/9781848608078.n6 DOI: https://doi.org/10.4135/9781848608078.n6
Dahler-Larsen, P. (2009). Learning-Oriented Educational Evaluation in Contemporary Society. In K.E. Ryan, & J.B. Cousins (Eds.), The SAGE International Handbook of Educational Evaluation. (pp. 307-322). Thousand Oaks: SAGE Publications. https://doi.org/10.4135/9781452226606.n17 DOI: https://doi.org/10.4135/9781452226606.n17
Dahler-Larsen, P. (2012). The Evaluation Society. Stanford. https://doi.org/10.1515/9780804778121 DOI: https://doi.org/10.1515/9780804778121
Davis, S. B. (2009). Introduction to the special issue on creative evaluation. Digital Creative Journal, 20(3), 133-139. Routledge. https://doi.org/10.1080/14626260903083561 DOI: https://doi.org/10.1080/14626260903083561
Daykin, N., Gray, K., McCree, M., & Willis, J. (2017). Creative and credible evaluation for arts, health and well-being: opportunities and challenges of co-production. Arts & Health, 9(2), 123-138. https://doi.org/10.1080/17533015.2016.1206948 DOI: https://doi.org/10.1080/17533015.2016.1206948
Dong, A., Garbuio, M., & Lovallo, D. (2016). Generative sensing in design evaluation. Design Studies, 45, 68-91. https://doi.org/10.1016/j.destud.2016.01.003 DOI: https://doi.org/10.1016/j.destud.2016.01.003
Edmonds, E., Bilda, Z., & Muller, L. (2009). Artist, evaluator and curator: three viewpoints on interactive art, evaluation and audience experience. Digital Creativity, 20(3), 141-151. https://doi.org/10.1080/14626260903083579 DOI: https://doi.org/10.1080/14626260903083579
Ellamil, M., Dobson, C., Beeman, M., & Christoff, K. (2012). Evaluative and generative modes of thought during the creative process. Neuroimage, 59(2), 1783-1794. https://doi.org/10.1016/j.neuroimage.2011.08.008 DOI: https://doi.org/10.1016/j.neuroimage.2011.08.008
Evans, M., Wallace, D., Cheshire, D., & Sener, B. (2005). An evaluation of haptic feedback modelling during industrial design practice. Design Studies, 26(5), 487-508. https://doi.org/10.1016/j.destud.2004.10.002 DOI: https://doi.org/10.1016/j.destud.2004.10.002
Fetterman, D.M. (2015). Empowerment Evaluation. In Elsevier Inc. International Encyclopedia of the Social & Behavioral Sciences: Second Edition. [Online]. Elsevier Inc. https://doi.org/10.1016/B978-0-08-097086-8.10572-0 DOI: https://doi.org/10.1016/B978-0-08-097086-8.10572-0
Fetterman, D. M. (2019). Empowerment evaluation: A stakeholder involvement approach. Health Promotion Journal of Australia, 30(2), 137-142. John Wiley and Sons Inc. https://doi:10.1002/hpja.243 DOI: https://doi.org/10.1002/hpja.243
Finke, R.A., Ward, T.B., Smith, S.M., (1992). Creative Cognition: Theory, Research and Applications. MIT Press. https://doi.org/10.7551/mitpress/7722.001.0001 DOI: https://doi.org/10.7551/mitpress/7722.001.0001
Fitzpatrick, J. L. (2012). An introduction to context and its role in evaluation practice. New Directions for Evaluation, 2012(135), 7-24. https://doi.org/10.1002/ev.20024 DOI: https://doi.org/10.1002/ev.20024
Frierson, H. T., Hood, S., Hughes, G. B., & Thomas, V. (2010). A guide to conducting culturally-responsive evaluations. In J. Frechtling (Ed.), The 2010 user-friendly hand-book for project evaluation (pp. 75-96). Washington, DC: National Science Foundation.
Gerrits, L., & Verweij, S. (2015). Taking stock of complexity in evaluation: A discussion of three recent publications. Evaluation, 21(4), 481-491. https://doi.org/10.1177/1356389015605204 DOI: https://doi.org/10.1177/1356389015605204
Guba, E. G., & Lincoln, Y. S. (1989). Fourth Generation Evaluation. Sage Publications.
Haldane, V., Chuah, F. L., Srivastava, A., Singh, S. R., Koh, G. C., Seng, C. K., & Legido-Quigley, H. (2019). Community participation in health services development, implementation, and evaluation: A systematic review of empowerment, health, community, and process outcomes. PloS one, 14(5), e0216112. https://doi.org/10.1371/journal.pone.0216112 DOI: https://doi.org/10.1371/journal.pone.0216112
Harper, L. M. & Dickson, R. (2019). 'Using developmental evaluation principles to build capacity for knowledge mobilisation in health and social care', Evaluation, 25(3), pp. 330-348. https://doi.org/10.1177/1356389019840058 DOI: https://doi.org/10.1177/1356389019840058
Harvey, S., & Kou, C. Y. (2013). Collective engagement in creative tasks: The role of evaluation in the creative process in groups. Administrative science quarterly, 58(3), 346-386 https://doi.org/10.1177/0001839213498591 DOI: https://doi.org/10.1177/0001839213498591
Hood, S., Hopson, R. and Kirkhart, K. E. (2015). Culturally Responsive Evaluation. In K. E. Newcomer, H. P. Hatry, & Wholey J. S. (Eds.), Handbook of practical program evaluation (4th ed., p. 281. Jossey-Bass. https://doi.org/10.1002/9781119171386.ch12 DOI: https://doi.org/10.1002/9781119171386.ch12
House, E. R., & Howe, K. R. (1998). The Issue of Advocacy in Evaluations. American Journal of Evaluation, 19(2), 233-236. https://doi.org/10.1177/109821409801900209 DOI: https://doi.org/10.1177/109821409801900209
House, E. R., & Howe, K. R. (2000). Deliberative democratic evaluation. New Directions for Evaluation, 2000(85), 3-12. https://doi.org/10.1002/ev.1157 DOI: https://doi.org/10.1002/ev.1157
Howe, K.R. & Ashcraft, C. (2005). Deliberative Democratic Evaluation: Successes and Limitations of an Evaluation of School Choice. Teachers College Record, 107(10), 2274-2297. https://doi.org/10.1111/j.1467-9620.2005.00592.x DOI: https://doi.org/10.1177/016146810510701004
Hsiao, S. W., Hsu, C. F., & Lee, Y. T. (2012). An online affordance evaluation model for product design. Design Studies, 33(2), 126-159. https://doi.org/10.1016/j.destud.2011.06.003 DOI: https://doi.org/10.1016/j.destud.2011.06.003
Isaksson, O., Keski-Seppälä, S., & Eppinger, S. D. (2000). Evaluation of design process alternatives using signal flow graphs. Journal of Engineering Design, 11(3), 211-224. https://doi.org/10.1080/09544820050020996 DOI: https://doi.org/10.1080/095448200750020996
Jandhyala, K. (2012). Ruminations on evaluation in the Mahila Samakhya programme. Indian Journal of Gender Studies, 19(2), 211-231. https://doi.org/10.1177/097152151201900203 DOI: https://doi.org/10.1177/097152151201900203
Jordanous, A. (2012). A standardised procedure for evaluating creative systems: Computational creativity evaluation based on what it is to be creative. Cognitive Computation, 4(3), 246-279. https://doi-org.ezproxy.lancs.ac.uk/10.1007/s12559-012-9156-1 DOI: https://doi.org/10.1007/s12559-012-9156-1
Kania, J. & Kramer, M. (2013). Embracing Emergence: How Collective Impact Addresses Complexity. Retrieved November 26, 2020, from www.ssireview.org
Kim, S. J., Kara, S., & Kayis, B. (2014). Economic and environmental assessment of product life cycle design: volume and technology perspective. Journal of cleaner production, 75, 75-85. https://doi.org/10.1016/j.jclepro.2014.03.094 DOI: https://doi.org/10.1016/j.jclepro.2014.03.094
Lawrence, R. B., Rallis, S. F., Davis, L. C., & Harrington, K. (2018). Developmental evaluation: Bridging the gaps between proposal, program, and practice. Evaluation, 24(1), 69-83. https://doi.org/10.1177/1356389017749276 DOI: https://doi.org/10.1177/1356389017749276
Lincoln, Y. S., & Guba, E. (1981). Do Evaluators Wear Grass Skirts? "Going Native" and Ethnocentrism as Problems in Utilization. Austin.
Lincoln, Y. S., & Guba, E. G. (2004). The Roots of Fourth Generation Evaluation: Theoretical and Methodological Origins. In M. C. Alkin (Ed.), Evaluation Roots (pp. 226-241). SAGE Publications, Inc. https://doi.org/10.4135/9781412984157 DOI: https://doi.org/10.4135/9781412984157.n15
Lindberg, M. & Portinson Hylander, J. (2017). 'Boundary dimensions of social innovation: negotiating conflicts and compatibilities when developing a national agenda', Innovation: The European Journal of Social Sciences, 30(2), pp. 168-181. https://doi.org/10.1080/13511610.2016.1204534 DOI: https://doi.org/10.1080/13511610.2016.1204534
Manohar, A. K., Smith, M. & Calvo, M., (2016). Capturing the "How": Showing the value of co-design through creative evaluation. Proceedings of the 50th Anniversary DRS Conference 2016, Future-Focused Thinking, 27-30 Jun 2016, Brighton, UK. DOI: https://doi.org/10.21606/drs.2016.469
Manzini, E. (2015). Design, When Everybody Designs: An Introduction to Design for Social Innovation. The MIT Press. https://doi.org/10.7551/mitpress/9873.001.0001 DOI: https://doi.org/10.7551/mitpress/9873.001.0001
Marentakis, G., Pirrò, D., & Weger, M. (2017, June). Creative evaluation. In Proceedings of the 2017 Conference on Designing Interactive Systems (pp. 853-864). https://doi.org/10.1145/3064663.3064710 DOI: https://doi.org/10.1145/3064663.3064710
Mertens, D. M. (2008). Transformative Research and Evaluation. Guilford Publications. http://ebookcentral.proquest.com/lib/lancaster/detail.action?docID=371196
Mertens, D. M., & Hesse-Biber, S. (2013). Mixed Methods and Credibility of Evidence in Evaluation. John Wiley & Sons, Incorporated. http://ebookcentral.proquest.com/lib/lancaster/detail.action?docID=1216101 https://doi.org/10.1002/ev.20053 DOI: https://doi.org/10.1002/ev.20053
Mokros, J. R. (1983). Creative Evaluation: by Michael Quinn Patton. Evaluation and Program Planning. Volume 6, Issue 1, Pages 80-81. https://doi.org/10.1016/0149-7189(83)90049-6 DOI: https://doi.org/10.1016/0149-7189(83)90049-6
Newcomer, K. E., Hatry, H. P., & Wholey, J. S. (2015). Handbook of practical program evaluation. Jossey-Bass. https://doi.org/10.1002/9781119171386 DOI: https://doi.org/10.1002/9781119171386
Nikander, J. B., Liikkanen, L. A., & Laakso, M. (2014). The preference effect in design concept evaluation. Design Studies, 35(5), 473-499 https://doi.org/10.1016/j.destud.2014.02.006 DOI: https://doi.org/10.1016/j.destud.2014.02.006
O'Sullivan, R. G. (2004) Practicing Evaluation: A Collaborative Approach. Sage Publications.
O'Sullivan, R. G. (2012). Collaborative Evaluation within a framework of stakeholder-oriented evaluation approaches. Evaluation and Program Planning, 35(4), 518-522. https://doi.org/10.1016/j.evalprogplan.2011.12.005 DOI: https://doi.org/10.1016/j.evalprogplan.2011.12.005
Owen, C. L. (2007). Evaluation of complex systems. Design Studies, 28(1), 73-101. https://doi.org/10.1016/j.destud.2006.06.001 DOI: https://doi.org/10.1016/j.destud.2006.06.001
Patrizi, P., Heid Thompson, E., Coffman, J., & Beer, T. (2013). Eyes wide open: Learning as strategy under conditions of complexity and uncertainty. The Foundation Review, 5(3), 7. 50-65. https://doi.org/10.9707/1944-5660.1170 DOI: https://doi.org/10.9707/1944-5660.1170
Patton, M.Q. (1981). Creative Evaluation. London: SAGE Publications.
Patton, M. Q. (1994). Developmental evaluation. Evaluation Practice,15(3), 311-319. https://doi.org/10.1177/109821409401500312 DOI: https://doi.org/10.1177/109821409401500312
Patton, M. Q. (2002). 'A Vision of Evaluation that Strengthens Democracy', Evaluation, 8(1), pp. 125-139. https://doi.org/10.1177/1358902002008001740 DOI: https://doi.org/10.1177/1358902002008001740
Patton, M. Q. (2005). The challenges of making evaluation useful. Ensaio (Rio De Janeiro, Brazil),13(46), 67-78. https://doi.org/10.1590/S0104-40362005000100005 DOI: https://doi.org/10.1590/S0104-40362005000100005
Patton, M. Q. (2011a). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: London: Guilford Press.
Patton, M. Q. (2011b). Essentials of utilization-focused evaluation. Sage Publications. Thousand Oaks: Sage Publications.
Patton, M. Q. (2020, June 5) Rules for Privileged Straight White Males. Utilization-Focused Evaluation. https://www.utilization-focusedevaluation.org/blog/2020/6/5/rules-for-privileged-straight-white-males
Picciotto, R. (2020). Towards a Complexity Framework for Transformative Evaluation. Journal of MultiDisciplinary Evaluation, 16(35), 54-76. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/643 https://doi.org/10.56645/jmde.v16i35.643 DOI: https://doi.org/10.56645/jmde.v16i35.643
Podems, D. R. (2010). Feminist Evaluation and Gender Approaches: There's a Difference? Journal of MultiDisciplinary Evaluation, 6(14), 1-17. https://doi.org/10.56645/jmde.v6i14.199 DOI: https://doi.org/10.56645/jmde.v6i14.199
Preskill, H. (2008). Evaluation's second act: A spotlight on learning. American Journal of Evaluation, 29(2), 127-138. https://doi.org/10.1177/1098214008316896 DOI: https://doi.org/10.1177/1098214008316896
Preskill, H., & Gopal, S., Mack, K., & Cook, J. (2014) Evaluating Complexity: Propositions for Improving Practice. Retrieved November 11, 2020 https://www.fsg.org/publications/evaluating-complexity
Rist, R., Fernández, Ana María, & Martin, Frederic P. (2016). Poverty, inequality and evaluation: Changing perspectives. Washington, DC: World Bank Group. https://doi.org/10.1596/978-1-4648-0703-9 DOI: https://doi.org/10.1596/978-1-4648-0703-9
Rodríguez-Campos, L., & Rincones-Gómez, R. (2012). Collaborative Evaluations : Step-By-Step, Second Edition (2nd ed.). Stanford University Press.
Rubin, A. (1989). Book Reviews: Creative Evaluation / Needs Assessment. 34(2), 172. Social Work, Volume 34 (2), p.172.
Sanderson, I. (2001). Performance Management, Evaluation and Learning in 'Modern' Local Government. Public Administration, 79(2), 297-313. https://doi.org/10.1111/1467-9299.00257 DOI: https://doi.org/10.1111/1467-9299.00257
Schwandt, T. A. (2019). Post-normal evaluation? Evaluation (London, England. 1995), 25(3), 317-329. https://doi.org/10.1177/1356389019855501 DOI: https://doi.org/10.1177/1356389019855501
Scriven, M. (1991). Evaluation thesaurus. Newbury Park, CA: Sage.
Scriven, M. (2008). The Concept of a Transdiscipline: And of Evaluation as a Transdiscipline. Journal of MultiDisciplinary Evaluation, 5(10), 65-66. https://journals.sfu.ca/jmde/index.php/jmde_1/article/view/161 https://doi.org/10.56645/jmde.v5i10.161 DOI: https://doi.org/10.56645/jmde.v5i10.161
Searle, M. J., Merchant, S., Chalas, A., & Lam, C. Y. (2017). A Case Study of the Guiding Principles for Collaborative Approaches to Evaluation in a Developmental Evaluation Context. Canadian Journal of Program Evaluation, 31(3). 350-373. https://doi.org/10.3138/cjpe.328 DOI: https://doi.org/10.3138/cjpe.328
Shadish, W. R., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: theories of practice. Sage.
Shakibamanesh, A., & Ghorbanian, M. (2017). Toward time-based design: Creating an applied time evaluation checklist for urban design research. Frontiers of Architectural Research, 6(3), 290-307. https://doi.org/10.1016/j.foar.2017.05.004 DOI: https://doi.org/10.1016/j.foar.2017.05.004
Shulha, L. M., & Cousins, J. B. (1997). Evaluation Use: Theory, Research and Practice Since 1986. Evaluation Practice, 18(3), 195-208. https://doi.org/10.1177/109821409701800302 DOI: https://doi.org/10.1016/S0886-1633(97)90027-1
Sielbeck-Bowen, K. A., Brisolara, S., Seigart, D., Tischler, C., & Whitmore, E. (2002). Exploring feminist evaluation: The ground from which we rise. New Directions for Evaluation, 2002(96), 3-8. https://doi.org/10.1002/ev.62 DOI: https://doi.org/10.1002/ev.62
Simons, H., & McCormack, B. (2007). Integrating arts-based inquiry in evaluation methodology: Opportunities and challenges. Qualitative Inquiry, 13(2), 292-311. https://doi.org/10.1177/1077800406295622 DOI: https://doi.org/10.1177/1077800406295622
Smith, N. L. (2010). Characterizing the Evaluand in Evaluating Theory. American Journal of Evaluation, 31(3), 383-389. https://doi.org/10.1177/1098214010371820 DOI: https://doi.org/10.1177/1098214010371820
Steenis, N. D., van Herpen, E., van der Lans, I. A., Ligthart, T. N., & van Trijp, H. C. (2017). Consumer response to packaging design: The role of packaging materials and graphics in sustainability perceptions and product evaluations. Journal of Cleaner Production, 162, 286-298. https://doi.org/10.1016/j.jclepro.2017.06.036 DOI: https://doi.org/10.1016/j.jclepro.2017.06.036
Stern, E., Stame, N., Mayne, J., Forss, K., Davies, R., & Befani, B. (2012). Broadening the range of designs and methods for impact evaluations. (Working Paper 38). London: DFID https://doi.org/10.22163/fteval.2012.100 DOI: https://doi.org/10.22163/fteval.2012.100
Stuart, K., Maynard, L., & Rouncefield, C. (2015). Evaluation practice for projects with young people. 55 City Road, London: SAGE Publications Ltd. https://doi.org/10.4135/9781473917811 DOI: https://doi.org/10.4135/9781473917811
Stufflebeam D.L. (2003) The CIPP Model for Evaluation. In T. Kellaghan, & D. L. Stufflebeam (Eds.), International Handbook of Educational Evaluation. Kluwer International Handbooks of Education, vol 9. Springer, Dordrecht. https://doi.org/10.1007/978-94-010-0309-4_4 DOI: https://doi.org/10.1007/978-94-010-0309-4_4
Suhariyanto, T. T., Wahab, D. A., & Rahman, M. N. A. (2018). Product design evaluation using life cycle assessment and design for Assembly: a case study of a water leakage alarm. Sustainability, 10(8), 2821. https://doi.org/10.3390/su10082821 DOI: https://doi.org/10.3390/su10082821
Tromp, N., & Hekkert, P. (2016). Assessing methods for effect-driven design: Evaluation of a social design method. Design Studies, 43, 24-47. https://doi.org/10.1016/j.destud.2015.12.002 DOI: https://doi.org/10.1016/j.destud.2015.12.002
Vedung, E. (2010). Four Waves of Evaluation Diffusion. Evaluation, 16(3), 263-277. https://doi.org/10.1177/1356389010372452 DOI: https://doi.org/10.1177/1356389010372452
Waapalaneexkweew (Nicole Bowman, Mohican/Lunaape), & Dodge-Francis, C. (2018). Culturally responsive Indigenous evaluation and tribal governments: Understanding the relationship. New Directions for Evaluation, 2018(159), 17-31. https://doi.org/10.1002/ev.20329 DOI: https://doi.org/10.1002/ev.20329
Walton, M. (2016). Expert views on applying complexity theory in evaluation: Opportunities and barriers. Evaluation, 22(4), 410-423. https://doi.org/10.1177/1356389016667890 DOI: https://doi.org/10.1177/1356389016667890
Ward Hood, D., & Cassaro, D. A. (2002). Feminist evaluation and the inclusion of difference. New Directions for Evaluation, 2002(96), 27-40. https://doi.org/10.1002/ev.64 DOI: https://doi.org/10.1002/ev.64
World Health Organization (1978, September 12). Declaration of Alma Ata. International conference on primary health care. Alma Ata, USSR.