Dialogic and Generative Reflection: An Application of the Critical Appreciative Process in Program Evaluation
Main Article Content
Abstract
Background: Even though the positive potential of reflective practice is widely acknowledged across professional fields, it has been recognized that reflective practice may be carried out primarily as an individual-based exercise, and at the technical or descriptive level without generative impact. Dialogic reflective processes involving both evaluators and program directors are far from being systematically implemented or examined.
Purpose: The purpose of this article is to share our experiences engaging in dialogic and generative reflections as the project director and program evaluators of a K-12 teacher education program using the critical appreciative process. Building upon the reflective practice traditions in both disciplinary areas, we introduce the use of the critical appreciative process as a promising model to guide dialogic and generative reflection to support the co-design and improvement of the program and accompanying evaluation efforts.
Setting: The project director and evaluators are engaged in a grant-funded teacher preparation project designed to prepare teachers for K-12 English learners and dual language learners. The project builds upon partnerships between the university teacher preparation program and two local school districts. The evaluation plan was designed based on culturally responsive, collaborative, and use-focused evaluation approaches and theory. In 2020, the project team faced critical decisions in the context of the COVID-19 pandemic.
Intervention: Not applicable.
Research Design: We applied self-study methodology to guide data collection and analysis in this study. The primary data source included individual written reflections and group critical friend dialogues guided by the critical appreciative process. Both the reflections and meeting notes were analyzed to identify convergent and divergent perspectives shared throughout the critical appreciative process and to highlight implications for both the evaluation and the program moving forward.
Findings: Convergent and divergent perspectives from both the project director and the evaluators were shared based on the 4-D critical appreciative process: Discover, Dream, Design, and Deliver. Based on this shared experience, we illustrate how the dialogic reflective process entails reflexivity and requires pausing; how reflective practice in program evaluation situates our dialogues as learning-oriented rather than a mere accountability discussion; and how reflective action can create a dialogic and generative virtuous cycle.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Allen, J. M., & Innes, M. (2013). Using appreciative inquiry to frame the appraisal of an Australian initial teacher education program. Australian Journal of Teacher Education, 38(11), 1-16. https://doi.org/10.14221/ajte.2013v38n11.8 DOI: https://doi.org/10.14221/ajte.2013v38n11.8
American Evaluation Association. [AEA] (2018, April 5). AEA competencies. https://www.eval.org/page/competencies.
Anderson, J. C., II., Thorson, C. J., & Kelinsky, L. R. (2016). An appreciative inquiry approach to evaluating culture, structure, and power in agricultural teacher education program reform. Journal of Agricultural Education, 57(1), 179-194. https://doi.org/10.5032/jae.2016.01179 DOI: https://doi.org/10.5032/jae.2016.01179
Archibald, T., Neubauer, L. C., & Brookfield, S. D. (2018). The critically reflective evaluator: Adult education's contributions to evaluation for social justice. In A. T. Vo & T. Archibald (Eds.), Evaluative Thinking. New Directions for Evaluation, 158, 109-123. https://doi.org/10.1002/ev.20319 DOI: https://doi.org/10.1002/ev.20319
Archibald, T., Sharrock, G., Buckley, J., & Young, S. (2018). Every practitioner a "knowledge worker": Promoting evaluative thinking to enhance learning and adaptive management in international development. In A. T. Vo & T. Archibald (Eds.), Evaluative Thinking. New Directions for Evaluation. 158, 73-91. https://doi.org/10.1002/ev.20323 DOI: https://doi.org/10.1002/ev.20323
Arias, M. B. (2020). Internet disparity challenges schooling for all. Center for Applied Linguistics. https://www.cal.org/news-and-events/in-the-news/internet-disparity-challenges-6_1_2020.
Attard, K. (2017). Personally driven professional development: Reflective self-study as a way for teachers to take control of their own professional development. Teacher Development, 21(1), 40- 56. https://doi.org/10.1080/13664530.2016.1218363 DOI: https://doi.org/10.1080/13664530.2016.1218363
Barnes, D. (1998). Looking forward: The concluding remarks at the Castle Conference. In M. L.Hamilton, with S.Pinnegar, T.Russell, J.Loughran, & V.LaBoskey (Eds.), Reconceptualizing teaching practice: Self-study in teacher education (pp. ix-xiv). London: Falmer Press.
Beauchamp, C. (2006). Understanding reflection in teaching: A framework for analyzing the literature. (doctoral thesis). https://digitool.library.mcgill.ca/webclient/StreamGate?folder_id=0&dvs=1579034720433~329
Beauchamp, C. (2015). Reflection in teacher education: Issues emerging from a review of current literature. Reflective Practice, 16(1), 123-141. https://doi.org/10.1080/14623943.2014.982525 DOI: https://doi.org/10.1080/14623943.2014.982525
Boje, D. M. (2010). Side shadowing appreciative inquiry: One storyteller's commentary. Journal of Management Inquiry, 19(3), 238-241. https://doi.org/10.1177/1056492610369627 DOI: https://doi.org/10.1177/1056492610369627
Bushe, G.R. (2001) Five theories of change embedded in appreciative inquiry. In Cooperrider, D. L., Sorenson, P., Whitney, D. & Yeager, T. (eds.) Appreciative inquiry: An emerging direction for organization development (117-127). Stipes Publishing.
Bushe. G.R. (2013), Generative process, generative outcome: The transformational potential of appreciative inquiry, In D.L. Cooperrider, D.P. Zandee, L.N. Godwin, M. Avital & B. Boland (eds.) Organizational generativity: The appreciative inquiry summit and a scholarship of transformation (Advances in Appreciative Inquiry, Volume 4), Emerald Group Publishing Limited (89-113). https://doi.org/10.1108/S1475-9152(2013)0000004003 DOI: https://doi.org/10.1108/S1475-9152(2013)0000004003
Chouinard, J. A., Cram, F. (2019). Situating culturally responsive approaches to evaluation: Empirical implications for theory and practice. Thousand Oaks, Sage Publishing. https://doi.org/10.4135/9781506368559 DOI: https://doi.org/10.4135/9781506368559
Coburn, C. E., Penuel, W. R., & Geil, K. E. (2013). Research-practice partnerships at the district level: A new strategy for leveraging research for educational improvement. William T. Grant Foundation.
Coglan, A. T., Preskill, H., & Catsambas, T. T. (2003). An overview of appreciative inquiry in evaluation. New Directions for Evaluation, 100, 5-22. https://doi.org/10.1002/ev.96 DOI: https://doi.org/10.1002/ev.96
Collin, S., & Karsenti, T. (2011). The collective dimension of reflective practice: The how and why. Reflective Practice, 12(4), 569-581. https://doi.org/10.1080/14623943.2011.590346 DOI: https://doi.org/10.1080/14623943.2011.590346
Collin, S., Karsenti, T., & Komis, V. (2013). Reflective practice in initial teacher training: Critiques and perspectives. Reflective Practice, 14(1), 104-117. https://doi.org/10.1080/14623943.2012.732935 DOI: https://doi.org/10.1080/14623943.2012.732935
Cooperrider, D. L., & Srivastva, S. (1987). Appreciative inquiry in organizational life. Research in Organizational Change and Development, 1, 129-169.
Cooperrider, D. L. & Whitney, D. (2001) A positive revolution in change. In Cooperrider, D. L. Sorenson, P., Whitney, D. & Yeager, T. (eds.) Appreciative inquiry: An emerging direction for organization development (9-29). Stipes Publishing.
Cooperrider D. L., Whitney, D. & Stavros, J. M. (2008). Appreciative inquiry handbook (2nd ed.). Brunswick Crown Custom Publishing.
Cousins, J. B. (Ed.). (2019). Global test drive of principles for collaborative approaches to evaluation (CAE). Thousand Oaks, Sage Publishing.
Cousins, J. B., Whitmore, E., & Shulha, L. (2013). Arguments for a common set of principles for collaborative inquiry in evaluation. American Journal of Evaluation, 34(1), 7-22. https://doi.org/10.1177/1098214012464037 DOI: https://doi.org/10.1177/1098214012464037
Dewey, J. (1933). How we think: A restatement of the relation of reflective thinking to the educative process. Henry Regnery.
Fendler, L. (2003). Teacher reflection in a hall of mirrors: Historical influences and political reverberations. Educational Researcher, 32,16-25. https://doi.org/10.3102/0013189X032003016 DOI: https://doi.org/10.3102/0013189X032003016
Fitzgerald, S.P., Oliver, C. & Hoxsey, J.C. (2010) Appreciative inquiry as shadow process. Journal of Management Inquiry, 19(3), 220-233. https://doi.org/10.1177/1056492609349349 DOI: https://doi.org/10.1177/1056492609349349
Freeman, M., & Vasconcelos, E. F. S. (2010). Critical social theory: Core tenets, inherent issues. New Directions for Evaluation, 2010(127), 7-19. https://doi.org/10.1002/ev.335 https://doi.org/10.1002/ev.335 DOI: https://doi.org/10.1002/ev.335
Ganly, T. (2018). Taking time to pause: Engaging with a gift of reflective practice. Innovations in Education and Teaching International, 55(6), 713-723. DOI: https://doi.org/10.1080/14703297.2017.1294492
Gergen, K. J. (1978). Toward generative theory. Journal of Personality and Social Psychology, 36(11), 1344-1360. https://doi.org/10.1037//0022-3514.36.11.1344 DOI: https://doi.org/10.1037//0022-3514.36.11.1344
Gergen, K. J. (2014). From mirroring to world-making: Research as future forming. Journal for the Theory of Social Behaviour, 45(3), 287-310. https://doi.org/10.1111/jtsb.12075 https://doi.org/10.1111/jtsb.12075 DOI: https://doi.org/10.1111/jtsb.12075
Grant, S., & Humphries, M. (2006). Critical evaluation of appreciative inquiry: Bridging an apparent paradox. Action Research, 4(4), 401-418. https://doi.org/10.1177/1476750306070103 DOI: https://doi.org/10.1177/1476750306070103
Hatton, N., & Smith, D. (1995). Reflection in teacher education: Towards definition and implementation. Teaching and Teacher Education, 11(1), 33-49. https://doi.org/10.1016/0742-051X(94)00012-U DOI: https://doi.org/10.1016/0742-051X(94)00012-U
Henrick, E. C., Cobb, P., Penuel, W. R., Jackson, K., & Clark, T. (2017). Assessing partnerships five dimensions of effectiveness. New York, NY: William T. Grant Foundation.
King, J. A., & Stevahn, L. (2013). Interactive evaluation practice. Thousand Oaks, Sage Publishing.
LaBoskey, V. K. (2004). The methodology of self-study and its theoretical underpinnings. In J. J. Loughran, M. L. Hamilton, V. L. LaBoskey, & T. Rusell (Eds.), International handbook of self-study of teaching and teacher education practices (pp. 817-870). Dordrecht: Kluwer. https://doi.org/10.1007/978-1-4020-6545-3_21 DOI: https://doi.org/10.1007/978-1-4020-6545-3_21
Loughran, J. J., & Northfield, J. (1998). A framework for the development of self-study practice. In M. L.Hamilton, with S. Pinnegar, T.Russell, J.Loughran, & V. K.LaBoskey (Eds.), Reconceptualizing teaching practice: Self-study in teacher education (pp. 7-18). London: Falmer.
Ludema, J.D. (2002) Appreciative storytelling: A narrative approach to organization development and change. In Fry, R et.al. Appreciative inquiry and organizational transformation: reports from the field (239-261). Quorum.
Lyons, N. (2006). Reflective engagement as professional development in the lives of university teachers. Teachers and Teaching: Theory and Practice, 12(2), 151-168. https://doi.org/10.1080/13450600500467324 https://doi.org/10.1080/13450600500467324 DOI: https://doi.org/10.1080/13450600500467324
McCarthy, M. D. (2018). Critically teaching criticality?: Modeling social and pedagogical inquiry with literary texts. Studying Teacher Education, 14(2), 174-193. https://doi.org/10.1080/17425964.2018.1449103 DOI: https://doi.org/10.1080/17425964.2018.1449103
Mohr, B. J., Smith, E., & Watkins, J. M. (2000). Appreciative inquiry and learning assessment: An embedded evaluation process in a transitional pharmaceutical company. OD Practitioner, 32(1), 36-52.
Odell, M. (2002). "Beyond the box": An innovative habitat for humanity paradigm for participatory planning, monitoring and evaluation-measuring and increasing program impacts with appreciative inquiry. Habitat for Humanity International.
Orem, S. L., Binkert, J., & Clancy, A. L. (2007). Appreciative coaching: A positive process for change. Jossey-Bass.
Osterman, K.F. & Kottkamp, R.B. (2004) Reflective practice for educators: Professional development to improve student learning. Corwin Press, Thousand Oaks.
Patel, L. (2015). Decolonizing education research: From ownership to answerability. Routledge. https://doi.org/10.4324/9781315658551 DOI: https://doi.org/10.4324/9781315658551
Patton, M. Q. (1997). Utilization-focused evaluation: The new century text. (3rd ed.) Thousand Oaks, Sage Publishing.
Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. Guilford Press.
Patton, M. Q. (2012). Essentials of utilization-focused evaluation (3rd ed.). Thousand Oaks, Sage Publishing.
Poland, B. D., Frohlich, K., Haines, R. J., Mykhalovskiy, E., Rock, M., & Sparks, R. (2006). The social context of smoking: The next frontier in tobacco control? Tobacco Control, 15, 59-63. https://doi.org/10.1136/tc.2004.009886 DOI: https://doi.org/10.1136/tc.2004.009886
Poom-Valickis, K., & Mathews, S. (2013). Reflecting others and own practice: An analysis of novice teachers' reflection skills. Reflective Practice,14(3), 420-434. https://doi.org/10.1080/14623943.2013.767237 DOI: https://doi.org/10.1080/14623943.2013.767237
Preskill, H. S. (2008). Evaluation's second act: A spotlight on learning. American Journal of Evaluation, 29(2), 127-138. https://doi.org/10.1177/1098214008316896 https://doi.org/10.1177/1098214008316896 DOI: https://doi.org/10.1177/1098214008316896
Preskill, H. S. & Caracelli, V. (1997). Current and developing conceptions of use: Evaluation use TIG survey results. Evaluation Practice, 18(3), 209-225. https://doi.org/10.1016/S0886-1633(97)90028-3 DOI: https://doi.org/10.1016/S0886-1633(97)90028-3
Preskill, H. S., & Torres, R. T. (1999). Evaluative inquiry for learning in organizations. Thousand Oaks, Sage Publishing. https://doi.org/10.4135/9781452231488 DOI: https://doi.org/10.4135/9781452231488
Rallis, S. F., & Rossman, G. B. (2000). Dialogue for learning: Evaluator as critical friend. New Directions for Evaluation, 2000(86), 81-92. https://doi.org/10.1002/ev.1174 https://doi.org/10.1002/ev.1174 DOI: https://doi.org/10.1002/ev.1174
Rama, J. A., Falco, C., & Balmer, D. F. (2018). Using Appreciative Inquiry to inform program evaluation in graduate medical education. Journal of Graduate Medical Education, 10(5), 587-590. https://doi.org/10.4300/JGME-D-18-00043.1 DOI: https://doi.org/10.4300/JGME-D-18-00043.1
Ridley-Duff, R. J. & Duncan, G. (2015). What is critical appreciation? Insights from studying the critical turn in an appreciative inquiry. Human Relations, 68(10), 1579-1599. https://doi.org/10.1177/0018726714561698 DOI: https://doi.org/10.1177/0018726714561698
Ruffinelli, A., Hoz, S., & Álvarez, C. (2020) Practicum tutorials in initial teacher training: conditions, strategies, and effects of reflective practice, Reflective Practice, 21(1), 54-67. https://doi.org/10.1080/14623943.2019.1708712 DOI: https://doi.org/10.1080/14623943.2019.1708712
Russell, T. (2013). Has reflective practice done more harm than good in teacher education? Phronesis, 2, 80-88. https://doi.org/10.7202/1015641ar DOI: https://doi.org/10.7202/1015641ar
Samaras, A. P. (2011). Self-study teacher research: Improving your practice through collaborative inquiry (1st ed.). Sage. https://doi.org/10.4135/9781452230481 DOI: https://doi.org/10.4135/9781452230481
Schön, D. (1983). The reflective practitioner. Basic Books.
Schwandt, T. A. (2015). Evaluation foundations revisited: Cultivating a life of the mind for practice. Stanford University Press.
Smith, T. L., Barlow, P. B., Skolits, G. J., & Peters, J. M. (2015). Demystifying reflective practice: Using the DATA model to enhance evaluators' professional activities. Evaluation and Program Planning, 52(2015), 142-147. https://doi.org/10.1016/j.evalprogplan.2015.04.004 DOI: https://doi.org/10.1016/j.evalprogplan.2015.04.004
Smith, T. L., & Skolits, G. J. (in press). Conceptualizing and engaging in reflective practice: Experienced evaluators' perspectives. American Journal of Evaluation.
Taggart, G., & Wilson, A. P. (2005). Promoting reflective thinking in teachers. Thousand Oaks, Corwin.
Thompson, S., & Thompson, N. (2008). The critically reflective practitioner. Basingstoke: Palgrave Macmillan. https://doi.org/10.1007/978-0-230-36655-8 DOI: https://doi.org/10.1007/978-0-230-36655-8
Tuck, E. (2015). Foreword. In L. Patel (Ed.), Decolonizing education research: From ownership to answerability (pp. xii-xv). Routledge.
Valli, L. (1992) Reflective teacher education. State University of New York Press.
van Draanen, J. (2017). Introducing reflexivity to evaluation practice: An in-depth case study. American Journal of Evaluation, 38(3), 360-375. https://doi.org/10.1177/1098214016668401 DOI: https://doi.org/10.1177/1098214016668401
Zeichner, K., & Liu, K. Y. (2010). A critical analysis of reflection as a goal for teacher education. In N. Lyons (Ed.), Handbook of reflection and reflective inquiry: Mapping a way of knowing for professional reflective inquiry (pp. 67-81). Springer. https://doi.org/10.1007/978-0-387-85744-2_4 https://doi.org/10.1007/978-0-387-85744-2_4 DOI: https://doi.org/10.1007/978-0-387-85744-2_4
Zeichner, K., & Liston, D. (1996). Reflective teaching: An introduction. Lawrence Erlbaum.