The Role of Theory in University Evaluation Coursework
Article Sidebar
Main Article Content
Abstract
This paper examines how theory is conveyed to novice evaluators in university coursework, providing a new perspective on the role of theory in evaluation. Using a survey of university evaluation instructors in the United States, the authors examined instructor perspectives on the role of evaluation theory as well as the evaluation content in assigned textbooks. Analysis indicated a similar lack of coherence and consistency in the use of theory to that seen in evaluation practice. The findings support three takeaways for evaluation educators, practitioners, and scholars. First, the field must continue to clarify the role of theory in evaluation. Second, an in-depth analysis of authors’ approaches to discussing theory in textbooks is warranted. Third, textbook authors and publishers could consider the themes that emerged from instructors’ responses to inform book proposals and revisions of textbook editions to better suit the range of instructor needs.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Alkin, M. C. (Ed.). (2013). Evaluation roots: A wider perspective of theorists' views and influences (2nd ed.). Sage Publications.
Alkin, M. & Vo, A. (2018). Evaluation essentials: From A to Z (2nd ed.). The Guilford Press.
Altschuld, J. W. (2014). Bridging the gap between asset/capacity building and needs assessment: Concepts and practical applications. Sage. https://doi.org/10.4135/9781483398945 DOI: https://doi.org/10.4135/9781483398945
Bryk, A. S., Gomez, L. M., Grunow, A., & LeMahieu, P. G. (2015). Learning to improve: How America's schools can get better at getting better. Harvard Education Press.
Christie, C. A. (2003). What guides evaluation? A study of how evaluation practice maps onto evaluation theory. New Directions for Evaluation, 97, 7-36. https://doi.org/10.1002/ev.72 DOI: https://doi.org/10.1002/ev.72
Christie, C. A., & Alkin, M. C. (2008). Evaluation theory tree re-examined. Studies in Educational Evaluation, 34(3), 131-135. https://doi.org/10.1016/j.stueduc.2008.07.001 DOI: https://doi.org/10.1016/j.stueduc.2008.07.001
Christie, C. A., & Lemire, S. T. (2019). Why evaluation theory should be used to inform evaluation policy. American Journal of Evaluation, 40(4), 490-508. https://doi.org/10.1177/1098214018824045 DOI: https://doi.org/10.1177/1098214018824045
Coryn, C. L. S., Noakes, L. A., Westine, C. D., & Schröter, D. C. (2011). A systematic review of theory-driven evaluation practice from 1990 to 2009. American Journal of Evaluation, 32(2), 199-226. https://doi.org/10.1177/1098214010389321 DOI: https://doi.org/10.1177/1098214010389321
Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart and Winston.
Davidson, E. J. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Sage. https://doi.org/10.4135/9781452230115 DOI: https://doi.org/10.4135/9781452230115
Davies, R., & MacKay, K. (2014). Evaluator training: Content and topic valuation in university evaluation courses. American Journal of Evaluation, 35(3), 419-429. https://doi.org/10.1177/1098214013520066 DOI: https://doi.org/10.1177/1098214013520066
D'Agostino, J. V. (2001). Increasing the Role of Educational Psychology Theory in Program Development and Evaluation. Educational Psychologist, 36(2), 127-132. https://doi.org/10.1207/S15326985EP3602_7 DOI: https://doi.org/10.1207/S15326985EP3602_7
Fink, A. (2005). Evaluation fundamentals: Insights into the outcomes, effectiveness, and quality of health programs (2nd ed.). Sage Publications.
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2011). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Pearson.
Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories: AMEE guide no. 67. Medical teacher, 34(5), e288-e299. https://doi.org/10.3109/0142159X.2012.668637 DOI: https://doi.org/10.3109/0142159X.2012.668637
Galport, N., & Azzam, T. (2017). Evaluator training needs and competencies: A gap analysis. American Journal of Evaluation, 38(1), 80-100. https://doi.org/10.1177/1098214016643183 DOI: https://doi.org/10.1177/1098214016643183
Gertler, P. J., Martinez, S., Premand, P., Rawlings, L. B., & Vermeersch, C. M. J. (2011). Impact evaluation in practice. The World Bank. https://doi.org/10.1596/978-0-8213-8541-8 DOI: https://doi.org/10.1596/978-0-8213-8541-8
Grinnell, R. M., Gabor, P. A., & Unrau, Y. A. (2019). Program evaluation for social workers: Foundations of evidence-based programs. Oxford University Press.
Gullickson, A. M., King, J. A., LaVelle, J. M., & Clinton, J. M. (2019). The current state of evaluator education: A situation analysis and call to action. Evaluation and program planning, 75, 20-30. https://doi.org/10.1016/j.evalprogplan.2019.02.012 DOI: https://doi.org/10.1016/j.evalprogplan.2019.02.012
Harris, M. J. (2016). Evaluating public and community health programs (2nd ed.). Jossey-Bass.
Hodges, B. C., & Videto, D. M. (2011). Assessment and planning in health programs. Jones & Bartlett Learning.
Hood, S., Hopson, R., & Frierson, H. (Eds.). (2014). Continuing the journey to reposition culture and cultural context in evaluation theory and practice. IAP. DOI: https://doi.org/10.1108/978-1-62396-937-0
Keating, S. B. (2017). Curriculum development and evaluation in nursing education (4th ed.). Springer Publishing Company.
Knowlton, L. W., & Phillips, C. C. (2012). The logic model guidebook: Better strategies for great results (2nd ed.). Sage.
Leedy, P. D., & Ormrod, J. E. (2015). Practical research: Planning and design (11th ed.). Pearson.
LaVelle, J. M. (2014). An examination of evaluation education programs and evaluator skills across the world [Doctoral dissertation, The Claremont Graduate University].
LaVelle, J. M. (2020). Educating evaluators 1976-2017: An expanded analysis of university-based evaluation education programs. American Journal of Evaluation, 41(4), 494-509. https://doi.org/10.1177/1098214019860914 DOI: https://doi.org/10.1177/1098214019860914
LaVelle, J. M., & Donaldson, S. I. (2010). University-based evaluation training programs in the United States 1980-2008: An empirical examination. American Journal of Evaluation, 31(1), 9-23. https://doi.org/10.1177/1098214009356022 DOI: https://doi.org/10.1177/1098214009356022
Leeuw, F. L., & Donaldson, S. I. (2015). Theory in evaluation: Reducing confusion and encouraging debate. Evaluation, 21(4), 467-480. https://doi.org/10.1177/1356389015607712 DOI: https://doi.org/10.1177/1356389015607712
Linfield, K. J., & Posavac, E. J. (2019). Program evaluation: Methods and case studies (9th ed.). Routledge. https://doi.org/10.4324/9781315102429 DOI: https://doi.org/10.4324/9781315102429
McDavid, J. C., Huse, I., & Hawthorn, L. R. L. (2018). Program evaluation and performance measurement: An introduction to practice. Sage. https://doi.org/10.4135/9781071878897 DOI: https://doi.org/10.4135/9781071878897
McKenzie, J., Neiger, B., & Thackeray, R. (2017). Planning, implementing, and evaluating health promotion programs (7th ed.). Pearson.
Mertens, D. M., & Wilson, A. T. (2019). Program evaluation theory and practice: A comprehensive guide (2nd ed.). The Guilford Press.
Newcomer, K., Harry, H., & Wholey, J. Handbook of practical program evaluation (4th ed.). Jossey-Bass.
Oermann, M. H., & Gaberson, K. B. (2019). Evaluation and testing in nursing education (6th ed.). Springer Publishing Company. https://doi.org/10.1891/9780826135759 DOI: https://doi.org/10.1891/9780826135759
Patton, M. Q., McKegg, K., & Wehipeihana, N. (Eds.). (2016). Developmental evaluation exemplars: Principles in practice. The Guilford Press.
Perrin, K. M. (2016). Essentials of planning and evaluation for public health. Jones & Bartlett Learning.
Posavac, E. J., & Carey, R. G. (1989). Program evaluation: Methods and case studies (3rd ed.). Prentice Hall.
Rossi, P. H., & Freeman, H. E. (1989). Evaluation: A systematic approach (4th ed.). Sage Publications.
Rossi, P. H., Lipsey, M. W., & Henry, G. T. (2018). Evaluation: A systematic approach (8th ed.). Sage Publications.
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach (7th ed.). Sage Publications.
Royse, D., Thyer, B. A., & Padgett, D. K. (2016). Program evaluation: An introduction to an evidence-based approach (6th ed.). Cengage Learning.
Rubin, A. (2020). Program evaluation: Pragmatic methods for social work and human service agencies. Cambridge University Press. https://doi.org/10.1017/9781108870016 DOI: https://doi.org/10.1017/9781108870016
Rubin, A., & Babbie, E. R. (2015). Essential research methods for social work (4th ed.). Cengage Learning.
Russ-Eft, D., & Preskill, H. (2001). Evaluation in organizations: A systematic approach to enhancing learning, performance, and change. Basic Books.
Saldaña, J. (2015). The coding manual for qualitative researchers (3rd ed.). Sage
Schwandt, T. A. (2015). Evaluation foundations revisited: Cultivating a life of the mind for practice. Stanford Business Books.
Shadish, W. R. (1998). Evaluation theory is who we are. American Journal of Evaluation, 19, 1-19. https://doi.org/10.1177/109821409801900102 DOI: https://doi.org/10.1016/S1098-2140(99)80177-5
Shaw, J. (2019). Teaching note-A call for including theories of evaluation in program evaluation courses taught in schools of social work. Journal of Social Work Education, 55(3), 596-601. https://doi.org/10.1080/10437797.2019.1567417 DOI: https://doi.org/10.1080/10437797.2019.1567417
Smith, N. L. (2010). Characterizing the evaluand in evaluating theory. American Journal of Evaluation, 31(3), 383-389. https://doi.org/10.1177/1098214010371820 DOI: https://doi.org/10.1177/1098214010371820
Stake, R. E. (2003). Standards-based & responsive evaluation. Sage. https://doi.org/10.4135/9781412985932 DOI: https://doi.org/10.4135/9781412985932
Stufflebeam, D. L., Madaus, G. F., & Kellaghan, T. (Eds.). (2000). Evaluation models: Viewpoints on educational and human services evaluation (2nd ed.). Kluwer Academic Publishers.
Stufflebeam, D. L., & Coryn, C. L. S. (2014). Evaluation theory, models, and applications (2nd ed.). Jossey-Bass.
Sylvia, R. D., & Sylvia, K. (2012). Program planning and evaluation for the public manager (4th ed.). Waveland Press.
Thomas, N. (2018). An analysis of program evaluation course content in CSHSE-accredited human services baccalaureate programs. Studies in Educational Evaluation, 59, 187-194. https://doi.org/10.1016/j.stueduc.2018.08.001 DOI: https://doi.org/10.1016/j.stueduc.2018.08.001
Thomas, V. G., & Campbell, P. B. (2021). Evaluation in today's world: Respecting diversity, improving quality, and promoting usability. Sage Publications.
Thomas, V. G., & Madison, A. (2010). Integration of social justice into the teaching of evaluation. American Journal of Evaluation, 31(4), 570-583. https://doi.org/10.1177/1098214010368426 DOI: https://doi.org/10.1177/1098214010368426
Varier, D., Marshall, D., & Kuehn, K. (2019, November). Teaching theory in evaluation courses: An exploratory study. American Evaluation Association, Minneapolis, MN, United States.
Weiss, C. H. (1997). How can theory-based evaluation make greater headway? Evaluation Review, 21(4), 501-524. https://doi.org/10.1177/0193841X9702100405 DOI: https://doi.org/10.1177/0193841X9702100405
Weiss, C. H. (1998). Evaluation (2nd ed.). Prentice-Hall.
Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Sage Publications.