Using Program Theory to Evaluate a Graduate College Student Development Program
Main Article Content
Abstract
Background: The “3 Minute Presentation” is a graduate student competition based off the more popular “3 Minute Thesis” competition. The program aims to help graduate students learn to inform others of their research in a quick and accessible manner. Programs to engage graduate students more deeply in their education require evaluation to determine if they are useful and effective at meeting their intended goals. Evaluation literature in graduate educational programs is currently limited, but increasingly needed for both the field and the students served.
Purpose: Development and testing of a program-theory evaluation to understand participation, recruitment, preparation, training, skills, and confidence of graduate students engaging in a “3 Minute Presentation” competition at a state university.
Setting: Institution of Higher Education
Intervention: 3 Minute Presentation competition
Research Design: Mixed-method program-theory evaluation
Data Collection and Analysis: Direct observations and closed-ended survey analyzed through qualitative coding, descriptive statistics, group comparisons, and correlation analysis.
Findings: Overall, the program evaluation found, with a possible lack of diversity in participants, that the program components of recruitment, preparation, and skill development work as expected. Additionally, engagement in preparation was associated with competition scores and the perceived helpfulness of preparation was related to students’ confidence in their presentation skills. This evaluation was deemed useful for program improvement and capacity building in the program’s continuation at the university.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
3MT (Producer). (2015, September 9). Promoting yourself with 3MT [video file]. Retrieved from https://vimeo.com/138709200
Astin, A. W. (1984). Student involvement: A developmental theory for higher education. Journal of College Student Personnel, 25(4), 297-308. Retrieved from https://www.researchgate.net/profile/Alexander_Astin/publication/220017441_Student_Involvement_A_Development_Theory_for_Higher_Education/links/00b7d52d094bf5957e000000/Student-Involvement-A-Development-Theory-for-Higher-Education.pdf
Bickman, L. (1987). The functions of program theory. New Directions for Program Evaluation, 33, 5-18.https://doi.org/10.1002/ev.1443 DOI: https://doi.org/10.1002/ev.1443
Boyle, P., & Boice, B. (1998). Systematic mentoring for new faculty teachers and graduate teaching assistants. Innovative Higher Education, 22(3), 157-179. https://doi.org/10.1023/A:1025183225886 DOI: https://doi.org/10.1023/A:1025183225886
Brousselle, A., & Champagne, F. (2011). Program theory: Logic analysis. Evaluation and Program Planning, 34(1), 63-78. https://doi.org/10.1016/j.evalprogplan.2010.04.001 DOI: https://doi.org/10.1016/j.evalprogplan.2010.04.001
Chen, H. T. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.
Chen, H. T. (1997). Applying mixed methods under the framework of theory-driven evaluations. New directions for evaluation, 1997(74), 61-72. https://doi.org/10.1002/ev.1072 DOI: https://doi.org/10.1002/ev.1072
Creswell, J. W. (2014). A concise introduction to mixed methods research. Sage Publications.
Deci, E. L., & Ryan, R. M. (2000). The 'what' and 'why' of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11, 227-268. https://doi.org/10.1207/S15327965PLI1104_01 DOI: https://doi.org/10.1207/S15327965PLI1104_01
Donaldson, S. I. (2007). Program theory-driven evaluation science: Strategies and applications. New York: Erlbaum. https://doi.org/10.4324/9780203809730 DOI: https://doi.org/10.4324/9780203809730
Fitz-Gibbon, C.T., & Morris, L.L. (1975). Theory-based evaluation. The Journal of Educational Evaluation, 5(1). Retrieved from http://www.cem.org/attachments/publications/CEMWeb022%20Theory%20Based%20Education.pdf
Fitzpatrick, J. L., Sanders, J. R., & Worthen, B. R. (2012). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Upper Saddle River, NJ: Pearson.
Gardner, S. K., & Barnes, B. J. (2007). Graduate student involvement: Socialization for the professional role. Journal of College Student Development, 48(4), 369-387. https://doi.org/10.1353/csd.2007.0036 DOI: https://doi.org/10.1353/csd.2007.0036
Hartnett, R. T. (1965). Involvement in extra-curricular activities as a factor in academic performance. Journal of College Student Development, 6(5), 272-274. Retrieved from https://muse.jhu.edu/journal/238
Kuh, G. D., Schuh, J. H., Whitt, E., Andreas, R. E., Lyons, J. W., Strange, C. C., ... & MacKay, K. A. (1991). Involving colleges. San Francisco: Jossey-Bass.
Lovitts, B. E. (2001). Leaving the ivory tower: The causes and consequences of departure from doctoral study. Lanham, MD: Rowman & Littlefield. DOI: https://doi.org/10.5040/9798216409847
Okahana, H., Feaster, K., & Allum, J. (2016). Graduate enrollment and degrees: 2005 to 2015. Washington, DC: Council of Graduate Schools. Retrieved from http://cgsnet.org/ckfinder/userfiles/files/Graduate%20Enrollment%20%20Degrees%20Fall%202015%20Final.pdf
Owen, J.M. (2006). Program evaluation: Forms and approaches (3rd ed.). New York: The Guilford Press.
Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students (Vol. 2). K. A. Feldman (Ed.). San Francisco, CA: Jossey-Bass.
Renn, K. A., & Reason, R. D. (2013). College students in the United States: Characteristics, experiences, and outcomes. San Francisco, CA: John Wiley & Sons.
Schuh, J.H. (2009). Assessment methods for student affairs. San Francisco, CA: Jossey-Bass.
Sheldon, K., Ryan, R., & Reis, H.T. (1996). What makes for a good day? Competence and autonomy in the day and in the person. Personality and Social Psychology Bulletin, 22, 1270-1279. https://doi.org/10.1177/01461672962212007 DOI: https://doi.org/10.1177/01461672962212007
Stockdill, S.H., Baizeman, M., & Compton, D.W. (2002). Toward a definition of the ECB process: A conversation with the ECB literature. New Directions for Evaluation, 93, 7-26. https://doi.org/10.1002/ev.39 DOI: https://doi.org/10.1002/ev.39
Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage.
Strauss, A., & Corbin, J. (1994). Grounded theory methodology. In N.K. Denzin & Y.S. Lincoln (Eds.), Handbook of qualitative research (pp. 217-285). Thousand Oaks, CA: Sage.
Terenzini, P. T., & Pascarella, E. T. (1991). Twenty years of research on college students: Lessons for future research. Research in Higher Education, 32(1), 83-92. https://doi.org/10.1007/BF00992835 https://doi.org/10.1007/BF00992835 DOI: https://doi.org/10.1007/BF00992835
Thomas, G., Clewell, B., & Pearson Jr, W. GRE Board. (1992). The role and activities of American graduate schools in recruiting, enrolling, and retaining United States' Black and Hispanic students. (Report No. 87-08) Princeton, NJ: Educational Testing Service, Graduate Record Examinations Board
Tinto, V. (1993). Building community. Liberal Education, 79(4), 16-21. Retrieved from https://www.aacu.org/liberaleducation https://doi.org/10.1177/105382599301600204 DOI: https://doi.org/10.1177/105382599301600204
University of Queensland. (2017) Three Minute Thesis. Retrieved from https://threeminutethesis.uq.edu.au/about
Weidman, J. C., Twale, D. J., & Stein, E. L. ERIC. (2001). Socialization of Graduate and Professional Students in Higher Education: A Perilous Passage? ASHE-ERIC Higher Education Report, (Volume 28, Number 3)., San Francisco, CA: Jossey-Bass, Publishers.
Weiss, C. H. (1997). Theory-based evaluation: Past, present, and future. New Directions for Evaluation, 76, 41-55. https://doi.org/10.1002/ev.1086 DOI: https://doi.org/10.1002/ev.1086
Weiss, C.H. (2000). Which links in which theories shall we evaluate? New Directions in Evaluation, 87, 35-45. https://doi.org/10.1002/ev.1180 DOI: https://doi.org/10.1002/ev.1180