Evaluating Faculty Perceptions of Student Learning Outcomes: A Rasch Measurement Analysis
Main Article Content
Abstract
Background: The importance of assessing student learning outcomes has demanded attention from most everyone involved in the higher education enterprise, as accreditation and funding implications are often linked to the results. Faculty, however, are often critical of the assessment process because outcomes assessment is costly with regard to time, energy and other resources, and evidence of its effectiveness is not always noticeable.
Purpose: The purpose of this article is to evaluate faculty perceptions of various student learning outcomes in order to determine which types outcomes are most valued by faculty.
Setting: United States.
Intervention: Not applicable.
Research Design: The HERI Faculty Survey was utilized to capture a nationally representative snapshot of faculty perceptions of student learning outcomes.
Data Collection and Analysis: The Rating Scale Model, a Rasch measurement model, was used to analyze survey data of 7,356 respondents.
Findings: Faculty from virtually all disciplines are primarily concerned with the intellectual growth of students. All other types of student learning outcomes (i.e., social, emotional, cultural growth) are of lesser concern. These findings suggest higher education institutions seeking “faculty buy-in” may want to consider focusing more on intellectual types of outcomes as these outcomes appear to best resonate with faculty, and in turn may result in less faculty resistance.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Alpert, D. (1985). Performance and paralysis: The organizational context American research university. Journal of Higher Education, 56, 241-81. https://doi.org/10.1080/00221546.1985.11780690 DOI: https://doi.org/10.1080/00221546.1985.11780690
Andrich, D. (1978). A rating scale formulation for ordered response categories. Psychometrika, 43, 561- 573. https://doi.org/10.1007/BF02293814 DOI: https://doi.org/10.1007/BF02293814
Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers (2nd ed.). San Francisco: Jossey-Bass.
Becher, T. (1987). Disciplinary shaping of the profession. In B. R. Clark (Ed.), The academic profession: National, disciplinary, and institutional settings. Los Angeles, CA: University of California Press. https://doi.org/10.1525/9780520311329-009 DOI: https://doi.org/10.1525/9780520311329-009
Biglan, A. (1973a). The characteristics of subject matter in different academic areas. Journal of Applied Psychology, 57, 195-203. https://doi.org/10.1037/h0034701 DOI: https://doi.org/10.1037/h0034701
Biglan, A. (1973b). Relationships between subject matter characteristics and the structure and output of university departments. Journal of Applied Psychology, 57, 3, 204-213. https://doi.org/10.1037/h0034699 DOI: https://doi.org/10.1037/h0034699
Bond, T. & Fox, C. (2001). Applying the Rasch model: Fundamental measurement in the human sciences. Mahwah, NJ; Lawrence Erlbaum. https://doi.org/10.4324/9781410600127 DOI: https://doi.org/10.4324/9781410600127
Braxton, J. M., & Nordvall, R. C. (1985). Selective liberal arts colleges: Higher quality as well as higher prestige? Journal of Higher Education, 5, 538- 554. https://doi.org/10.1080/00221546.1985.11778717 DOI: https://doi.org/10.1080/00221546.1985.11778717
Clark, B. R. (1980). Academic culture. New Haven, CT: Higher Education Research Group, Yale University.
Donald, J. (1990). Knowledge and the university curriculum. In C. F. Conrad & J. G. Haworth (Eds.), Curriculum in transition: Perspectives on the undergraduate experience. Needham Heights, MA: Ginn Press.
Fox, E. A. (1997). Examining factors which influence and significantly impact faculty teaching goals. (Doctoral dissertation, University of South Carolina, 1997) Dissertation Abstracts International, 58(03A), 0763.
Franklin, J., & Theall, M. (1992). Disciplinary differences: Instructional goals and activities, measures of student performance and student ratings of instructors. Paper presented at the meeting of the American Educational Research Association, San Francisco.
Gaff, J. G., & Wilson, R. C. (1971). Faculty cultures and interdisciplinary studies. Journal of Higher Education, 42(3), 186-201. https://doi.org/10.1080/00221546.1971.11776806 DOI: https://doi.org/10.1080/00221546.1971.11776806
Holland, J. L. (1966). The psychology of vocational choice. Waltham, MA: Blaisdell.
Jervis, F. M., & Cogdon, R. G. (1958). Student and faculty perceptions of educational values. American Psychologist, 13, 464-66. https://doi.org/10.1037/h0046996 DOI: https://doi.org/10.1037/h0046996
Kolb, D. A. (1980). Learning styles and disciplinary differences. In A. W. Chickering & Associates (Eds.), The modern American college. San Francisco, CA: Jossey-Bass.
Ladd, E. C., & Lipset, S. M. (1975). Technical report:1975 Survey of the American professorate. Storrs, CT: The University of Connecticut, Social Science Data Center.
Lattuca, L. & Stark, J. (1994) Will disciplinary perspectives impede curricular reform? Journal of Higher Education, 65, 401-426. https://doi.org/10.1080/00221546.1993.11778508 DOI: https://doi.org/10.1080/00221546.1993.11778508
Lawrence, J. H., Hart, K. A., Mackie, C., Muniz, D., & Dickmann, E. (1990). A comparison of the teaching goals, assumptions, and practices of faculty in eight liberal arts disciplines. Paper presented at the Annual Meeting of the Association for the Study of Higher Education, Portland, OR. (ERIC Document Reproduction Service No. ED 326 115)
Lee, J. J. (2004). Comparing institutional relationships with academic departments: A study of five academic fields. Research in Higher Education, 45(6) 603-624. https://doi.org/10.1023/B:RIHE.0000040265.86668.1a DOI: https://doi.org/10.1023/B:RIHE.0000040265.86668.1a
Leverenz, T. R., & Lewis, B. R. (1981). An analysis of faculty consistency in the academic professions. (ERIC Document Reproduction Service No. ED 201 254).
Liebert, R. J., & Bayer, A. E. (1975). Goals in teaching undergraduates: Professional reproductions and client- centeredness. The American Sociologist, 10, 195-205.
Linacre, J. M. (2010). Winsteps® (Version 3.69.1) [Computer Software]. Beaverton, Oregon: Winsteps.com.
Neumann, R., Parry, S., & Becher, T. (2002). Teaching and learning in their disciplinary contexts: A conceptual analysis. Studies in Higher Education, 27, 4, 405-417. https://doi.org/10.1080/0307507022000011525 DOI: https://doi.org/10.1080/0307507022000011525
Platt, G. M., Parsons, T., & Kirshstein, R. (1976). Faculty teaching goals, 1968- 1973. Social Problems, 24, 298-307. https://doi.org/10.2307/800347 DOI: https://doi.org/10.1525/sp.1976.24.2.03a00140
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Copehagen, Denmark: Danmarks Paedogogische Institut.
Royal, K. D., Eli, J. A., & Bradley, K. D. (2010). Exploring community college faculty perceptions of student outcomes: Findings of a pilot study. Community College Journal of Research & Practice, 34(7), 523-540. https://doi.org/10.1080/10668920701827025 DOI: https://doi.org/10.1080/10668920701827025
Royal, K. D., & Bradley, K. D. (2008). Rethinking measurement in higher education research. (ERIC Document Reproduction Service No. ED 506 513).
Royal, K. D. (2010). Making meaningful measurement in survey research: A demonstration of the utility of the Rasch model. IR Applications Available at: http://www.airweb.org/page.asp?page=566.
Smart, J. C. (2005). Attributes of exemplary research manuscripts employing quantitative analyses. Research in Higher Education, 46, 4, 461-477. https://doi.org/10.1007/s11162-005-2970-5 DOI: https://doi.org/10.1007/s11162-005-2970-5
Smart, J. C. (1982). Faculty teaching goals: A test of Holland's theory. Journal of Educational Psychology, 74(2), 180-188. https://doi.org/10.1037/0022-0663.74.2.180 DOI: https://doi.org/10.1037//0022-0663.74.2.180
Smart, J. C., & Elton, C. F. (1975). Goal orientations of academic departments: A test of Biglan's theory. Journal of Applied Psychology, 60, 580-588. https://doi.org/10.1037/0021-9010.60.5.580 DOI: https://doi.org/10.1037//0021-9010.60.5.580
Smart, J. C. & Ethington, C.A. (1995) Disciplinary and institutional differences in undergraduate education goals. In N. Hativa & M. Marincovich (Eds.), Disciplinary differences in teaching and learning: Implications for practice (pp. 49-57). San Francisco, CA: Jossey-Bass. https://doi.org/10.1002/tl.37219956408 DOI: https://doi.org/10.1002/tl.37219956408
Smart, J. C., Feldman, K. A., & Ethington, C. A. (2000). Academic disciplines: Holland's theory and the study of college students and faculty. Nashville, TN: Vanderbilt University Press.
Stark, J. S., & Morstain, B. R. (1978). Educational orientations of faculty in liberal arts colleges: An analysis of disciplinary differences. Journal of Higher Education, 49, 420-437. https://doi.org/10.1080/00221546.1978.11780393 DOI: https://doi.org/10.1080/00221546.1978.11780393
Swenson, C. D. (1997). Teaching goals of practitioner-educators: Improving teaching and learning in continuing higher education. Continuing Higher Education Review, 61, 88-103.
Wilson, R. C., Gaff, J. G., Dienst, E. R., Wood, L., & Bavry, J. L. (1975). College professors and their impact on students. New York, NY: Wiley.