Investigating the Efficacy of a Professional Development Program in Formative Classroom Assessment in Middle School English Language Arts and Mathematics
Main Article Content
Abstract
Background: Teachers who engage in formative classroom assessment using practices that accurately measure student learning should be better positioned to diagnose the instructional needs of their students and to act on that information. For this reason, there has been increased interest in formative classroom assessment in recent years. Although some researchers have found indications that some assessment practices may raise student achievement, evaluations of professional development programs designed to increase teacher assessment skill have not found differences in student performance. However, few studies of this type have been performed.
Purpose: The purpose of this study was to determine the efficacy of a professional development program in formative classroom assessment on teacher assessment knowledge and their students’ achievement.
Setting: The professional development program was implemented in state-identified, low performing middle schools from November 2005 through April 2008.
Intervention: Researchers investigated a professional development program for teachers designed to increase their skill in creating and using assessments to support student learning. In Year 1, the professional development was implemented by an assessment coach in the treatment schools. No professional development was provided in the control schools. In Years 2 and 3, levels of treatment were investigated such that the professional development was implemented by an assessment coach or a relatively untrained facilitator.
Research Design: Year 1 involved a multi-site, cluster randomized trial where schools were randomly assigned to either the treatment or control group. Year 2 and Year 3 involved a quasi-experimental design.
Data Collection and Analysis: Researchers collected pretest and posttest teacher measures and analyzed the data using a split-plot ANOVA each year. Summative large scale assessment data was collected for students and analyzed using hierarchical linear modeling (HLM).
Findings: Findings from this study indicate that the professional development program increased teacher assessment skill regardless of whether the program was implemented with a trained assessment coach or a relatively untrained facilitator. However, students of teachers participating in the professional development tended to demonstrate lower achievement than a matched set of students whose teachers did not receive the professional development. Implications for how teachers use assessment data to guide reteaching are discussed.
Keywords: formative assessment; classroom assessment; student achievement
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
Funding data
-
Institute of Education Sciences
Grant numbers R305M050270B
References
Andrade, H., & Boulay, B. (2003). The role of rubric-referenced self assessment in learning to write. Journal of Educational Research, 97(1), 21-34. https://doi.org/10.1080/00220670309596625 DOI: https://doi.org/10.1080/00220670309596625
Andrade, H. L., Du, Y., & Wang, X. (2008). Putting rubrics to the test: The effect of model, criteria generation, and rubric-referenced self assessment on elementary school students' writing. Educational Measurement: Issues and Practice, 27(2), 3−13. https://doi.org/10.1111/j.1745-3992.2008.00118.x DOI: https://doi.org/10.1111/j.1745-3992.2008.00118.x
Andrade, H. (2010). Students as the definitive source of formative assessment: Academic self-assessment and the self-regulation of learning. In H. Andrade & G. Cizek (Eds.), Handbook of formative assessment. (pp. 90-105). New York: Routledge.
Aschbacher, P. R. (1999). Developing indicators of classroom practice to monitor and support school reform. University of California, Los Angeles: CRESST Technical Report 513. University of California, Los Angeles: Center for Research on Educational Standards and Student Testing. Retrieved January 30, 2009, from http://research.cse.ucla.edu/Reports/TECH513.pdf.
Barton, P., & Coley, R. (1994). Testing in America's schools. Princeton, NJ: Educational Testing Services.
Bell, C., Steinberg, J., Wiliam, D., & Wylie, C. (2008, March). Formative assessment and teacher achievement: Two years of implementation of the Keeping Learning on Track Program. Paper presented at the annual meeting of the National Council on Measurement in Education, New York.
Brookhart, S. M. (2005, April). Research on formative classroom assessment. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada.
Brookhart, S. M., Moss, C. M., & Long, B. A. (2007). A cross-case analysis of teacher inquiry into formative assessment practices in six Title I reading classrooms. CASTL Technical Report Series No. 1-07. Retrieved August 18, 2008, from http://www.castl.duq.edu/Castl_Tech Reports.htm
Brookhart, S. M., Moss, C. M., & Long, B. A. (2008, March). Professional development in formative assessment: Effects on teacher and student learning. Paper presented at the annual meeting of the National Council on Measurement in Education, New York.
Brookhart, S.M. (2009). Editorial. Educational Measurement: Issues and Practice, (28)3, 1-3. https://doi.org/10.1111/j.1745-3992.2009.01131.x DOI: https://doi.org/10.1111/j.1745-3992.2009.00148.x
Brookhart, S. M. (2010). Mixing it up: Combining sources of classroom achievement information for formative and summative purposes. In H. Andrade & G. Cizek (Eds.), Handbook of formative assessment (pp. 279-296). New York: Routledge.
Carter, K. (1984). Do teachers understand the principles for writing tests? Journal of Teacher Education, 35, 57-60. https://doi.org/10.1177/002248718403500613 DOI: https://doi.org/10.1177/002248718403500613
D'Amico, L., Hardee, S., Morgan, G., Yap, C. C., & Ishikawa, T. (2008). An evaluation of teacher quality research: Investment, implementation, and long-term student achievement effects. Columbia, SC: University of South Carolina, Office of Program Evaluation.
Fleming, M., & Chambers, B. (1983). Teacher-made tests: Windows on the classroom. In W. E. Hathaway, (Ed.), Testing in the schools. New directions for testing and measurement (pp. 29-38). San Francisco: Jossey-Bass.
Haydel, J. B., Oescher, J., & Banbury, M. (1995, April). Assessing classroom teachers' performance assessments. Paper presented at the annual meeting of the American Educational Research Association, San Francisco.
Heritage, M., Kim, J., Vendlinski, T., & Herman, J. (2009). From evidence to action: A seamless process in formative assessment? Educational Measurement: Issues and Practice, (28)3, 24-31. https://doi.org/10.1111/j.1745-3992.2009.00151.x DOI: https://doi.org/10.1111/j.1745-3992.2009.00151.x
Huynh, H., Meyer, J. P., & Barton, K. (2000). Technical documentation for the 1999 Palmetto Achievement Challenge Tests of English Language Arts and Mathematics, grades 3 - 8. Columbia, South Carolina: University of South Carolina, South Carolina Department of Education.
Kane, M. T. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement: Fourth edition. (pp. 17-64). Westport, CT: Praeger Publishers.
Linacre, J. M. (2003). A user's guide to WINSTEPS MINISTEP Rasch-model computer program. Author.
Llosa, L. (2005). Assessing English learners' language proficiency: A Qualitative Investigation of Teachers' Interpretations of the California ELD standards. The CATSOEL Journal, 17(1), 7-18. https://doi.org/10.5070/B5.36337 DOI: https://doi.org/10.5070/B5.36337
Marso, R. N., & Pigge, F. L. (1993). Teachers' testing knowledge, skills, and practices. In S. L. Wise (Ed.), Teacher training in measurement and assessment skills (pp. 129-185).
Lincoln, NE: Buros Institute of Mental Measurements, University of Nebraska-Lincoln.
McCoach, D. B. (2010). Hierarchical linear modeling. In G. R. Hancock & R. O. Mueller (Eds.), The reviewer's guide to quantitative methods in the social sciences (pp.123-140). New York: Routledge.
Meisels, S., Atkins-Burnett, S., Xue, Y., Nicholson, J., Bickel, D. D., & Son, S. (2003). Creating a system of accountability: The impact of instructional assessment on elementary children's achievement scores. Educational Policy Analysis Archives, 11(9), Retrieved March 23, 2007, from http://epaa.asu.edu/epaa/v11n9/ https://doi.org/10.14507/epaa.v11n9.2003 DOI: https://doi.org/10.14507/epaa.v11n9.2003
Nasstrom, G. (2009). Interpretations of standards with Bloom's revised taxonomy: A comparison of teachers and assessment experts. International Journal of Research & Method in Education, 32(1), 39-51. https://doi.org/10.1080/17437270902749262 DOI: https://doi.org/10.1080/17437270902749262
Newmann, F., Bryk, A. S., & Nagaoka, J.K. (2001). Authentic intellectual work and standardized tests: Conflict or coexistence? Retrieved June 27, 2007, fromhttp://ccsr.uchicago.edu/publications/p0a02.pdf
Nichols, P. D., Meyers, J. L., & Burling, K. S. (2009). A framework for evaluating and planning assessments intended to improve student achievement. Educational Measurement: Issues and Practice (28)3, 14-33. https://doi.org/10.1111/j.1745-3992.2009.00150.x DOI: https://doi.org/10.1111/j.1745-3992.2009.00150.x
Oescher, J., & Kirby, P. C. (1990). Assessing teacher-made tests in secondary math and science classrooms. Paper presented at the annual meeting of the National Council on Measurement in Education, Boston. (ERIC Document Number ED 322 169).
Plake, B. S., & Impara, J. C. (1997). Teacher assessment literacy: What do teachers know about assessment? In G. D. Phye (Ed.), Handbook of classroom assessment, learning, adjustment, and achievement. San Diego, CA: Academic Press. https://doi.org/10.1016/B978-012554155-8/50005-3 DOI: https://doi.org/10.1016/B978-012554155-8/50005-3
Ragland, S., Schneider, M. C., Yap, C. C., & Kaliski, P. K. (2008, April). The effect of classroom assessment professional development on English language arts and mathematics student achievement: Year 2 results. Paper presented at the 2008 meeting of the National Council on Measurement in Education, New York, NY.
Ross, J. A., Hogaboam-Gray, A., & Rolheiser, C. (2002). Student self evaluation in grade 5-6 mathematics effects on problem solving achievement. Educational Assessment, 8(1), 43-59. https://doi.org/10.1207/S15326977EA0801_03 DOI: https://doi.org/10.1207/S15326977EA0801_03
Schneider, M. C., Meyer, J. P., Miller, B. J., & Kaliski, P. K. (2007, April). The effect of classroom assessment professional development on English language arts and mathematics achievement. Paper presented at the 2007 meeting of the National Council on Measurement in Education, Chicago, IL.
Schneider, M. C., & Randel, B. (2010). Research on characteristics of effective professional development programs for enhancing educators' skills in formative assessment. In H. Andrade & G. Cizek (Eds.), Handbook of formative assessment (pp. 251-276). New York: Routledge.
Shepard, L. A. (2009). Commentary: Evaluating the validity of formative and interim assessment. Educational Measurement: Issues and Practice (28)3, 32-37. https://doi.org/10.1111/j.1745-3992.2009.00152.x DOI: https://doi.org/10.1111/j.1745-3992.2009.00152.x
Sobolewski, K. B. (2002). Gender equity in classroom questioning. Unpublished doctoral dissertation, South Carolina State University, Orangeburg.
Stout, W. F. (1987). A nonparametric approach for assessing latent trait dimensionality. Psychometrika, 52, 589-617. https://doi.org/10.1007/BF02294821 DOI: https://doi.org/10.1007/BF02294821
Webb, N. M., Herman, J. L., & Webb, N. L. (2007). Alignment of mathematics' state-level standards and assessments: The role of reviewer agreement. Educational Measurement: Issues and Practice, 26, 17-29. https://doi.org/10.1111/j.1745-3992.2007.00091.x DOI: https://doi.org/10.1111/j.1745-3992.2007.00091.x
Wiliam, D., Lee, C., Harrison, C. & Black, P. (2004). Teachers developing assessment for learning: Impact on student achievement. Assessment in Education, 11(1), 49-65. https://doi.org/10.1080/0969594042000208994 DOI: https://doi.org/10.1080/0969594042000208994
Wolfe, E. W., Viger, S. G., Jarvinen, D. E., & Linksman, J. (2007). Validation of scores from a measure of teachers' efficacy toward standards-aligned classroom assessment. Educational and Psychological Measurement, 67(3), 460-474. https://doi.org/10.1177/0013164406292091 DOI: https://doi.org/10.1177/0013164406292091
Yap, C. C., Whittaker, L., Liao, C., & D'Amico, L. (2006). Evaluation of a professional development program in classroom assessment: 2005-06. Columbia, SC: University of South Carolina, Office of Program Evaluation.
Yap, C. C., Pearsall, T., Morgan, G., Wu, M., Maganda, F., Gilmore, J., Lewis, A., Halladay, K., & D'Amico, L. (2007). Evaluation of a professional development program in classroom assessment: 2006-07. Columbia, SC: University of South Carolina, Office of Program Evaluation.