Research on Evaluation Outside Journal Publications: An Analysis of Proposals Accepted for the 2019 American Evaluation Association Annual Conference
Article Sidebar
Main Article Content
Abstract
Many evaluators may be conducting and presenting research on evaluation (RoE) without realizing it. Proposals accepted to present at the AEA 2019 conference were analyzed for whether or not they were RoE (i.e., systematic, empirical, and focused on evaluation). RoE proposals were then coded using a framework by Mark (2008) to determine the common trends across those identified as RoE. Non-RoE proposals were coded for why they were not RoE, including that they were not systematic, empirical, and/or focused on evaluation. A total of 15% of the 732 proposals analyzed were coded as RoE; most proposals examined evaluation activities using the descriptive mode of inquiry. Furthermore, most non-RoE proposals were coded as not RoE because they presented an evaluation rather than a research study on evaluation or because they were reflections on evaluation. This study presents multiple opportunities and future directions for RoE.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
American Evaluation Association. (2018a). American Evaluation Association 2018 member survey report. https://www.eval.org/Portals/0/Docs/2018%20AEA%20Survey%20Report%20Final%20(002).pdf
American Evaluation Association. (2018b). Guiding principles for evaluators. https://www.eval.org/About/Guiding-Principles
American Evaluation Association. (2018c). AEA evaluator competencies. https://www.eval.org/About/Competencies-Standards/AEA-Evaluator-Competencies
Brandon, P. R. (2015), Editor’s notes. New Directions for Evaluation, 148, 3–6. https://doi.org/10.1002/ev.20152 DOI: https://doi.org/10.1002/ev.20152
Brandon, P. R., & Fukunaga, L. L. (2014). The state of the empirical research literature on stakeholder involvement in program evaluation. American Journal of Evaluation, 35(1), 26–44. https://doi.org/10.1177/1098214013503699 DOI: https://doi.org/10.1177/1098214013503699
Christie, C. A. (2003). What guides evaluation? A study of how evaluation practice maps onto evaluation theory. New Directions for Evaluation, 97, 7–36. https://doi.org/10/fwg4xq DOI: https://doi.org/10.1002/ev.72
Coryn, C. L. S., Ozeki, S., Wilson, L. N., Greenman, G. D., Schröter, D. C., Hobson, K. A., Azzam, T., & Vo, A. T. (2016). Does research on evaluation matter? Findings from a survey of American Evaluation Association members and prominent evaluation theorists and scholars. American Journal of Evaluation, 37(2), 159–173. https://doi.org/10/f8qqvk DOI: https://doi.org/10.1177/1098214015611245
Coryn, C. L. S., Wilson, L. N., Westine, C. D., Hobson, K. A., Ozeki, S., Fiekowsky, E. L., Greenman, G. D., & Schröter, D. C. (2017). A decade of research on evaluation: A systematic review of research on evaluation published between 2005 and 2014. American Journal of Evaluation, 38(3), 329–347. https://doi.org/10/ggf5td DOI: https://doi.org/10.1177/1098214016688556
Henry, G. T., & Mark, M. M. (2003). Toward an agenda for research on evaluation. New Directions for Evaluation, 97, 69–80. https://doi.org/10/b5d6xg DOI: https://doi.org/10.1002/ev.77
Lincoln, Y. S., & Guba, E. G. (2004). The roots of fourth generation evaluation. In M. C. Alkin (Ed.), Evaluation roots: Tracing theorists’ views and influences (pp. 225–241). Sage.
Linnell, D. (2020, April 23). What is research on evaluation (RoE)? [Blog post]. https://www.danalinnell.com/blog/whatisroe
Linnell, D. (2023, July 30). Research, evaluation, research ON evaluation?! AEA365. https://aea365.org/blog/research-evaluation-research-on-evaluation-by-dana-linnell/
Mark, M. M. (2008). Building a better evidence base for evaluation theory: Beyond general calls to a framework of types of research on evaluation. In N. L. Smith & P. R. Brandon (Eds.), Fundamental issues in evaluation (pp. 111–134). Guilford.
Mata, H., Latham, T. P., & Ransome, Y. (2010). Benefits of professional organization membership and participation in national conferences: Considerations for students and new professionals. Health Promotion Practice, 11(4), 450–453. https://doi.org/10.1177/1524839910370427 DOI: https://doi.org/10.1177/1524839910370427
Nolton, E. C., Harnar, M., Moore, T. R., Mahato, S., & Wanzer, D. L. (2022, November 11). Exploring RoE across TIGs: RoE within accepted 2019 AEA conference proposals. In M. Harnar & M. M. Mark (Chairs), (Re)shaping our understanding of research on evaluation (RoE) [Multipaper presentation]. American Evaluation Association 2022 Annual Conference, New Orleans, LA, United States.
Szanyi, M., Azzam, T., & Galen, M. (2013). Research on evaluation: A needs assessment. The Canadian Journal of Program Evaluation, 27(1), 39–64. https://doi.org/10.3138/cjpe.027.002 DOI: https://doi.org/10.3138/cjpe.027.002
Tseng, V. (2012). The uses of research in policy and practice. Social Policy Report, 26(2). https://doi.org/10/ggf6bc DOI: https://doi.org/10.1002/j.2379-3988.2012.tb00071.x
Yarbrough, D. B., Shulha, L. M., Hopson, R. K., & Caruthers, F. A. (2010). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Sage.