Adaptation and Use of a Five-Task Model for Evaluability Assessment
Main Article Content
Abstract
Background: Over four decades have passed since the concept of evaluability was introduced; however, the availability and accessibility of methodology, frameworks, checklists and other guidance on evaluability assessments remains limited (Smith, 2005). Evaluators who wish to conduct an evaluability assessment must adopt one of few existing models or operate without guidance. This case provides an example of one model and how it was utilized to conduct an evaluability assessment of an ongoing program intervention.
Purpose: This article provides a real-world example of how an evaluability assessment was conducted using a five-task model from the criminal justice field.
Setting: A criminal justice program operating with at-risk adolescent youth and police officers who patrol their neighborhoods.
Intervention: NA
Research Design: NA
Data Collection and Analysis: The original evaluability assessment collected data through interviews with program personnel, observations by the evaluability researchers, and analysis of survey data provided by the program (i.e., program participants’ responses to survey questions about the program).
Findings: The evaluability assessment conducted in this case study example identified multiple changes necessary for the program to be evaluable. Technical assistance was provided in order to support the efforts of the program to prepare for evaluation, though, the evaluability assessment identified risks and costs for the program to consider prior to proceeding. The findings were developed through a five-task process, which evaluators may be able to adapt for use with other types of programs undergoing evaluability assessment.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
Funding data
-
Office of Juvenile Justice and Delinquency Prevention
Grant numbers 2013-PB-FX-0004
References
Davies, R. (2013). Planning Evaluability Assessments: A Synthesis of the Literature with Recommendations. Report of a Study Commissioned by the Department for International Development. Retrieved on February 22, 2016 from https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/248656/wp40-planning-eval-assessments.pdf
Hopkins, N. (1994). School pupils' perceptions of the police that visit schools: Not all police are pigs. Journal of Community and Applied Psychology, 4, 189-207. https://doi.org/10.1002/casp.2450040306 DOI: https://doi.org/10.1002/casp.2450040306
International Development Research Centre. (1996). Guidelines for Preparing an Evaluation Assessment. Retrieved February 22, 2016 from http://web.idrc.ca/en/ev-29113-201-1-DO_TOPIC.html
Kaufman-Levy, D. & Poulin, M. (2003). Evaluability Assessment: Examining the Readiness of a Program for Evaluation. Washington, DC: Justice Research and Statistics Association. Retrieved from http://www.jrsa.org/pubs/juv-justice/evaluability-assessment.pdf
LaMotte, V., Ouellette, K., Sanderson, J., Anderson, S.A., Kosutic, I., Griggs, J., et al. (2010). Effective police interactions with youth: A program evaluation. Police Quarterly, 13, 161-179. https://doi.org/10.1177/1098611110365689 DOI: https://doi.org/10.1177/1098611110365689
Leiber, M., Nalla, M. & Farnworth, M. (1998). Explaining juveniles' attitudes toward the police. Justice Quarterly, 15, 151-174. https://doi.org/10.1080/07418829800093671 DOI: https://doi.org/10.1080/07418829800093671
McEvoy, K. & Newburn, T. (Eds.) (2014). Criminology, Conflict Resolution and Restorative Justice. London: Palgrave Macmillan.
Menkel-Meadow, C. (2014). Alternative and appropriate dispute resolution in context. In Coleman, P.T., Deutsch, M., & Marcus, E.C. (Eds.) Handbook of Conflict Resolution (3rd Ed.) (pp.971-997). Hoboken, NJ: John Wiley & Sons, Incorporated.
Nihart, T., Lersch, K., Sellers, C. & Mieczkowski, T. (2005). Kids, cops, parents, and teachers: Exploring juvenile attitudes toward authority figures. Western Criminology Review, 6, 79-88.
Office of Justice Programs. (n.d.). OJJDP Model Programs Guide [website]. Available at http://www.ojjdp.gov/mpg/
Peersman, G., Guijt, I., and Pasanen, T. (2015). Evaluability Assessment for Impact Evaluation. A Methods Lab publication. London: Overseas Development Institute.
Smith, M.F. (1989). Evaluability Assessment: A practical approach. New York: Springer. https://doi.org/10.1007/978-94-015-7827-1 DOI: https://doi.org/10.1007/978-94-015-7827-1
Smith, M. F. (2005). Evaluability assessment. In Mathison, S. (Ed.), Encyclopedia of Evaluation (pp. 136-139). Thousand Oaks, CA: Sage.
Staub, E. (2014). Reconciliation between Groups In Coleman, P.T., Deutsch, M., & Marcus, E.C. (Eds.) Handbook of Conflict Resolution (3rd Ed.) (pp.971-997). Hoboken, NJ: John Wiley & Sons, Incorporated.
The Evaluation Center. (n.d.). Evaluation Checklists [website]. Available at http://www.wmich.edu/evaluation/checklists
Trevisan, M.S. & Huang, Y.M. (2003). Evaluability assessment: a primer. Practical Assessment, Research & Evaluation, 8(20). Available from http://PAREonline.net/getvn.asp?v=8&n=20
Tyler, T.R. (1990). Why People Obey the Law. New Haven: Yale University Press.
Tyler, T.R. (2004). Enhancing police legitimacy. The Annals of the American Academy of Political and Social Science, 593, 84-99. https://doi.org/10.1177/0002716203262627 DOI: https://doi.org/10.1177/0002716203262627
Tyler, T.R. & Fagan, J. (2008). Legitimacy and cooperation: Why do people help the police fight crime in their communities? Ohio State Journal of Crime, 231-275.
UN Women. (2015). How to Manage Gender-Responsive Evaluation: Evaluation Handbook. New York: United Nations Women Independent Evaluation Office.
Welsh, W.N., Harris, P.W., & Jenkins, P.H. (1996). Reducing overrepresentation of minorities in juvenile justice: Development of community-based programs in Pennsylvania. Crime and Delinquency, 42(1), 76-98. https://doi.org/10.1177/0011128796042001005 DOI: https://doi.org/10.1177/0011128796042001005
Wholey, J.S. (1979). Evaluation: Promise and performance. Washington, DC: The Urban Institute.
Wholey, J.S. (2004). Evaluability assessment. In Wholey, J.S., Hatry, H.P., Newcomer, K.E. (Eds.), Handbook of Practical Program Evaluation (2nd ed.) (pp.33-62). San Francisco: Jossey Bass.
Zandniapour, L. (2014). Impact Evaluability Assessment Tool. Washington, DC: Corporation for National and Community Service. Retrieved February 22, 2016 from http://www.nationalservice.gov/sites/default/files/resource/SIF_Impact_Evaluability_Assessment_Tool_Final_Draft_for_Distribution.pdf