Michael Scriven and “The Missing Half of Quantitative Evaluation”

Article Sidebar

cover page
PDF
Published: Jun 21, 2024
DOI: https://doi.org/10.56645/jmde.v20i47.1007
Keywords:
costs, cost analysis, cost-inclusive evaluation (CIE), Cost-benefit analysis, Cost-Effectiveness Analysis (CEA), economic evaluation, Michael Scriven

Main Article Content

Brian Yates
Department of Psychology, American University, Washington, DC United States
https://orcid.org/0000-0003-0555-6393
Nadini Persaud
Sagicor Cave Hill School of Business and Management, The University of the West Indies, Cave Hill Campus Barbados, W.I.
https://orcid.org/0000-0003-1827-2867

Abstract

Although Michael Scriven is one of the most recognized parents of program evaluation in general, it may be surprising to learn that he also was a founder of cost-inclusive evaluation (CIE) in particular. CIE incorporates into evaluations information on the amounts, types, and monetary values of resources used by programs to achieve outcomes, and, increasingly, the monetary value of those outcomes (Herman et al., 2009; Persaud & Yates, 2023; Yates, 1996). Michael Scriven blazed a trail toward routine use of cost, cost-effectiveness, and cost-benefit analyses by emphasizing that evaluations should measure costs as well as nonmonetary and monetary outcomes, and should compare costs and outcomes for alternative programs in cost-effectiveness and cost-benefit analyses. Scriven also was emphatic that CIE be recognized as a core evaluator competency. Noting that “cost analysis” in evaluation was “the missing half of quantitative evaluation” (Scriven, 2008a, p. iii), he did not immediately succeed in convincing most evaluators that program costs, and monetary program outcomes, were essential to assess. However, his publications, his supervision, and his encouragement of junior colleagues legitimized and recognized CIE as nothing else could.

Downloads

Download data is not yet available.

Article Details

How to Cite
Yates, B., & Persaud, N. (2024). Michael Scriven and “The Missing Half of Quantitative Evaluation”. Journal of MultiDisciplinary Evaluation, 20(47), 24–27. https://doi.org/10.56645/jmde.v20i47.1007
Issue
Vol. 20 No. 47 (2024): Special Issue: Remembering Michael Scriven
Section
In Memoriam
Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright and Permissions

Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org

 

Author Biographies

Brian Yates, Department of Psychology, American University, Washington, DC United States

Brian Yates is a tenured Professor at American University, Washington DC. Most of his 104 publications, including 6 books, apply cost-effectiveness or cost-benefit analyses to formative evaluation of health and human services, including as youth mentoring and parent training initiatives, prevention and treatment of substance abuse, computer-based cognitive-behavioral therapy for alcohol abuse, suicide prevention, and treatments for obesity, depression, and seasonal affective disorder. Dr. Yates also teaches online asynchronous courses in self-management, personality theories, and health psychology, having also developed and taught traditional-format courses in community psychology, behavioral medicine, advanced theories of personality, research methods, and program evaluation.

Nadini Persaud, Sagicor Cave Hill School of Business and Management, The University of the West Indies, Cave Hill Campus Barbados, W.I.

Dr. Nadini Persaud Lecturer in Project Management and Evaluation

Dr. Nadini Persaud is a Lecturer in Project Management, Project Evaluation, and Accounting in the Sagicor Cave Hill School of Business and Management, and Deputy Dean, Research and Innovation and Entrepreneurship. She is the Programme Co-ordinator for MSc. Project Management and Evaluation, and was instrumental in restructuring the programme to meet the certification criteria of the USA based Project Management Institute (PMI).

Dr. Persaud brings a wealth of project management experience to the classroom having worked as a Project Officer with the Economic Planning, Project Coordination and Social Policy Development Unit, Ministry of Finance, Planning & Development, Government of St. Vincent and the Grenadines. She also worked on several evaluation projects in the United States whilst doing her doctoral studies in the United States.

Dr. Persaud serves on a number of international boards including Director, Training and Professional Development, Caribbean Evaluators International, Council Member, Chartered Professional Accountants Canada—Barbados Chapter Advisory Council, and the 3F Advisory Committee of the Faster Forward Fund, a private US-based foundation established by the late Professor Michael Scriven (one of the three founding fathers of the discipline of project evaluation) to accelerate the theory, practice, and profession of evaluation, She has also served in several positions on the American Evaluation Association including Program Chair of the Costs, Effectiveness, Benefits and Economics Topical Interest Group, and Chair of the Finance Advisory Board (2014, 2019).

She is the co-author of three books, the most recent being 'Cost-Inclusive Evaluation: Planning It, Doing It, Using It (2023) with Professor Brian Yates of the USA. A citizen of St. Vincent and the Grenadines, Dr. Persaud loves spending time with family. Her hobbies include photography and arts and craft.

References

Coryn, C. L. S., & Scriven, M. (2007). Are national-level research evaluation models valid, credible, useful, cost-effective, and ethical? Journal of MultiDisciplinary Evaluation, 4(8), 92–96. https://doi.org/10.56645/jmde.v4i8.32 DOI: https://doi.org/10.56645/jmde.v4i8.32

Davidson, J. E. (2005). Evaluation methodology basics: The nuts and bolts of sound evaluation. Sage. https://doi.org/10.4135/9781452230115 DOI: https://doi.org/10.4135/9781452230115

Herman, P., Avery, D. J., Schemp, C. S., & Walsh, M. E. (2009). Are cost-inclusive evaluations worth the effort? Evaluation and Program Planning, 32(1), 55–61. https://doi.org/10.1016/j.evalprogplan.2008.08.008 DOI: https://doi.org/10.1016/j.evalprogplan.2008.08.008

Kazdin, A. E. (2003). Research design in clinical psychology (4th ed.). Allyn and Bacon.

King, J. (2018). Expanding theory-based evaluation: Incorporating value creation in a theory of change. Evaluation and Program Planning, 89, 101963. https://doi.org/10.1016/j.evalprogplan.2021.101963 DOI: https://doi.org/10.1016/j.evalprogplan.2021.101963

King, J. (2019). Evaluation and value for money: Development of an approach using explicit evaluative reasoning [Doctoral dissertation, Melbourne Graduate School of Education]. http://hdl.handle.net/11343/225766

Kirkhart, K. E., & Scriven, M. (1989). Conceptualizing evaluation: The Key Evaluation Checklist. In D. M. Mertens (Ed.), Creative ideas for teaching evaluation: Activities, assignments, and resources (pp. 49–71). Springer Dordrecht. https://doi.org/10.1007/978-94-015-7829-5_3 DOI: https://doi.org/10.1007/978-94-015-7829-5_3

Levin, H. M. (1975). Cost-effectiveness analysis in evaluation research. In M. Guttentag & E. L. Struening (Eds.), Handbook of evaluation research (Vol. 2, Chap. 5). Sage.

Levin, H. M., McEwan, P. J., Belfield, C., Bowden, A. B., & Shand, R. (2018). Economic evaluation in education: Cost-effectiveness and benefit-cost analysis (3rd. ed.). Sage. https://doi.org/10.4135/9781483396514 DOI: https://doi.org/10.4135/9781483396514

New Zealand Treasury. (2005). Cost benefit analysis primer. The Treasury.

Persaud, N. (2007). Conceptual and practical analysis of costs and benefits in evaluation: Developing a cost analysis tool for practical program evaluation [Doctoral dissertation, Western Michigan University]. ProQuest. https://www.proquest.com/docview/304815206/

Persaud, N. (2018). A practical framework and model for promoting cost-inclusive evaluation. Journal of MultiDisciplinary Evaluation, 14(30), 88–104. https://doi.org/10.56645/jmde.v14i30.494 DOI: https://doi.org/10.56645/jmde.v14i30.494

Persaud, N., & Yates, B. T. (2023). Cost-inclusive evaluation: Planning it, doing it, using it. Guilford.

Rogers, P. J., Stevens, K., & Boymal, J. (2009). Qualitative cost-benefit evaluation of complex, emergent programs. Evaluation and Program Planning, 32, 83–90. https://doi.org/10.1016/j.evalprogplan.2008.08.005 DOI: https://doi.org/10.1016/j.evalprogplan.2008.08.005

Scriven, M. (1969). An introduction to meta-evaluation. Educational Projects Report, 2, 36–38.

Scriven, M. (1991). Evaluation thesaurus (4th ed.). Sage.

Scriven, M. (1993). Parting perspectives. New Directions for Program Evaluation, 58, 87–92. https://doi.org/10.1002/ev.1646 DOI: https://doi.org/10.1002/ev.1646

Scriven, M. (1996). Types of evaluation and types of evaluator. Evaluation Practice, 17(2), 151–61. https://doi.org/10.1177/109821409601700207 DOI: https://doi.org/10.1016/S0886-1633(96)90020-3

Scriven, M. (2007). Key Evaluation Checklist. https://wmich.edu/sites/default/files/attachments/u350/2014/key%20evaluation%20checklist.pdf

Scriven, M. (2008a). Editorial: Cost analysis in evaluation. Journal of MultiDisciplinary Evaluation, 5(9), iii. https://doi.org/10.56645/jmde.v5i9.176 DOI: https://doi.org/10.56645/jmde.v5i9.176

Scriven, M. (2008b). The economist’s fallacy. Journal of MultiDisciplinary Evaluation, 5(9), 74–76. https://doi.org/10.56645/jmde.v5i9.184 DOI: https://doi.org/10.56645/jmde.v5i9.184

Scriven, M. (2009). Meta-evaluation revised. Journal of MultiDisciplinary Evaluation, 6(11), iii–viii. https://doi.org/10.56645/jmde.v6i11.220 DOI: https://doi.org/10.56645/jmde.v6i11.220

Scriven, M. (2015). Key Evaluation Checklist. https://wmich.edu/sites/default/files/attachments/u1105/2023/kec-scriven.pdf

Scriven, M. (2019). The checklist imperative. New Directions for Evaluation, 163. https://doi.org/10.1002/ev.20374 DOI: https://doi.org/10.1002/ev.20374

Yates, B. T. (1994). Toward the incorporation of costs, cost-effectiveness analysis, and cost-benefit analysis into clinical research. Journal of Consulting and Clinical Psychology, 62, 729–736. https://doi.org/10.1037/0022-006X.62.4.729 DOI: https://doi.org/10.1037/0022-006X.62.4.729

Yates, B. T. (1996). Analyzing costs, procedures, processes, and outcomes in human services: An introduction. Sage Publications. https://doi.org/10.4135/9781412983358 DOI: https://doi.org/10.4135/9781412983358

Yates, B. T. (2012). Step arounds for common pitfalls when valuing resources used versus resources produced. New Directions in Program Evaluation, 133, 43–52. https://doi.org/10.1002/ev.20005 DOI: https://doi.org/10.1002/ev.20005

Yates, B. T. (2020). Research on improving outcomes and reducing costs of psychological interventions: Toward delivering the best to the most for the least. Annual Review of Clinical Psychology, 16, 125–150. https://doi.org/10.1146/annurev-clinpsy-071519-110415 DOI: https://doi.org/10.1146/annurev-clinpsy-071519-110415

Yates, B. T. (2021). Toward collaborative cost-inclusive evaluation: Adaptations and transformations for evaluators and economists. Evaluation and Program Planning, 89, 101993. https://doi.org/10.1016/j.evalprogplan.2021.101993 DOI: https://doi.org/10.1016/j.evalprogplan.2021.101993

Yates, B. T. (2023a). Program evaluation: Outcomes and costs of putting psychology to work. In H. Cooper, M. N. Coutanche, L. M. McMullen, A. T. Panter, D. Rindskopf, & K. J. Sher (Eds.), APA handbook of research methods in psychology: Research designs: Quantitative, qualitative, neuropsychological, and biological (2nd ed., pp. 725–743). American Psychological Association.

Yates, B. T. (2023b). Cost-inclusive research on health psychology interventions: Why, how, and what next. Health Psychology, 42(3), 139–150. https://psycnet.apa.org/doi/10.1037/hea0001280 DOI: https://doi.org/10.1037/hea0001280