A Review of the DAC Evaluation Criteria: Upgrading Efficiency
Main Article Content
Abstract
Background: Efficiency has remained for long time the Cinderella of evaluation practice. According to several meta-evaluations only a small number of evaluations include robust and meaningful assessments in this field.
Purpose: Following the review of the Development Assistance Committee’s (DAC) evaluation criteria this article reviews the main causes of efficiency remaining in a secondary role in evaluation practices and proposes: 1) upgrading it through broadening the current economic definition of efficiency, and 2) proposing a set of four evaluation dimensions revolving around the principle of sound financial management, social perceived value, result-based budgeting and interconnectedness with other criteria.
Setting: Not applicable.
Intervention: Not applicable.
Research design: Literature review and empirical pilot testing of proposed evaluation methodology.
Data collection & analysis: Qualitative analysis.
Findings: This paper calls for a shift in focus from efficiency to a broader enriched principle of sound financial management including a strong partner-based focus and highlighting management as a pre- condition “sine qua non” for an intervention to be efficient. Efficiency assessments would then be twofold: managerial (procedures, policies and practices that lay out the requirements for efficiency to take place) and substantial (context, circumstances and reasoning for the existing budget balance between results and target groups). The key aspects of the proposal involve four proposed dimensions to be analysed under this renewed criteria that imply audits and evaluations come much closer and look at each other.
Downloads
Article Details

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
References
Bennett, A. (2010). Process Tracing and Causal Inference. In Rethinking Social Inquiry: Diverse Tools Shared Standards, 2nd edition, ed. Henry E. Brady and David Collier, 207-220. Lanham: Rowman and Littlefield Publishers Inc. DOI: https://doi.org/10.5040/9798216385530.ch-010
Committee of Sponsoring Organisations of the Treadway Commission (2013). Internal Control - Integrated Framework. American Institute of CPAs.
Council of Europe (2005). Result based budgeting manual. Version 3.1. Retrieved from: https://www.coe.int/t/budgetcommittee/Source/RBB_SEMINAR/RBB_Manual_en.pdf
Cugelman, B. and Otero, E. (2010). Basic Efficiency Resource: A framework for measuring the relative performance of multi-unit programs. Leitmotiv consultoria social. Retrieved from: http://www.leitmotivsocial.com/Resources/BER%20whitepaper_v23.pdf
D'Emidio, F.; Wallace, T.; Henon, S; Buckles, D. (2017). Value for money in ActionAid: creating an alternative. Action Aid. Retrieved from: https://www.actionaid.org.uk/sites/default/files/publications/actionaid_-_value_for_money_-_creating_an_alternative.pdf
De Greve, P. In theory, practice is simple: A few reflections on efficiency analysis. The Spindle. Retrieved from: https://thespindle.org/wp-content/uploads/2019/01/Pol-de-Greve-In-theory-practice-is-simple.pdf
De Kemp, A. (2017). Efficiency analysis: Balancing between rigour and realism. The Spindle. Retrieved from: https://thespindle.org/wp-content/uploads/2019/01/Antonie-de-Kemp-Balancing-between-rigour-and-realism-1.pdf
Duflo, E., Glennerster, R., & Kremer, M. (2007). Using randomization in development economics research: A toolkit. Handbook of development economics, 4, 3895-3962. Retrieved from: http://www.povertyactionlab.org/sites/default/files/documents/Using%20Randomization%20in%20Development%20Economics.pdf https://doi.org/10.1016/S1573-4471(07)04061-2 DOI: https://doi.org/10.1016/S1573-4471(07)04061-2
Earl, S.; Carden, F. and Smutylo, T. (2001). Outcome mapping: building learning and reflection into development programs. Ottawa: International Development Research Centre. Retrieved from: https://idl-bnc-idrc.dspacedirect.org/bitstream/handle/10625/32122/IDL-32122.pdf?sequence=1&isAllowed=y
European Commission (2006). Study on the Use of Cost-Effectiveness Analysis in EC's Evaluations. Lyon: Centre for European Expertise and Evaluation. Retrieved from: https://ec.europa.eu/smart-regulation/evaluation/docs/cea_finalreport_en.pdf
European Court of Auditors (2017). Performance Audit Manual. Directorate of Audit Quality Control Committee (DQC). Retrieved from: https://www.eca.europa.eu/Lists/ECADocuments/PERF_AUDIT_MANUAL/PERF_AUDIT_MANUAL_EN.PDF
European Union (2012). Regulation (EU, EURATOM) no 966/2012 of the European Parliament and of the Council of 25 October 2012 on the financial rules applicable to the general budget of the Union and repealing Council Regulation (EC, Euratom) No 1605/2002. Official Journal of the European Union L 298/1. Retrieved from: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32012R0966&from=EN#d1e4140-1-1
Hedblom; Hildenwall (2008). Strengthening the Results-Orientation in Sweden's Engagement in Multilateral Development Cooperation: An Evaluation of UNDP's Country Level Evaluation Activities. Karlstad: Swedish Agency for Development Evaluation.
Heider, C. (2017). Rethinking Evaluation - Efficiency, Efficiency, Efficiency [Blog]. Retrieved from http://ieg.worldbankgroup.org/blog/rethinking-evaluation-efficiency
Independent Commission for Aid Impact (2018). DFID's approach to value for money in programme and portfolio management: a performance review. Retrieved from: https://icai.independent.gov.uk/wp-content/uploads/ICAI-VFM-report.pdf
International Organisation of Supreme Audit Institutions (2016). Guidelines on Central Concepts for Performance Auditing. International Standards of Supreme Audit Institutions 3100. Vienna: INTOSAI General Secretariat. Retrieved from: http://www.issai.org/issai-framework/4-auditing-guidelines.htm
Mayne, J. (2008) Contribution Analysis: An approach to exploring cause and effect, ILAC methodological brief, Retrieved from: https://hdl.handle.net/10568/70124
OECD (2002). Glossary of Key Terms in Evaluation and Results Based Management. Paris: OECD. Retrieved from: http://www.oecd.org/dac/evaluation/2754804.pdf
OECD (2008) "The Accra Agenda for Action", endorsed at the 3rd High-Level Forum on Aid Effectiveness, Accra. Retrieved from: http://www.oecd.org/dac/effectiveness/45827311.pdf
OECD (2011). The Busan Partnership for effective Development cooperation. Fourth High-Level Forum on Aid Effectiveness, Busan. Retrieved from: http://www.oecd.org/development/effectiveness/busanpartnership.htm
OECD (2018). OECD DAC Evaluation Criteria: Summary of consultation responses. Retrieved from: https://oecd.us15.list-manage.com/track/click?u=e0b8510009af73b60000d11e8&id=79365f511d&e=4c10394d2f
OECD DAC (2010). Evaluating Development Co-Operation: Summary of Key Norms and Standards. Paris: OECD. Retrieved from: http://www.oecd.org/development/evaluation/dcdndep/41612905.pdf
OECD/UNDP (2013). Guide to the Monitoring Framework of the Global Partnership. Global Partnership for Effective Development Co-operation. Retrieved from: http://effectivecooperation.org/wp-content/uploads/2016/08/Guide-to-the-GP-Monitoring-Framework.pdf
OECD/UNDP (2016). Making development cooperation more effective: 2016 progress report. Paris: OECD. Retrieved from: https://doi.org/10.1787/9789264266261-en DOI: https://doi.org/10.1787/9789264266261-en
Palenberg, M. (2011). Tools and Methods for Evaluating the Efficiency of Development Interventions. Evaluation Working Papers. Bonn: Bundesministerium für wirtschaftliche Zusammenarbeit und Entwicklung. Retrieved from: http://www.managingforimpact.org/sites/default/files/resource/bmz_wp_tools_methods_evaluating_efficiency.pdf
United Nations (1998). Guide to results based budgeting. Version 1.1.
United Nations (2005). Concept Paper on Results Based Budgeting (RBB) in the Organizations of the United Nations System. Chief Executives Board for Co-ordination, CEB/2005/HLCM/R.6.
United Nations (2016). The Sustainable Development Goals Report. New York. Retrieved from: https://unstats.un.org/sdgs/report/2016/#
Weiss (1998). Evaluation: Methods for Studying Programs and Policies. Upper Saddle River, N.J.: Prentice Hall.
Wilson-Grau, R. and Britt, H. (2012). Outcome Harvesting. Cairo: Ford Foundation. Retrieved from: http://outcomeharvesting.net/outcome-harvesting-brief/
World Bank, Independent Evaluation group (2010). Cost-Benefit Analysis in World Bank Projects. Washington: World Bank. Retrieved from: http://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/cba_full_report1.pdf
World Commission on Environment and Development (1987). Our common Future. Report of the World Commission on Environment and Development. G. H. Brundtland. Retrieved from: https://sustainabledevelopment.un.org/content/documents/5987our-common-future.pdf