A Review of the DAC Evaluation Criteria: Upgrading Efficiency

Main Article Content

Joaquín de la Concha González

Abstract

Background: Efficiency has remained for long time the Cinderella of evaluation practice. According to several meta-evaluations only a small number of evaluations include robust and meaningful assessments in this field.


Purpose: Following the review of the Development Assistance Committee’s (DAC) evaluation criteria this article reviews the main causes of efficiency remaining in a secondary role in evaluation practices and proposes: 1) upgrading it through broadening the current economic definition of efficiency, and 2) proposing a set of four evaluation dimensions revolving around the principle of sound financial management, social perceived value, result-based budgeting and interconnectedness with other criteria.


Setting: Not applicable.


Intervention: Not applicable.


Research design: Literature review and empirical pilot testing of proposed evaluation methodology.


Data collection & analysis: Qualitative analysis.


Findings: This paper calls for a shift in focus from efficiency to a broader enriched principle of sound financial management including a strong partner-based focus and highlighting management as a pre- condition “sine qua non” for an intervention to be efficient. Efficiency assessments would then be twofold: managerial (procedures, policies and practices that lay out the requirements for efficiency to take place) and substantial (context, circumstances and reasoning for the existing budget balance between results and target groups). The key aspects of the proposal involve four proposed dimensions to be analysed under this renewed criteria that imply audits and evaluations come much closer and look at each other.

Downloads

Download data is not yet available.

Article Details

How to Cite
de la Concha González, J. (2020). A Review of the DAC Evaluation Criteria: Upgrading Efficiency. Journal of MultiDisciplinary Evaluation, 16(34), 44–55. https://doi.org/10.56645/jmde.v16i34.587
Section
Research on Evaluation Articles

References

Bennett, A. (2010). Process Tracing and Causal Inference. In Rethinking Social Inquiry: Diverse Tools Shared Standards, 2nd edition, ed. Henry E. Brady and David Collier, 207-220. Lanham: Rowman and Littlefield Publishers Inc. DOI: https://doi.org/10.5040/9798216385530.ch-010

Committee of Sponsoring Organisations of the Treadway Commission (2013). Internal Control - Integrated Framework. American Institute of CPAs.

Council of Europe (2005). Result based budgeting manual. Version 3.1. Retrieved from: https://www.coe.int/t/budgetcommittee/Source/RBB_SEMINAR/RBB_Manual_en.pdf

Cugelman, B. and Otero, E. (2010). Basic Efficiency Resource: A framework for measuring the relative performance of multi-unit programs. Leitmotiv consultoria social. Retrieved from: http://www.leitmotivsocial.com/Resources/BER%20whitepaper_v23.pdf

D'Emidio, F.; Wallace, T.; Henon, S; Buckles, D. (2017). Value for money in ActionAid: creating an alternative. Action Aid. Retrieved from: https://www.actionaid.org.uk/sites/default/files/publications/actionaid_-_value_for_money_-_creating_an_alternative.pdf

De Greve, P. In theory, practice is simple: A few reflections on efficiency analysis. The Spindle. Retrieved from: https://thespindle.org/wp-content/uploads/2019/01/Pol-de-Greve-In-theory-practice-is-simple.pdf

De Kemp, A. (2017). Efficiency analysis: Balancing between rigour and realism. The Spindle. Retrieved from: https://thespindle.org/wp-content/uploads/2019/01/Antonie-de-Kemp-Balancing-between-rigour-and-realism-1.pdf

Duflo, E., Glennerster, R., & Kremer, M. (2007). Using randomization in development economics research: A toolkit. Handbook of development economics, 4, 3895-3962. Retrieved from: http://www.povertyactionlab.org/sites/default/files/documents/Using%20Randomization%20in%20Development%20Economics.pdf https://doi.org/10.1016/S1573-4471(07)04061-2 DOI: https://doi.org/10.1016/S1573-4471(07)04061-2

Earl, S.; Carden, F. and Smutylo, T. (2001). Outcome mapping: building learning and reflection into development programs. Ottawa: International Development Research Centre. Retrieved from: https://idl-bnc-idrc.dspacedirect.org/bitstream/handle/10625/32122/IDL-32122.pdf?sequence=1&isAllowed=y

European Commission (2006). Study on the Use of Cost-Effectiveness Analysis in EC's Evaluations. Lyon: Centre for European Expertise and Evaluation. Retrieved from: https://ec.europa.eu/smart-regulation/evaluation/docs/cea_finalreport_en.pdf

European Court of Auditors (2017). Performance Audit Manual. Directorate of Audit Quality Control Committee (DQC). Retrieved from: https://www.eca.europa.eu/Lists/ECADocuments/PERF_AUDIT_MANUAL/PERF_AUDIT_MANUAL_EN.PDF

European Union (2012). Regulation (EU, EURATOM) no 966/2012 of the European Parliament and of the Council of 25 October 2012 on the financial rules applicable to the general budget of the Union and repealing Council Regulation (EC, Euratom) No 1605/2002. Official Journal of the European Union L 298/1. Retrieved from: https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:32012R0966&from=EN#d1e4140-1-1

Hedblom; Hildenwall (2008). Strengthening the Results-Orientation in Sweden's Engagement in Multilateral Development Cooperation: An Evaluation of UNDP's Country Level Evaluation Activities. Karlstad: Swedish Agency for Development Evaluation.

Heider, C. (2017). Rethinking Evaluation - Efficiency, Efficiency, Efficiency [Blog]. Retrieved from http://ieg.worldbankgroup.org/blog/rethinking-evaluation-efficiency

Independent Commission for Aid Impact (2018). DFID's approach to value for money in programme and portfolio management: a performance review. Retrieved from: https://icai.independent.gov.uk/wp-content/uploads/ICAI-VFM-report.pdf

International Organisation of Supreme Audit Institutions (2016). Guidelines on Central Concepts for Performance Auditing. International Standards of Supreme Audit Institutions 3100. Vienna: INTOSAI General Secretariat. Retrieved from: http://www.issai.org/issai-framework/4-auditing-guidelines.htm

Mayne, J. (2008) Contribution Analysis: An approach to exploring cause and effect, ILAC methodological brief, Retrieved from: https://hdl.handle.net/10568/70124

OECD (2002). Glossary of Key Terms in Evaluation and Results Based Management. Paris: OECD. Retrieved from: http://www.oecd.org/dac/evaluation/2754804.pdf

OECD (2008) "The Accra Agenda for Action", endorsed at the 3rd High-Level Forum on Aid Effectiveness, Accra. Retrieved from: http://www.oecd.org/dac/effectiveness/45827311.pdf

OECD (2011). The Busan Partnership for effective Development cooperation. Fourth High-Level Forum on Aid Effectiveness, Busan. Retrieved from: http://www.oecd.org/development/effectiveness/busanpartnership.htm

OECD (2018). OECD DAC Evaluation Criteria: Summary of consultation responses. Retrieved from: https://oecd.us15.list-manage.com/track/click?u=e0b8510009af73b60000d11e8&id=79365f511d&e=4c10394d2f

OECD DAC (2010). Evaluating Development Co-Operation: Summary of Key Norms and Standards. Paris: OECD. Retrieved from: http://www.oecd.org/development/evaluation/dcdndep/41612905.pdf

OECD/UNDP (2013). Guide to the Monitoring Framework of the Global Partnership. Global Partnership for Effective Development Co-operation. Retrieved from: http://effectivecooperation.org/wp-content/uploads/2016/08/Guide-to-the-GP-Monitoring-Framework.pdf

OECD/UNDP (2016). Making development cooperation more effective: 2016 progress report. Paris: OECD. Retrieved from: https://doi.org/10.1787/9789264266261-en DOI: https://doi.org/10.1787/9789264266261-en

Palenberg, M. (2011). Tools and Methods for Evaluating the Efficiency of Development Interventions. Evaluation Working Papers. Bonn: Bundesministerium für wirtschaftliche Zusammenarbeit und Entwicklung. Retrieved from: http://www.managingforimpact.org/sites/default/files/resource/bmz_wp_tools_methods_evaluating_efficiency.pdf

United Nations (1998). Guide to results based budgeting. Version 1.1.

United Nations (2005). Concept Paper on Results Based Budgeting (RBB) in the Organizations of the United Nations System. Chief Executives Board for Co-ordination, CEB/2005/HLCM/R.6.

United Nations (2016). The Sustainable Development Goals Report. New York. Retrieved from: https://unstats.un.org/sdgs/report/2016/#

Weiss (1998). Evaluation: Methods for Studying Programs and Policies. Upper Saddle River, N.J.: Prentice Hall.

Wilson-Grau, R. and Britt, H. (2012). Outcome Harvesting. Cairo: Ford Foundation. Retrieved from: http://outcomeharvesting.net/outcome-harvesting-brief/

World Bank, Independent Evaluation group (2010). Cost-Benefit Analysis in World Bank Projects. Washington: World Bank. Retrieved from: http://ieg.worldbankgroup.org/sites/default/files/Data/Evaluation/files/cba_full_report1.pdf

World Commission on Environment and Development (1987). Our common Future. Report of the World Commission on Environment and Development. G. H. Brundtland. Retrieved from: https://sustainabledevelopment.un.org/content/documents/5987our-common-future.pdf