The "Usability" of Evaluation Reports: A Precursor to Evaluation Use in Government Organizations

Abstract

Background: According to the Treasury Board of Canada’s Policy on Evaluation (2009), evaluations produced by federal government departments must contribute to decision-making at an organizational level (mainly summative) as well as a program level (mainly formative). Previous research shows that although the formative objectives of evaluation are generally reached, the use of evaluation for broader, budgetary management is limited. However, little research has been conducted thus far on this issue.

Purpose: This study investigates the extent to which program evaluation is used in the Canadian federal government for budgetary management purposes.

Setting: This paper outlines the results obtained following the first component of a two-pronged research strategy focusing on evaluation use in Canadian federal government organizations.

Intervention: N/A

Research Design: Two federal agencies were recruited to participate in organizational case studies aiming to identify the factors that facilitate the use of evaluation for budgetary reallocation exercises.

Data Collection and Analysis: This report presents the findings from a detailed analysis of evaluation reports published by both agencies between 2010-2013. The data were collected from public evaluation reports and analyzed using NVivo.

Findings: The preliminary findings of the study show that instrumental use has occurred or can be expected to occur, based on the types of recommendations outlined in the reports reviewed and on the responses to the evaluations produced by program managers.

Keywords: evaluation use; organizational evaluation capacity; instrumental use; evaluation reports; Canadian federal government; document review; organizational decision-making.