Evaluation for an Open Society: Then and Now

Abstract

Background: Karl Popper’s views about science and political economy remain relevant to evaluation theory and practice. His Open Society opus inspired pioneering contributions to experimental evaluation and shaped the evaluation discipline. Yet, his ideas are not widely known without the evaluation community even though populist leaders are once again threatening to undermine democracy.

Purpose: To define the Open Society, probe its epistemological tenets, confirm that they remain valid as the foundation of evaluation practice, identify the ways in which the operating environment for evaluation has changed and, against this background, propose a policy change agenda relevant to the contemporary evaluation discipline.

Setting: The Open Society is once again being undermined. Modern authoritarianism is tightening its grip. The lure of the strong man is once again gaining traction. The dominance of an international order grounded in democracy, human rights, and the rule of law is giving way to a world in which leaders are pursuing narrow nationalist and vested interests. In this troubled context, policy making has become more complex than when evaluation emerged out of the ashes of World War II. Economic and social dysfunctions have led to extraordinary concentration of wealth and privilege.  Ominous environmental threats loom. The architecture of international relations designed in the mid-1940’s has become obsolete.

Research Design: To design this commentary about the prospects of the evaluation discipline, the author drew on his personal experience as evaluation academic, international development practitioner, manager of the World Bank’s Independent Evaluation Group for two consecutive five-year terms and senior independent evaluation adviser to governments and international development agencies.

 Intervention: As an intervention, this article adds value to evaluation theory and practice by showing why and how the Popper/Campbell mandate for evaluation needs to be upgraded to protect the public interest in a new operating environment.  Specifically, Popper’s piecemeal social experimentation concept should be refined to forge links between small scale experiments and the broader fabric of society. In addition, the ambiguity regarding the relationship between the Open Society and evaluation should be lifted through a reconsideration of the democratic evaluation model.

Data Collection and Analysis: The author conducted an extensive review of the literature and consulted with a wide range of evaluation thinkers to examine the extent to which Popper’s philosophy remains relevant to the evaluation discipline.

Findings: Popper’s Open Society ideas aimed at avoiding the rise and perpetuation of autocracy and remain highly relevant. But the current threats to democracy call for a more ambitious and detailed remit for the evaluation occupation. Beyond the promotion of evaluation in democracy and of democracy in evaluation, evaluation for democracy should be pursued. This implies putting value, ethics, and the public interest at the very center of the evaluation occupation; breaking free of Popper’s parsimonious piecemeal social engineering concept to inform systemic social reform; bringing peace to a methodologically divided house; systematic mixing of evaluation methods and models; and the promotion of evaluation independence through professionalization.