Why Does the Evaluation of Research Need to be Reformed?

Abstract

The evaluation of scientific research already has an extensive and well-structured literature on its objects, motives, methods and procedures, criteria and standards, difficulties, and results. It hardly needs elaborate philosophical underpinnings as there is normally, but not always, a consensus around what is truly important and valuable research. Simultaneously, most governments around the world recognize that current methods for evaluating research for funding purposes are not sufficient for current needs, and they are now funding efforts to find new and improved methods. In the United States alone, as evidenced by the concerns of the American Council on Competitiveness, the initiative of the White House Office of Science and Technology Policy, and the National Science Foundation’s funding of studies to build a science of science and innovation policy, a primary interest is assessment to understand how to improve research so that it can effectively contribute to national goals. Similar efforts exist throughout Europe, in Japan, and in Korea. In early 2008, a volume of New Directions for Evaluation (NDE), guest edited by myself and Michael Scriven, titled “Reforming the Evaluation of Research” is scheduled for publication. This volume, devoted to reforming the evaluation of research, is intended to contribute to the process of addressing this analytical need. It represents the thinking and work of some of the world’s leading scholars and practitioners who have devoted themselves to improving the way that research is evaluated. Without staking any great claim about reforming research itself, a field of action that has rarely been out of the public eye, ultimately the suggestions found in that volume should indirectly result in increasing the quality of, and payoff from, research that is done, reducing the cost of doing it, and lending public credibility to the manner in which research is funded.