Assessing the Impact of Planned Social Change*
Main Article Content
Abstract
Originally published as Paper #8, Occasional Paper Series, December, 1976*
Reprinted with permission of The Public Affairs Center, Dartmouth College.
Preparation of this paper has been supported in part by grants from the Russell Sage Foundation and the National Science Foundation, Grant Number SOC-7103704-03. A version of this paper was presented to the Visegrad, Hungary, Conference on Social Psychology, May 5-10, 1974.
Downloads
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
Copyright and Permissions
Authors retain full copyright for articles published in JMDE. JMDE publishes under a Creative Commons Attribution-NonCommercial 4.0 International License (CC BY - NC 4.0). Users are allowed to copy, distribute, and transmit the work in any medium or format for noncommercial purposes, provided that the original authors and source are credited accurately and appropriately. Only the original authors may distribute the article for commercial or compensatory purposes. To view a copy of this license, visit creativecommons.org
Funding data
-
National Science Foundation
Grant numbers SOC-7103704-03
References
Anderson, J.K. Evaluation as a process in social systems. Unpublished doctoral dissertation, Northwestern University, Department of Industrial Engineering & Management Sciences, 1973.
Baldus, D. C. Welfare as a loan: An empirical study of the recovery of public assistance payments in the United States. Stanford Law Review, 1973. 25.123-250. (No. 2) https://doi.org/10.2307/1227967
Barnow, B. S. The effects of Head Start and socioeconomic status on cognitive development of disadvantaged children. Unpublished doctoral dissertations, University of Wisconsin, Department of Economics, 1973. (252 pp.)
Bauman, R. A., David, M. H., & Miller, R. F. Working with complex data files: II. The Wisconsin assets and incomes studies archive. In R. L. Bisco (Ed.), Data bases, computers, and the social sciences. New York: Wiley- Interscience, 1970, 112-136.
Beck, B. Cooking the welfare stew. In R. W. Habenstein (Ed.), Pathways to data: Field methods for studying ongoing social organizations. Chicago: Aldine, 1979.
Becker, H. M., Geer, B., & Hughes, E. C. Making the grade. New York: Wiley, 1968
Becker,H.W.Sociologicalwork:Method and substance. Chicago: Aldine, 1970.
Berliner, J. S. Factory and manager in the U.S.S.R. Cambridge Mass.: Harvard University Press, 1957. https://doi.org/10.4159/harvard.9780674188273
Blau, P. M. The dynamics of bureaucracy. (Rev. Ed.) Chicago: University of Chicago Press, 1963.
Boruch, R. F., & Campbell, D. T. Preserving confidentiality in evaluative social research: Intrafile and interfile data analysis. Paper presented at the meeting of the American Association for the Advancement of Science, San Francisco, February/March 1974.
Box,G.E.P.,&Tiao, G.C. Achangein level of non-stationary time series. Biometrika, 1965, 52.181-192. https://doi.org/10.1093/biomet/52.1-2.181
Box, G. E. P., & Jenkins, G. M. Time- series analysis: Forecasting and control. San Francisco: Holden Day, 1970.
Campbell, D. T. Pattern matching as an essential in distal knowing. In K. R. Hammond (Ed.), The psychology of Egon Brunswik. New York: Holt, Rinehart & Winston, 1966, 81-106.
Campbell, D. T. Administrative experimentation, institutional records, and nonreactive measures. In J. C. Stanley & S. M. Elam (Eds.), Improving experimental design and statistical analysis. Chicago: Rand McNally, 1967, 257-291.Reprinted: in W. M. Evan (Ed.), Organizational experiments: Laboratory and field research. New York: Harper & Row, 1971, 169-179.
Campbell, D. T. A phenomenology of the other one: Corrigible, hypothetical and critical. In T. Mischel (Ed.), Human action: conceptual and empirical issues. New York: Academic Press, 1969a, 41-69.
Campbell, D. T. Reforms as experiments. American Psychologist, 1969b, 24.409-429. (No. 4, April) https://doi.org/10.1037/h0027982
Campbell, D. T. Considering the case against experimental evaluations of social innovations. Administrative Science Quarterly, 1970, 15.110-113 (No. 1, March) https://doi.org/10.2307/2391194
Campbell, D. T. Methods for an experimenting society. Paper presented to the Eastern Psychological Association, April 17, 1971, and to the American Psychological Association, Sunday, September 5, 12:00, Diplomat Room, Shoreham Hotel, Washington D.C. to appear when revised in the American Psychologist.
Campbell, D. T. Experimentation revisited: A conversation with Donald T. Campbell (by S. Salasin). Evaluation, 1973, 1.7-13. (NO. 1)
Campbell, D. T. Qualitative Knowing in Action Research. Journal of Social Issues, probably 1975.
Campbell, D. T., & Boruch, R. F. Making the case for randomized assignment to treatments by considering the alternatives: Six ways in which quasi- experimental evaluations in compensatory education trend to underestimate effects. Lecture delivered at the conference on Central Issues in Social Program Evaluation, C. A. Bennett & A. Lumsdaine, coordinators, Battelle Human Affairs Research Center, Seattle, Washington, July 1973. Publication pending.
Campbell, D. T., Boruch, R. F., Schwartz, R. D., & Steinberg, J. Confidentiality- preserving modes of useful access to files and to interfile exchange for statistical analysis, National Research Council, Committee on Federal Agency Evaluation Research, Final Report, Appendix A. 1975.
Campbell, D. T., & Erlebacher, A. E. How regression artifacts in quasi- experimental evaluations can mistakenly make compensatory education look harmful. In J. Hellmuth (Ed.), Disadvantaged child. Vol. 3 Compensatory education: A national debate. New York: Brunner/Mazel, 1970.
Campbell, D. T., & Fiske, D. W. Convergent and discriminant validation by the multitrate- multimethod matrix. Psychological Bulletin, 1959, 56.81-105. (Also, Bobbs-Merrill Reprint series in the social sciences, S-354.) https://doi.org/10.1037/h0046016
Campbell, D. T., Siegman, C. R., & Rees, M. B. Direction-of-wording effects in the relationships between scales. Psychological Bulletin, 1967, 68.293- 303. (No. 5) https://doi.org/10.1037/h0020187
Campbell, D. T. & Stanley, J. C. Experimental and quasi-experimental designs for research. Chicago: Rand McNaly, 1966.
Caporso, J. A., & Roos, L. L., Jr. (Eds.) Quasi-experimental approaches: Testing theory and evaluating policy. Evanston, Ill.: Northwestern University Press, 1973.
Caro, F. G. (Ed.) Readings in evaluation research. New York: Russell Sage Foundation, 1972.
Cicirelli, V. G., et al. The impact of Head Start: An evaluation of the effects of Head Start on children's cognitive and affective development. A report presented to the Office of Economic Opportunity pursuant to Contract B89-4536, June 1969. Westinghouse Learning Corporation, Ohio University. (Distributed by Clearinghouse for Federal Scientific and Technical Information, U.S. Department of Commerce, National Bureau of Standards, Institute for Applied Technology. PB 184 328).
Conner, R. F. A methodological analysis of twelve true experimental program evaluations. Unpublished doctoral dissertation, Northwestern University, August, 1974.
Cook, T. D., Appleton, H. , Conner, R., Shaffer, A., Tamkin, G., & Weber, S. J. Sesame Street revisited: A case study in evaluation research. New York: Russell Sage Foundation,1975.
Cook, T. D., & Campbell, D. T. The design and conduct of quasi-experiments and true experiments in field settings. In M. D. Dunnette & J. P. Campbell (Eds.), Handbook of industrial and organizational research. Chicago: Rand McNally, 1975.
Cronbach, L. J. Response sets and test validity. Educational and Psychological Measurement, 1946, 6.475-494. https://doi.org/10.1177/001316444600600405
Cronbach, L. J. Further evidence on response sets and test design. Educational and Psychological Measurement, 1950, 10.3-31. https://doi.org/10.1177/001316445001000101
David, H. P. Family planning and abortion in the socialist countries on Central and Eastern Europe. New York: The Population Council, 1970. Romanian data based primarily on Anuarul Statistic al Republicii Socialiste Romania.
Douglas, J. D. The social meanings of suicide. Princeton, N.J.: Princeton University Press, 1967.
Edwards, A. L. The social desirability variable in personality assessment and research. New York: Dryden, 1957.
Fairweather, G. W. Methods for experimental social innovation. New York: Wiley, 1967
Fischer, J. L. The uses of Internal Revenue Service data. In M.E. Borus (Ed.), Evaluating the impact of manpower programs. Lexington, Mass.: D. C. Health, 1972, 177-180
Gardiner, J. A. Traffic and the police: Variations in law-enforcement policy. Cambridge, Mass.: Harvard University Press, 1969. https://doi.org/10.4159/harvard.9780674491618
Garfinkel, H. "Good" organizational reasons for "bad" clinic records. In H. Garfinkel, Studies in ethnomethodology. Englewood Cliffs, N.J.: Prentice Hall, 1967, 186-207.
Glaser, D. Routinizing evaluation: Getting feedback on effectiveness of crime and delinquency programs. Washington, D.C.: Center for Studies of Crime and Delinquency, NIMH., 1973. (Available from U.S. Government Printing Office, Washington, D.C. 20402; publication number 1724-00319, $1.55.)
Glass, G. V., Willson, V. L., & Gottman, J. M. Design and analysis of time-series experiments. Boulder, Colo.: Laboratory of Educational Research, University of Colorado, 1972.
Goldberger, A. S. Selection bias in evaluating treatment effects: Some formal illustrations. Discussion Papers, 123-72. Madison: Institute for Research on Poverty, University of Wisconsin, 1972.
Gordon,A.C., Campbell,D.T., et al. Recommended accounting procedures for the evaluation of improvements in the delivery of state social services. Duplicated paper, Center for Urban Affairs, Northwestern University, 1971.
Goslin,D.A. Guidelines for the collection, maintenance, and dissemination of pupil records. New York: Russell Sage Foundation, 1970.
Granick, D. Management of the industrial firm in the U.S.S.R. New York: Columbia University Press, 1954. https://doi.org/10.7312/gran91908
Guttentag, M. Models and methods in evaluation research. Journal for the TheoryofSocialBehavior,1971,1.75- 95. (No. 1, April) https://doi.org/10.1111/j.1468-5914.1971.tb00166.x
Guttentag, M. Evaluation of social intervention programs. Annals of the New York Academy of Sciences, 1973, 218.3-13. https://doi.org/10.1111/j.1749-6632.1973.tb47616.x
Guttentag, M. Subjectivity and its use in evaluation research. Evaluation, 1973, 1.60-65. (No. 2)
Hatry, H. P., Winnie, R. E., & Fisk, D. M. Practical program evaluation for state and local government officials. Washington, D.C.: The Urban Institute (2100 M. Street NW, Washington, DC 20037), 1973.
Heller, R. N. The uses of social security administration data. In M. E. Borus (Ed.), Evaluating the impact of manpower programs. Lexington, Mass.: D. C. Heath, 1972, 197-201.
Ikeda, K., Yinger, J. M., & Laycock, F. Reforms as experiments and experiments as reforms. Paper presented to the meeting of the Ohio Valley Sociological Society, Akron, May 1970.
Jackson, D. N., & Messick, S. Response styles on the MMPI: Comparison of clinical and normal samples. Journal of Abnormal and Social Psychology, 1962, 65.285-299. https://doi.org/10.1037/h0045340
Kepka, E. J. Model representation and the threat of instability in the interrupted time series quasi- experiment. Doctoral dissertation, Northwestern University, Department of Psychology, 1971.
Kershaw, D. N. A negative income tax experiment. Scientific American, 1972, 227.19-25. https://doi.org/10.1038/scientificamerican1072-19
Kershaw, D. N. The New Jersey negative income tax experiment: a summary of the design, operations, and results of the first large-scale social science experiment. Dartmouth/OECD Seminar on "Social Research and Public Policies," Sept. 13-15, 1974 (this volume).
Kershaw, D. N., & Fair, J. Operations, surveys, and administration, Volume IV of the Final Report of the New Jersey Graduated Work Incentive Experiment. Madison, Wisconsin: Institute for Research on Poverty, December 1973. Duplicated, 456 pp. (Mathematica, Inc., Princeton, N.J. as Vol. IV of the Report of the New Jersey Negative Income Tax Experiment) December, 1973. (To be published by Academic Press)
Kitsuse, J. K., & Circourel, A. V. A note on the uses of official statistics. Social Problems, 1963, 11.131-139. (Fall) https://doi.org/10.1525/sp.1963.11.2.03a00020
Kuhn, T. S. The structure of scientific revolutions. (2nd ed.) Chicago: University of Chicago Press, 1979. (Vol. 2, No. 2, International Encyclopedia of Unified Science.)
Kutchinsky, B. The effect of easy availability of pornography on the incident of sex crimes: The Danish experience. Journal of Social Issues, 1973, 29.163-181. (No. 3) https://doi.org/10.1111/j.1540-4560.1973.tb00094.x
Levenson, B., & McDill, M. S. Vocational graduates in auto mechnaics: A follow- up study of Negro and white youth. Phylon, 1966, 27.347-357. (No. 4) https://doi.org/10.2307/273615
Lohr, B. W. An historical view of the research on the factors related to the utilization of health services. Duplicated Research Report, Bureau for Health Services Research and Evaluation, Social and Economic Analysis Division, Rockvill, Md., January 1973, 34 pp. In press, U.S. Government Printing Office.
Lord, F. M. Large-scale covariance analysis when the control variable is fallible. Journal of the American Statistical Association, 1960, 55.307- 321. https://doi.org/10.1080/01621459.1960.10482065
Lord, F. M. Statistical adjustments when comparing preexisting groups. Psychological Bulletin, 1969, 72.336- 337. https://doi.org/10.1037/h0028108
McCain, L. J. Data analysis in the interrupted time series quasi- experiment. Master's thesis, Northwestern University, Department of Computer Sciences, in preparation.
Miller, A. R. The assault on privacy. Ann Arbor, Mich.: University of Michigan Press, 1971.
Miller, S. M., Roby, P., & Steenwijk, A. A. V. Creaming the poor. Trans-action, 1970, 7.38-45. (No. 8, June) https://doi.org/10.1007/BF02805485
Morgenstern, O. On the accuracy of economic observations. (2nd ed.) Princeton, N.J.: Princeton University Press, 1963.
Morrissey, W. R. Nixon anti-crime plan undermines crime statistics. Justice Magazine, 1972, 1.8-11, 14. (No. 5/6, June/July) (Justice Magazine, 922 National Press Building, Washington, D.C. 20004)
Porter, A. C. The effects of using fallible variables in the analysis of covariance. Unpublished doctoral dissertation, University of Wisconsin, 1967. (University Microfilms, Ann Arbor, Michigan, 1968)
Ridgeway, V. Dysfunctional consequences of performance measures. Administrative Science Quarterly, 1956, 1.240-247. (No. 2, September) https://doi.org/10.2307/2390989
Riecken, H. W., Boruch, R. F., Campbell, D.T., Caplan, N., Glennan, T.K., Pratt, J., Rees, A., & Williams, W. Social Experimentation: a method for planning and evaluating social intervention. New York: Academic Press, 1974.
Rivlin, A. M. Systemic thinking for social action. Washington, D.C.: The Brookings Institution, 1971.
Ross, H. L. Law, science and accidents: The British Road Safety Act of 1967. Journal of Legal Studies, 1973, 2.1-75. (No. 1; American Bar Foundation) https://doi.org/10.1086/467491
Rossi, P. H. Practice, method, and theory in evaluating social-action programs. In J. L. Sundquist (Ed.), On fighting poverty. New York: Basic Books, 1969, 217-234. (Chapter 10)
Rossi, P. H., & Williams, W. (Eds.) Evaluating social programs: Theory, practice, and politics. New York: Seminar Press, 1972.
Ruehausen, O. M., & Brim, O. G., Jr. Privacy and behavioral research. Columbia Law Review, 1965, 65.1184- 1211. https://doi.org/10.2307/1120683
Sawyer, J., & Schecter, H. Computers, privacy, and the National Data Center: The responsibility of social scientists. American Psychologists, 1968, 23.810-818. https://doi.org/10.1037/h0026719
Schwartz, R. D., & Orleans, S. On legal sanctions. University of Chicago Law Review, 1967, 34.274-300. https://doi.org/10.2307/1598934
Seidman, D., & Couzens, M. Crime statistics and the great American anti-crime crusade: Police misreporting of crime and political pressures. Paper presented at the meeting of the American Political Science Association, Washington, D.C., September 1972. (To appear in Law & Society Review).
Segall, M. H., Campbell, D. T., & Herskovits, M. J. The influence of culture on visual perception. Indianapolis, Ind.: Bobbs-Merrill, 1966.
Shaver, P., & Staines, G. Problems facing Campbell's "Experimenting Society." Urban Affairs Quarterly, 1971, 7.173- 186. (No. 2, December) https://doi.org/10.1177/107808747100700203
Skolnick, J. H. Justice without trial: Law enforcement in democratic society. New York: Wiley, 1966. (Chapter 8, 164-181)
Smith, M. S., & Bissell, J. S. Report analysis: The impact of Head Start. Harvard Educational Review, 1970, 40.51-104. https://doi.org/10.17763/haer.40.1.f3h6164m816t2552
Stake, R. E. Testing hazards in performance contracting. Phi Delta Kappan, 1971. 52.583-588. (No. 10, June)
Suchman, E. Evaluation research. New York: Russell Sage Foundation, 1967.
Sween, J. A. The experimental regression design: An inquiry into the feasibility of non-random treatment allocation. Unpublished doctoral dissertation, Northwestern University, 1971.
Thompson, C. W. N. Administrative experiments: The experience of fifty- eight engineers and engineering managers. IEEE Transactions on Engineering Management, 1974, EM- 21, pp. 42-50. (No. 2, May) https://doi.org/10.1109/TEM.1974.6448465
Thorndike, R. L. Regression fallacies in the matched groups experiment. Psychometrika, 1942, 7.85-102. https://doi.org/10.1007/BF02288069
Twigg, R. Downgrading of crimes verified in Baltimore. Justice Magazine, 1972, 1, 15, 18. (No. 5/6, June/July) (Justice Magazine, 922 National Press Building, Washington, D.C. 20004)
Watts, H. W., & Rees, A. (Eds.) Final report of the New Jersey graduated work incentive experiment. Volume I. An overview of the labor supply results and of Central labor-supply results (700 pp.), Volume II. Studies relating to the validity and generalizability of the results (250 pp.), Volume III. Response with respect to expenditure, health, and social behavior and technical notes (300 pp.) Madison, Wisconsin: Institute for Research on Poverty, University of Wisconsin. December, 1973. (Duplicated)
Weber, S. J., Cook, T. D., & Campbell, D. T. The effects of school integration on the academic self-concept of public schoolchildren.Paperpresentedatthe meeting of the Midwestern Psychological Association, Detroit, 1971.
Weiss, C. H. (Ed.) Evaluating action programs: Readings in social action and education. Boston: Allyn & Bacon, 1972a.
Weiss, C. H. Evaluation research. Englewood Cliffs, N.J.: Prentice-Hall, 1972b.
Weiss, R. S., & Rein, M. The evaluation of broad-aim programs: A cautionary case and a moral. Annals of the American Academy of Political and Social Science, 1969, 385. 133-142. https://doi.org/10.1177/000271626938500112
Weiss, R. S., & Rein, M. The evaluation of broad-aim programs: Experimental design, its difficulties, and an alternative. Administrative Science Quarterly, 1970, 15.97-109. https://doi.org/10.2307/2391193
Westin, A. F. Privacy and freedom. New York: Atheneum, 1967.
Wheeler, S. (Ed.) Files and dossiers in American life. New York: Russell Sage Foundation, 1969.
Wholey, J. S., Nay, J. N., Scanlon, J. W., Schmidt, R. E. If you don't care where you get to, then it doesn't matter which way you go. Dartmouth/OECD Seminar on Social Research and Public Policies, Sept. 13-15, 1974 (this volume).
Wholey, J. S., Scanlon, J. W., Duffy, H. G., Fukumoto, J., & Vogt, L. M. Federal evaluation policy. Washington, D.C.: The Urban Institute, 1970.
Wilder, C. S. Physician Visits, volume and interval since last visit, U.S. 1969. Rockville, Md.: National Center for Health Statistics, series 10, No. 75, July 1972 (DHEW Pub. No. [HSM] 72- 1064).
Williams, W., & Evans, J. W. The politics of evaluation: The case of Head Start. The Annals, 1969, 385.118-132. (September) https://doi.org/10.1177/000271626938500111
Wortman, C. B., Hendricks, M., & Hillis, J. Some reactions to the randomization process. Duplicated research report, Northwestern University, Department of Psychology, 1974.
Zeisel, H. And then there were none: The diminunition of the Federal jury. University of Chicago Law Review, 1971, 38.710-724. https://doi.org/10.2307/1598870
Zeisel, J. The future of law enforcement statistics: A summary view. In Federal statistics: Report of the President's Commission. Vol. II, 1971, 527-555. Washington, D.C.: U.S. Government Printing Office. (Stock no. 4000-0269)