Skip to main content
SpringerLink
Log in

On ensuring the availability of evaluative data for secondary analysis

  • Published:
Policy Sciences Aims and scope Submit manuscript

Abstract

This paper is based on a programmatic effort to elicit and reanalyze data stemming from the evaluation of social programs. The stress is on identifying problems and negotiating for data and analyzing it. This includes: researchers' reluctance to disclose data for documentation and data control systems; vague or nonexistent policy for assuring access. The possible solutions presented here are based on governmental regulation and policy for publicly supported evaluation.

Individuals who kindly provided their reactions to earlier drafts of this paper and/or advice and information in other ways include: Ole Engberg, John Evans, Harold B. King, Sheldon Laube, William Madow, Richard Marciano, Margaret E. Martin, Keith Marvin, Marjorie Powell, Alice Robbin, Peter Rossi, Lee Sechrest, Ernst W. Stromsdorfer, and Frederick Mosteller. Not all their reactions and criticisms have been accommodated, and we alone bear responsibility for the opinions registered here.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Boruch, R. F. (1974), “On common contentions about randomized field experiments.” In R. F. Boruch and H. W. Riecken (eds.), Experimental Testing of Public Policy. Boulder, Colo.: Westview Press.

    Google Scholar 

  • Boruch, R. F., and Wortman, P. M. (1976). Project on Secondary Analysis: Progress Report. Unpublished manuscript. Evanston, Il.: Northwestern University, Psychology Department.

    Google Scholar 

  • Boruch, R. F. and Cecil, J. S. (1978). Assuring privacy and confidentiality in social research. Philadelphia: University of Pennsylvania (in press).

    Google Scholar 

  • Borus, M. E. and Hardin, E. (1971). The Economic Benefits and Costs of Retraining. Lexington, Mass.: Heath Lexington Books.

    Google Scholar 

  • Bryant, F. B., and Wortman, P. M. (1978). Secondary analysis: The case for data archives. American Psychologist, 33: 381–387.

    Google Scholar 

  • Bunker, J. P., Barnes, B. A., and Mosteller, F. (eds.). (1977). Costs, Risks, and Benefits of Surgery. New York: Oxford University Press.

    Google Scholar 

  • Burstein, L. (1976). “The choice of unit of analysis in the investigation of school effects: IEA in New Zealand,” New Zealand Journal of Educational Studies, 11, 11–24.

    Google Scholar 

  • Campbell, A. (1970). “Some questions about the New Jerusalem.” In R. L. Bisco (ed.), Data Bases, Computers, and the Social Sciences. New York: Wiley and Sons.

    Google Scholar 

  • Campbell, D. T. (1977). One perspective on Cronbach day (October 28, 1976) at Northwestern. Unpublished memo. Evanston, Il.: Northwestern University, Psychology Department.

    Google Scholar 

  • Campbell, D. T., and Erlebacher, A. (1970). “How regression artifacts in quasi-experimental evaluations can mistakenly make compensatory education look harmful.” In J. Hellmuth (ed.), Compensatory Education: A National Debate, Vol. 3 of Disadvantaged Child. New York: Brunner-Mazel.

    Google Scholar 

  • Campbell, D. T., Boruch, R. F., Schwartz, R. D. and Steinberg, J. (1977). “Confidentiality preserving modes of access to files and to interfile exchange for useful statistical analysis,” Evaluation Quarterly, 1, 269–299.

    Google Scholar 

  • Cecil, J. (1976). Memo on attempts to secure data from Howard Earle's police training evaluation. Unpublished memo. Evanston, Il.: Northwestern University, Psychology Department.

    Google Scholar 

  • Center for Political Studies, University of Michigan. (1972). The CPS 1972 national election study. Ann Arbor, Michigan: ICPR.

    Google Scholar 

  • Collins, B. E. (1974). “Four components of the Rotter Internal-External Scale: Belief in a difficult world, a just world, a predictable world, and a politically responsive world.” Journal of Personality and Social Pyschology, 29, 381–391.

    Google Scholar 

  • Cook, T. D. (1974). “The potential and limitations of secondary analysis.” In M. W. Apple, M. J. S. Subkoviak, and H. S. Lufler Jr. (eds.), Educational Evaluation: Analysis and Responsibility. Berkeley, Calif.: McCutchan.

    Google Scholar 

  • Cook, T. D., Appleton, H., Conner, R. F., Sheffer, A., Tomkin, G., and Weber, S. J. (1975). “Sesame Street” Revisited. New York: Russell Sage Foundation.

    Google Scholar 

  • Cronbach, L. J. (1976). Research on classrooms and schools: Formulation of questions, design, and analyses. Stanford Evaluation Consortium, August.

  • Dansk Data Kriv (Danish Data Archive), (1976). Information on the Danish Data Archive. Copenhagen: DDA (H. C. Andersens Blvd., 38 mezz).

    Google Scholar 

  • Deniston, O. L., and Rosenstock, I. M. (1972). The Validity of Designs for Evaluating Health Services. Research report. Ann Arbor, Michigan: University of Michigan, School of Public Health, March.

    Google Scholar 

  • Director, S. (1974). Evaluating the Impact of Manpower Training Programs. Ph.D. dissertation, Northwestern University, Evanston, Il.

    Google Scholar 

  • Earle, H. H. (1973). Police Recruit Training: Stress vs. Nonstress. Springfield, Il.: Charles C. Thomas.

    Google Scholar 

  • Ehrlich, I. (1975). “The deterrent effect of capital punishment: A question of life or death,” American Economic Review, 65, 397–417.

    Google Scholar 

  • Evans, J. W. (1977). Correspondence, October 21.

  • Flaherty, D. L. (1978). “Final report of the Bellagio conference on privacy, confidentiality, and the use of government microdata for research and statistical purposes,” Statistical Reporter, 8, 274–278.

    Google Scholar 

  • Flaherty, D. L., and Ennis, J. (1977). Report of the Project on Privacy and Access to Government Microdata. Unpublished. London, Ontario, Canada: University of Western Ontario.

    Google Scholar 

  • Gerard, H. B., and Miller, N. (1975). School Desegregation: A Long Term Study. New York: Plenum Press.

    Google Scholar 

  • Grasso, J. T. (1974). Documentation of statistical data sets: The case of the National Longitudinal Surveys. Paper presented at the NBER Workshop on Documentation of Large Machine-Readable Statistical Data Sets, April.

  • Greenwald, A. G. (1976). “An editorial,” Journal of Personality and Social Psychology, 33, 1–7.

    Google Scholar 

  • Greenwald, A. G. (1977). Personal communication, January.

  • Gurin, P., Gurin, G., Lao, R., and Beattie, M. (1969). “Internal-external control in the motivational dynamics of Negro youth,” Journal of Social Issues, 25, 29–53.

    Google Scholar 

  • Hedrick, T. E. (1976). Memo on attempts to secure data on a Career Education Program. Unpublished memo. Evanston, Il.: Northwestern University, Psychology Department.

    Google Scholar 

  • Hyman, H. H. (1971). Secondary Analysis of Sample Surveys: Principles, Procedures, and Potentialities. New York: Wiley.

    Google Scholar 

  • IASSIST: International Association for Social Science Information Service and Technology. (1976). Newsletter, 1.

  • Kallek, S. (1975). “Potential applications of Census Bureau Economic Series in microdate analysis,” American Economic Review, 65 (Suppl.), 257–262.

    Google Scholar 

  • King, H., and Krasny, M. (1975). “A standard description for magnetic tape files,” Annals of Economic and Social Measurement, 4, 449–454.

    Google Scholar 

  • Magidson, J. (1977). “Toward a causal model approach to adjusting for pre-existing differences in the nonequivalent control group situation,” Evaluation Quarterly, 1, (3), 1977.

    Google Scholar 

  • Merton, R. K. (1969). “Behavior patterns of scientists,” American Scholar, 38, 197–225.

    Google Scholar 

  • Mirels, H. L. (1970). “Dimensions of internal versus external control,” Journal of Counseling and Clinical Psychology, 34, 226–228.

    Google Scholar 

  • Mosteller, F., and Moynihan, D. P. (1972). On Equality of Educational Opportunity. New York: Random House.

    Google Scholar 

  • National Opinion Research Center (1976). National data program for the social sciences: Codebook for the Spring 1976 General Social Survey. Chicago, Il.: University of Chicago, NORC, 1976.

    Google Scholar 

  • Page, M. (1971). Deceiving the deceivers: Compliance, reference group identification, and reactions to deception among social pyschological researchers. Unpublished manuscript. University of Nebraska, Psychology Department.

  • Panel on Deterrent and Incapacitative Effects (1976). (Committee on Research on Law Enforcement and Criminal Justice, National Research Council). Third draft of the panel summary report. December.

  • Primack, J., and von Hippel, F. (1974). Advice and dissent. New York: Basic Books.

    Google Scholar 

  • Privacy Protection Study Commission. (1977). Draft recommendations for social research and statistics. Washington, D.C.: The Commission, December.

    Google Scholar 

  • Research Notes. (1976). Report on Educational Research, June 16, p. pp8.

  • Reuss, H. (1964). Interview with Henry Reuss. Congressional Record, 115, 34743.

    Google Scholar 

  • Rezmovic, E. L. (1977). Some design, measurement, and treatment issues in correctional evaluation. Paper presented to the National Academy of Sciences Panel on Rehabilitation of Criminal Offenders, Woods-Hole, Mass., July.

  • Rezmovic, V. (1977). Memo on attempt to acquire data from the Beginning Teaching Study. Unpublished memo. Evanston, Il.: Northwestern University, Psychology Department.

    Google Scholar 

  • Rindskopf, D. (1976). Memo on acquisition of evaluation data from 20 studies supported by the U.S. Office of Education in 1974. Unpublished memo. Evanston, Il.: Northwestern University, Psychology Department.

    Google Scholar 

  • Robbin, A. (1974). “Managing information access through documentation of the data base,” SIGSOC Bulletin, 6, 74–75; 56–68.

    Google Scholar 

  • Rogson, M. M. (1976). Documentation in social science experiments. The Rand Paper Series, January.

  • Roistacher, R. C. (1976). The Data Interchange File: A first report (Document No. 207). Urbana-Champaign, Il.: University of Illinois, Center for Advanced Computation, June 21.

    Google Scholar 

  • Roistacher, R. C. (1977). L.E.A.A. Research Support Center machine readable source documentation system user's guide (draft 1). Urbana-Champaign, Il.: University of Illinois, Center for Advanced Computation, July 29.

    Google Scholar 

  • Roistad, M., (1974). NIE's proposed voluntary data sharing policy. Presented at NBER Workshop on Documentation of Large Machine-Readable Statistical Data Sets, April.

  • Ross, J. (1976). Memo on acquisition of I. Ehrlich's data on deterrent effects of capital punishment. Unpublished memo. Evanston, Il.: Northwestern University, Psychology Department.

    Google Scholar 

  • Rotter, J. B. (1966). “Generalized expectancies for internal versus external control of reinforcement.” Psychological Monographs, 80, 1 (Whole No. 609).

    Google Scholar 

  • Royal Netherlands Academy of Arts and Sciences: Social Science Information and Documentation (1974). Steinmetz Archives. Amsterdam: North-Holland Publishing Company.

    Google Scholar 

  • Schoenfeldt, L. F. (1970). “Data archives as resources for instruction, research, and policy planning,” American Psychologist, 25, 609–616.

    Google Scholar 

  • Summers, A. A., and Wolfe, B. L. (1975). “Which school resources help learning? Efficiency and equity in Philadelphia public schools.” Federal Reserve Bank of Philadelphia Review, February 1975.

  • Taeuber, R. C. (1973) “Problems of access: Some comments,” Annals of Economic and Social Measurement, 2, 215–220.

    Google Scholar 

  • Université Catholique de Louvain. (1976). B.A.S.S.: Belgian Archives for the Social Sciences. Louvain-La-Neuve.

  • Watson, J. D. (1968). The Double Helix. New York: Atheneum Publishers.

    Google Scholar 

  • Watts, H. W. (1972). “Microdata: Lessons from the SEO and Graduated Work Incentive Experiment,” Annals of Economic and Social Measurement, 1, 183–191.

    Google Scholar 

  • Wolins, L. (1962). “Responsibility for raw data,” American Psychologist, 17, 657–658.

    Google Scholar 

  • Wortman, P. M. (1976). Memo on attempt to acquire data from the Federal Reserve Bank of Philadelphia. Unpublished memo. Evanston, Il.: Northwestern University, Psychology Department.

    Google Scholar 

  • Wortman, P. M., Reichardt, C. S., and St. Pierre, R. (1977). The first year of the Educational Voucher Demonstration: A secondary analysis of student achievement test scores. Technical report, unpublished. Evanston, Il.: Northwestern University, Psychology Department.

    Google Scholar 

  • Zeisset, P. T. (1974). Some views on good documentation. Paper presented at the NBER Workshop on Documentation of Large Machine-Readable Statistical Data Sets, April.

  • Zyzanski, S. J. (1962). Analysis of variance applied to factors which do not have comparable scales. Ph.D. dissertation, Iowa State University.

Download references

Author information

Authors and Affiliations

  1. Kent State University, Kent, UK

    Terry E. Hedrick

  2. Northwestern University, UK

    Robert F. Boruch & Jerry Ross

Authors
  1. Terry E. Hedrick
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert F. Boruch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jerry Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Development of this paper has been supported by NIE Contract C-74-0115. All unpublished documentation discussed here is on file at the Methodology and Evaluation Research Division (Lucina Gallagher, Librarian).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hedrick, T.E., Boruch, R.F. & Ross, J. On ensuring the availability of evaluative data for secondary analysis. Policy Sci 9, 259–280 (1978). https://doi.org/10.1007/BF00136830

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00136830

Keywords

Search

Navigation