Protecting Individual Privacy in Evaluation Research (1975)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

THE NEED FOR PROTECTING INDIVIDUAL PRIVACY IN EVALUATION RESEARCH BACKGROUND OF THE COMMITTEE ON FEDERAL AGENCY EVALUATION RESEARCH The Committee on Federal Agency Evaluation Research (COFAER) was set up in 1971 by the National Academy of Sciences at the request of (and with funding from) the U.S. Office of Economic Opportunity (OEO). The Committee was given a broad mandate to examine the government's activities in evaluation research and social policy experimentation and to recommend ways of making these activ- ities more effective and useful. In particular, it was asked to examine the problem of confidentiality of data collected in the course of evaluation re- search and social experimentation and to recommend ways of protecting the privacy of individuals who provide such information to researchers. The concern of the Office of Economic Opportunity with these issues arose both from a specific incident and from its general role as the spearhead of the government's "war on poverty." The Office of Economic Opportunity had been created as an experimental agency that would try out new programs, which if successful, might be transferred to more permanent agencies. Identification of such successful programs clearly necessitated substantial efforts to eval- uate OEO-sponsored activities and, therefore, substantial interest in the problems of program evaluation. The particular incident that precipitated OEO's acute concern with confi- dentiality occurred during the New Jersey income maintenance project—a major social experiment designed to measure the impact of various negative income tax plans on labor force behavior and other activities of low-income families. Participants in the experiment were asked to complete detailed questionnaires about their income, employment, and other activities and were promised by the researchers that this information would be kept absolutely confidential. How- ever, local law enforcement officers saw in the experiment an opportunity to check on welfare cheating and endeavored to get hold of the confidential infor- mation. The researchers in charge of the experiment found themselves in a David N. Kershaw and Joseph C. Small, "Data Confidentiality and Privacy: Lessons from the New Jersey Negative Income Tax Experiment," Public Policy, 20:257-280 (Spring 1972).

difficult position. They had promised confidentiality to respondents in good faith, but it was by no means clear that they could honor that pledge if a court saw fit to issue a subpoena. Distressed by the dilemmas of this incident, OEO officials asked the Com- mittee to give special attention to the problem of confidentiality and to make recommendations for ways of ensuring confidentiality of data collected in future evaluations and experiments. THE NEED FOR EVALUATION The Office of Economic Opportunity was only the most conspicuous of the many federal agencies that were involved in the enormous expansion of social welfare programs in the 1960's, programs that provide cash, in-kind benefits, or human services directly to individuals and families. These programs — in- cluding Social Security, public assistance, Medicare and Medicaid, Head Start, manpower training, housing, and urban renewal — grew from about $23 billion or one-quarter of the federal budget in Fiscal Year 1960 to about $150 billion or roughly one-half of the federal budget in Fiscal Year 1975. And in such areas as drug abuse control and the administration of criminal justice, federal in- volvement and concern has become far more pervasive than is suggested by the relatively small amounts of federal money involved. If the public is to make an intelligent assessment of how well the federal government is doing its job, it needs information on how federal programs are operating: who is being reached; how effective the programs are; at what cost results are being produced. It is also essential for the public and the Con- gress to have estimates of the costs and effects of alternative future programs before making decisions on the adoption of those programs, and sometimes the best way of estimating how a new policy would work is to carry out a small- scale experiment. Evaluation research and social experimentation are extremely difficult. It is hard to identify precisely the objectives of government programs or to find valid measures of their effects. It is also difficult to design and exe- cute experiments that accurately represent what would happen if a new program were actually adopted. Despite the difficulties, however, major efforts should be made to improve both the quantity and the quality of evaluation results available to decision makers. Indeed, for government officials, legislators, and the public to make decisions about national programs without careful con- sideration of the costs and probable effects of present and future policies would be both inefficient and irresponsible. Accountability to the public requires not only that evaluations of public programs be carried out, but that they be available for public scrutiny. Fur- thermore, government officials shouldAbe given the sole responsibility for evaluating their own programs. Therefore, it is essential that government officials or agencies other than the program sponsor and researchers in the private sector engage in independent evaluations and that they have access to experimentation and evaluation data collected by the program sponsors (whether government agencies or contractors) so that they can reanalyze and reinterpret them.

PROTECTION OF PRIVACY The need for these data runs into conflict with the deeply held American value of personal privacy. Louis D. Brandeis called the right to be let alone "the right most valued by civilized men."3 The framers of the Bill of Rights took great pains to guard private citizens against arbitrary and unreasonable intrusions of government into their personal affairs. In recent years, however, new technologies—such as wiretapping, electronic surveillance, and computers— have posed threats to individual privacy undreamed of by the authors of the Constitution and have challenged the legal and political system to devise new rules and procedures to protect that privacy. The large-scale collection of information about individuals for the purpose of experimenting with and eval- uating government programs also poses a threat to individual privacy. The problem is complicated by the fact that the most important questions about the effectiveness of human service programs relate to their effects over a fairly long period of time. Evaluating such programs requires following the same people to see what happened to them—how their training affected their in- come or how their treatment in prison affected their criminal record. It may also require obtaining a great many different pieces of information together on one person, perhaps from different sources, so that the effects of a program can be sorted out from the effects of other influences on that person's life. Thus, evaluating transfers and services to people may involve building dossiers on individuals and using them for a long time. Federal programs affecting people are likely to grow both absolutely and relatively in the future. Hence, informed decision making about federal activ- ities will require more and more evaluations based on sensitive information about individual behavior. Improved survey design and management, including improved statistical sampling procedures, questionnaire design, analytic tech- niques, and data processing capabilities make such evaluation feasible, but the increased collection of data from and about individuals increases the threat to privacy and the danger that these data will be misused. A desire for openess and accountability in government further compounds the problem. The use of outside evaluators and the reanalysis of data by outside analysts increase the danger that individual information will fall into the wrong hands and be used for purposes other than evaluation. It is necessary to devise workable procedures for facilitating evaluation and accountability while continuing to protect individual privacy. The compet- ing goals must be continually balanced: it is not in the national interest to sacrifice privacy for the sake of evaluation and accountability, nor would it be desirable to write such strict procedures to safeguard individual privacy that it becomes impossible to evaluate or reevaluate public programs or to gain in- sights into the potential costs and effects of new policies. Ultimately, effective protection of individual privacy is a sine qua non of usable evaluation and experimentation. People simply will not cooperate in giving accurate information about themselves to researchers and evaluators un- less they are assured that the information will not be used against them by bill collectors, tax collectors, or neighborhood gossips. Hence, some means of ^Dissent in Olmstead v. U.S., 277 U.S. 438, 48 S.Ct. 564 (1928).

assuring that information will be kept confidential is essential, not only to protect individual privacy as an end in itself, but also to make possible valid experimentations and evaluations of public programs. FINDINGS AND RECOMMENDATIONS The Committee does not believe it is possible, at this time, to formulate definitive procedures that will ensure both protection of individual privacy and adequate data for experimentation and evaluation. The Committee's mandate, though broad, covered only federal activities, and the Committee has concluded that a definitive review of the problem must take into account the full range of research activities. The Committee also believes that a much broader public debate on the competing values of individual privacy and government account- ability by means of evaluation and experimentation is needed. Therefore, the Committee decided that its most useful contribution to this complex problem would be to present some specific proposals that can be the basis for wide dis- cussion by those concerned with the problem both inside and outside the federal government. The specific proposals included in this report cover two broad areas, corresponding to two distinct risks to the privacy of individuals who provide information about themselves to evaluators of social programs. One might be called the risk of unauthorized misuse of sensitive data. This is the risk that identifiable information on people collected for purposes of evaluation will fall into the hands of unauthorized persons who want to use it against an individual—blackmailers, sales people, bill collectors, etc. The danger is that such persons might get hold of the actual records showing individual names and addresses as well as private information or that they might be able to identify an individual even without the name and address from the information itself, perhaps by matching information with public records. The problem here is one of physical protection of the data—devising ways of reducing the number of people with access to identifiable records and of reducing the probability that individuals can be identified even after names and addresses or other identifiers, such as Social Security numbers, have been removed. The other risk might be described as that of official misuse for law en- forcement or other official purposes of data originally obtained from people for evaluation purposes. Here the problem is how to provide legal protection so that people who cooperate with an evaluation by providing information about their behavior can be assured that the information they provide will not be used for any other purpose, including law enforcement. Early discussions convinced the Committee of the need for new procedures for the physical protection of confidential data, especially when data files are to be made available by the collecting agency for reanalysis by outside re- searchers and when evaluation requires the combination of information on the same people from more than one source. Therefore, the Committee asked Donald T. Campbell, Richard D. Schwartz, Robert F. Boruch, and Joseph Steinberg to consider this range of problems and make recommendations. Their paper, enti- tled "Confidentiality-Preserving Modes of Access to Files and to Interfile Ex- change for Useful Statistical Analysis," is reproduced as Appendix A to this Report. The Committee deems it a valuable and major contribution to the debate and hopes it will be widely read and discussed by those concerned with the problem.

The Committee was also convinced by its consultation with experts in the field that legal protection of confidential information collected in the course of evaluation is required, but that difficult and controversial decisions would have to be made before the specific form of that protection could be formulated. One suggested form of legal protection is a "shield statute" for researchers, and the Committee invited one of its members, Paul Nejelski, and his colleague, Howard Peyser, to draft a statute and to submit a paper discussing the major issues that had to be resolved in drafting the statute and the reasons they favored the particular resolutions reflected in their proposed statute. Their paper, entitled "A Researcher's Shield Statute: Guarding Against the Compul- sory Disclosure of Research Data," is reproduced as Appendix B to this Report. The Committee believes they have made a major contribution and that their statute is worthy of strong consideration and debate. Based on its review of these papers and its own deliberation, the Commit- tee recommends: (1) that all federal agencies engaging in evaluation research adopt rigorous procedures to ensure that data collected about individuals in the course of such research are kept strictly confidential and are not used for purposes other than such research or released in any way that permits identification of individuals; (2) that consideration be given to enactment of a federal statute that would protect from subpoena information collected from individuals in the course of federal evaluation research and thus prevent such information from being used in law enforcement or other legal pro- ceedings . The remainder of this Report serves as an introduction to the appended papers and summarizes the Committee's views about physical and legal protection of confidential information in social experimentation and evaluation research. Because of the origin and chief concern of the Committee, the Report deals mainly with federally funded evaluation research, but the problems discussed arise in connection with other types of research on human behavior and the Committee believes that any solutions offered should be applicable to all such research. It should also be noted that the Committee focused only on the pri- vacy of individuals; the more complex problem of the privacy of groups or insti- tutions (e.g., schools, unions) was not studied.