Protecting Student Records and Facilitating Education Research: A Workshop Summary (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 Reflections and Next Steps PLANNING COMMITTEE REFLECTIONS In the final session, members of the workshop planning committee reflected on what they had learned about reconciling access, privacy, and confidentiality of education records and offered suggestions for future research and policy. Felice Levine noted that, on one hand, the workshop had described several different models that allow researchers to access education and other administrative records while protecting confiden- tiality and had also illuminated the benefits of research using education record data. On the other hand, she said, the workshop discussions had clarified the challenges that result from the failure of the Family Educa- tional Rights and Privacy Act (FERPA) to reconcile privacy with research access in an effective way. Robert Boruch reiterated his earlier call for researchers to use more uniform language to communicate with the public and clarify the distinc- tion between statistical and administrative uses of administrative data. He said that the discussions about models of research access were very helpful, asking researchers and public agencies to share their formal data- sharing agreements and memoranda of understanding. Boruch argued that sharing these agreements is essential to alleviate fears about comply- ing with FERPA and other privacy laws. Boruch said that changes to the FERPA law or regulations, such as redefining “educational institution” to include state education agencies, as called for by Steven Winnick (see Chapter 2), would help to facilitate researchers’ access to data. And he said that the process of obtaining a waiver of the FERPA informed consent 75 

76 PROTECTING STUDENT RECORDS requirement from a state or local education agency presents an opportu- nity to forge a research partnership. Thomas Plewes (National Research Council) observed that federal agencies have developed innovative approaches to providing access to data for research purposes, including research data centers, data enclaves, and the data licensing agreements pioneered by the National Center for Education Statistics. However, these innovations at the federal level have not yet been tried by state or local education agencies. He noted that the No Child Left Behind Act generates the need to gather data on student performance and also drives the need for more education research. Constance Citro (National Research Council) agreed that the workshop had highlighted the value of different models of access tailored to different types of data, such as the Census Bureau’s research data cen- ters, which severely limit access, as is appropriate for the sensitive data they maintain. Noting that state and local education agencies are most affected by FERPA, Citro asked which of the federal models would be most effective in helping these agencies provide data access while protect- ing confidentiality. Martin Orland said he had learned that “federalism is alive and well” through the workshop discussions, highlighting the question of what the federal role should be. He noted that Congress had not anticipated the possibility that education records might be used for research purposes when it wrote FERPA. He called for changes in the law that would rec- ognize the value of using school records to benefit research and improve education policies and practices. He cautioned against concluding that “all is well,” simply because presenters had described a few successful models of research access. In addition to changing FERPA, he said, suc- cess in using education record data for research purposes requires four critical conditions: 1. There is researchable data. 2.  here is a confluence of interest between a researcher and an edu- T cation agency. 3.  ime and commitment are available to build trust between the T researcher and the agency. 4.  he education agency has the technical capacity to share data while T protecting confidentiality of individual information. Helen Ladd said that the critical issues of research access and con- fidentiality protection revolve around the use of state education data, reflecting the reality that education is a state function under the U.S. Constitution. She called for increased clarity in the Department of Educa- tion’s guidance about FERPA, noting that state and local education agen- 

REFLECTIONS AND NEXT STEPS 77 cies’ nervousness about data-sharing agreements poses a barrier to good research. Ladd agreed with Orland that FERPA is not the only barrier to high-quality education research, observing that community colleges in North Carolina have information technology systems that are not capable of providing deidentified data to researchers. In addition, she reminded participants that Florida’s education agency, with one of the most devel- oped education data systems in the nation, lacks staff to respond to the many requests for access from individual researchers. Finally, she said that both researchers and education agencies must continually work to build trust. Barbara Schneider noted that many speakers had agreed on the need to fix FERPA, because of its “chilling effect” on education and public health research. At the same time, she said that the workshop discussions had clarified the tensions between the benefits of research access and the harm that could come to an individual from an “inevitable” breach of sensitive personal information. Schneider repeated her earlier call for professional development about privacy and confidentiality in the educa- tion research community. REFLECTIONS BY KENNETH PREWITT The workshop planning committee invited Kenneth Prewitt, a for- mer Census Bureau director and long-time member of the Committee on National Statistics, to provide concluding reflections at the end of the workshop. Administrative Data and Survey Data Prewitt said that researchers must use administrative data because they have so much to offer. He said that, although many social science researchers are not very familiar with data mining, this technique for analyzing administrative data represents a more important methodologi- cal breakthrough than any advance in survey methodology. For example, the Bureau of Economic Analysis now purchases credit card data from banks, mining the data for information to incorporate into its economic modeling. Administrative and survey data differ in several important ways, he said. Administrative records are “theory indifferent,” poor in num- ber of variables, and rich in number of cases. Because they are gathered for administrative purposes, these data are not based on any theory of human behavior and include only a very limited number of variables, which are related to the administrative purposes. For example, an agency may gather data on an individual’s earnings for purposes of providing 

78 PROTECTING STUDENT RECORDS TABLE 6-1 Administrative and Survey Data Characteristic Administrative Records Survey Data Theory Indifferent Embedded Number of variables Poor Rich Number of cases Rich Poor Source: Presentation by Kenneth Prewitt (2008). social benefits, without gathering data on any other variables related to that individual. This leads to creation of very large data sets that are thin in the number of variables and are not designed to test any particu- lar theory. In contrast, survey data are “theory embedded”—that is, the design of the survey and therefore the characteristics of the resulting data are informed by theory—and rich in variables. Survey data are poor in number of cases, because of the expense of administering a survey; for example, a major national survey of public health includes only 5,000 cases (see Table 6-1). These characteristics affect privacy and confidentiality, Prewitt explained. Because administrative data sets are thin in variables, research- ers must often link them to other data sets in order to address important research questions. But this linkage magnifies the risk that individual information could be identified. The two data sources also vary in quality, Prewitt said. The accuracy of variables in administrative data depends in part on the goals and mis- sions of agencies that gather these data. For example, the Social Security Administration is unlikely to tolerate errors in age data, while the Depart- ment of Housing and Urban Development may be less concerned about accuracy in this variable. Prewitt noted that survey theory and practice are well developed, with a history dating back to the 1930s, and today students can take many classes to learn about how to gather high-quality survey data. Asking how many classes are available on the quality of administrative data, Prewitt called for increased study and discussion focusing on the quality of these very important new data sources. Developing Research Partnerships Prewitt observed that, at the federal level, relationships between researchers and data-producing agencies have matured over the past 15 years. Their ongoing discussions of research access, privacy, and confi- dentiality have informed the development of new models that recon- cile access with protections, such as data enclaves and data licensing agreements. Prewitt suggested that a similar dialogue was needed at the 

REFLECTIONS AND NEXT STEPS 79 state level, in order to support development of similar models there. He encouraged researchers to try to understand the motivations and ethical and emotional concerns of state officials as part of that dialogue. Echoing Boruch, Prewitt also called for increased sharing of formal memoranda of understanding between researchers and education agen- cies, in order to address state and local officials’ fears about sharing data while complying with FERPA and other privacy laws. When considering concerns about privacy and data confidentiality, he said, it is important to distinguish between the harm to an individual whose data might be released to the public and the harm that could result to the research enter- prise. He said that he had participated in a discussion earlier that day about whether and how the National Center for Health Statistics might release DNA data. A knowledgeable expert said that, if individual DNA data ever were inadvertently released and this became publicly known, this would effectively mean the end of the National Center for Health Statistics. Prewitt urged the research community to assume responsibility for protecting data confidentiality, along with the data providers, in order to avoid harm to both research and agency missions. Prewitt called for greater clarity about the different uses of adminis- trative data. These data are used for administrative purposes, for research purposes, and also for program evaluation and policy design, he said. For example, the No Child Left Behind Act requires states to gather data for program evaluation. He asked the research community to avoid a tendency to project the way it wants to use administrative data onto all of these other uses. Next Steps Prewitt said that the workshop had been valuable in developing new ideas related to access to school record data for use in education research, going beyond a narrow focus on changing the legislative language in FERPA. He urged participants to offer comments to the Department of Education on its proposed revisions to the FERPA regulations. Based on his experience at the Census Bureau, he said, “I can tell you that com- ments are read carefully,” as agencies try to be responsive to interested stakeholders. He noted that similar conversations about balancing data access for research and confidentiality were going on in health and other sectors, and he urged continued discussion across sectors. Such conversations about the complexities and challenges—both across sectors and across levels of government—will help to make agencies and researchers more comfortable with sharing data, he said. Finally, he said he thinks of the current situation as involving two sides. On one side are the school sys- 

80 PROTECTING STUDENT RECORDS tem, parents, and grandparents wishing for a better public school system. On the other side is a large education research enterprise, including the 25,000 experts who participate each year in the American Educational Research Association’s annual conference. Somewhere in between the analytic capacity of the researchers and the concerned students and fami- lies are the rich education data sets. The workshop discussions, he said, were about the way in which this research capacity can address the social challenge of improving education by using the rich data sets to generate evidence. final discussion Michael Feuer (National Research Council) responded to Prewitt’s remarks about the quality of administrative data, focusing particularly on educational test score data. When thinking about linking test data with other data sets, Feuer said, it is important to consider how the validity of these test data might be compromised by the linkage. In addition, he cau- tioned that students’ responses to certain conditions—such as high stakes attached to certain tests—may compromise their performance, affecting the quality of the resulting score data. Moving on to FERPA, Feuer warned researchers to avoid exaggerat- ing their complaints about privacy regulations, because this appears self- serving. He suggested considering other stakeholders’ views of privacy and access issues, including the views of legal experts. Felice Levine responded that, as currently administered, FERPA does not address responsible research access to the data and is “almost exclusionary.” Levine responded to Prewitt’s comments about the different uses of administrative data by observing that, while there has been more atten- tion to these different uses for large administrative data sets, the federal government has provided little guidance. For example, the Common Rule provides guidance on how institutional review boards should review research proposals, but it devotes far less attention to how to protect privacy and confidentiality as the research plan is executed and when the results are being disseminated. In contrast, she said, she attended a recent meeting on biosecurity at which the entire discussion focused on dissemination and on the potential for unwanted uses of information. For example, if a researcher wants to publish the invention of an aerosol spray that could potentially be used to disperse airborne spores that could annihilate populations, biosecurity rules provide guidance on how to present the research findings in a way that would avoid this unwanted use. Levine then invited a team of researchers and public school repre- sentatives from Baltimore, Maryland, who participated in the workshop 

REFLECTIONS AND NEXT STEPS 81 and have formed a research partnership, to offer any comments about the workshop or pose any final questions. Ike Diibor (Baltimore City Public Schools) responded that the work- shop had been very helpful in increasing his understanding of current activities in the research community. He said that he and other school officials have interpreted FERPA with the guidance of the school system’s legal counsel, without devoting much thought to researchers’ perspec- tive on the law. Observing that the team was in the process of creating a formal research consortium, he said that the information gathered at the workshop about confidentiality and privacy would be very valuable. Stephen Plank said that the workshop participants had seen the team of researchers and school officials from Baltimore try to practice the pro- cess of building trust and developing shared understandings. Reflecting on earlier discussions, he agreed with Helen Ladd that researcher–school system partnerships should not depend only on trust and personal rela- tionships but should be codified in formal memoranda of understanding.