The Computer Science and Telecommunications Board (CSTB) convened a workshop on July 21-22, 2015, to advance dialogues on privacy between technical and policy staff of the Intelligence Community (IC) and outside experts from academia and the private sector. CSTB is a standing board of the National Academies of Sciences, Engineering, and Medicine. The workshop was sponsored by the Office of the Director of National Intelligence (ODNI).
To conduct this workshop, a workshop steering committee was appointed to identify potential speakers and design the workshop agenda. The committee and Academies staff worked with ODNI to invite staff from IC agencies. Approximately 40 participants, including the steering committee, invited panelists, IC staff and officials, and Academies staff, participated in the 1½-day workshop held in Washington, D.C.
This report has been prepared by the workshop rapporteur as a factual summary of what occurred at the workshop. The steering committee’s role was limited to planning and convening the workshop. The views contained in the report are those of individual workshop participants and do not necessarily represent the views of their employers, the workshop participants as a whole, the steering committee, the Academies, the sponsor, or any other affiliated organizations.
The workshop was designed around the following three major areas:
- Privacy implications of emerging technologies,
- Public and individual preferences and attitudes toward privacy and the social science and behavioral economics of privacy, and
- Ethical approaches to data collection and use.
Two panels were devoted to the first topic, one panel was devoted to the second and one to the third.
The workshop was designed to be as interactive as possible, with an emphasis on discussion and engagement rather than lengthy presentations. Opening remarks were delivered by Fred H. Cate, workshop steering committee chair and C. Ben Dutton Professor of Law at Indiana University, and Alexander W. Joel, civil liberties protection officer, ODNI. Each panel was moderated by a member of the steering committee, with the following format: panelists each presented 5 minutes of opening remarks, participated in a moderated panel discussion, and then engaged in open discussion with all workshop participants. The workshop concluded with a final wrap-up panel, during which participants summarized and discussed key points and reflections from the proceedings.
Workshop discussions focused on privacy implications of various technologies and practices, individual privacy preferences and behaviors, privacy policies and practices of organizations, and the broader societal impacts of privacy. In particular, the invited speakers from academia and the private sector each provided his or her expertise and/or perspectives, with content ranging from academic research on consumer privacy
behaviors to corporate strategies for privacy assessment; lessons from academia and the private sector were often discussed in the context of the work of the IC.
Several themes recurred throughout the workshop. To assist the reader, recurring themes are briefly discussed below.
Many participants noted that “privacy” means different things to different people, and that this can vary highly with context. It was suggested that the term is often used to connote a range of associated values or principles; throughout the workshop, a range of examples emerged, such as trust, security, the right to be forgotten, freedom, and anonymity.
One panelist identified a common definition of privacy as “the ability to control what happens to one’s information.” Others defined privacy violations, or issues, in terms of whether a given practice, policy, or action regarding personal information might be perceived as negative by a stakeholder (for example, individuals or regulators).
Participants discussed different conceptions of privacy, touching on both legal and philosophical considerations. Several participants suggested that it may not be possible or practical to develop a universal definition of privacy largely due to its contextual nature. Someone suggested that the inability to be defined is an intrinsic characteristic of privacy, and that it is something that society must struggle over. A panelist pointed out that an inability to clearly define or quantify privacy could confound those working toward protecting it. Another participant suggested that privacy might be easier to understand and address by focusing on one of its associated values or principles at a time.
Two of the panels focused on the privacy implications of emerging technologies, touching on the Internet of Things, smart and connected vehicles, mobile communications and devices, biometrics, health information technology (IT), cloud/edge computing, big data analytics (data mining, aggregation, etc.), and online advertising. It was noted that computing technologies have become ubiquitous, and that there are more and more ways and places that data are being collected.
One of the panelists noted a shift in some emerging technologies from privacy by trust (where individuals must have faith that technology services will not misuse their data) to privacy by design (where privacy is considered during every phase of the design process to minimize potential privacy issues), citing the evolution of approaches to smart and connected vehicles.
In general, participants noted that privacy implications of emerging technologies can be hard to anticipate. In particular, a panelist pointed out that transformative technologies are often fundamentally new, so it can be hard to predict how they will be used and what privacy implications could emerge. One of the panelists described a tool developed in her research group to help software developers identify potential privacy and fairness issues in their code.
Moving Beyond Legal Compliance
Multiple participants suggested that organizational compliance with existing laws and regulations around data practices is not sufficient to protect privacy and/or preserve public trust in an organization that works with potentially sensitive data. Laws and regulations take time to create, so they often lag behind technological advances. Several participants suggested that the fact that a practice or action is not illegal does not make it acceptable; privacy is defined by values, not the laws that aim to uphold them, and people have reacted negatively to perceived as well as actual privacy violations. Multiple participants suggested that
organizations must develop and continuously adapt their own internal policies and practices to protect privacy—beyond those that are legally mandated—in order to be effective and maintain the trust of their stakeholders and the public. A participant suggested that the public wants to see evidence that their data are treated with care and respect.
Many workshop participants suggested that transparency is critical for building trust in an organization or a technology. Several panelists suggested that individuals may be more likely to trust a given tool or service if they are (1) provided contextual information on how it works and how it uses individuals’ data, or (2) given more control over this use.
It was also pointed out that, due to the generally secret nature of its mission, the IC likely does not have access to the same transparency-enabling tools or mechanisms available to private sector organizations. Several participants noted that transparency does not necessarily require direct disclosure of an organization’s practices or the specific data that it is using. For example, an organization could provide illustrative rather than actual examples of its practices, or it could provide transparency to a trusted third-party or oversight body, who might then provide assurance to the public that a given practice is considered and appropriate.
There was also discussion about how public perception impacts trust. A participant pointed out that transparency about a given practice can generate trust, and make individuals more likely to give an organization the benefit of the doubt in future cases.
Several participants implied that trust can be justified or misplaced, and productive or undermining to privacy. At least one participant raised the issue of “pseudo-transparency,” when individuals falsely believe (or are led to believe) that they have an accurate understanding of (and/or control over) how their data are being used, which could lead to misguided trust and complacency. Another participant suggested that publicity about pseudo-transparency could potentially lead to public outcry.
Multiple participants suggested that building trust is not simple, and that it takes time. Many suggested that an organization can build trust by being more transparent—for example, about how data are used and the value they generate (to consumers or society), steps taken to protect privacy, and oversight mechanisms. Several participants suggested that building trust requires a long-term commitment to clear and accurate communication, both within an organization and externally.
There was much discussion about strategies that organizations have taken, or might take, to protect privacy and improve trust. Several participants from the private sector noted that it can take time for an organization to develop the rigorous data management practices needed to protect privacy and build trust with users or constituents. They suggested that the process is iterative, and that an organization’s practices can develop and improve with time through sustained effort, evaluation, engagement with stakeholders, and adjustment. Multiple participants pointed out various factors, such as changes in technological capabilities, legal requirements, or individuals’ privacy expectations, that make it difficult to anticipate potential privacy challenges or concerns. Several participants suggested that privacy is a moving target, and that an organization must be willing to continuously revisit, evaluate, and adapt its practices to best accommodate the changing privacy landscape.
There was some discussion of how organizations might operationalize privacy decision-making. Several participants discussed the strategy of asking whether revealing a given data practice would embarrass the organization. Several participants from the private sector noted the contextual nature of privacy and suggested that each decision must be evaluated individually, with consideration for all stakeholder impacts.
There was some discussion of important roles within an organization. For example, individuals can be designated to promote privacy as a core value beyond simple legal compliance, propose alternative strategies,
and anticipate future challenges. Organizations can establish internal mechanisms for privacy oversight. Several participants pointed out that some organizations have limited resources; a panelist suggested that a small core group of privacy professionals might be augmented by designating one or more people within each operational unit as privacy liaisons to the core.
There was some discussion of internal vs. external regulation and oversight of government and private sector organizations’ data privacy practices. Several participants suggested that internal regulation could be more responsive, agile, and thorough than external regulation. Another participant suggested that external regulations might prompt organizations to focus on compliance rather than outcomes, and also might lag behind current technologies. Others suggested that internal regulation is subject to bias toward an organization’s own interest, and that external regulation is necessary for transparency. Many participants noted that external input, guidance, or oversight could help to bring balance, and to build trust among those external stakeholders whose privacy is at stake.
Privacy Research Results, Challenges, and Needs
One of the functions of the workshop was to expose members of the IC to outside research related to privacy. Multiple panelists discussed their own research, as described below.
- Fuming Shih, senior product manager, Oracle Cloud, discussed his research around smart phone user privacy preferences and behaviors.
- Steven M. Bellovin, Percy K. and Vidal L. W. Hudson Professor of Computer Science, Columbia University, noted that his research involves creating a new formal definition of privacy and the harms that result from various activities.
- Carl Gunter, professor of computer science, University of Illinois, provided insights from his work on privacy and security in health IT.
- Roxana Geambasu, assistant professor of computer science, Columbia University, discussed her research aimed at increasing privacy online, including the development of tools to help users understand how their personal data are tracked and used, and to help programmers detect “privacy bugs” while developing applications.
- Idris Adjerid, assistant professor of management, University of Notre Dame, discussed his research on the economics of privacy with a focus on behavioral economics.
- Jessica Staddon, associate professor of computer science, North Carolina State University, discussed some of her work related to user perceptions of transparency tools.
- Joseph Turow, Robert Lewis Shayon Professor of Communication at the Annenberg School for Communication, University of Pennsylvania, discussed his survey research related to digital relationships and surveillance in the context of marketing and retailing, including results of a recent survey addressing consumer attitudes about private sector tracking and collection of their data.
Workshop discussions also addressed the broader privacy research landscape, and multiple participants highlighted challenges associated with work in this area. Several suggested that massive private sector data sets are generally underutilized for research purposes, probably because of disincentives for such research in the private sector. One participant suggested that academic researchers tend to be limited to small data sets and generally lack access to private-sector data.
Several participants suggested that some studies on privacy preferences and behaviors have yielded conflicting results. It was pointed out that surveys and studies must be carefully designed, and results and
individual behaviors carefully interpreted, in order to yield meaningful conclusions. A participant identified the need for a culture of repeatability, and for consistency and objectivity in measurement. Several participants noted existing strategies for such design and interpretation of experiments and surveys, and reiterated the highly contextual nature of privacy.
Several participants called attention to areas in which little research has been done and where more would be helpful, including the following:
- How individuals feel about their own privacy vs. that of others,
- Whether secrecy undermines trust,
- Privacy preferences among different demographic groups, such as lower-income populations and minority groups, and
- Social (rather than individual) costs and benefits of privacy.
Several participants discussed the notion of a “science of privacy.” A member of the workshop steering committee suggested that, within this framework, grand challenge problems could be identified and data sets could be developed and shared to advance privacy research. Another pondered whether such a formal framework might help organizations develop tools for operationalizing privacy decision-making. One participant suggested that the contextual nature of privacy could make this very difficult, and another pointed out that the field of ethics already offers a rigorous basis for deriving actionable principles. Many suggested that more research about privacy is needed.
A member of the workshop steering committee suggested that researchers might be able to make progress on some of the IC’s privacy challenges if given “toy problems,” or representative problems, that can be shared publicly but embody critical challenges. Another member suggested that deeper engagement between the IC and academia could facilitate stronger communication of the IC’s commitment to compliance around privacy.
Individual Preferences and the Privacy Paradox
Several participants pointed out the phenomenon of individuals who say they care about privacy but nonetheless seemingly act against their own interest, termed by one as the “privacy paradox.” Possible reasons for such behavior were discussed. In particular, several participants suggested that individuals often do not have the time or knowledge to deduce the privacy implications of their actions or to learn what they may do to enhance privacy. They also may not understand how a given technology works, or what companies are doing with their data. A panelist noted that humans do not always behave rationally in the economic sense, especially when it comes to privacy decisions. It was also suggested that there may be tension between what an individual believes to be “the right thing,” and what he or she wants in the moment. Another panelist suggested that an individual’s intuition or level of familiarity with a given app or service might play a large role in individual decision-making.
Several participants cautioned against the common assumption that people are comfortable giving up their data in exchange for the benefits of using a given technology or service, suggesting that this is a faulty assumption. One panelist suggested that fair trade-offs between privacy and utility are not feasible, due to the limited number of options provided for how to use a technology or a service, and because the value of an individual’s data depends upon what other data exist that they might be combined with and how, and is thus always changing and difficult to pin down. The panelist also suggested that, for these reasons, any notion of a “privacy market” will fail. Another panelist noted survey evidence suggesting that many individuals actually feel resigned to the fact that their data are being collected, and feel that this condition is simply beyond their control. Several participants suggested that consumers may feel that the use of various technologies, such as the Internet or mobile devices, is an all-or-nothing proposition: Either they get the convenience of these technologies while giving away data they would actually prefer not to share, or simply do not get to use the technology.
There was also some discussion of the idea of “tipping points,” that is, points at which individuals’ perspectives on a technology or practice shift, causing them to alter their behavior or attitudes, possibly by ceasing to use a given tool or by pushing back against a given practice. A participant suggested that tipping points likely occur at the individual rather than societal level, but that a series of mini-events that trigger tipping points could cause a critical mass of individuals to change their perspectives. There was some discussion of visible events that might fall into this category, such as the abuses investigated by the Church committee, the Office of Personnel Management (OPM) breach, the Ashley Madison breach, and the Snowden disclosures.
Many participants noted societal benefits of data collection and use, for example, to advance public health or national security. One participant suggested that the field of health IT could be a valuable evolving case study on balancing the use of information for public good (such as disease prevention) with individual privacy.
Several participants suggested that while privacy is often considered an individual value, privacy itself can also have important, collective societal benefits that are not always taken into account. For example, private and anonymous voting can help promote robust democracy, and privacy can empower individuals to explore non-majoritarian views and facilitate freedom of thought.
Several participants also suggested that certain demographic groups may be disproportionately impacted by privacy issues, and that such impacts may be undercounted. It was also pointed out that little research has been done on how privacy preferences vary between demographic groups. Several participants suggested that more research is needed in these areas.
Several participants noted that recent discussions around big data and privacy have emphasized protection of privacy via control of how data are used rather than by limiting their collection. One participant suggested that privacy advocates are uncomfortable with this notion, because even the best use control policies can be changed in ways that open up pathways for unintended or harmful use of stored data: If data do not exist, none can be abused. Another participant pointed out that the public seems to have accepted the practice of massive collection and aggregation of data even in the absence of a rigorous argument demonstrating that this is defensible, and suggested that there might be value in revisiting this acceptance.
There was also some discussion about evolving societal values. One participant suggested that the principles underlying existing societal norms were honed over time from important societal values, and they should not simply be discarded as technology advances. A participant questioned whether these underlying values were being upended by the rapid evolution of technology. Several participants cautioned against the notion that technology itself is an uncontrollable force, and suggested that we should focus not only on emerging technologies, but also on how they are deployed throughout society.
