Current Research at the Intersection of Usability, Security, and Privacy
Six workshop speakers who work at the forefront of usability, security, and privacy and associated fields were asked to discuss the challenges, applicable research, and potential research needs associated with usability, security, and privacy. Their remarks are summarized below.
USABLE PRIVACY (LORRIE FAITH CRANOR)
Privacy has been described as an “adjustment process” in which humans continuously adjust the views of themselves that they present to others. In the online world, humans often rely on software tools to help them manage this process. However, many currently available privacy tools are difficult to use. Lorrie Faith Cranor’s presentation addressed areas in which usability research is needed in order to provide more effective privacy protection and explored areas in which some privacy goals may appear to conflict with other privacy goals, usability goals, or security goals.
Cranor began her talk by observing that privacy is hard to define, and quoted from a paper by Robert C. Post in the Georgetown Law Journal: “Privacy is a value so complex, so entangled in competing and contradictory dimensions, so engorged with various and distinct meanings, that I sometimes despair whether it can be usefully addressed at all.”1 She went on to provide a variety of definitions of privacy that have been offered
by public figures and in legal and other academic literature. The myriad definitions have at their core the basic notions limiting access to and providing control over personal information or contact with individuals.2
Access and control can be provided through either technical or legal and regulatory measures. Access can be limited using either laws that prohibit or limit the collection and disclosure of information or technology that facilitates anonymous transactions or otherwise minimizes disclosure. One way to provide control over personal information is through laws and regulations that mandate choice, the choice either to opt in or to opt out. Another is the use of technology that facilitates informed consent, such as tools to keep track of and enforce privacy preferences.
Although work in the past has often focused on information collected by Web sites, a wide array of current and emerging technologies will have significant impacts on privacy, including behavioral advertising, social networks, deep packet inspection, server log files, and location sharing. All of these technologies raise questions about how to communicate meaningfully about the effects that these technologies will have on privacy and about how to help people understand privacy risks that may seem distant or not relevant to them today. Related to this, different rates and patterns of use and the acceptance of these technologies suggest that different types of communication may be necessary to reach people in different age groups, of different genders, or in different cultures.
Cranor drew the connection between privacy and usability, observing that the privacy concerns that people often express seem inconsistent with their actual behavior—that is, people say that they want privacy but do not always take the steps necessary to protect it. There are many possible explanations—for example, people may not actually care all that much about privacy, or they may favor short-term benefits that may come at the cost of privacy over the long-term consequences for their privacy. But there are other possible explanations for the gap between expressed concerns and behavior: people may not understand the privacy implications of their behavior; the cost of privacy protection may be too high (including the cost of figuring out what steps should be taken to protect their privacy); or users might think that they have taken steps to protect their privacy but misunderstood those steps and actually did not. All three possibilities directly implicate usability.
One case where usability issues impede privacy protection is the use of privacy policies, which are intended to inform consumers about pri-
vacy practices and to help them decide whether those practices are acceptable or whether to opt out. However, Cranor observed that most policies are difficult to read, long, and subject to frequent change, with the result that few people read privacy policies; this suggests that privacy policies of the sort common today do not really enable consumers to exercise effective control over their personal information. Meaningful control is only possible if individuals can understand what their options are and what the implications of these options are, if they have the means to exercise the options, and if the costs (in terms of money, time, convenience, and cost versus benefit) are reasonable. Cranor described a research effort in which she is involved that aims to address these issues through the development of standardized, easy-to-read labels akin to nutritional labeling on food.
Another case is privacy configuration management. How can the creation of privacy rules be simplified even though the context may be very complex? How can people be allowed to establish privacy preferences easily up front for a range of applications? How can people be helped to realize when adjustments to these settings are needed and to adjust them easily or automatically? Cranor described a research effort studying some of these privacy configuration issues: the location-finding service, Locaccino, developed at Carnegie Mellon University. The application includes capabilities for defining with whom, when, and where location information is shared. It also provides information about who has asked to view a user’s location and who can view that information currently, and it is instrumented to collect feedback on how comfortable users are with this information.
The compelling functionality as well as the significant privacy impacts of location-finding services is illustrative of the conflicts that can arise. How can the need to store information be balanced with the need to discard information to provide privacy? Examples of such conflicts involve not only information used to improve application functionality but also information used to automate privacy configurations. Similar tensions arise between privacy and other interests, such as the need to store access data for auditing purposes versus the need to protect employee privacy, or the needs of law enforcement versus the need to discard information to protect privacy. Are there technical solutions that can preserve privacy while enabling these functions?
Anonymity tools can enhance privacy in certain situations. These tools typically hide users in cover traffic or send traffic by way of a circuitous route that is difficult to trace back. Users typically give up speed, convenience, or functionality in exchange for this anonymity. The tools must also be turned on and off, which is cumbersome and requires explicit user action. Are there ways of providing anonymity without degrading the user experience?
Cranor ended her talk by presenting a series of slides listing a number of the research questions discussed above. She closed by posing three questions with broad implications for privacy and usability as well as future research on these topics:
As today’s youth grow up with their lives online, will they come to expect less privacy?
As we increasingly trade off privacy for convenience and functionality, are we doomed to a slow erosion of privacy that eventually leaves us with minimal expectations of privacy?
Can “usable privacy” be designed into technology to provide convenience and functionality without sacrificing privacy?
ECONOMIC ISSUES OF USABLE SECURITY AND PRIVACY (NICHOLAS ECONOMIDES)
The talk by Nicholas Economides addressed how the incentives of both users and companies with respect to usable security and privacy are not currently structured to maximize social benefit.3 Most users do not have sufficient incentives to secure their computers to prevent network-wide catastrophic events, and they might find it very difficult to implement sufficient security even if they had sufficient incentives. What economic and legal policies can be implemented to change the incentives of users, software and hardware companies, firms conducting electronic commerce, and companies providing online services such as search so that they are closer to maximizing social benefit? What are some possible economic motivators for usable security and privacy from the perspective of the end user, private companies, and society? How do economic incentives change when viewed domestically versus globally?
Economides began by noting the significant security deficiencies of computing devices and software today, the complexity of the interfaces that define security functionality, and the poor knowledge that users typically have about the level of privacy present in the software and services that they use. The Internet is widely understood to have both multiplied the security problems of connected devices and highly increased the global impact that results from a local lack of security. Indeed, typical users have a very limited understanding of the network capabilities of their computers and the possibilities of abuse in a network setting.
A similar phenomenon was noted in Don Davis, “Compliance Defects in Public-Key Cryptography,” Proceedings of the 6th Usenix Security Symposium, San Jose, Calif., 1996, pp. 171-178, available at http://world.std.com/~dtd/#compliance.
The question of incentives can be approached from a number of perspectives, such as those of the individual or residential user; private companies (which have different perspectives depending on the nature of their business); the overall network or societal interests; vendors of hardware, software, and services; and Internet service providers.
Even individual users face a myriad of choices with respect to their activities that depend on computing, communications, and storage capabilities. It is not clear that users do—or even can reasonably be expected to—understand the financial or other consequences to themselves or others from poor security in any of these choices. Do users have sufficient economic incentives (either rewards or penalties) to use sufficient security? Improved usability of security would make it possible for at least those users who aim for higher security to achieve it at reasonable cost.
Private firms’ views on security and privacy vary widely. Some firms, such as banks, investment brokers, and electronic commerce firms, generally desire higher levels of security and have found various private solutions to make their transactions more secure. (The level of security achieved and the investment that they make reflect such firms’ view of the costs and benefits and will not necessarily provide a level of security demanded by broader societal interests.) Other firms, such as online advertisers, tend to favor more retention or disclosure of private information so that they can use this information to identify products and services that better match consumer preferences. Economides observed that, as a result, a very secure online world in which users are made fully aware of the impact of disclosures of their private information would cut into the profits of these firms. Other firms that produce operating systems and other software have not fully adjusted to today’s world in which the exploitation of even small security flaws can have global consequences. Operating systems’ producers do not face full liability for the damage that may be caused by security flaws. Once sold, many systems will persist for years; security issues and questions about incentives apply not only at the time of purchase but also throughout the useful life of the product. Internet service providers (ISPs) have an interest in furthering the security of end users, given that breaches can affect their own networks; ISPs may also view security as an attractive value-added business. Given these diverse perspectives, Economides observed that a consensus among companies on security and privacy is unlikely.
From a societal point of view, the value of security is much higher for the network than it is for an individual user. That is, users, left on their own, will generally tend to achieve lower security than what society desires. Low security at the nodes can lead to catastrophic network events that are much more damaging to society than to the individual node. The owner of the node does not face the network-wide financial and other
liability that low security at the node causes. The lack of security at a node is, therefore, a negative externality to the network. Similar considerations apply to the vendors of hardware, software, and services.
Economides posed related questions about the incentives for security:
What legal and economic policy changes would help improve the usability of the security of operating systems, Web sites and services, or Internet service providers?
How can the usability of security be improved (and thus its cost reduced) so that users who aim for higher security are better able to achieve it?
When usable security is available, how can economic incentives be created so that users will aim for sufficient security?
A variety of potential incentives might be considered. These include positive monetary incentives, awards and other nonmonetary positive incentives, and punishments. Negative incentives would include end-user liability for damage caused by insecure nodes, liability for vendors, or regulation. For example, regulations could prohibit computers that fail a basic security test from being connected to the Internet, or they could prohibit systems from being shipped with known insecure default settings. There are also thorny policy issues that apply in individual sectors. For example, blocking access on the basis of a security test limits to some extent the rights of computer owners. Also, there may be a tension between asking ISPs to play a greater role in limiting or preventing some attacks and ensuring that carriers comply with network neutrality principles such as not prioritizing content.
Economides closed by posing the following key questions regarding incentives for security and privacy:
How can society best deal with the negative externality for the network and society that is created by the lack of usable security of individual network nodes?
How can positive and negative, monetary, and nonmonetary incentives be provided to both users and private-sector firms to reduce or eliminate the negative externality?
How can the usability of security be improved so that the costs are lowered for users who aim to achieve higher security?
WHAT WOULD USER-CENTERED SECURITY LOOK LIKE? (ANGELA SASSE)
Angela Sasse started with the observation that user-centered approaches to designing technology start with understanding user requirements. To do that, researchers and developers try to establish the following:
The needs of the target users, plus specific capabilities or limitations that they have;
The tasks and business processes that the users have to perform; and
The physical, cultural, and situational context in which the interaction takes place.
However, since security is not a primary goal of users (protecting data, transactions, and systems is secondary to “getting the job done”), users often experience security as something that gets in the way of their activities as opposed to being something that is valuable. How can security be made less of a “barrier” that gets in the way of user goals? How can the user effort required be reduced? When is it reasonable to expect users to expend extra effort on security? What are existing user needs and values that could be connected to security?
Sasse then turned to the reasons that usability is important for security. She observed that the results of failure to make security usable are much more widespread than is generally realized. For users, this failure manifests itself as errors, frustration, annoyance, and individual loss of productivity. For organizations, there are the risks of system failure, the alienation of customers, and damage to organizations’ reputations and impacts on their business processes and performance. For society, security ends up being seen as an annoyance or obstacle rather than as something that should be valued. Poor security makes possible attacks that undermine trust and confidence.
Sasse offered a framework for thinking about usability that includes the following elements: the users and actors (including individuals and organizations), the activity (the goals of the interaction [the “what”] and the tasks and processes to be carried out to achieve those goals [the “how”]), and the context (including physical, situational, and cultural aspects). In addition, one must consider the system or technology platform in question.
gain access to services, accommodations should be made for user groups that have particular requirements.
In terms of activity, it is important realize that security is a secondary or enabling activity. From a user’s perspective, security at best slows down the completion of a task, and at worst it can prevent the user from achieving a goal. From an organization’s perspective, security consumes resources and slows down business processes; at worst it may stop business processes altogether. As a result, the needs of business processes and user tasks impose performance requirements on security tasks.
A number of contextual factors have a bearing on usability and privacy. These include the physical environment, situational factors such as the impact of interactions and failures, and cultural factors. Cultural factors include behavioral norms such as the acceptability of touching equipment, or reactions to the prohibitions on smiling associated with some face-recognition systems.
Security has both costs and benefits. Individual costs include the physical workload (e.g., additional keystrokes or mouse clicks) and mental workload (e.g., remembering passwords). Both actual and perceived costs are relevant. Organizational costs include the cost of operating security capabilities (including training and maintenance) and the cost when these capabilities fail. The impacts of security extend beyond business efficiency to employee behavior, trust, and goodwill. These costs and benefits are weighed in each decision about whether or not to comply with security measures. Such decisions are affected by the design of the security system, the organizational culture, and the extent of monitoring and the possibility of sanctions for noncompliance.
Sasse closed by listing the following as key research challenges:
Identifying and understanding trade-offs,
Developing ways to quantify and compare costs for different usability and security criteria and for different stakeholders,
Identifying and reconciling individual and collective goals with respect to security, and
Developing a better understanding of the short- and long-term impact of security measures on individuals, businesses, and society.
SECURITY IN VIRTUAL WORLDS (FRANK GREITZER)
Social media such as blogs, microblogs (e.g., Twitter), social networking sites (e.g., Facebook), and virtual worlds provide new tools for individuals to communicate, play, and work. Because these virtual communities are being used for many of the same things that people do in real life, they are becoming plagued by many problems and crimes of
the real world—including theft of identities and virtual assets. Identity and access management is a particular challenge in virtual environments because it is difficult to establish that an online identity is in fact the real-life person that it claims to be. Moreover, online tools do not necessarily provide protection that is strong enough to protect confidential discussions (and it may be appropriate today to shift such activities to a private environment).
This suggests the need for a better understanding of the security issues that threaten trust and privacy in these environments and for a better understanding of the role played by usability. Frank Greitzer noted several conventional cybersecurity challenges that may play out in different ways in virtual environments. These include what sorts of authentication and credentials are most appropriate in virtual worlds, who should be responsible for managing credentials and verification, and how authentication and identification can best be manifested in a virtual environment.
One of the most important research questions concerns the human factors and usability implications of proposed solutions. How can someone trust that the person (avatar) with whom he or she is interacting is accountable? For any particular solution, how can the solution be made usable and trustworthy for individuals who participate in virtual worlds? Finally, Greitzer underscored that validation—how to evaluate the effectiveness of proposed solutions—is essential.
FEEDING PRACTICE BACK INTO RESEARCH (MARY ELLEN ZURKO)
Mary Ellen Zurko discussed how to integrate lessons learned from practice into research thinking, noting that not only should research results inform practice, but practice and real-world experience with development, deployment, and use also should inform research. Issues that can only be understood in this context include scaling; performance; usability, accessibility, and user experience; and the total cost of ownership and return on investment.
For example, the security weaknesses of text passwords have been revealed by understanding their use and changes in their use. In the early days, passwords were used primarily by a handful of professionals to access a single computer. Today, people make use of passwords for a wide array of services, each of which has different strength requirements and management policies. The result is that almost all forms of deployed security using passwords are weak in terms of both usability and the security that results. Researchers are exploring alternatives to passwords for authentication, but these have many barriers to deployment, such as those
associated with the scale of enrollment and the need to retrofit complex infrastructures that only support passwords.
Another connection between practice and research is the real-world constraints that affect the deployment of research results. For example, a researcher might come up with a better way of presenting a user with information about how much trust to place in the claimed sender of an e-mail message. In the real world, the space available for presenting this information may be significantly constrained by an e-mail client’s user interface. Products routinely have a number of features competing for space in the user interface, with designers making decisions based on factors such as primary use cases, sales criteria, organizational politics, esthetics, technical difficulty, and maintenance. Such trade-offs, commonplace in practice, need to inform research so that researchers can successfully transfer their results into practice and products. Such technology transfer depends on the development of tools and best practices that allow practitioners to incorporate research results on user-centered security into the systems that they design, build, and operate. It also depends on the development of criteria and approaches for evaluating how usably secure a system or approach is likely to be. The transfer into practice can be facilitated through standards groups such as the Web Security Context Working Group of the World Wide Web Consortium. Intellectual property concerns can also be a barrier to uptake.
Zurko proposed a number of ideas that would encourage a greater emphasis on technology transfer concerns within the context of the research environment. Most obviously, funding specifically targeted at usable security research addressing uptake issues would drive progress in that area. Venues for publishing the results of such research are critical, as one of the main activities of researchers is to publish. Framing devices such as use cases, frameworks, and challenges can inspire and structure potential research and its results.
Zurko suggested several opportunities for research to be informed by experiences with deployed systems, including the following:
Conducting user studies of deployed technology, including contextual analysis;
Measuring changes in user behavior in response to changes in services;
Using open-source and free-product betas as a source of information on user behavior; and
Studying the characteristics of deployed security, through such techniques as tiger-teaming.
The presentation closed with the observation that although there is no substitute for the ground truth of real-world experiments, there are also constraints on what can be done in these settings. One should not be able to make changes that deliberately impair materially the security of an operational system. As a result, experiments with security in real-world settings require controls and oversight, much as efficacy and safety considerations govern the conduct of drug trials.
CYBERSECURITY INSIDER THREAT (DEANNA CAPUTO)
Deanna Caputo began her presentation by discussing the problem posed by trusted insiders. Espionage, intellectual property theft, and sabotage involving computer networks are among the most pressing cybersecurity challenges that threaten government and the private sector. Surveys reveal that current or former employees and contractors are the second-greatest cybersecurity threat, exceeded only by hackers. The insider threat is manifested when human behavior departs from compliance with established policies, regardless of whether it results from malice (malicious insiders) or a disregard for security policies.
Because insiders can make use of the privileges that they have been granted, they do not need to engage in behaviors that break explicit rules, making it difficult to detect these actions. What are the possible signatures of lawful but suspicious activities? How can these detection mechanisms be made usable by security analysts? How can the interests in detecting suspicious behavior be balanced with the privacy interests of employees?
Caputo went on to describe work being done at the MITRE Corporation to address these questions. This work includes the development, testing, and piloting of a prototype detection system known as Exploit Latent Information to Counter Insider Threats (ELICIT). It uses sensors to collect information used to detect and prioritize potential threats. It is based on a characterization of how trusted insiders use information, and it uses information about both the user and the information context to differentiate malicious and legitimate activities. Caputo commented that the resulting information allows time-consuming and costly threat validation and forensic investigation to be concentrated on a small number of prioritized cases.
Work on ELICIT prompted a team of social scientists and engineers to explore experimentally how malicious insiders use information differently from how a benign baseline group uses information.4 Caputo
discussed preliminary results from the double-blind study of malicious insiders, which revealed some counterintuitive results. One surprise was that malicious insiders tend to grab and go—favoring quantity over quality—contrary to expectations that insiders would be “low and slow,” working meticulously to avoid raising suspicions. Caputo also offered some essential aspects gleaned from these efforts of approaches for detecting insider threats. The work has also involved the development of test data to represent both malicious and benign users.
The work has also informed practical guidance developed by MITRE for handing these threats.5 The following measures can be used by organizations to defend against the insider threat:
Make employees the first line of defense. Educate them about spotting suspicious behavior. Understand that satisfied workers are less likely to be disgruntled insiders.
Pay attention to employee behavior. Look for signs of vulnerability, unexplained wealth, and so on.
Prioritize assets. Concentrate monitoring resources where it matters most.
Know what baseline behaviors on the network look like so that anomalies can be recognized. Enumerate trust relationships with other organizations because their insiders can become your insiders.
Divide responsibilities. Separate duties for key functions to reduce exposure.
Grant least privileges, and audit for privilege overentitlement.
Prepare for recovery through continuity of operations and data backup plans.
Caputo also described work on the insider threat by several other research groups. Shari Lawrence Pfleeger and Joel Predd at RAND have developed a framework for understanding the insider threat and a taxonomy for describing insider actions, and they are developing a framework for response to the insider threat. Frank Greitzer at the Pacific Northwest National Laboratory is looking at behavioral data to support predictive modeling and analysis in order to improve situational awareness for the security analyst, facilitate response coordination, and help the analyst focus on the highest-risk activities. A prototype system is under development that provides enhanced visual analytics and a multilayered user
Mark Maybury, How to Protect Digital Assets from Malicious Insiders, The MITRE Corporation and Institute for Information Infrastructure Protection. Available at http://www.thei3p.org/research/mitremi.html; accessed February 25, 2010.
interface encompassing displays for high-level status as well as detailed monitoring.
In terms of areas for further research, Caputo posed the following questions:
What trade-offs associated with insider threat monitoring are there between the individual’s right to privacy and the organization’s need to protect its assets?
What are the implications of pre-interventional activities such as monitoring and the collection of data and predictive modeling? How might they affect morale or violate employee trust or legal guidelines? What is the potential for false accusations or misuse?
What is the impact of user profiling, and what are the ethical and legal issues surrounding this approach?
Finally, Caputo noted that research on the insider threat would be aided by good operational data samples.