CHALLENGE SUMMARY
As data is ingested from and about our daily lives using smartphones, computers, and other sources, society is on a clear trajectory to have various degrees of “Full Life Recording.” “Extreme Lifelogging” is a on a decade course to allow an individual to record everything they see (Caprani et al., 2001), hear and much more with the aid of GPS tracking and on body health monitoring aids (Topol, 2012).
A log of our information interactions is a step toward management and integration of our information and ourselves. A log that includes GPS encoding, near-field interactions with devices, pointers to pictures we’ve taken, or videos we’ve recorded begins to approach the fidelity of a lifelog with a potential to serve as a lifelong complement to our internal memory and our digital selves. This lifetime log is the basis for the creation of understanding and stories.
Being able to reliably store and retrieve a lifetime of information within personal devices and the cloud is practical, inexpensive, and inevitable. This inevitably raises issues in every aspect from recording though the lifetime use and hereafter life storage that research and dozens of companies are forming to solve.
As we struggle to find, organize, and use life records, one challenge is information fragmentation of our personal information or our cyberself. We need to know where our cyberself is located, who owns and can access it, and when (now and in the future). Of course control involving permanency, privacy, and security of our digital selves is always a concern as we
continue to allow or offer more public access to our cyberself. For people’s lives that are maintained by public institutions such as universities and national libraries (British Library, 2009), maintaining life records of hybrid analog-digital people is a challenge. For example, who will be in public digital lifeboats?
We also want our logs to have existence and persistence independent of the applications and devices we happen to be using at any time and held anywhere. Our digital life is forever.
Key Questions
Will “extreme lifelogging” actually occur as determined by whether such systems can be built that serve a useful purpose i.e., are able to create a market? What privacy laws or concerns will inhibit their existence?
How can meaningful structures emerge as an effortless by-product of our interactions (with our information and with other people)? E.g., E-mail threads tell useful stories of discussions which extend across and “over” time. “Stories” are perhaps the most useful structures about a person. How can these be constructed automatically?
How much can a person’s eMemory help that person in the event of declining function including various neurological diseases?
What are the costs to store my cyberself forever? Assuming I pay for this upfront, how can I guarantee my forever existence without a physical self?
What mechanisms e.g., standards, laws, technology is required to insure the long term accessibility of digital lives such that these personal bits will be always readable?
Will the BCI (Brain Computer Interface) play into such systems? Ideally, a person’s eMemory is a person’s real and lasting memory, and a URL and metadata to access this content.
Reading
Bell G. and Gemmell J. Total recall: Your life uploaded. Dutton, Penguin: New York 2009.
British Library. First digital lives research conference: personal digital archives for the 21st century, 9 February 2009.
Caprani N, O’Connor N, and Gurrin C. Motivating lifelogging practices through shared family reminiscence. In: CHI 2011 workshop: Bridging practices, theories, and technologies to support reminiscence, 8 May 2011: Vancouver, Canada.
Jones W. (XooML: XML in support of many tools working on a single organization of personal information. In Proceedings of the 2011 iConference 2011: Seattle, Washington;478-488.
Ringel M, Cutrell E, Dumais S, Horvitz E. Milestones in time: The value of landmarks in retrieving information from personal stores. Proceedings of Interact 2003: Ninth International Conference on Human-Computer Interaction September 2003: Zürich, Switzerland.
Topol E. The creative destruction of medicine: How the digital revolution will create better health care. Basic Books: New York, 2012.
IDR TEAM MEMBERS
- Carole R. Beal, University of Arizona
- Robert M. Bilder, UCLA
- Kim T. Blackwell, George Mason University
- Chris Forsythe, Sandia National Laboratories
- Robert A. Greenes, Arizona State University
- Cathal Gurrin, Dublin City University
- Ning Lu, North Carolina State University
- Ashley Taylor, New York University
- Anthony D. Wagner, Stanford University
IDR TEAM 3
Ashley Taylor, Science Writing Scholar New York University
IDR Team 3 was asked to define the trajectory, value, and risk of extreme lifelogging.
What Is Extreme Lifelogging?
Lifelogging is simply recording any kind of information about one’s life. Older forms of lifelogs include photo albums and diaries. More modern types of lifelogs include Facebook timelines and Twitter feeds. In all these examples of lifelogging, an individual selects moments to record. Extreme lifelogging, as defined by Gordon Bell, a Microsoft researcher, lifelogger, and co-author of a book on the subject, is when you capture “everything you see and hear.” Bell thinks extreme lifelogging would be possible by 2020. By Bell’s definition, extreme lifelogging is a thing of the future. However,
several devices are bringing people closer to extreme lifelogging, and these devices collect information about one’s life in a different way than the kinds of lifelogs—Facebook, diaries—mentioned before. These lifelogging devices to collect data about one’s life and do so automatically, at periodic intervals or in response to certain stimuli, rather than handpicking moments to record. Moreover, they go beyond Bell’s definition by including other dimensions of data about the individual beyond experiences seen and heard, such as movement, location, body function, and interactions with other people or the environment. Two types of lifelogging devices are lifelogging cameras, which are worn around the neck and take photos automatically, and high-tech pedometers that automatically sync to phones and USB drives.
Lifelogging cameras
Two prominent lifelogging cameras are Microsoft’s SenseCam and the forthcoming Memoto, made by a Swedish company. The SenseCam is a camera about the size of a deck of cards that hangs around the neck and automatically takes photos depending on how you set it. It was invented in 2003 by Lyndsay Williams, of Microsoft Research. One can program the SenseCam to periodically take photos, for example, to take a photo every 30 seconds. One can also set it to take photos in response to certain environmental stimuli, such as body heat and changes in light levels.
The SenseCam and software to view the photos are currently available only to researchers, such as Gordon Bell, who has worn the SenseCam since 2003 and Cathal Gurrin of Dublin City University, who has worn a SenseCam since 2006 and has logged over 9 million photos. A commercial version of the SenseCam is available as the Vicon Revue; however, it is being discontinued at the end of 2012, according to the product website.
Though some of the hardware for extreme lifelogging is already on the market, the software for extracting useful information from lifelog data is limited. One can buy software to view photos taken by the Vicon Revue. Viewing photos in order creates a flipbook effect. However, the software to tag those images and search them remains in the research stages.
Memoto, founded in 2012 and funded, in part, by a $550,000 Kickstarter campaign, is taking pre-orders for its version of a lifelogging camera and expects to deliver its first batch of orders in April 2013. The camera, a 1.4-inch square box that clips onto clothing, will take a photo every 30 seconds and record where and when the photo was taken. Memoto has iPhone and Android apps for organizing the lifelogging data, and these
apps can connect to social networking websites if one chooses. Memoto will store customers’ lifelogging data in the cloud for a monthly fee.
Pedometers/physical monitors
Another kind of lifelogging device is a pedometer that records steps taken and other measures and wirelessly uploads them to smartphones or USB drives. One of these pedometers is FitBit One, a device about the size of a flash drive that uses its three-axis accelerometer and altimeter to record steps taken, miles traveled, and stairs climbed, and also to monitor sleep quality. It calculates calories burned. Unlike older pedometers, it syncs automatically to smartphones and to a USB drive and comes with applications for analyzing the data.
The BodyMedia armband is another automatic fitness tracker with more sensors than the FitBit. It records calories consumed, steps taken, and sleep quality, and includes an online Activity Manager and smartphone app. Unlike the FitBit, it also measures skin temperature, the heat dissipated from the skin, and the skin’s electrical conductance (a function of sweating) in order to better calculate calories burned, claiming over 90 percent accuracy. With a strap purchased from another company, the device can also measure heart rate.
State of the science
As the above examples indicate, some of the hardware for extreme lifelogging is already on the market, and more is coming. However, the software for extracting useful information from lifelog data is limited. The software to tag and search images recorded by lifelogging cameras remains in the research stages. Generally, tools for aggregating one’s own data over time and from different sensing devices are in the development stage, as are tools for aggregating the data of many people. Furthermore, there are no laws governing how lifelogging data will be used and safeguarding, or at least regulating, the privacy of personal data.
To address these deficits, IDR Team 3 came up with several proposals for how to manage lifelogging data, on a personal level, through apps, and on a societal level, by developing regulations to safeguard privacy of lifeloggers and those they capture on camera. Before delving into discussion of how to deal with lifelogging data, the IDR Team began by considering the pros and cons of the practice.
Is Lifelogging a Good Idea?
The IDR Team acknowledged that with so much data collected about people already, through Google, smartphone usage and GPS tracker logs, and our various transactions such as online purchases, for example, life logs about people are already being developed, like it or not, though individuals may not have access to the information. The question is whether people will choose to collect minutely detailed data about themselves, and if they do, how they will use the information
Potential downsides of extreme lifelogging
A digital life log could give a false sense of completeness. Though a lifelogging camera will record one’s external experiences in great detail, it does not capture one’s emotional reactions. In order to make lifelog photos searchable, they need to be tagged with words, and the tags, in standardizing and categorizing things people see, will fail to capture all the richness of life experiences.
Furthermore, losing some information could actually be a desirable feature of our biological memories, some IDR Team members thought. Forgetting embarrassing moments or unpleasant experiences is a coping mechanism. Do we want a record of everything? Do we want to actively participate in creating a record of our pasts that might come to haunt us?
The IDR Team identified several risks of collecting terabytes of personal data, primarily with respect to threats to privacy. Lifelogging would probably increase the risk and magnitude of identity theft; someone who stole one’s lifelogging data would have much more information than the thief could get from someone who was not lifelogging. Employers or insurers, if they accessed the data, might use it to discriminate against people. Would employers ask to look at life log data or provide strong incentives for employees to share the information? Would insurers give discounts to people who collect life log data and share the information with them? Lastly, life log data would infringe on the privacy of people who are not lifeloggers but are recorded in other people’s logs, for example, as figures captured by other people’s SenseCams.
Potential advantages of extreme lifelogging
That said, the IDR Team saw several potential advantages of lifelogging and identified three categories of stakeholders who would benefit from the
practice in different ways: individuals, government/society, and corporations. At the individual and societal levels, the team focused mostly on how lifelogging might be useful to improve health.
Lifelogging could help individuals better understand and predict the consequences of their actions. Whether the information was about eating habits, blood pressure or anxiety, it could help people interested in self-improvement by providing a more accurate picture of how they are now. Our group focused on the potential of extreme lifelogging to improve health, both of individuals and of society overall. The general idea is that if a rational person knew, to take a made-up example, that a certain level of exercise reduced one’s chances of a heart attack by a certain amount, that person, using a FitBit, might exercise at the recommended level and improve his/her cardiac health. Individuals improving their health would lower healthcare costs overall, which would be good for society.
That example is, of course, oversimplified. There are several caveats to the idea that extreme lifelogging could help people improve their health. First, how strong are the correlations between behaviors and outcomes? To take the previous example, how certain are scientists that exercise can avert heart attacks? The group did not discuss these kinds of correlations. Theoretically, aggregation of extreme lifelogging data about exercise combined with records of heart attacks could produce evidence in support of such a correlation, and the more data went into it, the more specific the correlation would be. Yet even if aggregate lifelogging data suggested that a person who exercised regularly had a lower chance of a heart attack, that wouldn’t change the fact that anyone could have a heart attack at any time. Lifelogging data might tell an individual the most likely outcomes of their behavior, but even then, what is most likely is not always what happens. Furthermore, people are not rational and don’t always act rationally on the information they already have about their behavior and health. More data, from extreme lifelogging, may thus not make people more rational. People’s tendencies to be irrational would limit the potential usefulness of lifelogging data.
In addition to health monitoring, individuals could use lifelogging as a memory aid, something that would be particularly useful to people with memory loss. Microsoft’s SenseCam has been shown to improve the memory of a patient with amnesia, who was better able to remember events after recording them with the SenseCam and reviewing the footage.
Corporations could use lifelogging data to see how people are using their products and generally to observe consumer behavior. People wearing
a SenseCam would be collecting data not only about their own lives but also about all the people and things they encounter in their environment. Everyone will collect information irrelevant to them that could be important to someone else. The IDR Team discussed the likely possibility that people could sell their data to corporations.
The IDR Team proposes three ideas for making lifelogging data more and for establishing standards for how the data can be used: a conference, at which people would discuss a Consumer Bill of Rights for lifeloggers; an open-source platform where people could share software for analyzing life log data; and a competition to create lifelogging apps. These proposals, outlined below, would also serve to develop a community of people interested in lifelogging.
Solutions
The IDR Team decided that it would be important to develop a policy framework for laws to protect people from abuses of life log data. For example, the IDR Team thought it would be important to develop laws to prohibit employers and insurers from discriminating against people based on lifelog data, analogous to a 2008 law prohibiting them from discriminating based on genetic data, the Genetic Information Nondiscrimination Act. In order to develop these regulations, the IDR Team proposed a conference where participants would develop a Consumer Bill of Rights for lifeloggers.
In order to allow people to explore correlations in lifelog data and draw strong conclusions, it will be necessary to gather data from many people and aggregate it. The IDR Team proposed developing an open-source platform where people who developed software related to presenting and analyzing lifelog data could share their work. People could post software components that would be useful in making apps for analyzing lifelog data. An open-source platform would aid development of software by facilitating cooperation. It could save individual app developers from creating software in parallel when they could just use existing software and focus their time on developing original programs. In addition to promoting the creation of apps, such a platform would help create a community of people interested in lifelogging.
The IDR Team then proposed a competition to encourage the development of apps that would make lifelog data useful—for example, a fitness app that would use data from physiological sensors to make health recom-
mendations. The IDR Team would provide “contestants” with data, and they would need to develop an app to analyze it.
Future Work
All three proposals are first steps. After development of a Consumer Bill of Rights would come implementation. After the creation of open-source software would come app creation. Once apps were available to the public we could expect the more exciting part: where people start aggregating and analyzing data from large numbers of participants to learn things about human health and behavior. Finally, time will tell if people will act on the information gathered from lifelogging. An outstanding question is what happens to the data when people die, something that’s an issue today and will be a larger one as people leave larger digital traces.
Lifelogging of data is a technology that will increase people’s information about their lives, and the recommendations of the IDR Team will help people to use that information wisely.