Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 34
4
Credibility, Authenticity,
and Reputation
T
he use of social media to disseminate information, both official and
unofficial, during disasters raises questions about how to assess the
information’s credibility and authenticity. For example, although
the reach of an official message may be widened greatly if it is redistrib-
uted (e.g., retweeted), the message might have been modified in ways
not anticipated or desired by its originators. Paul Resnick, University
of Michigan; Dan Roth, University of Illinois, Urbana-Champaign; and
David Stephenson, Stephenson Strategies, examined credibility, authentic-
ity, and reputation in the context of social media and disaster response.
REPUTATION SYSTEMS
Paul Resnick observed that credibility problems have arisen in many
online systems and that a variety of approaches have been explored to
address them. One such approach is a reputation system, which formalizes
the process of gathering, aggregating, and distributing information about
individuals’ past behavior. The electronic commerce firm eBay operates
one of the largest and best-known online reputation systems, which pro-
vides buyers with a history of a seller’s past transactions along with feed-
back from individuals who purchased items from the seller. Reputation
systems have three principal functions:
• Inform participants about other participants, to help them deter-
mine if a particular participant is trustworthy.
34
OCR for page 35
CREDIBILITY, AUTHENTICITY, AND REPUTATION 35
• Create an incentive for good behavior. If participants know that
they will be rated and that the rating is publicly available, they are more
likely to provide accurate information (e.g., product listings), good ser-
vice, and so on.
• Provide a selection effect. If participants know that good behavior
will be noticed and rewarded, they are more likely to join the system. Sim-
ilarly, would-be malicious participants will know that any incompetence
or deliberate disruption will be made public—a deterrent to misbehavior.
Resnick cited two main challenges to reputation systems: the ability
to create new pseudonyms and the high cost or other barriers to entry
for newcomers. Most online sites use pseudonyms, and there are several
valid reasons for not requiring “real” names.1 A user of a reputation sys-
tem who develops a bad reputation can often easily create a new pseud-
onym. However, reputation systems can still succeed even when users
are easily able to create pseudonyms, because those that establish positive
reputations will continue to use their account, thus ensuring that positive
information is available in the system. Similarly, a user who establishes
a positive reputation has a disincentive to suddenly shift behaviors. The
lack of reputation limits that user’s ability to participate in transactions
since having a high approval rating with one transaction is much less
valuable than having a high approval rating with 200 transactions.
Resnick also noted that the low value of having little or no reputation
information creates barriers for newcomers. Research shows that it is not
likely that one can treat each newcomer as having a positive reputation
until they misbehave: when newcomers are treated as if they have a posi-
tive reputation, system managers become overwhelmed with the number
of new, poorly behaved users that must be removed from the system.
As a result, there seems to be no alternative to having newcomers pay
their dues by developing a positive reputation over time. This tradeoff,
between the utility of a well-managed reputation system and the high
cost to newcomers, is a challenge to the growth of a reputation system,
said Resnick.
Turning to the usefulness of reputation systems in the context of
disaster response, Resnick commented that some participants may be
able to develop a positive reputation through interactions before a disas-
ter occurs, whereas other participants who may in fact have very useful
information will not necessarily have established a prior reputation nor be
able to establish their reputation quickly during an event. In such cases,
additional measures are needed.
1 For example, see National Research Council, The Internet’s Coming of Age, National Acad-
emy Press, Washington, D.C., 2001.
OCR for page 36
36 PUBLIC RESPONSE TO ALERTS AND WARNINGS USING SOCIAL MEDIA
ENCOURAGING SELF-CORRECTION
Another approach to enhancing credibility is to encourage Web users
to spread correct information. When rumors of an event or disaster first
appear, authorities and trusted information brokers are often tasked with
broadcasting corrected information. However, rumors typically continue
to spread despite having been found to be false—a problem that is not
unique to the Internet or social media. Resnick mentioned that he has
begun to examine the use of social processes in the context of politi-
cal campaigns and elections. Traditional news media outlets have estab-
lished online sites that examine the truthfulness of statements made by
candidates. Although such sites are helpful to those who are already
knowledgeable and are involved in politics, few others may visit these
sites to verify information. One possibility is to use social media to mobi-
lize individuals to create pointers to such correct information. A person
motivated to correct false information could, for instance, begin to search
tweets containing misinformation and then invite other users to respond
to these incorrect tweets, including supplying links to where the informa-
tion is corrected.
COMPUTATIONAL CLAIM VERIFICATION
Evaluating the trustworthiness of a particular piece of informa-
tion can require connecting to many other pieces of information (either
reinforcing or contradictory) from a wide variety of sources (e.g., news
reports, official statements, blogs, wikis, and social media messages) that
provide useful information. Metadata, such as embedded geographical
coordinates or network activity associated with a piece of information,
can also provide valuable information. Dan Roth observed that manually
inspecting these potentially large amounts of data is difficult, a situation
that prompted his research group to develop a tool to integrate relevant
information to score the trustworthiness of claims and sources. 2
Roth’s research aims to create a tool that judges trustworthiness in
a manner similar to how a person might. Interestingly, accuracy is not
the only important factor, because information can be technically accu-
rate yet misleading. Furthermore, simply counting the number of times
information is repeated is not sufficient either. Rather, a decision on the
trustworthiness of claims and sources should, according to Roth, be based
on several characteristics: support for a given piece of information across
multiple trusted sources, source characteristics (such as reputation), the
2 V.G.Vinod Vydiswaran, ChengXiang Zhai, and Dan Roth. Content-driven trust propaga-
tion framework. Proceedings of the 18th ACM SIGKDD International Conference on Knowledge
Discovery and Data Mining (KDD’11), 2011, pp. 974-982.
OCR for page 37
CREDIBILITY, AUTHENTICITY, AND REPUTATION 37
organizational type of the source (e.g., public interest, government, or
commercial entity), verifiability of the provided information, and prior
beliefs and background knowledge of the source.
Recognizing that a single metric based on accuracy is inadequate,
Roth’s research group proposed three measures of trustworthiness:
truthfulness (focusing on importance-weighted accuracy); completeness
(thoroughness of a collection of claims); and bias (which results from
supporting a favored position with untruthful statements or targeted
incompleteness/lies of omission).3
Roth’s group’s initial design was based on computing trustworthiness
using sources and claims. However, this approach proved too simplistic,
and other factors had to be added. These included information on the
certainty (and uncertainty) of the system’s technical ability to extract
information, similarity across claims, attributes of group memberships,
independence of sources, and so on. The system also had to incorporate
prior knowledge, including some commonsense understanding and an
understanding of how claims interact with one another—in order to rec-
oncile competing truth claims, for instance.
Another aspect of checking trustworthiness is incorporating evidence.
In adding another system layer for evidence, natural-language-processing
techniques are needed to help determine what a message is actually
saying—is it supporting or countering a specific claim?
APPLYING THE “CITIZEN SCIENCE” MODEL
TO DISASTER MANAGEMENT
Past research at the University of Colorado, Boulder, and at the Uni-
versity of Delaware has found that during disasters individuals act largely
in a self-directed, collaborative way to create emergent behavior. Their
decentralized, pluralistic decision making, David Stephenson reported,
finds imaginative and innovative ways to cope with the contingencies that
typically appear in major disasters. Combining these emergent behaviors
with social media tools could provide a significant opportunity to incor-
porate the public into disaster response, suggested Stephenson.
The success of citizen science initiatives, a form of crowdsourcing
that harnesses individual observations to assist in scholarly research,
suggests that similar techniques could be very useful in harnessing the
public for help in coping with disasters. The concept is not new—the
3 J. Pasternack and D. Roth. Comprehensive Trust Metrics for Information Networks. 27th
Army Science Conference, November 29-December 2, 2010, Orlando, Florida.
OCR for page 38
38 PUBLIC RESPONSE TO ALERTS AND WARNINGS USING SOCIAL MEDIA
112th Audubon Christmas Bird Count4 was held in December 2011—but
smart phones and similar technology have made it much easier to collect
information.
Although the primary goal in citizen science projects is to produce
research and knowledge, these projects also serve as effective outreach
mechanisms, Stephenson observed. Citizen science is a powerful educa-
tional tool because it involves volunteers directly in the research process
and requires the creation of simple, easy-to-follow educational programs
that enable nonexpert volunteers to participate effectively.
Today, new technologies make the reporting of information much eas-
ier. For example, in a National Weather Service (NWS) program,5 Twitter
users are asked to report on weather conditions using the hashtag #WX.
Through it, users can contribute information that may help the NWS bet-
ter understand very localized conditions such as microbursts that may be
missed by conventional instrumentation and sources. In another initia-
tive, Tweak the Tweet,6 a special syntax using simple, short, and easy-
to-remember hashtags was developed to make Twitter messages more
focused and machine readable during disasters (Figure 4.1), thus making
it easier for people to contribute information in a form useful in a disaster
response. The syntax was rushed into service during the 2010 Haiti earth-
quake recovery and helped provide needed structure to the information
that residents and aid workers were reporting from the scene.
Many other possibilities are evident for the use of recent technolo-
gies in disaster reporting. Smart phones generally have precise location
information, and this can be made available along with a person’s mes-
sages (if a user activates this feature), making it possible for emergency
managers to map the sources of reports. Another useful capability is the
ability to send still images and video. Multiple pictures or video clips
shot from multiple perspectives could provide authorities with a virtual
comprehensive view of an event.
4 The Audubon Christmas Bird Count is a census of birds performed annually by volun-
teer birdwatchers.
5 See http://www.nws.noaa.gov/stormreports/ and http://www.nws.noaa.gov/
stormreports/twitterStormReports_SDD.pdf.
6 Kate Starbird, Leysia Palen, Sophia B. Liu, Sarah Vieweg, Amanda Hughes, Aaron
Schram, Kenneth Mark Anderson, Mossaab Bagdouri, Joanne White, Casey McTaggart,
and Chris Schenk, Promoting structured data in citizen communications during disaster
response: An account of strategies for diffusion of the “Tweak the Tweet” syntax, Christine
Hagar (Ed.), Crisis Information Management: Communication and Technologies, pp. 43-63,
Chandos Publishing, 2012; K. Starbird and J. Stamberger, 2010, Tweak the tweet: Leverag-
ing microblogging proliferation with a prescriptive grammar to support citizen reporting,
short paper presented at the 7th International Information Systems for Crisis Response
and Management Conference, Seattle, Wash., May 2010. Also see http://epic.cs.colorado.
edu/?page_id=11.
OCR for page 39
CREDIBILITY, AUTHENTICITY, AND REPUTATION 39
FIGURE 4.1 An example of Tweak the Tweet syntax used during the 2010 Haiti
earthquake. SOURCE: Project Epic Web site. Reprinted by permission.
OCR for page 40
40 PUBLIC RESPONSE TO ALERTS AND WARNINGS USING SOCIAL MEDIA
OBSERVATIONS OF WORKSHOP PARTICIPANTS
Observations on credibility, authenticity, and reliability offered by
workshop panelists and participants in the discussion that followed the
panel session included the following:
• There are many information brokers, including many not tradition-
ally viewed as official sources of news, who can serve as trusted sources in
social media, such as individuals active in a particular geographical area
or in organizations such as the Standby Task Force. Many of these bro-
kers are taking steps to verify information. Those following these trusted
brokers can in turn share this information with others who may not know
who is a trusted broker.
• Technology allows us to use distributed approaches to establishing
trust. For example, although an individual’s information may not be trust-
worthy, greater trust can be established if multiple reports with similar
information can be found.
• The ways in which people seek information during a disaster can
be different from the ways they seek information normally. For example,
research has shown that during a disaster or mass emergency situation,
people have a greater willingness to follow individuals who are different
from themselves than they do under normal circumstances. Also, they
tend to seek firsthand, “on the ground” information. Locality and hyper-
locality matter. On Twitter, formal emergency response agencies or local
media are retweeted more often than others.7
• What can be learned from past research on emergent behavior
(where groups of individuals collectively complete complex tasks they
could not do independently)? What prior results apply to emergent
behavior with social media, and what aspects might be different?
• At the same time that they are learning how to evaluate informa-
tion provided by the public, officials must also find ways to build their
credibility with the public. An effort to build credibility can be as simple
as an acknowledgment that an organization is listening to the public. This
kind of direct communication between officials and volunteers builds a
network of trust.
7 KateStarbird and Leysia Palen. Pass it on?: Retweeting in a mass emergency. Proceedings
of the Conference on Information Systems for Crisis Response and Management (ISCRAM 2010).
Seattle, Wash., May 2010.
