National Academies Press: OpenBook
« Previous: 7 Threats to Science's Reputation
Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×

8

Evaluating Science Communication

Image

Well-designed and rigorous evaluations can help answer critical questions in science communication. What types of narratives and images produce accurate impressions and lasting memories? How does a presentation’s impact vary by audience? How should statistics and human narratives be combined for maximum impact? As the narrator of the session on evaluating science communication, Arthur Lupia, Hal R. Varian Collegiate Professor of Political Science at the University of Michigan, put it, “many researchers and science organizations have limited resources to devote to communication. Many of you are being asked to do more with

Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×

less. How can we create effective science communication strategies in these circumstances?”

INFLUENCE ON TWITTER

James Fowler, professor in the Political Science Department and in the Global Public Health Division of the Department of Medicine at the University of California, San Diego, addressed the evaluation of a specific kind of communication: tweets that involve science. Many measures exist of the influence of scientists on social media, Fowler said, including levels of activity, the responses generated by a person’s activity, and a person’s influence within a network of people (Riquelme and González-Cantergiani, 2016). However, none of these measures necessarily mean that a person is influential. “You could be saying things that are completely banal to millions of people and not changing their lives at all,” said Fowler. “That’s where a lot of these measures fall down, because they’re just measuring quantity.”

On Twitter, measures of quantity specifically include tweets, replies, retweets, favorites, mentions, follows, and network indicators. However, these measures have the same problem as with other social media: they do not measure influence. “It reminds me of an ancient university proverb: ‘Academic deans can’t read, but they sure can count,’” said Fowler. “The literature has been focusing more on what a dean would focus on.”

Fowler described a way of measuring influence that incorporates quality as well as quantity. The method begins with the measure of information in a tweet. For example, if a girl gets socks from her aunt every Christmas, opening that year’s present and finding socks does not provide much information. But if the present is a brand new bicycle, that girl’s ideas about her aunt will change, because her aunt has done something that she has never done before.

Similarly, the quality of a tweet can be quantified by measuring how likely it is that a tweet would have occurred. According to information theory, the less probable something is, the more information it contains. Specifically, the negative logarithm of the probability can be summed across different pieces of information to get the total information in a message.

To measure the probability of a tweet, Fowler took the simple and transparent method of looking at all the words on Twitter that have occurred in the past 24 hours. He then measured all the words among responders for a 24-hour period and classified how likely it was that a given tweet would have randomly drawn from the set of all the words on

Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×

Twitter. “This is a good place to start because there will be some information in a tweet based on the kinds of words that are used” in responses to that tweet.

SCIENTISTS ON TWITTER

Scientists tend to be in the middle of the distributions of user favorites, followers, and friends among all Twitter users, Fowler noted. However, they tend not to tweet as much. “We’re busy, and tweeting isn’t necessarily the first thing we do.” As a result, scientists get fewer retweets per person, fewer replies per person, and fewer quotes per person. “That’s not surprising. We have lower activity, so there’s going to be fewer responses,” Fowler said. They also get less information into the Twittersphere, because they tweet less.

However, the tweets of scientists have higher average information scores than do the tweets of nonscientists. “The moral of the story is that we don’t tweet as much as other people, but when we do tweet, we tweet with a higher amount of information,” he noted.

This higher information content can result in great impact, Fowler observed. As the information score of a tweet goes up, the likelihood of retweets, replies, and quotes increases. Furthermore, this increased interest can have concrete outcomes. A study of Facebook users, for example, found that messages on Facebook led about one-third of 1 million more people to vote in 2010 than would have been the case otherwise (Bond et al., 2012).

This is the kind of approach that is needed to move beyond quantity to quality, Fowler concluded. Researchers need to “try to think of creative ways where, at scale, we can measure the quality of [messages]. It’s only then that we’re going to be able start to think about how one scientist’s actions might be able to change other people’s opinions.” For instance, could messages from scientists change policy or encourage people to become involved in citizen science? “I feel optimistic, given the comparisons that we’re now just starting to make.”

In addition, measures of quality may encourage scientists to communicate more and build more of a following, further increasing their influence. Scientists “are late comers to social media,” said Fowler, “but now a lot of them have adopted it.”

Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×
Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×
Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×

This page intentionally left blank.

Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×
Page 55
Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×
Page 56
Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×
Page 57
Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×
Page 58
Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×
Page 59
Suggested Citation:"8 Evaluating Science Communication." National Academy of Sciences. 2018. The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium. Washington, DC: The National Academies Press. doi: 10.17226/24958.
×
Page 60
Next: 9 Communicating Uncertainty »
The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity: Proceedings of a Colloquium Get This Book
×
Buy Paperback | $45.00 Buy Ebook | $36.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Successful scientists must be effective communicators within their professions. Without those skills, they could not write papers and funding proposals, give talks and field questions, or teach classes and mentor students. However, communicating with audiences outside their profession - people who may not share scientists' interests, technical background, cultural assumptions, and modes of expression - presents different challenges and requires additional skills. Communication about science in political or social settings differs from discourse within a scientific discipline. Not only are scientists just one of many stakeholders vying for access to the public agenda, but the political debates surrounding science and its applications may sometimes confront scientists with unfamiliar and uncomfortable discussions involving religious values, partisan interests, and even the trustworthiness of science.

The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity summarizes the presentations and discussions from a Sackler Colloquium convened in November 2017. This event used Communicating Science Effectively as a framework for examining how one might apply its lessons to research and practice. It considered opportunities for creating and applying the science along with the barriers to doing so, such as the incentive systems in academic institutions and the perils of communicating science in polarized environments. Special attention was given to the organization and infrastructure necessary for building capacity in science communication.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!