2

Mapping the Landscape

The term “citizen science” is often applied to a wide range of projects with different goals, participants, and modes of participation. All involve people, typically not professional scientists, who participate in and make use of scientific processes, data, and knowledge. The fact that all citizen science includes participation in some form of science activity and thinking means that all citizen science projects have the potential to advance science learning. The range of citizen science opportunities also means that these possibilities vary by the project, and are influenced by the goals, participants, and modes of participation of the project.

Though the committee declined to specifically define citizen science and instead elected to describe citizen science activity (as discussed in Chapter 1), many scholars have attempted to create typologies that characterize and define citizen science. Understanding the evolution of these typologies helps to understand the differing goals, participants, and modes of participation, and how citizen science can support science learning. In this chapter, we describe the evolution of how scholars have defined and characterized citizen science in order to explain the breadth of contemporary understandings of citizen science. We then turn to our own description of similarities and variations in citizen science projects and kinds of participation in citizen science. By exploring these ideas, this chapter lays the groundwork for our later discussions that connect contemporary understandings of science learning and design to the diverse kinds of citizen science.

THE HISTORY OF CITIZEN SCIENCE: EVOLVING DEFINITIONS AND TYPOLOGIES

Citizen science has antecedents in the desire to collect regular, repeated information about the natural world. In some ways, the idea of citizen science has roots in practices before science was professionalized: observations of nature as part of indigenous knowledge, agricultural and pastoral practice, and historical record keeping all bear similarities to citizen science. The notion of “gentleman” science, where people with privilege and means engaged in science and science activities as hobbyists, recalls citizen science. Officially, however, the field of citizen science emerged in reaction to the formal institution of science as a mechanism for engaging the public: A central tenet of citizen science is that science is not the sole providence of professional scientists.

The gathering of natural history data by both expert and nonexpert participants predates the development of “scientist” as a category in the mid-19th century (Jardine, Secord, and Spary, 1996), and many other historical antecedents can be found (e.g., Vetter, 2011). Miller-Rushing, Primack, and Bonney (2012, p. 286) point to deep historical traditions regarding the systematic collection of observations and information by publics, including millennia-old records documenting natural phenomena over time (referred to as phenological datasets):

Likewise, indigenous peoples around the world have and continue to develop knowledge of the natural world through their own knowledge systems that utilize systematic observation and interaction with their environments (Cajete, 2000).

Citizen science also has antecedents in the desire to affect change, and in nonscientists using scientific methods and data, often data they collected, to motivate or guide that change. Community groups have long collected data in their neighborhoods and used that data to improve lives and livelihoods (Miller-Rushing, Primack, and Bonney, 2012). Recent advances in technology, including low-cost sensors and Internet-enabled data management and communication, have contributed to an explosion in community-

led collection and analysis of data (Haklay, 2013). In the following section, we discuss the history of defining and characterizing citizen science in order to set the stage for our description of the diversity of citizen science projects and types of participation in citizen science.

Definitions and Typologies

As described in Chapter 1 of this report, the field of citizen science has not yet codified into a discipline with clearly defined criteria for what “counts.” Attempts to define the field have led in divergent directions, with multiple scholars positing different meanings and practices for the work of citizen science. In this section, we describe attempts to define and characterize citizen science. In unpacking the variation present in these efforts, the committee demonstrates the challenges inherent in trying to find a single, clear definition for citizen science. Along those same lines, by describing the variations in what citizen science signifies to different parties, the committee is setting the stage for our later discussions about how project designers can leverage variation in citizen science to support science learning.

The term citizen science has two distinct, but related root definitions. In Citizen Science: A Study of People, Expertise and Sustainable Development, Irwin (1995) employs the term “citizen science” in reference to the relationship people have to ongoing environmental concerns that benefit from a scientific understanding. Irwin employs the term as a critique of the institution of Western science, arguing that science must be accessible if it is to be useful to individuals and communities. In 1996, Bonney articulated a different use for the term: “Scientific work undertaken by members of the general public, often in collaboration with or under the direction of professional scientists and scientific institutions.”

Eitzel and colleagues (2017) compare citizen science to crowdsourcing, where a large number of people are recruited to contribute “services, ideas or content” to a project through “microtasking,” without necessarily understanding the full import of the work. For science projects, this may mean that participants are conducting tasks without engaging with the underlying science concepts, as in the gamification of a project.

Bonney and colleagues (2009b, p. 977) describe a kind of citizen science focused on large-scale data collection, where citizen participation is driven by the scales of space and time beyond any one individual that the data patterns describe:

These elements of scale have also been pointed out by Cooper and colleagues (2007) and by Danielsen and colleagues (2009) as seminal to the modern phenomena of citizen science, whether in service of encapsulating the geographic entirety of a scientific question or working at larger scales of data analysis and interpretation allowing enlightened decision making at both local and regional scales. Haklay (2013) defines “geographical citizen science” as projects explicitly collecting location information, often as part of the meta-data attached to the sample. For instance, the geolocation stamp a cell phone attaches to a photograph sent to iNaturalist (Bowser et al., 2014), a digital image data storage platform that houses more than 2,000 citizen science projects centered on crowdsourcing both image collection and subsequent species identification.

In working to define citizen science (within a youth-focused context but applying it more generally), Ballard, Dixon, and Harris (2017) emphasizes the contribution to basic research or resource management and distinguishes citizen science from projects that result only in new awareness, understanding, and skill development only on the part of the participant.

In terms of typologies that attempt to classify citizen science, one common way of describing the range of citizen science projects is to describe how the extent of control that project participants have over the direction of the project is correlated with the degree of participation (a concept often traced to Arnstein’s [1969] “ladder of participation”). In contributory projects, participants focus on data collection; collaborative projects also include participants in data analysis, interpretation and/or dissemination; and co-created projects mix the involvement of scientists and participants throughout all aspects of the work (Bonney et al., 2009a, see also Shirk et al., 2012). Danielsen et al. (2009) and Haklay (2013) point out the degree of control over any-to-all aspects of the project as crucially important, referring variously to “autonomous local monitoring” and “extreme citizen science” to describe projects in which citizens control most aspects of the project.

Wiggins and Crowston (2011, p. 1) consider citizen science typologies derived from the goal(s) of citizen science projects and project participants, arguing that a singular focus on the level of participant engagement pays “little attention to sociotechnical and macrostructural factors influencing the design and management of participation.” Thus, their schema incorporates goals of science, of individuals (via education), and of community in five mutually exclusive types identified through a cluster analysis of 80 possible attributes assessed more than 28 intentionally selected projects. They further link this schema to educational opportunities and posit the following:

• Investigation projects focus on physical data collection according to scientific standards and methods and often provide volunteers with scaffolded learning opportunities.
• Virtual projects adhere to scientific standards and methods but are entirely mediated through information and communication technologies.
• Conservation projects are primarily ecologically focused, support natural resource management or stewardship, are designed by content experts, and involve volunteers as data collectors.
• Action projects are grassroots, community projects that may employ participatory action research methods in service of addressing local concerns.
• Education projects span formal and informal learning providing youth learners with opportunities to engage in the practice of science in order to contribute data to a larger scientific effort. Within the latter type, Zoellick, Nelson, and Schauffler (2012) define Scientist-Teacher-Student Partnerships as engaging in authentic science practice, from study question through dissemination, with the students as the focus and with facilitated interaction with scientists and teachers at all stages.

Haklay (2013) divides citizen science projects involving technology into “volunteered computing” or projects using the computer resources of millions of individuals across the globe to process otherwise intractable problems without any direct interaction with the owner; “participatory sensing” in which the smart phone or other personal data-recording device is automatically used to collect environmental information with, or without additional direct input of the owner; and “volunteered thinking” or projects where the participant is trained to perform some task (e.g., image classification or analysis as in Zooniverse; Masters et al., 2016).

Summary

Clearly, no single definition can encompass the broad range of activities that exist under the umbrella of citizen science. In this section, we have described several efforts to define citizen science in order to demonstrate the complexity of codifying on the activities that occur across this broad range of projects. For some, the term citizen science refers to people contributing observations and efforts to conducting science. Those holding this view may see citizen science as a new research tool, which facilitates larger scale research. For others, the term encompasses the democratization of science, allowing people outside of the mainstream scientific establishment to conduct and govern science. Still others see citizen science as including

elements of civic education and expanding the public understanding of science (Eitzel et al., 2017). We do not attempt to define the term in a way that excludes projects; rather, we seek to understand the different projects to which the terms can apply and link those differences to different, and similar, opportunities for science learning.

Thinking about the way in which citizen science projects are constructed, the activities of the participants, and their different levels of engagement all help understand the learning that occurs in citizen science and how to design to influence learning. In the following section, the committee attempts to illuminate the complex landscape of citizen science activities by describing how projects may be similar or different across a range of axes. In describing this space, the committee found it most helpful to think in terms of the specific activities that are carried out by participants, how often those activities are carried out, and how those activities are supported. These activities in turn depend on the project’s stated goal or desired outcome, and by the degree of public participation or project control by nonscientists.

PROJECT SIMILARITIES AND VARIATIONS

For the purposes of analyzing learning, we focused on elements and attributes of citizen science that speak to advancing the educational, scientific, and community-action oriented goals of these activities. We divided these elements and attributes into two categories—those that were common across citizen science projects and those that varied among citizen science projects. Here, projects are our unit of analysis and are often identified by a specific name, a framing scientific question, a pool of participants, and multiple activities in which those participants engage. Broad similarities across projects seem to hint at necessary prerequisites for a project to be considered citizen science, and these prerequisites were validated against available research describing citizen science, including the definitions summarized above. In surveying research and projects, we also identified tensions or continua, which span the space that is collectively citizen science. All citizen science projects possess the common traits, but no single project can encompass all the tensions or exemplify the whole continuum. Like traditional science, citizen science is an inherently social phenomenon, with many actors, roles, and interactions, all of which bear on how each of these traits play out in project implementation.

Common Traits of Citizen Science Projects

In the following section, we describe several of the traits the committee identified as largely common across citizen science projects. Again, it is

important to note that the committee does not consider these traits to be inclusion or exclusion critieria; a project need not possess all these traits to “count” as citizen science. Rather, the committee describes these traits in order to help readers develop a general sense of commonalities across citizen science projects.

Citizen Science Projects Actively Engage Participants

Active engagement refers to the personal effort from the participant required to either physically and/or intellectually take part in the science. This can include a myriad of activities, including defining the problem, issue, or question; developing hypothesis; designing the study or protocol; receiving training; collecting data or samples; advising on analysis; doing data analysis and interpreting results; drawing conclusions and disseminating results/conclusions; asking new questions and taking action. Passive activities such as allowing a software program to use one’s personal computer for automated analysis (e.g., SETI at home, Anderson et al., 2002) or wearing sensors that automatically collect information about personal (Milenković, Otto, and Jovanov, 2006) or environmental health (Piedrahita et al., 2014) may be useful contributions to ongoing scientific endeavors, but the committee suggests that participating in citizen science involves active engagement and thus the possibility of learning through action. The committee also opted not to count projects where the participants are subjects of the research, even if they were contributing data (Reade et al., 2017).

Citizen Science Projects Engage Participants with Data

Projects collect data in myriad forms and, in turn, may provide access to these data to support science learning opportunities and activities. These activities may include collecting and submitting data, formulating hypotheses based on data, asking and answering questions with data, data interpretation and analysis, and using data as evidence in decision making or to back a scientific claim. Projects where participants are solely engaged in science communication or science policy but do not have a direct connection to the data generation or analysis and application do not fall under our description of citizen science.

Citizen Science Projects Use a Systematic Approach to Producing Reliable Knowledge

Citizen science projects meet widely recognized standards of scientific integrity and follow practices common in science. For example, hypotheses and interpretations are rigorously weighed against available evidence—data

are not changed, ignored, or selectively subsampled to prove a certain idea. It is important to note that these values are not exclusive to science, which means a project could be considered citizen science even if the values are anchored in traditions, cultures, and epistemologies that are not part of what is sometimes referred to as Western¹ science.

Participants in Citizen Science Projects Are Primarily Not Project-Relevant Scientists

Participants may range from children to adults. Many will not have degrees in science or extensive formal training in the content and skills of the project they elect to join; although many participants may have some scientific training or a desire for such training. Depending on the project, some participants may be classified as hobbyists (Jones et al., 2017), enthusiasts (Boakes et al., 2016), or amateurs with a high degree of expertise (Cooper and Smith, 2010). While professional scientists are typically involved in citizen science as project organizers, they may also be participants. Professional scientists involved as participants may be working outside their professional role or field of study and may be motivated by personal concerns rather than career interests.

Citizen Science Projects Help Advance Science

Here, advancing science is broadly defined. Citizen science may lead to novel discoveries. However, just as with science conducted by professional scientists, advances can include documenting known phenomenon within a novel context, replicating findings, or using science to create a local impact. This means that citizen science projects are not designed solely to educate the participants about known scientific knowledge. Neither is the purpose of these projects solely to transform nonscientists into scientists, such as is the case for many internship programs. Citizen science projects include multiple shared goals between the organizers and the participants and advancing science may be only one of those goals.

Participants in Citizen Science Can Benefit from Participation

Often, participants choose to become involved with a project because it provides tangible or intangible benefits aligning with their values and

___________________

¹ In the same way committee members are ambivalent about the term citizen science but used it because of its ubiquity, we are using Western science but noting that this term fails to acknowledge or make room for past and future contributions to science from Eastern, Islamic, and indigenous communities, among others.

motivations.² The project could simply be a venue to do something they thoroughly enjoy (e.g., “hobbyists”). The project may be personally meaningful in ways that lend intrinsic purpose to their effort. The project results could be useful to them, for example, satisfying intellectual curiosity or providing information that guides other activities and practices in their lives. The project may serve as a conduit to scientific data or information that can benefit a community or larger group more broadly. An example of this larger group value would be environmental monitoring that could impact environmental management and improve environmental conditions in the local area, for example, the Alliance for Aquatic Resource Monitoring ([ALLARM], 2018), or a project that allows people to share health information to improve their health outcomes (Wicks et al., 2010).

Citizen Science Projects Communicate Results

To support both scientific and participant benefits, an important feature of citizen science projects is that the results are communicated. Participants are more likely to persist in participating if they are aware of how the results of their work are being used (Eveleigh et al., 2014). Citizen science projects can often hold relevance for communities, policies, and scientific advancement. The potential utility of the information learned from the project is a motivation for communicating it to community members, managers, policy makers, scientists, and other interested parties. It is worth noting that decisions about how project data are handled (i.e., the extent to which it is open and accessible to the public or blocked from public use) may inform how project results are communicated.

Summary

The committee has attempted to describe above several of the common traits of citizen science. While a project does not need to possess all these traits in order to be considered citizen science (although many do), the committee noted these general trends across projects. In summary, citizen science projects tend to actively engage participants, engage participants with data, use a systematic approach to producing reliable knowledge, engage participants that are primarily not project-relevant scientists, help advance science, offer some kind of benefit to participants, and communicate results.

___________________

² In some cases, especially for youth, participants might join a citizen project because it is part of a formal or informal educational experience (e.g., Girl Scouts). The committee recognizes that this kind of participation can provide valuable contributions to citizen science projects, regardless of whether participants have complete autonomy or agency over the nature of their participation. We discuss the role of choice in participation later in this chapter in our section on Free-Choice, Voluntary, and Compensated Participation.

VARIATION IN CITIZEN SCIENCE PROJECTS

Though citizen science projects largely possess the common traits listed above, there is also considerable variation across projects. The committee delineated a number of axes across which citizen science might vary. In this section, we discuss several types of variation that we encountered in citizen science projects. In Chapter 6 of this report, we discuss the ways that these variations can influence project design and learning outcomes.

Duration of Participation

Some projects are designed to be a one-time only activity such as BioBlitzes (National Geographic Society, 2018), while others like monitoring streams (e.g., ALLARM, 2018; Dickinson et al., 2012; Wilderman, 2007) or weather (e.g., CoCoRHaS, Community Collaborative Rain, Hail, and Snow Network, 2018; Reges et al., 2016) request that participants engage multiple times over an extended period of time. Because of the regular meeting structure of formal educational contexts such as science classes, citizen science activities that can be conducted within these spaces may provide even more opportunities for repeated, sustained engagement with learners. Many citizen science projects effectively support a mix of one-time and repeated participation, as many participants drop out after a short time (Sauerman and Franzoni, 2015). This difference in time commitment can impact what participants learn and projects designed for one-time engagement are less likely to include extensive and in-depth training.

For this reason, it is worthwhile to consider the frequency or intensity of engagement in the project. Boakes and colleagues (2016, p. 2) use the terms “dabblers,” “steady,” and “enthusiasts” to categorize the likelihood that a participant will return more than once to a project. Many studies of participant involvement have found the majority of individuals spend little time—down to a single data collection session—involved in a given citizen science project. Sauerman and Franzoni (2015) analyzed the seven “most-played” Zooniverse projects and found that 90 percent of the players within each project contributed less than 20 percent of the classifications, and in some projects less than 10 percent. Jones and colleagues (2017) used interviews to characterize “hobbyists” (akin to the enthusiasts of Boakes et al., 2016) within citizen science (specifically birders and amateur astronomers) as individuals who have pursued their hobby for at least a decade; began their pursuit in childhood or early adolescence often sparked by parents, grandparents, or other family members; and continue to deepen their interest through a personalized learning ecology involving TV, the Internet, reading materials, interactions with experts or mentors, and other informal science education opportunities. Other taxonomies of participants can

come from the “serious leisure” literature, where hobbies can be described as social worlds with four levels of participants: strangers, tourists, regulars, and insiders (Unruh, 1980). Similarly, Edwards (2014) distinguishes among “volunteers,” “citizens,” and “amateurs.” These studies collectively suggest that citizen science provides opportunities for a range of different kinds of participants, from social individuals to those less interested in ongoing social interaction, and from individuals who sample widely to those who dive deeply into a single pursuit.

Modes of Communication

Although communication of project results is a common trait of citizen science (as described in the preceding section), the modes for how projects can communicate with their participants and how participants communicate with one another are quite diverse, including but not limited to Web-based, social media, in-person, telephone, and print. Within each of these modes, the topics of communication with and among participants also vary, including recruiting, training, testing, project data and results, calls to action, and social interaction. With respect to participant training, the mode of communication may impact learning and/or engagement. For instance, Gallo and Waitt (2011) found that of the total pool of 338 participants recruited to a hands-on invasive species project (Invaders of Texas), 43 percent of those attending a training workshop went on to submit observations whereas only 9 percent of participants trained online followed through to begin observations. Opportunities for in-person communication are especially high for projects that are conducted in formal education settings, where regular attendance is the norm.

Finally, modes of project communication with participants can include written information, graphic displays of information or visualization, and audio or audio-visual presentation. Kermish-Allen (2017) suggested that fewer, and simpler, modes of communication are best for reliably conveying information to participants in an online project. However, a single mode of communication will be limiting when not all participants have equal access (e.g., Internet-based communications in locations where participants do not have easy, free, or reliable access to the Internet, social media platforms of which not all members are a participant, or in-person meetings where the participants are geographically dispersed or transportation is not universally accessible) and/or where different channels are used to convey different messages/information (Parrish et al., 2017). Multiple types of visual communication can be tuned to specific content and skills learning needs within the participant community (Snyder, 2017).

Online, In-Person, and Hybrid Modes of Participation

An online project refers to one in which all aspects of the project occur virtually (e.g., projects in the Zooniverse, Masters et al., 2016). These projects can offer participants unique opportunities to investigate natural systems and phenomena that would typically be inaccessible to them, from the charismatic megafauna of East Africa (Wildcam Gorongosa, 2018) to the planets and solar systems beyond our own (Sungrazer, 2018) to the inner workings of cells and proteins (Foldit, 2018). In-person citizen science projects refer to those in which all participant-involved activities (usually training and data collection) are done physically rather than virtually. A hybrid program can mix elements from either. For example, training could be online for an in-person water monitoring project, where the samples are mailed to a lab for analysis (e.g., Global Microplastics Initiative, 2018), or training could be in person for a coastal monitoring project, in which data are later uploaded to an online database (e.g., COASST; Haywood, Parrish, and Dolliver, 2016). This variance in project type has implications for accessibility of the project to different groups of participants. Projects that have an outdoor component, for example, need to provide different things to be accessible to people with physical disabilities than do projects in online environments. Both projects, however, will benefit from improving all learners’ ability to participate.

Individual to Community-Scale Activities

Some activities may be conducted by a single person, while others require pairs, a small group, or an entire community. The level of social interaction can influence whether someone chooses to participate. Jones and colleagues (2017) reported that birders most often listed “environmental awareness” and the “opportunity to exercise” as their primary reasons for continuing to bird (~62% of the interview population), over “opportunities to socialize” (45% of the interviewees). By contrast, the Hudson River Estuary Program—a citizen science project that engages people in counting and releasing American eels along tributaries of the Hudson River—requires at least two people to monitor together and participants report that the social interactions are one of the most enjoyable aspects of the project as well as a great source of learning (Phillips, 2017).

Some projects are built around the participation of a large group of people. For social and environmental justice projects, a community focus is often paramount. For instance, the West Oakland Environmental Indicators Project—a community action program centered on issues of local air quality—explicitly focuses on bringing together community members to document and work toward eliminating toxic sources in their neighbor-

hood (Environmental Protection Agency, 2018). Social interaction can also contribute to growing community beyond the citizen science project itself. Haywood, Parrish, and Dolliver (2016) reported that 18 percent of 80 interviewees in COASST—a beached bird monitoring project in the Pacific Northwest—called out the ability to come together as a community as valuable: “One of the things that has really been a benefit for us is the ability to get together and have these kinds of conversations and have this community that has grown out of it.”

Role of Location

Centralized projects operate in a specific location, such as a park, museum, zoo/aquarium, or other informal learning center, all of which allow dozens to thousands of participants to access science activities that are connected to the context in which they occur (e.g., FrogWatch USA [Association of Zoos & Aquariums, 2018]). Citizen science projects centered in a specific geographic or ethnographic community may encourage learning about particular interactions between science and society within that context. As a result, participants in these projects may take political or advocacy actions based on the science they learned through participation (Chari et al., 2017).

Decentralized citizen science activities are conducted over a wide geographic range, which may be habitat based, as in the coral reef fish project at the Reef Environmental Education Foundation (2018), more broadly taxon based as in the birding project eBird (Sullivan et al., 2009), or even process based as in the natural event timing project at the National Phenology Network (2018).

Free-Choice, Voluntary, and Compensated Participation

Free choice occurs when participants actively and freely choose the what, where, when, and with whom of their participation. Free-choice learning (Falk, Storksdieck, and Dierking, 2007) is a concept linked to citizen science as part of informal science education, or science learning outside of the classroom. Citizen science as truly free-choice learning implies that all individuals can elect to join, stay, or leave a project. In reality, a range of challenges and barriers restrict choice. Joining requires awareness: Individuals must know about an opportunity to take advantage of it. Some projects may intentionally exclude individuals, for example not allowing the participation of minors (e.g., Patients Like Me, 2018); or unintentionally exclude individuals, for instance as a function of disability, language, or economic hardship (Conrad and Hilchey, 2011). The structure of a project or the training necessary to participate in the project may not be designed to

be culturally responsive, and its leadership may not be culturally competent (Ladson-Billings, 1995). Some participants may be closely supervised or directed by someone else, as in students required to participate in a citizen science project as part of their class activities and for which they receive a grade. Another aspect of free choice is informed choice, for instance an activity that participants believe is purely educational and may be unaware that data are also being gathered on them (e.g., Project Implicit [Xu, Nosek, and Greenwald, 2014] or Perfect Pitch Test [Wiggins and Crowston, 2011]).

Most citizen participants are involved as volunteers. However, in some citizen science programs, participants may be compensated, either because the activity is woven into their job, or in recognition of their specialized expertise, as in traditional ecological knowledge. Finally, some individuals pay to participate in science projects, as in the small fee for Project FeederWatch or other eco-tourist and conservation tourism activities such as EarthWatch (Chandler, 2017; Halpenny and Caissie, 2003).

Citizen Science vs. Using Citizen Science Practices and Activities

The practices that are common to citizen science such as creating data, using data, and displaying/analyzing data can also be used outside of citizen science. For example, an educator may teach his or her students to use a citizen science protocol to analyze water quality in a local stream, which presents a variety of opportunities for different kinds of learning but is not necessarily citizen science in itself. However, many classes take their participation to the next level by making contributions to citizen science projects when they not only collect data according to a prescribed protocol but also go on to share their data with a common project database. The use of these activities, whether fully participating in the common feature of contributing knowledge to a larger project or not, represents one way that the practices of citizen science can influence science education, so it is one of the ways the committee investigated learning from citizen science in this report.

Another mode of citizen science supports participants to act as apprentice scientists, with a goal of developing scientific skills and practices through participation in the overall activities of science, For example, Kids Survey Network at TERC (see https://www.terc.edu/display/Projects/Kids%27+Survey+Network [May 2018]) created a set of activities for youth in after school programs to ask and answer questions in social science by creating and taking surveys from each other (Kids Survey Network, 2018). The project provided opportunities to learn some of the activities of scientific inquiry, but with considerable support, and without any assumption that the results would be publishable. Again, the committee would not call this particular example citizen science, because the students did not contribute to a larger investigation, but we see the potential for similar

apprentice models to be used in the context of citizen science (e.g., Freitag, Meyer, and Whiteman, 2016).

Longitudinal Monitoring to Experimental Science

Many citizen science projects involve collecting measurements to monitor the state of something, as in many environmental quality projects. At the local level, monitoring projects may address specific environmental concerns, particularly where the issue is one of safety (i.e., environmental justice). At larger geographic scales, monitoring projects can be used to document patterns, as in the distribution and abundance of birds globally (eBird; Sullivan et al., 2009). Other projects are specifically designed to answer a particular question, and may involve an experimental design, even if the individual participant may be unaware of all aspects of the experimental work. These projects may be bounded in time, that is, they end when the question has been answered (Oliveira, Jun, and Reinecke, 2017).

Community-Based Decision Making vs. Citizen Science

In terms of how projects are designed and resulting decisions are made, citizen science projects may be led by scientists or may be led primarily by citizens. The citizen science community seems to agree that citizen science includes community-based and community-driven projects that bring professional scientists into the project either to conduct or facilitate particular tasks (e.g., analysis advice within collaborative monitoring with local data interpretation, see Danielsen et al., 2009). There is less agreement that projects that engage in the discourse about or use of science findings in a decision-making capacity without engaging directly in science practices, such as data collection, knowledge creation, or priority setting are also citizen science. For the purposes of the report, the committee is not considering projects that focus entirely on science communication or science-based decision making, as citizen science, and so we did not investigate science learning in those projects.

Summary

The preceding section describes variation across types of citizen science projects, all of which must be considered when project designers are making choices about how to set up and implement their projects. Later in this report, we will describe how decisions about these variations matter for what kinds of learning are possible through participation in citizen science, and how project designers might leverage these decisions to support specific learning outcomes.

WHO IS INVOLVED AND HOW ARE THEY INVOLVED?

A second factor that influences whether and how learning might occur is the role of the participant, and how that role is perceived by other participants. In the section below, we discuss different types of roles often embodied by project participants, and some implications for how participating in that role could lead to science learning. We conclude with investigation into who participates in citizen science and discuss how the demography of participation can inform our understanding of how to support science learning through project participation.

Variations in Types of Participation

The committee observed several different ways that an individual might enter into participation in citizen science. Given the variations in project type described above, participants must make a number of decisions about what kind of citizen science experience is important for them. In this section, we highlight a few different kinds of participations and offer insight into what that might imply about participants’ experiences and opportunities for learning.

Participants as Observer and Data Provider

Bonney and colleagues (2009a) identify one of the most common paradigms of participation: that of observer and data gatherer. Their examples, drawn from the field of ornithology, include highly structured, predigital projects such as the Audubon Christmas Bird Count, as well as more current work in which amateur birders enter their birding data collected via a variety of individualized methods online (e.g., eBird).

Data collection can involve minimal expertise on the part of participants or demand specialized training and the development of specific skills in order to acquire usable scientific data (Dickinson, Zuckerberg, and Bonter, 2010). Virtual observations are also possible, as in the annotation of digital files (still images, videos, and audio files). For example, the Galaxy Zoo project was able to categorize the morphology of nearly 1 million galaxies in the Sloan Digital Sky Survey with the assistance of online volunteers (Lintott et al., 2010).

Even within the participant as data collector, there is a huge range of ways to participate. Pocock and colleagues (2017) scored 509 environmental or ecological citizen science projects on 32 attributes and found a broad distribution of methodological approaches to data collection. They argued that there is a continuum from projects requiring regular monitoring and featuring elaborate approaches to data collection requiring written pro-

tocols and specialized equipment, to “mass participation” projects allowing one-off participation and featuring relatively simple tasks.

Participants as Competitor or Gamer

In some scientific areas, gamification (Deterding et al., 2011) can allow people to participate in science as recreation or competition. For example, Foldit is an online game in which players construct portions of large, complex proteins of unknown structure according to a set of rules and with the goal of finding the lowest energy configuration (Khatib et al., 2011). In Phylo, participants align gene sequences with no knowledge of the underlying scientific questions and answers (Kawrykow et al., 2012). Here the main objective is often to solve a problem or challenge, rather than explicit science learning.

Participants as Stakeholder/Partner

Some citizen science projects involve partnerships between the science community and nonscientists whose goals either instrumentally involve science or overlap with those of scientists. For example, the Nature’s Notebook project, run by the USA National Phenology Network, engages groups who have instrumental needs for phenology data or findings. They partner with amateur scientists, but also with professional natural resource managers who might need phenology data for their environmental management, or hiking clubs such as the Appalachian Mountain Club whose members might need phenology data to help schedule long-distance treks (Schwartz, Betancourt, and Weltzin, 2012).

Participants as Cultural Guides

Some projects engage participants not only as researchers or drivers of inquiry, but also as guides to a culture that is relevant to the project. One example is described by Charitonos and Kukulska-Hulme (2017) in which heritage learners in a language school conducted projects to study language and cultural heritage. The research model was one of action research, in which the learners were simultaneously studying and participating in the cultures and practices being examined. In these cases, participants have a critical role in not only conducting the research, but also interpreting it, and in bringing meaning-making (both personal and collective) to the work. As crisply pointed out by Medin and Bang (2014, p. 34), “If participation in cultural practices is central to our development as humans, then these practices will influence how we learn and practice science.” They argue for the importance of engaging diverse participants in science not only out of

some sense of fairness or equity but also because the diversity of cultural perspectives contributes to project outcomes.

Summary

As with variations in types of projects, there are noteworthy distinctions in how participants can engage in citizen science activities. Each of these variations impact what participants may be likely to learn through participation in citizen science. In subsequent chapters, we will discuss how different kinds of participation may be leveraged in pursuit of specific learning outcomes.

CONSIDERING THE DEMOGRAPHICS OF CITIZEN SCIENCE

In order to fully understand how citizen science can support science learning, it is essential to consider who has access to citizen science, especially in terms of groups that have been historically underrepresented in science. For this reason, the committee devoted considerable time and energy, including a review of participation literature (see Appendix A), to understanding who participates in citizen science. As Appendix A details, existing data are relatively limited, and the data we have undercount youth-focused projects and projects designed to advance community goals; the available data suggest that members of communities historically underrepresented in science, people with less formal education, and people of color are underrepresented in citizen science as well. Some projects also have an underrepresentation of women.

Despite the limited data described in Appendix A, the committee believes that it is possible to make some narrow observations about the demographics of citizen science, with the appropriate caveats. First, of course, it is important to note that more comprehensive demographic data would assist in a more comprehensive understanding of participation if more programs knew and shared who their participants were, even in an aggregate way, researchers could investigate trends in participation for a more diverse group of participants.

Second, the field of citizen science is in danger of reproducing the inequities, biases, and underrepresentation that has plagued science. Our interpretation of available evidence suggests that the majority of projects that are being studied/profiled in the peer-reviewed scholarly literature have a participant base that is well-educated, middle to upper class, older in age, and almost entirely white.

It is worth recalling the danger of underparticipation in science. A science community that is less diverse than society is less likely to engage in research relevant to the full diversity of society and less likely to do work

reflecting the priorities of those groups underrepresented, or unrepresented, in the current scientific mainstream (Hurtado, Carter, and Kardea, 1998). Less diverse science settings marginalize cultural knowledge from members of underrepresented groups (Calabrese Barton, 2012) and privilege cultural knowledge and practices from dominant groups. Indeed, some indigenous people argue that science has been used to oppress their communities (Deloria and Wildcat, 2001). The committee’s investigations suggest that these trends, all too common in professional science and formal and informal science education, reach into citizen science.

Moreover, there is no research to suggest that some groups of people are inherently less able to participate in citizen science projects because of some perceived deficit—cultural, social, educational, linguistic, or otherwise. Rather, the committee emphasizes that all participants need some encouragement or scaffolding to participate in citizen science, regardless of demography or prior experience. On the other hand, conducting citizen science in partnership with underrepresented groups, welcomed both as experts in their own culturally inflected perspectives and equal participants with something to contribute to scientific process, does allow a diversity of epistemologies, interpretations, and questions. For example, work by Bang and Medin (2010) illustrates how European-American and Native American learners interpret the relationship between self and nature differently, and how incorporating these differences can enhance ecological science work for all students. More generally, there is a robust literature from community science that confirms the educational value of respectfully welcoming participants’ prior knowledge and experience (Ballard, Dixon, and Harris, 2017; Calabrese Barton, 2012; Carlone et al., 2015; Mueller, 2009; Rahm, 2002) and recognizing that experience for its contributions to scientific understanding.

Consideration of these questions—what kinds of scaffolding are necessary and in what context—all revolve around the fundamental question of who is designing citizen science experiences for whom. As we will discuss in Chapter 7, designing in ways that remove barriers connected to assumptions about physical ability, economic resources, linguistic ability, and neurodiversity is design that respects every individual’s right to choose to engage in citizen science or science.

Summary

The preceding sections have detailed the substantial differences and similarities across the range of citizen science projects and types of participation in citizen science. These descriptions are intended to demonstrate the complex landscape of citizen science, and set the stage for our later

discussions of how project designers can make particular choices in order to achieve specific science learning outcomes.

Our analysis of the characteristics of citizen science mostly focuses on who participates and how participation takes place, and not on the kind of scientific questions that are asked in the projects. This is reflective of the state of research and practice in the field of citizen science: We know of no analysis that either looked at learning outcome explicitly in terms of the nature of the scientific question asked, nor have we seen a typology of learning in citizen science based on scientific question. Yet, as we will see in subsequent chapters on learning outcomes and design, there is strong evidence that the nature of scientific learning is influenced by the kind of question or investigation asked, and the questions asked are often part of the explicit or implicit design process. More research in this arena could shed light on a potential relationship between the kind of scientific question asked and the nature of participation and activity.

REFERENCES

Broeder, L., Devilee, J., Van Oers, H., Schuit, J., and Wagemakers, A. (2016). Citizen science for public health. Health Promotion International, 33(3), 505-514. https://doi.org/10.1093/heapro/daw086 [May 2018].

Burgess H., DeBey L.B., Froehlich H., Schmidt, N., Theobald, E.J., Ettinger A.K., Hille Ris Lambers J., Tewksbury J., and Parrish, J.K. 2017. The science of citizen science: Exploring barriers to use as a primary research tool. Biological Conservation, 208, 113-120.

Cajete, G. (2000). Indigenous knowledge: The Pueblo metaphor of Indigenous education. In M. Battiste (Ed.), Reclaiming Indigenous Voice and Vision (pp. 181-191). Vancouver: University of British Columbia Press.

Calabrese Barton, A.M. (2012). Citizen(s’) science. A response to “The Future of Citizen Science.” Democracy and Education, 20(2), 12.

Carlone, H.B., Huffling, L.D., Tomasek, T., Hegedus, T.A., Matthews, C.E., Allen, M.H., and Ash, M.C. (2015). “Unthinkable” selves: Identity boundary work in a summer field ecology enrichment program for diverse youth. International Journal of Science Education, 37(10), 1524-1546.

Chandler, M., Rullman, S., Cousins, J., Esmail, N., Begin, E., Venicx, G., Eisenberg, C., and Studer, M. (2017). Contributions to publications and management plans from 7 years of citizen science: Use of a novel evaluation tool on Earthwatch-supported projects. Biological Conservation, 208, 163-173.

Chari, R., Matthews, L., Blumenthal, M., Edelman, A., and Jones, T. (2017) The Promise of Community Citizen Science. Santa Monica, CA: RAND Corporation. Available: https://www.rand.org/pubs/perspectives/PE256.html [May 2018].

Charitonos, K., and Kukulska-Hulme, A. (2017). Community-Based Interventions for Language Learning among Refugees and Migrants. Paper from the 8th International Conference on Communities and Technologies (CandT 2017). Available: http://oro.open.ac.uk/49677/4/ACM_HCI%20Refugees_Charitonos%2BKukulska-Hulme.pdf [November 2018].

Chuine, I., Yiou, P., Viovy, N., Seguin, B., Daux, V., and Ladurie, E.L.R. (2004). Historical phenology: Grape ripening as a past climate indicator. Nature, 432(7015), 289.

Community Collaborative Rain, Hail, and Snow Network. (2018). Available: https://www.cocorahs.org [September 2018].

Conrad, C.C., and Hilchey, K.G. (2011). A review of citizen science and community-based environmental monitoring: Issues and opportunities. Environmental Monitoring and Assessment 176(1-4), 273-291.

Cooper, C.B., and Smith, J.A. (2010). Gender patterns in bird-related recreation in the USA and UK. Ecology and Society 15(4), 4.

Cooper, C.B., Dickinson, J., Phillips, T., and Bonney, R. (2007). Citizen science as a tool for conservation in residential ecosystems. Ecology and Society, 12(2), 11-22.

Danielsen, F., Burgess, N.D., Balmford, A., Donald, P.F., Funder, M., Jones, J.P.G., Alviola, P., Balete, D.S., Blomley, T., Brashares, J., Child, B., Enghoff, M., Fjelds, J., Holt, S., Hübertz, H., Jensen, A.E., Jensen, P.M., Massao, J., Mendoze, M.M., Ngaga, Y., Poulsen, M.K., Rueda, R., Sam, M., Skielboe, T., Stuart-Hill, G., Topp-Jorgensen, E., and Yonten, D. (2009). Local participation in natural resource monitoring: A characterization of approaches. Conservation Biology, 23(1), 31-42.

Deloria, V., Jr., and Wildcat, D.R. (2001). Power and Place: Indian Education in America. Golden, CO: Fulcrum.

Deterding, S., Dixon, D., Khaled, R., and Nacke, L. (2011). From game design elements to gamefulness: Defining gamification. In Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments (pp. 9-15). New York: Association for Computing Machinery.

Dickinson, J.L., Zuckerberg, B., and Bonter, D.N. (2010). Citizen science as an ecological research tool: Challenges and benefits. Annual Review of Ecology, Evolution, and Systematics, 41, 149-172.

Dickinson, J.L., Shirk, J., Bonter, D., Bonney, R., Crain, R.L., Martin, J., Phillips, T., and Purcell, K. (2012). The current state of citizen science as a tool for ecological research and public engagement. Frontiers in Ecology and the Environment, 10(6), 291-297.

Edwards, R. (2014). The “citizens” in citizen science projects: Educational and conceptual issues. International Journal of Science Education, Part B, 4(4), 376-391. doi:10.1080/ 21548455.2014.953228.

Eitzel, M.V., Cappadonna, J.L., Santos-Lang, C., Duerr, R.E., Virapongse, A., West, S.E., Kyba, C.C. M., Bowser, A., Cooper, C.B., Sforzi, A., Metcalfe, A.N., Harris, E.S., Thiel, M., Haklay, M., Ponciano, L. Roche, J., Ceccaroni, L., Shilling, F.M., Dörler, D., Heigl, F., Kiessling, T., Davis, B.Y., and Jiang, Q. (2017). Citizen science terminology matters: Exploring key terms. Citizen Science: Theory and Practice, 2(1).

Environmental Protection Agency (2018). Available: https://www.epa.gov/c-ferst/california-west-oakland-environmental-indicators-project-former-epa-care-project [May 2018].

Eveleigh, A., Jennett, C., Blandford, A., Brohan, P., and Cox, A.L. (2014). Designing for dabblers and deterring drop-outs in citizen science. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2985-2994). New York: Association for Computing Machinery.

Falk, J.H., Storksdieck, M., and Dierking, L.D. (2007). Investigating public science interest and understanding: Evidence for the importance of free-choice learning. Public Understanding of Science, 16(4), 455-469.

Foldit. (2018). Available: http://dx.doi.org/10.5334/cstp.6https://fold.it/portal [September 2018].

Freitag, A., Meyer, R., and Whiteman, L. (2016). Strategies employed by citizen science programs to increase the credibility of their data. Citizen Science: Theory and Practice, 1(1) 2. doi: http://doi.org/10.5334/cstp.6.

Gallo, T., and Waitt, D. (2011). Creating a successful citizen science model to detect and report invasive species. BioScience 61(6), 459-465.

Global Microplastics Initiative. (2018). Available: http://www.adventurescientists.org/microplastics.html [May 2018].

Haklay, M. (2013). Citizen science and volunteer geographic information: Overview and typology of participation. In D.Z. Sui, S. Elwood, and M.F. Goodchild (Eds.), Crowdsourcing Geographic Knowledge: Volunteered Geographic Information (VGI) in Theory and Practice (pp. 105-122). Berlin: Springer.

Halpenny, E.A., and Caissie, L.T. (2003). Volunteering on nature conservation projects: Volunteer experience, attitudes and values. Tourism Recreation Research, 28(3), 25-33.

Haring M.J. (1999). The case for a conceptual base for minority mentoring programs. Peabody Journal of Education, 74, 5-14. doi: 10.1207/s15327930pje7402_2.

Haywood, B.K., Parrish, J.K., and Dolliver, J. (2016). Place-based and data-rich citizen science as a precursor for conservation action. Conservation Biology, 30(3), 476-486.

Hopkins, A.D. (1918). Periodical Events and Natural Law as Guides to Agricultural Research and Practice (No. 9). Washington, DC: U.S. Government Printing Office.

Hurtado, S., Carter, D.F., and Kardia, D. (1998). The climate for diversity: Key issues for institutional self-study. New Directions for Institutional Research, 98, 53-63.

Irwin, A. (1995). Citizen Science: A Study of People, Expertise, and Sustainable Development. New York: Routledge.

Jardine, N., Secord, J.A., and Spary, E.C. (1996). Cultures of Natural History. Cambridge, England: Cambridge University Press.

Jones, M.G., Corin, E.N., Andre, T., Childers, G.M., and Stevens, V. (2017). Factors contributing to lifelong science learning: Amateur astronomers and birders. Journal of Research in Science Teaching, 54(3), 412-433.

Kawrykow, A., Roumanis, G., Kam, A., Kwak, D., Leung, C., Wu, C., Zarour, E., Phylo players, Sarmenta, L., Blanchette, M., and Waldispühl, J. (2012). Phylo: A citizen science approach for improving multiple sequence alignment. PloS ONE, 7(3), e31362.

Kermish-Allen, R. (2017). Design principles of online learning communities in citizen science. Maine Policy Review, 26(2), 80-85, Available: https://digitalcommons.library.umaine.edu/mpr/vol26/iss2/16 [May 2018].

Khatib, F., Cooper, S., Tyka, M. D., Xu, K., Makedon, I., Popović, Z., Baker, D. and Foldit Players (2011). Algorithm discovery by protein folding game players. Proceedings of the National Academy of Sciences of the United States of America, 108(47), 18949-18953.

Kids Survey Network. (2018). Available: https://www.terc.edu/display/Projects/Kids%27+Survey+Network [May 2018].

Ladson-Billings, G. (1995). Toward a theory of culturally relevant pedagogy. American Educational Research Journal, 32(3), 465-491.

Lintott, C., Schawinski, K., Bamford, S., Slosar, A., Land, K., Thomas, D., Edmondson, E., Masters, K., Nichol, R.C., Raddick, M.J., Szalay, A., Andreescu, D., Murray, P., and Vandenberg, J. (2010). Galaxy Zoo 1: Data release of morphological classifications for nearly 900,000 galaxies. Monthly Notices of the Royal Astronomical Society, 410(1), 166-178.

Masters, K.L., Oh, E.Y., Cox, J., Simmons, B., Lintott, C., Graham, G., Greenhil, A., and Holmes, K. (2016). Science learning via participation in online citizen science. Journal of Science Communication, Special Issue: Citizen Science, Part II, (15)03. doi: https://doi.org/10.22323/2.15030207.

Medin, D.L., and Bang, M. (2014). Who’s Asking?: Native Science, Western Science, and Science Education. Cambridge, MA: MIT Press.

Meymaris, K., Henderson, S., Alaback, P., and Havens, K. (2008, December). Project Budburst: Citizen Science for All Seasons. In the abstracts for the American Geophysical Union, Fall Meeting.

Milenković, A., Otto, C., and Jovanov, E. (2006). Wireless sensor networks for personal health monitoring: Issues and an implementation. Computer Communications, 29(13-14), 2521-2533.

Miller-Rushing, A., Primack, R., and Bonney, R. (2012). The history of public participation in ecological research. Frontiers in Ecology and the Environment, 10(6), 285-290.

Mueller, M.P. (2009). Educational reflections on the “ecological crisis”: Ecojustice, environmentalism, and sustainability. Science and Education, 18(8), 1031-1056.

National Geographic Society. (2018). BioBlitz. Available: https://www.nationalgeographic.org/projects/bioblitz/ [November 2018].

National Phenology Network. (2018). Available: https://www.usanpn.org/ [September 2018].

Oliveira, N., Jun, E., and Reinecke, K. (2017) Citizen science opportunities in volunteer-based online experiments. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (pp. 6800-6812). New York: Association for Computing Machinery.

Parrish, J.K., Litle, K., Dolliver, J., Hass, T., Burgess, H., Frost, E., Wright, C. and Jones, T. (2017). Defining the baseline and tracking change in seabird populations: The Coastal Observation and Seabird Survey Team (COASST). In J.A. Cigliano and H.L. Ballard (Eds.), Citizen Science for Coastal and Marine Conservation (pp. 37-56). New York: Routledge.

Patients Like Me. (2018). Available: https://www.patientslikeme.com [May 2018].

Pfund, C., Maidl Pribbenow, C., Branchaw, J., Miller Lauffer, S., and Handelsman, J. (2006). Professional skills: The merits of training mentors. Science, 311(5760), 473-474.

Phillips, T.B. 2017. Engagement and Learning in Environmentally Based Citizen Science: A Mixed-Methods Comparative Case Study. (Doctoral dissertation.) Cornell University, Ithaca, NY.

Piedrahita, R., Xiang, Y., Masson, N., Ortega, J., Collier, A., Jiang, Y., Li, K., Dick, R.P., Lv, Q, Hannigan, M., and Shang, L. (2014). The next generation of low-cost personal air quality sensors for quantitative exposure monitoring. Atmospheric Measurement Techniques, 7(10), 3325-3336.

Pocock, M.J.O., Tweddle, J.C., Savage, J., Robinson, L.D., and Roy, H.E. (2017). The diversity and evolution of ecological and environmental citizen science. PLoS One, 12(4), e0172579.

Primack, R.B., Higuchi, H., and Miller-Rushing, A.J. (2009). The impact of climate change on cherry trees and other species in Japan. Biological Conservation, 142(9), 1943-1949.

Rahm, J. (2002). Emergent learning opportunities in an inner-city youth gardening program. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 39(2), 164-184.

Reade, S., Spencer, K., Sergeant, J.C., Sperrin, M., Schultz, D.M., Ainsworth, J., Lakshminarayana, R., Hellman, B., James, B., McBeth, J., Sanders, C., and Dixon, W.G. (2017). Cloudy with a chance of pain: Engagement and subsequent attrition of daily data entry in a smartphone pilot study tracking weather, disease severity, and physical activity in patients with rheumatoid arthritis. JMIR mHealth and uHealth, 5(3), e37.

Reef Environmental Education Foundation. (2018). Available: https://www.reef.org [September 2018].

Reges, H.W., Doesken, N., Turner, J., Newman, N., Bergantino, A., and Schwalbe, Z. (2016). CoCoRHaS: The evolution and accomplishments of a volunteer rain gauge network. Bulletin of the American Meteorological Society, 97(10), 1831-1846.

Rose, D. (2000). Universal design for learning. Journal of Special Education Technology, 15(3), 45-49.

Russell, S.H., Hancock, M.P., and McCullough, J. (2007). Benefits of undergraduate research experiences. Science, 316, 548-549. doi: 10.1126/science.1140384.

Sauermann, H., and Franzoni, C. (2015). Crowd science user contribution patterns and their implications. Proceedings of the National Academy of Sciences of the United States of America, 112(3), 679-684.

Schwartz, M.D., Betancourt, J.L., and Weltzin, J.F. (2012). From Caprio’s lilacs to the USA National Phenology Network. Frontiers in Ecology and the Environment, 10(6), 324-327.

Shirk, J.L., Ballard, H.L., Wilderman, C.C., Phillips, T., Wiggins, A., Jordan, R., McCallie, E., Minarchek, M., Lewenstein, B.V., Krasny, M.E., and Bonney, R. (2012). Public participation in scientific research: A framework for deliberate design. Ecology and Society, 17(2), 29.

Snyder, J. (2017). Vernacular visualization practices in a citizen science project. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (pp. 2097-2111). New York: Association for Computing Machinery.

Sobel, D. (2004). Place-based education: Connecting classroom and community. Nature and Listening, 4, 1-7.

Stolle-McAllister, K. (2011). The case for summer bridge: Building social and cultural capital for talented black STEM students. Science Educator, 20(2), 12-22.

Sullivan, B.L., Wood, C.L., Iliff, M.J., Bonney, R.E., Fink, D., and Kelling, S. (2009). eBird: A citizen-based bird observation network in the biological sciences. Biological Conservation, 142(10), 2282–2292.

Sungrazer. (2018). Available: https://scistarter.com/project/529-The-Sungrazer-Project [May 2018].

Tian, H., Stige, L.C., Cazelles, B., Kausrud, K.L., Svarverud, R., Stenseth, N.C., and Zhang, Z. (2011). Reconstruction of a 1910-years-long locust series reveals consistent associations with climate fluctuations in China. Proceedings of the National Academy of Sciences of the United States of America, 108(35), 14521-14526.

Tsui, L. (2007). Effective strategies to increase diversity in STEM fields: A review of the research literature. The Journal of Negro Education, 76(4), 555-581.

Unruh, D.R. (1980). The nature of social worlds. Pacific Sociological Review, 23(3), 271-296.

Vetter, J. (2011). Introduction: Lay participation in the history of scientific observation. Science in Context, 24(02), 127-141. doi:10.1017/S0269889711000032.

Wicks, P., Massagli, M., Frost, J., Brownstein, C., Okun, S., Vaughan, T., Bradley, R., and Heywood, J. (2010). Sharing health data for better outcomes on PatientsLikeMe. Journal of Medical Internet Research, 12(2), 124-135.

Wiggins, A., and Crowston, K. (2011, January). From Conservation to Crowdsourcing: A Typology of Citizen Science. Paper presented at the 44th Hawaii International Conference on System Sciences.

Wildcam Gorongosa. (2018). Available: https://www.wildcamgorongosa.org/#/ [May 2018].

Wilderman, C.C. (2007). Models of Community Science: Design Lessons from the Field. Presentation at the Citizen Science Toolkit Conference, June 20-23, Cornell Laboratory of Ornithology, Ithaca, NY. Available: http://www.birds.cornell.edu/citscitoolkit/conference/toolkitconference/proceeding-pdfs/Full%20Proceedings.pdf {December 2018].

Xu, K., Nosek, B., and Greenwald, A. (2014). Psychology data from the Race Implicit Association Test on the Project Implicit Demo Website. Journal of Open Psychology Data 2(1).

Zoellick, B., Nelson, S.J., and Schauffler, M. (2012). Participatory science and education: bringing both views into focus. Frontiers in Ecology and the Environment, 10(6), 310-313.

Zooniverse. (2018). Available: http://www.zooniverse.org [September 2018].

This page intentionally left blank.