Measuring the Effectiveness of Public Involvement in Transportation Planning and Project Development (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

8 Literature Review The research team used a systematic methodology to review measures currently being used to assess the effectiveness of public involvement in transportation, as well as in other relevant fields. The phases and tasks outlined below describe how this systematic process informed the devel- opment and testing of a valid, easy-to-use measure of the effectiveness of public involvement. Identifying Documents for Review To identify documents for review, the research team took the following steps: • Conducted a literature search designed to collect information and documents to meet the search criteria. The goal of this phase was to compile documentation of others’ efforts to evaluate public involvement. General materials relating to public involvement were not included, but any documents referring to follow-up, evaluation, effectiveness, and measurement were included. While transportation was the main focus, the searches also allowed for research on public involvement in other domains. This first search included the following: – Review of all 50 state DOT websites and publications, as well as the U.S. DOT website and publications – Traditional academic journal databases via Google Scholar – LinkedIn groups (Public Involvement, Engagement, etc.) – TRB committees (Public Involvement, Research, etc.) – TRB publications database – American Evaluation Association – Institute for Local Government • Emailed a link for an online form to State DOT Communication Directors, Public Infor- mation Officers, Planners, and so forth (via the AASHTO Transportation Communication Subcommittee’s list of 154 members), allowing them to provide descriptions, paste links, and upload relevant documents that addressed measuring the effectiveness of public involvement. The team also conducted phone calls as necessary to fulfill this part of the search. State DOT employees from five states and one additional anonymous respondent provided information. • Compiled documents in a reference database using an online and desktop reference management tool (Mendeley). The tool was used to store PDFs, citation information, researcher-created tags, and notes directly on PDFs. Multiple researchers accessed and modified the database, allowing for easy collaboration between team members. This process resulted in a final database of 1,065 full documents and 182 citation entries without full documents (abstracts only). C H A P T E R 2 Literature Review

Literature Review 9 Reviewing Documents for Relevance The team developed a process for identifying documents and citation entries relevant to the research objectives, and codes were assigned to each document using a tagging scheme in qualitative data analysis software (Atlas.ti). The tagging scheme was based on Rowe and Frewer’s (2000) idea that the highest level of public involvement is a two-way process where agencies communicate information to the public and also elicit feedback from the public to inform final decision making. Lower levels of public involvement are generally a one-way relationship, in which agencies only communicate information without asking for any feedback. Based on this concept, the team’s schema categorized documents by their respective levels of public involvement from the highest to lowest (1 through 3), using the following definitions: 1. Explicit evidence of a two-way process, with goals to effectively communicate information to the public and to elicit feedback from the public to inform final decision making. 2. Goal is a one-way process where agencies only elicit feedback from the public to inform decision making. 3. Goal is a one-way process where agencies only communicate information to the public for awareness and educational purposes. After coding 1,247 documents and abstracts, a selection was chosen for in-depth review. All documents in the “1” or “2” public involvement category that had a methods section, an evaluation tool, or both, were selected. All citation entries (without PDFs) that were tagged with the “1” or “2” public involvement category were reread, and it was determined that 18 articles merited further review. A total of 260 full documents met the final criteria for in-depth analysis. A team of four researchers systematically reviewed and entered data from each of the 260 selected documents into an online database. Appendix B shows the variables that were coded and collected for all 260 documents. After the information from these documents was entered into the online database, the research team analyzed the data and narrowed the list to 68 documents that closely met the objective of providing evidence-based guidance on how to measure the effectiveness of a public involvement campaign or program. Of the 68 documents, 36 included frameworks or tools for under- standing the effectiveness of public engagement. (See Appendix C for an in-depth review of the 36 articles that included frameworks or measurement tools. See Appendix D for examples of these frameworks and tools.) Key Findings of the Literature Below are four key findings from the review of the 36 articles that contained frameworks or measurement tools: • Most of the measurement tools were evaluation frameworks with indicator goals and out- comes of what successful evaluation should provide. • There were common public involvement goals across the frameworks that are seen as important for measuring effective public involvement. These included such goals as representativeness, independence, early involvement, influence, and transparency. • Most of the tools were agency “self-assessments” and did not include going back to the public (citizens, participants, stakeholders) for their opinions on the effectiveness of the public involvement process. • Very few of the measurement tools or frameworks offered actual measures or methodologies for collecting the data to measure the indicators. These instruments and methodologies were also not tested for whether they validly measured the indicators of the framework.