Measuring the Effectiveness of Public Involvement in Transportation Planning and Project Development (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1 Purpose Consultation with the public is not only required, but fundamental to the development of transportation plans and projects. This outreach is also crucial for building public support for agency programs and securing adequate funding for transportation infrastructure. In addition to these “compliance motivated” reasons for assessing the effectiveness of public involvement, transportation agencies are also using performance measures to assess how well they serve the public. While there are widespread resources for conducting public outreach and a growing body of literature and experience on how to engage the public, there are few practical or validated methods to gauge the success of these public involve- ment approaches. This project addressed this gap by developing and testing survey and scoring tools that transportation agencies can use to measure the effectiveness of their public involvement efforts from the public’s and the transportation agencies’ perspectives. These tools can be used on all types of projects, such as new construction, infrastructure repair or change, and long-range plans. Creating the Public Involvement Effectiveness Survey The research team’s goal was to create a tool that measures the effectiveness of public involvement, while also being user-friendly and doable given the typical constraints faced by transportation agencies. The findings from the systematic literature review were used to develop questions (items), which were then categorized based on themes (indicators). These questions became the first draft of the survey. The survey was tested initially by conducting in-depth cognitive interviews with three participants who had attended a recent transportation project open house. The survey was further tested on the following three projects: • Washington State Department of Transportation Puget Sound Gateway Program (SR 509 and SR 167 extension) • Washington State Ferries’ Long-Range Plan Project • Washington State Department of Transportation I-405 Bellevue Widening and Express Toll Lane Project This testing highlighted the need to reduce the number of items in the survey not only to decrease the burden on the public when completing the survey, but also to enhance the validity and reliability of the survey. The length of the survey was reduced from S U M M A R Y Measuring the Effectiveness of Public Involvement in Transportation Planning and Project Development

2 Measuring the Effectiveness of Public Involvement in Transportation Planning and Project Development 47 questions to 38 questions by using several quantitative processes (factor analysis, principal components analysis, test of convergent validity, and reliability analysis), and qualitative processes (three focus groups with persons who participated in public involvement activities, an analysis of skipped survey items, and discussions among public involvement professionals). Two versions of the survey were created: one to be completed by the public and one to be completed by the agency. The public version and the agency version were designed to be compared with each other. The testing process also identified several challenges for implementation. These challenges included the following: • Getting all interested or affected project agencies and consultants to agree to use the evaluation tool. Transportation projects are complex, and some agencies may believe that adding a public involvement effectiveness measurement element into their project is too challenging. • Fear of exposure to criticism and the effects it could have on the credibility of an agency and its public involvement processes. • Critiques of public involvement processes may actually reflect other issues. For example, if a project faces strong public opposition, this may produce a negative bias among respondents, even if the public involvement activities are well executed. Or, because negative perceptions can be persistent, early missteps could continue to shape feedback about the public involvement activities that are conducted later in the process. In addition to the survey, a scoring tool was developed and tested. This tool allows agencies to input the survey results and automatically calculate a score to determine how well they are doing (and to compare the public results with the agency results). This scoring tool can be found on the TRB website (www.trb.org) by searching for “NCHRP Research Report 905”. Public Involvement Effectiveness Measurement Toolkit The toolkit includes a paper version of the survey for use with the public, an online version of the survey for use with the public, an online version of the survey for the agency to score itself (and provide documentation or evidence to objectively substantiate its ratings), the Excel scoring tool, and the guidelines PowerPoint presentation for using the surveys and the scoring tool. Survey for Use with the Public. This survey comprises 38 questions divided across several areas, including six indicators of effective public involvement (see Appendix A). Most of the questions are rated on a five-point scale: strongly disagree to strongly agree, with “don’t know” and “not applicable” response categories also available. The areas of the survey include the following: • Respondent demographics. These are measured in ways that allow for comparison with Census demographics (questions 1 through 9). • Influence and Impact. The goal of this indicator is to measure the extent to which public feedback has an impact on project decisions and ensure that agencies are not just eliciting feedback from the public as part of a “checklist” (questions 10a through 10g). • Transparency and Clarity. The goal of this indicator is to measure whether trust of government agencies has increased or improved as a result of the public involvement processes, and whether agencies were appropriately transparent about the project (questions 11a through 11f). • Timing. The goal of this indicator is to evaluate whether public involvement started early enough and was of sufficient length and frequency to be valuable (questions 12a through 12c).

Summary 3 • Inclusion. The goal of this indicator is to measure the extent to which the public involvement was inclusive and representative of all targeted and affected populations (questions 13a through 13c). • Targeted Engagement. The goal of this indicator is to measure the extent to which public involvement included locations relevant to the targeted and affected populations (questions 14a through 14c). • Accessibility. The goal of this indicator is to measure the extent to which public involve- ment activities used multiple methods for participation (questions 15a through 15d). • Overall satisfaction with the public involvement (question 16). • An open-ended question asking for the top three ways the public involvement could have been improved (question 17). • Types of public involvement activities the respondent had been involved in for the project (question 18). Online Survey for Use by the Transportation Agency. This survey comprises 39 ques- tions divided across several areas, including six indicators of effective public involvement. Most of the questions are rated on a five-point scale of strongly disagree to strongly agree, with “don’t know” and “not applicable” response categories also available. The agency version of the survey includes space for the respondents to indicate the type of documentation or evidence they have to substantiate their ratings. Since the indicator items on the agency version are identical to those on the public version, it is possible to compare the ratings on these items from both the public and agency perspectives. Scoring Tool. The scoring tool is in an Excel workbook that automatically calculates the geometric mean for each survey item and indicator. It also calculates the overall index score and a measure of discrepancy between public and agency scores. The higher the geometric means, the more effective the public involvement. This scoring tool also has the benefit of easy comparison of public involvement effectiveness at different times during the development and delivery of a project or plan. Guidelines. The guidelines PowerPoint presentation is titled Public Involvement Effectiveness Survey and Scoring Tool: Use and Scoring Instructions. It provides detailed instructions in text, diagrams, and visual examples for the following aspects of using the surveys and the scoring tool: • Definitions of terms • Overview of the survey contents • How to prepare and use the paper version of the survey with the public • How to prepare and use the online version of the survey with the public • Best practices for increasing the response rate • How to prepare and use the agency version of the survey • How to enter data into the scoring tool • How to use the scoring tool to measure public involvement effectiveness Conclusions Through this research process, user-friendly survey and scoring tools were developed to measure the effectiveness of public involvement. The survey and scoring tools can be used by transportation agencies—and non-transportation agencies—that conduct public involvement. The following is a list of resulting benefits: • The survey results provide a way to target areas for improvement and track such improvement over time. This aspect will be of particular interest to agencies adopting

4 Measuring the Effectiveness of Public Involvement in Transportation Planning and Project Development a performance measurement framework that includes the measurement of public involvement effectiveness. • Using the survey improves relationships with affected communities since the public will have a means for providing feedback not only on transportation projects and plans but also on the public involvement processes. • Transportation agencies that use these tools will demonstrate the seriousness with which they take their responsibilities to conduct effective public involvement.