Measuring the Effectiveness of Public Involvement in Transportation Planning and Project Development (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

21 Toolkit Contents The toolkit has four components: 1. Survey for use with the public (paper and online versions) 2. Survey for use by the transportation agency to score itself 3. Survey scoring tool 4. Guidelines for using the surveys and the scoring tool Survey for Use with the Public The survey for use with the public (see Appendix A) comprises 38 questions divided across several areas, including six indicators of effective public involvement. It also includes space on the introductory panel for the respondent to indicate the name of the transportation project, the location of the public involvement activity, and the date. Most of the questions are rated on a five-point scale of strongly disagree to strongly agree, with “don’t know” and “not applicable” response categories also available. The areas of the survey include the following: • Respondent demographics. These are measured in ways that allow for comparison to Census demographics (questions 1 through 9). • Influence and Impact. The goal of this indicator is to measure the extent to which public feedback has an impact on the project decisions, and that agencies are not just eliciting feedback from the public as part of a “checklist” (questions 10a through 10g). • Transparency and Clarity. The goal of this indicator is to measure whether trust of government agencies has increased or improved as a result of the public involvement processes, and whether agencies were appropriately transparent about the project (questions 11a through 11f). • Timing. The goal of this indicator is to evaluate whether public involvement started early enough and was of sufficient length and frequency to be valuable (questions 12a through 12c). • Inclusion. The goal of this indicator is to measure the extent to which the public involve- ment was inclusive and representative of all targeted and affected populations (questions 13a through 13c). • Targeted Engagement. The goal of this indicator is to measure the extent to which the public involvement included locations relevant to the targeted and affected populations (questions 14a through 14c). • Accessibility. The goal of this indicator is to measure the extent to which the public involvement activities used multiple methods for participation (questions 15a through 15d). • Satisfaction. Overall satisfaction with the public involvement (question 16). • Open-Ended Question. An open-ended question asks for the top three ways the public involvement could have been improved (question 17). C H A P T E R 5 Public Involvement Effectiveness Measurement Toolkit

22 Measuring the Effectiveness of Public Involvement in Transportation Planning and Project Development • Public Involvement Experience. This question asks for the types of public involvement activities the respondent had been involved in for the project (question 18). The paper version of the survey is designed to be printed double-sided on 11 × 17 in. paper and then folded into thirds. By doing this, the business reply postage artwork panel and the introductory panel are on the outside. Having a business reply postage panel allows the survey to be mailed back to the transportation agency. This is useful for those attending public involve- ment activities who wish to take the survey with them to be completed later, or when the survey is mailed to public involvement participants. The paper version can also be designed to allow for optical character recognition, to enable scanning for easy and accurate electronic data entry by vendors who provide such services. The online version of the survey to be used with the public is identical to the paper version, except that the information on name of transportation project, location of the public involve- ment activity, and date appear as the first three questions of the survey (thus, shifting the ques- tion numbering for the remaining questions relative to the paper version). An example of the online version appears in Appendix N. There are several well-respected, easy to use, and afford- able online survey software platforms available on the internet that will work well for creating the online version of the survey. The online version of the survey can be used at public involvement events via laptop computers and tablets with internet access. It can also be used by participants who wish to complete it on their smart phones, since most online survey platforms optimize surveys for use on such devices. The online version can also be used by mailing or emailing the link to the survey to public involvement participants. Finally, the online version can be used as part of an online open house or other type of online public involvement engagement activities by placing the link on the website. A distinct advantage of the online version is that the data entry is automatic and can be downloaded into an Excel spreadsheet, which can then be cut and pasted into the survey scoring tool. The research team is confident that online versions do not bias the results, because most people now have access to the internet through their computer, tablet, or smart phone. What have traditionally been referred to as underserved populations (lower income, minorities, and elderly) are increasingly accessing the internet through mobile devices, such as smart phones and tablets. According to the Pew Research Center (Perrin and Duggin 2015), 84% of American adults use the internet and “. . . a new analysis of 15 years-worth of data highlights several key trends: For some groups, especially young adults, those with high levels of education, and those in more affluent households, internet penetration is at full saturation levels. For other groups, such as older adults, those with less educational attainment, and those living in lower-income households, adoption has historically been lower but rising steadily, especially in recent years.” Survey for Use by the Transportation Agency to Score Itself The online survey for use by the transportation agency (see Appendix O) comprises 39 ques- tions divided across several areas, including six indicators of effective public involvement. Most of the questions are rated on a five-point scale of strongly disagree to strongly agree, with “don’t know” and “not applicable” response categories also available. The agency version of the survey also includes space for the respondent to indicate the type of documentation or evidence they have to substantiate their ratings. Since the indicator items on the agency version are identical to those on the public version, it is possible to compare the ratings on these items from both the public and the agency perspectives. The areas of the survey include the following: • Information on the transportation project name, project location, and date (questions 1 through 3).

Public Involvement Effectiveness Measurement Toolkit 23 • Influence and Impact. The goal of this indicator is to measure the extent to which public feed- back has an impact on the project decisions, and that agencies are not just eliciting feedback from the public as part of a “checklist” (questions 4a through 4g). • Transparency and Clarity. The goal of this indicator is to measure whether trust of government agencies has increased or improved as a result of the public involvement processes, and whether agencies were appropriately transparent about the project (questions 5a through 5f). • Timing. The goal of this indicator is to evaluate whether public involvement started early enough and was of sufficient length and frequency to be valuable (questions 6a through 6c). • Inclusion. The goal of this indicator is to measure the extent to which the public involve- ment was inclusive and representative of all targeted and affected populations (questions 7a through 7c). • Targeted Engagement. The goal of this indicator is to measure the extent to which the public involvement included locations relevant to the targeted and affected populations (questions 8a through 8c). • Accessibility. The goal of this indicator is to measure the extent to which the public involve- ment activities used multiple methods for participation (questions 9a through 9d). • Satisfaction. Overall satisfaction with the public involvement (question 10). • Open-Ended Question. This question asks for the top three ways the public involvement could have been improved (question 11). • Public Involvement Experience. This question asks for the types of public involvement activities the respondent had been involved in for the project (question 12). • Agency-Specific Items. These are agency-specific items that are not asked of the public (question 13a through 13g). Survey Scoring Tool The scoring tool is in an Excel workbook. A full description of the survey scoring tool and how to use it can be found in the guidelines PowerPoint presentation titled Public Involve- ment Effectiveness Survey and Scoring Tool: Use and Scoring Instructions. Both the scoring tool and guidelines presentation can be found on the TRB website (www.trb.org) by searching for “NCHRP Research Report 905”. The following is a high-level overview of the scoring tool: What It Contains • A codebook showing all items in the survey instrument • A data entry tab for each indicator • Scoring tab showing the index scores How It Works • Survey data for each respondent is entered into the scoring tool (see Figure 3). • Questions 1 through 9 are meant to be completed by just the public participants (as opposed to the agency version). Questions 1 through 8 are measures of participant demographics and can be compared with the most recent Census or American Community Survey data for the project area. This comparison indicates whether the public involvement participants are representative of people from areas affected by the project. Question number 9 is designed to determine whether those participating in the public involvement have any special needs that might limit their ability to participate in the public involvement process. • Responses for the items under each of the six indicators (questions 10 through 16) are given a score from 1 through 5 (i.e., ranging from strongly disagree to strongly agree), with “don’t know” or “not applicable” as options as well. • The scoring tool calculates the geometric mean for each item, each indicator, and the overall index score. The higher the geometric mean, the more effective the public involvement. The

24 Measuring the Effectiveness of Public Involvement in Transportation Planning and Project Development geometric mean is used instead of the arithmetic mean to avoid the problem of outliers masking the true results, because very high (or very low) scores in one area can dispropor- tionately pull the average up (or down). See Appendix P for further details on calculating the geometric mean. What It Produces • A score for each item in the index • A score for each indicator in the index • An overall index score aggregating scores for all indicator scores • A measure of discrepancy between public and agency scores • Optional discrepancy calculations comparing the survey respondent demographics with the Census demographics for the area affected by the transportation project are presented on a separate tab. The discrepancy results are presented in the format used by the EJSCREEN (see https://www.epa.gov/ejscreen for more details.) Benefits of the Excel Scoring Tool • Automatically summarizes survey data scores • Allows for easy comparison of public scoring and agency scoring • Allows for easy comparison at different times within a transportation project • Allows integration with overall environmental justice efforts Guidelines for Using the Surveys and the Scoring Tool The guidelines for using the surveys and scoring tool are contained in a PowerPoint presenta- tion that provides detailed instructions in text, diagrams, and visual examples for the following aspects of using the surveys and the scoring tool: • Definitions of important terms • Overview of the survey contents • How to prepare and use the paper version of the survey with the public • How to prepare and use the online version of the survey with the public Figure 3. Example of scoring results. Source: PRR, Inc.

Public Involvement Effectiveness Measurement Toolkit 25 • Best practices for increasing the response rate • How to prepare and use the agency version of the survey for scoring itself • How to enter data into the scoring tool • How to use the scoring tool to measure public involvement effectiveness The guidelines presentation can be found on the TRB website (www.trb.org) by searching for NCHRP Research Report 905”.