Integration of National-Level Geospatial Ecological Tools and Data (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

129 1. Usability Testing • Nine sessions scheduled. • Six sessions conducted with participants from Colorado, Texas, Washington State, California, and Missouri. Table I.1 lists participant numbers, testing dates, and organi- zations involved in the usability testing. The findings from the usability testing are listed below. The more significant issues are bolded. Home Page • Participants liked the visual appearance and simplicity of the home page. • It was not clear to some participants how to get started from the home page. There was a Get Started link at top, and also three compelling calls to action in the center area. • After reading the description of Eco-Plan and Eco-Plan Advanced at the bottom of the home page, many partici- pants had the impression that Eco-Plan was free and Eco-Plan Advanced was paid. Therefore, they hesitated to learn anything more about Eco-Plan Advanced. Get Started • Compared with the theme maps, participants liked the ability to quickly flip through different maps on the Get Started page. • Several participants suggested the capability to overlay multi- ple theme maps on one view. For example, users can see both protected areas and endangered species on the same map. • The seven accordion panels were not always successfully loaded. Some participants saw only the first four panels. • A couple of participants did not realize that the first panel was already opened by default, and wondered why the first panel was not clickable. • Several participants complained that the legend, when expanded, blocked a large area of the map. • There were too many colors in the legend. It was difficult for participants to distinguish the different shades of green or blue. • One participant pointed out that the colors could not be recognized by color-blind users, like someone in her orga- nization. She suggested showing the data type not only in the legend, but also in the pop-up box. • It was not apparent to participants that the colored areas on the map were clickable. • Most participants did not notice the left and right arrows in the pop-up box at the beginning. Some found the arrows after seeing “(1 of 5)” at the top bar. • It was confusing to participants that clicking on an area did not always pop up a content box with information in that area. Sometimes it showed information about the county that this area belonged to. • Participants pointed out that not all the data in the pop-up box were useful, such as IDs and distribution policy. One user suggested distinguishing the data for its owners and public users. • Population was mentioned as a helpful addition to the pop-up content box. Maps • Most participants didn’t understand the differences between the maps on the Get Started page and the maps on the gal- lery page. Their impression was that the gallery-page maps must be somehow different from the maps on Get Started. • The title “Map Gallery” made many participants feel that this section had just static map images or examples. • Participants did not understand how the maps were grouped together and suggested gallery titles that would be more descriptive than “Gallery 1,” “Gallery 2.” • It was easy for all participants to locate options at the top of the image, such as Details, Legend, Basemap, and Search box. In the meantime, most participants paid little attention to the three links below the map. A p p e n d i x i Test Results

130 • Most participants wondered if they could filter the data through the legend. • The map did not indicate which basemap was currently presented. • Several participants mentioned the text below the map was helpful. Some suggested making it more accessible, like moving it above the map. • Most participants were able to locate the link of “upload your transportation plan” on Map Details or Get Started page. Some would like to upload multiple Shape Files or project data in different formats. • The level of detail of the map data seemed all right for novice users but not sufficient for expert users. One participant particularly mentioned that he would usually look at high- resolution maps at less than 10-ft scale. • Expert GIS users found that the theme maps did not offer additional data that they didn’t have in their organization or could not be found elsewhere. • Many participants would like to have more theme maps. Data • The first paragraph introduction test on the Data Gallery page was confusing to most participants who read it. They didn’t know what “the Integrated Ecological Framework” was, and the last sentence in the paragraph was not straightforward. • Some participants expressed that they expected to view data and maybe images of the data/map on the details page, but found only metadata on the Eco-Plan site. • The metadata on the details was helpful. Participants were particularly interested in the publication date and how often the data set was updated. • Presenting the connections between the data sets and the theme maps on the Eco-Plan website would be helpful for many users. Eco-Plan Advanced • Most participants saw the descriptions about Eco-Plan and Eco-Plan Advanced on the home page but didn’t clearly understand the connections and differences. • Several participants thought Eco-Plan Advanced was the paid version of Eco-Plan, so additional functions were available there. • The page title of the Eco-Plan Advanced home page was “Eco-Plan.” Most participants who navigated to this page thought they were still on Eco-Plan. • When they were on the map details page of Eco-Plan and were looking for additional functions like filter and share, they could not find these functions. Neither did they intend to click “Open Eco-Plan Advanced” link, because the page didn’t indicate what was available there and users didn’t recall what they read on the home page at this moment. • Participants who were familiar with ArcGIS wondered what the differences were between Eco-Plan Advanced and ArcGIS. • Participants found that the filter, share, print, and search function and being able to view a larger map were all very useful. • The function to export or download data from a map was suggested by a few participants. Some would export/ download the data and customize the format of presenta- tion with their desktop tools. 2. Functionality Test Feedback Figures I.1 through I.3 summarize the functionality test feed- back. Tables I.2 through I.6 summarize the responses to the test questionnaire and comments on the scripts for the Eco-Plan Test and the Eco-Plan Advanced Test. Table I.1. Participant Data from the Usability Testing (2013) Participant Number Testing Date Organization Participant 1 February 11 North Front Range Metropolitan Planning Organization Participant 2 February 11 North Central Texas Council of Governments Participant 3 February 12 North Central Texas Council of Governments Participant 4 February 17 ICF International Participant 5 February 18 California Department of Transportation Participant 6 February 18 East-West Gateway Council of Governments Dissatisfied, 2, 12% Satisfied, 10, 63% Very Satisfied, 4, 25% Figure I.1. Ease of Use question results.

131 Neither, 3, 19% Satisfied, 8, 50% Very Satisfied, 5, 31% Figure I.2. Usefulness of Content question results. Dissatisfied, 2, 12% Neither, 2, 13% Satisfied, 8, 50% Very Satisfied, 4, 25% Figure I.3. Navigation question results. Eco-Plan Count Relevancy of the theme maps to your transportation planning Relevant 11 Already use the data 1 Too coarse 1 Relevant but some issues 1 Confusing content or terms Yes 5 No 6 Initially confusing 1 What do you like most? Information and Data 6 Ease/Simplicity 4 Getting Started 3 Maps 2 Speed 1 Design 1 What do you NOT like? Theme maps 2 Map legend 1 Map search not working 1 Map gallery 1 Navigation 1 Additional content or function you want Different data 4 Add maps to ArcDesktop 1 More layers 1 Keep location across pages 1 Geoprocessing tools 1 Dynamic legend 1 Full-screen maps 1 How would you use this site? Preliminary scoping 6 Data 2 Overview for nontechnical audience 2 Won’t use 1 Eco-Plan Advanced 1 Eco-Plan Advanced Count What functions do you like most? Interact with maps and add data 7 Save own maps 3 Add own data 3 Data 3 Speed 2 Basemaps 1 Ease of analysis 1 What additional functions would you like to have? Link accounts 1 Sketch or draw 1 Export layers 1 Additional symbology 1 Attribute tables 1 Default to my location 1 Create communities 1 How would you use Eco-Plan Advanced web application in your planning efforts? Share maps 3 Create maps 2 Upload data 2 Develop LRTP 2 Find areas of concern 1 Basic corridor planning 1 Browse layers 1 Are the differences between Eco-Plan and Eco-Plan Advanced clear to you? Yes 14 No 0 Maybe 1 Table I.2. Summary of Responses to Test Questionnaire

132 Table I.3. Comments on Eco-Plan Test Scripts Tester # Task 1: Review the Eco-Plan Website Task 2: Get Started Task 3: Review the Map Gallery Task 4: Explore Additional Map Functionality Task 5: Review Eco- Plan Data Tester 1 Would recommend removing Alaska/Hawaii specific leg- end info or putting it after that of the lower 48. Location search not working. Map defaults to Surry County, NC. Tester 2 Yes, page is well organized. The content box did not pop up. It appears this row is not active. When I hovered the mouse above this selection, it did not high- light like the other cells in the column. Can you put some informa- tion here to define “linear features” and “polygon features”? Did not see “Find address or place.” A window appears, but no content was in the window and does not respond when clicked. There is no “data” option in the top navigation window. Tester 3 Appears okay. Not great but not bad. This was confusing because the “accordion” does not always show. Yes, but maps don’t correlate with each other. Each one starts off somewhere else. Not immediately. Does not say if it is still processing. Appeared dead. Tester 4 Looks nice; logical categories. Maybe not “upcoming” any- more. Can scroll around map and zoom in and out. The Details and Legend links are a bit hard to find. Zoom worked, but the map lost the landcover display— GAP displays up to the 2 mi. scale. The address function did not work for the watershed map, critical habitat map, or protected areas map. Worked on one map that I tried (Protected Areas), but then the cursor was stuck on identify and I couldn’t move around the map or zoom. The identify function did not work on the GAP map. “Upcoming feature” pop box will allow user to upload a shapefile. (continued on next page)

133 Table I.3. Comments on Eco-Plan Test Scripts Tester # Task 1: Review the Eco-Plan Website Task 2: Get Started Task 3: Review the Map Gallery Task 4: Explore Additional Map Functionality Task 5: Review Eco- Plan Data Tester 5 The banners at top are too big; can’t get whole page on one screen without scrolling past them. Should at least be able to keep the drop-down menus on screen and look at map. Maybe put the EcoPlan banner on the left side? Suggest drop-down menus from the top level click- ables. Then a user can more easily map the site. No content box pops up. Can scroll the map to my area. Note that the protected areas color (reddish) is the same as the letters used to name the protected areas. So, hard to read because on top of each other. The legend should state what vegetation map is used. It’s just called USGS landcover. But, is that the NLCD? Is it National GAP? Is it NatureServe? The species for critical habi- tat are not defined. I like the clickable data! There was no housing + jobs layer, at least for the southern Bay Area section I was looking for. You have the road density, which is good. I would remove ref- erence to the activity den- sity unless you can include it. I would put the road density above the pro- tected area status in your web description. The text names “Map Details,” but the drop- down menu at top says “Maps.” I’d use the same term in both places. Need a sentence at top of Map Gallery telling people how to use the page, similar to the home and get started pages. Gallery 1 maps seem good. Gallery 2—I think you already found the bug. A couple days ago, you could only see some of the lay- ers that were in there. Generally it was diffi- cult to look at road density and activity density at the same time, and activity density seemed to not be working. You could leave it with road density, for a transportation planning. Now road density seems to be missing. I like the clickable for- mat to get info, but somewhere near the top there needs to be instructions that some layers need to be zoomed in before they can be seen/ used. In the current (2/14) version, I see that the road density is still in the “details” section, but no longer in the legend. Yes, but see notes above on mouse functionality and on having more than one land- cover or colored set of data (grids or polygons). No current upload capacity The UCD team has particu- lar interest in working with this function because it will allow comparison of what we found vs. what’s in national data sets. It might be a lot of work, but it might be nice to provide a thumbnail image of each data set—both for the zoomed out view and the detail view. (continued) (continued on next page)

134 Table I.3. Comments on Eco-Plan Test Scripts Tester # Task 1: Review the Eco-Plan Website Task 2: Get Started Task 3: Review the Map Gallery Task 4: Explore Additional Map Functionality Task 5: Review Eco- Plan Data I like that you removed the labels for the criti- cal habitats; they were previously in red, and unreadable on top of a red background. Suggest you determine the 3 to 4 things peo- ple can do on the web version of map crawling, and have a short written instruc- tion for those, maybe as another button next to “details and Legend.” This would be different from the “how to use” that is buried in the text. I think that section should include (as it does) things about how to add data, to go to ArcGIS online, types of analyses that can be done, etc. The button on the banner would tell people, “zoom in to see some data layers,” “click on locations to get data,” “scroll by . . .” things like that. Gallery 3—suggest you rank the protected areas by area pro- tected, or at least put the big 3 federal at top—NPS, BLM, USFS . . . possibly followed by other federal, then by state, and then by other. Same for National Map Gov Units. Gallery 4—it’s neat that the HUC units vary with the level zoomed in or out, but this is not clear in the leg- end. Needs to add wording like “varies with scale” or something. (continued) (continued on next page)

135 Table I.3. Comments on Eco-Plan Test Scripts Tester # Task 1: Review the Eco-Plan Website Task 2: Get Started Task 3: Review the Map Gallery Task 4: Explore Additional Map Functionality Task 5: Review Eco- Plan Data When scrolled out, the mouse stayed on “click,” calling up the Huc level and also scrolling the map. This is a glitch. Sug- gest that you use right mouse click to get info, and the left mouse click to navi- gate the maps. Doesn’t seem to work to have them both on the same command. Or, have an “info” but- ton in the banner that switches the mouse from navigate to data retrieval mode. Generally, there is work to clean up this leg- end as well. Right now the watermoni- toring icons jump out more than anything else. It this the thing that should be at the top of the Legend? What does “High Reso- lution” mean? How do you toggle between the USFWS categories and the many other, non- identified categories? That mouse glitch is really annoying! See above. Ok, but why two sets of streets? What does “oceans” mean? Generally I think more attention to the text communication would improve user utility. Tester 6 Looks good. There were no links to click that I could see. I don’t know if this is possi- ble, but it would be nice to be able to turn off and on the gov. units. The land cover overwhelms everything else on the map. How are the classifications done in the legend? The splits seem kind of random. I don’t see this (View Water- shed and wetlands maps). I don’t see this (Review the potential for development in unprotected areas). I don’t see this (Next steps). Are there supposed to be maps in galleries 3 and 4? And why are there multiple galler- ies? Is there a signifi- cance to them? Maybe bold the data set names or some- thing here to make them stand out more. Despite all the addresses/ places here, the map only seems to take you to selected places. Is there a place to go to view the metadata? If not, it is hard to decipher what the field names mean. “Cancel” was not found, just “X.” Error message when trying to draw one of the layers. I don’t see “View Current Data.” (continued) (continued on next page)

136 Table I.3. Comments on Eco-Plan Test Scripts Tester # Task 1: Review the Eco-Plan Website Task 2: Get Started Task 3: Review the Map Gallery Task 4: Explore Additional Map Functionality Task 5: Review Eco- Plan Data Tester 7 User tips, things to be aware of, and source links are extremely helpful for users. Good job adding those! This is a helpful feature and will make the data and site more usable. I searched for “Saint Louis, Missouri,” and an option for that location was available. I selected it, but it did not zoom me to selected area. Not sure if it’s supposed to zoom me to queried loca- tion, but I would expect it to. Tester 8 Link is broken or option is not completed. I also had difficulty scrolling down past #4 once I went out and came back to the page. I’m not sure if this is a problem with the pro- gram or an internal error with our server/Internet. The layer works fine, but the data are coarse. It was hard to see anything at the default scale. It was also difficult to distinguish some of the colors (e.g., shades of green). However, the ability to zoom in and data provided by the pop-up tables when you click on the individual features (e.g., wetlands) was very helpful and provided the informa- tion I was looking for. “Maps” loads, but it takes a while. The delay was long enough to make me think that the link was broken. I’m not sure if this is working properly. The content box comes up, but only says that the feature is coming. It does not mention the acceptable data format. Slow in opening on my com- puter, but there were Inter- net issues earlier in the day. In addition, I had an error message in trying to load the Layer STORET Water Quality Monitoring Loca- tions. The message said at first, the layer was not responding and then that the layer could not be added to the map. Tester 9 No need to click on this. When starting on the “Get Started” page, it’s already open. Maybe this is a limitation of the data, but this is not showing the types of species. At the original extent, the legend suggests that I should see a lot more; it’s not the case until I zoom close in. My only issue is that the maps are very narrow. A drop-down box with sug- gested addresses shows up, but I am not taken to the place I inputted. Pop-up comes up with “Upcoming Feature.” In IE8, a new browser opens up. Tester 10 I think that Eco-Plan and Eco-Plan + should be more visible, so the information on the page should be vertical and not horizontal. The map should be the U.S. so that the user does not have to do so much manipulation. I did not see the question or request for an email I do not find this information very useful. The layer came up, but because there are two layers of information “at-risk species”— and critical habitat— one of the layers needs to be more transparent. Also, it would be great if there was a way to enlarge the map. The map does not go to the address. The text box sits over the area so that changes that come up as different infor- mation is presented on the map; it is obscured by the text box. Not yet. It is really step 7, and then I would have to know to open it and look for Eco- Plan Advanced. (continued) (continued on next page)

137 Table I.3. Comments on Eco-Plan Test Scripts Tester # Task 1: Review the Eco-Plan Website Task 2: Get Started Task 3: Review the Map Gallery Task 4: Explore Additional Map Functionality Task 5: Review Eco- Plan Data The critical habitat mapping is variable—sometimes it comes up—sometimes it does not and often it is not correct—at-risk species is not helpful. It blends those with legal protection and those that do not, which does not provide very good data to an environ- mental planner. There were no wetlands shown on the map. The critical habitat layer is too variable and inaccu- rate—it works fine on ECOS and IPaC so not sure that issue is here. Yes—helpful because after I read this description, I realized that NatureServe layer is not helpful because too generic and G1/G2 mean nothing to me. The results of the analy- sis by NatureServe of species that are Imperiled (G1/G2) or listed under the U.S. Endangered Species Act (ESA). The At-Risk layer does not identify which species are endangered, only the number of at-risk spe- cies within each 12-digit Hydrologic Unit (HUC). Tester 11 Doesn’t work. This is really ownership. How is “city land” protected? It isn’t owned by the city. Is it possible to have the leg- end sensitive to what is visible in the map? There is a lot on “land cover” that doesn’t retain. Geography search—i.e., Colorado, rather than zoom in/out on map. An indicator telling you which basemap is depicted would be helpful. And a 3next to the selected map in the drop-down so you know where you are. Doesn’t work. Stayed at default location. Tester 12 I think the home page looks very nice and not too much information for the user to sift through and decide where to go next. The con- tent at the top and options are self- explanatory and correlate to what a user might be looking for. Right off the bat without clicking on it, I wasn’t sure what “Systems” means though. No, if you click on the “Com- ing Soon: Upload your Transportation Plan” in the upper right corner, a pop- up box is displayed that describes the upcoming feature and asks for an e-mail message. Yes, this works, but when you click on a polygon of at-risk species, the attri- bute table that pops up is hard to understand what each abbreviated attribute is. Example: AQ_TOT, WT_ TOT, etc . . . . not sure what these mean? I put in an address in the search box and could not get anything to come up on the map. It finds the address when I type it in but does not display on the map. (continued) (continued on next page)

138 Table I.3. Comments on Eco-Plan Test Scripts Tester # Task 1: Review the Eco-Plan Website Task 2: Get Started Task 3: Review the Map Gallery Task 4: Explore Additional Map Functionality Task 5: Review Eco- Plan Data Yes, this works, but when you click on a polygon of hous- ing + jobs, the attribute table that pops up is hard to understand what each abbreviated attribute is. Also—why is the Critical Habitat in Step 4 not dis- played in addition to the US Protected Areas, and USFWS Critical Habitat? It seems all of those would be important to determine risk of development. Tester 13 No e-mail prompt. Legend link could be bolder. Pop-up often blocks part of watershed, or blocks part of Legend. Hard to discern whether USGS Protected areas, NPScape Road Density, or SmartLocationDatabase is displayed. Perhaps use different “families” of col- ors for 3 data sets. Legends should be accessed the same way on all maps. Once left-clicked, the map moves with pointer until you right click. If you then move the pointer off the map and back again, pointer still moves the map. Link Upload your transportation plan misspelled. Number of terrestrial animal species: Obed River HUC_12 060102080305 For example, obscures entire watershed until zoomed out. I would recommend “light gray canvas” as default. Tester 14 Good color. I like the layout, as it is clear, concise, and easy to navigate. It draws the eye to the various parts of the home page. I didn’t see an e-mail mes- sage about the usefulness of this feature. Will you be renaming the galleries? Are you separating the maps into categories? Happy with the speed of the map showing the details in the cur- rent extent as you zoom in and out. Limited address/place finding ability. I was unable to find places/addresses in Alaska and Oregon. I was able to find places in Kansas and Missouri. The first three details in the box are a little confusing, and I would suggest maybe not showing them or renaming them. The next two—Source_DataDesc and Source_Originator— should be renamed as well. I am excited for this feature to be enabled. I like that you are using the ArcGIS Online. Tester 15 Nothing happens when you click on “Access GIS Systems.” Not able to click on “Find your area or load your project.” Nothing hap- pened. No content box popped up. No message about sending an e-mail. Recommend adding a description of what users can do with this page (i.e., look at map details). Search function does not work. You can enter a loca- tion, but the map doesn’t change. I tried several loca- tions in the U.S., and the map never changed. (continued) (continued on next page)

139 (continued)Table I.3. Comments on Eco-Plan Test Scripts Tester # Task 1: Review the Eco-Plan Website Task 2: Get Started Task 3: Review the Map Gallery Task 4: Explore Additional Map Functionality Task 5: Review Eco- Plan Data The “Map Details” link takes you to the Map Gallery—is this where it should take you? Map details vs. map gallery could be confusing. Recommend making more consistent. Or at the top of Map Gallery, state that people can click on a gal- lery to see map details. “Eco Planned Advanced” takes you to an ArcGIS online webpage and shows the same layers that the getting started map does. Is this meant to direct you somewhere else? Link to “Upload” works, but there is not a pop-up that describes the data formats. I think this is an upcoming feature? Tester 16 Page displayed as expected. Testers need clarifi- cation of what the third major icon refers to. Is it the AGO? Is it the lim- ited number of data sets available via ICF/host server? Typo first sentence, delete “or.” Panning is only search option. Search field would be nice. We did not see this feature/ did not work. Panning refresh is slow. No scroll bar on legend. Labeling obscures pro- tected areas. Labels do not align when scaling. Cannot determine pro- tected areas. St. Louis County mislabeled. (St. Louis City is not in St. Louis County.) Include in the legend only values displayed in the project area. Too clunky to be used by some to get an overview. Why just terrestrial fauna? Why not plant and aquatic? We clicked on HUC label and got info. More icons to choose from. Tried clicking on dot. Got info. Hand does not appear when hovering over an icon we want to click on, i.e., water quality monitoring sites. Or automatic label pop-up would be good. Green blobs are protected— poorly explained in legend. Identify tool needed. Would be nice to select just one color in the leg- end. Display box often cut off by window. “Map Details” takes you to “Map Gallery.” Titles should be the same. User knows what you are starting to look at, when you click on the map. The Get Started section doesn’t have enough information as to what you are looking at. Had to go to the legend. Legend wasn’t well defined. Very focused and direct for low end users. More advanced users will go to AGO. Not as jumbled. Framing, navigation good. Drawing a box option to select project area would be good selec- tion tool. Works; provides the explanation. Need legend narrowed to data that is pres- ent. The scroll bar nice. The legend for Critical areas does not include what the red lines indicate. Seems more responsive than previous. Basemaps not loading. Now it did, just took a while. Typo transpor- tation plan. Testers like the text below map. Description of data is good. This is more logical and user friendly. On protected areas and boundary theme. Can you make map full screen? No. Box doesn’t work—could not find Belleville, did not find St. Louis. Site locked up. Hand over clickable feature would be good. Navigated to St. Louis region. Would be more helpful if protected area was first record that popped up rather than state, etc.; rear- range layers. Falling spring. Good explanation of data. Edit to read “Upload your transportation plan shapefile.” Nothing being displayed. Directory of web service links is useful. Testers feel we would be more likely to use this feature, but we know what we are looking for. Users could look through the data to see the general area a project is in and to see impacts. Or someone without a lot of GIS support could use this. Small RPCs in Missouri would benefit from this feature; they cur- rently use MSDIS for GIS support.

140 141 Table I.4. Responses to Eco-Plan Questionnaire Tester # Browser Ease of use Usefulness of content Navigation Relevancy of the theme maps to your transportation planning Confusing content or terms What do you like most What do you NOT like Additional content or function you want How would you use this site Additional comments or suggestions Tester 1 Firefox Satisfied Very Satisfied Satisfied The maps are relevant to those looking for information on the environment and habitats. No The ease of use and the speed at which the map widget works I would like to be able to move the legend on the getting started map widget. The but- ton and the legend are large enough that they obscure a large portion of the map frame. It would be nice to be able to search and add the pre-made map galleries maps via Arc Map Desktop. Tester 2 IE10 Very Satisfied Very Satisfied Very Satisfied I noted the need to define “linear” and “polygon” features on the critical habitat layer map. Noted above They are all very good. The ability to zoom into an area is probably my favorite. na More layers As a source of existing data to alert us to areas of concern No, I would need to use the tool more. Tester 3 IE8 Dissatisfied Neither Neither They are relevant. How do you get to others? Will they be preloaded or do you have to go load? “Accordion” initially Maps Search. Does not go where you ask it to go. Maps should track so when you switch from one map to another you are in the same location. Not at this stage Tester 4 IE10 Satisfied Satisfied Neither The Critical Habitat map seems the most relevant. The other maps all show layers I already access in my GIS at work. The pop-up content boxes that appear when you click on an area in the map are confusing—they appear to be taken directly from the attribute table with no changes made to the field names. Ex. Num- ber of Terrestrial Animal Spe- cies 4-23 (legend) when you click on an area on the map, it is unclear which field is being used to popu- late that area. Is it TR_TOT_A? Hard to discern. Also, can we see which species are included in that? Or at least which category the spe- cies is in (ex. federally listed as threatened? endangered?) The “Get Started” function is nice and easy to follow, especially for novice GIS users. Once you can upload your transportation plan, it would be even more useful for experienced users. Also, the data portal is well orga- nized and pretty comprehensive. The Map Gallery is not something I would likely use. The zoom wasn’t working well and the information is not as useful for the analysis I would be doing. Some geoprocessing tools would be great. I would recommend the website to people who want preliminary scoping for projects who do not have access to GIS. I would check the data portal for possibly tracking down useful data I don’t already have. Tester 5 IE10 Satisfied Satisfied Dissatisfied I’m a university user. They seem relevant. See comments in table. I like the scalability, the info from the clicking; colors are pretty good. The linking of data retrieval and naviga- tion to the mouse caused the site to hang in some cases. Remains to be seen with the data upload capacity See above. Some fur- ther work on legends would make them more informative. Tester 6 IE11 Satisfied Satisfied Satisfied They seem pretty relevant. See comments above. The collection of data sets assembled to date Tester 7 Chrome Satisfied Satisfied Satisfied na No I like the explanatory informa- tion and metadata provided so that users know what they are looking at and the limitations of the data. Sim- plicity of use is helpful to the novice or non-GIS user. na (continued on next page)

142 143 Table I.4. Responses to Eco-Plan Questionnaire Tester # Browser Ease of use Usefulness of content Navigation Relevancy of the theme maps to your transportation planning Confusing content or terms What do you like most What do you NOT like Additional content or function you want How would you use this site Additional comments or suggestions Tester 8 IE10 Satisfied Satisfied Satisfied They are relevant for showing some general information and concepts. However, the data are very broad and coarse and would be of limited use if we couldn’t add local data and/or themes from the advanced version. The ability to add your own data and maps. It’d be fine to work from the advanced platform and save to the basic platform. The website would be helpful for providing an overview of resources on a broad level for nontechnical personnel. The data provide a decent overview, but would be of limited use given the wealth of information/data we have in Southern Oregon. How- ever, if we could upload our local data to the Eco-Plan Advanced, that would be incredibly useful. We would use this platform to share the database we created for the pilot project, maps, results, and other relevant information. In addition, there are other applica- tions that we are likely to test our process on and envision using this as the end user interface. Tester 9 IE8 Very Satisfied Satisfied Satisfied Maps are relevant. However, there may be the need for additional themes. No I like the availability of online data, and the mixture of background data is helpful to help clear up some of the data. Opening a map file that I saved took a little work and was not readily evident on where it was saved. The data is not con- sistently in the cor- rect location and moves as the image is changed. The legend is too long to be use- ful. Having the legend reflect only what is shown on the screen would be helpful. I would use Eco-Plan for presenta- tions, some research, and shar- ing maps with other people on the project. I would like to see a zoom window so that one can select an area and zoom in. Tester 10 IE8 Dissatisfied Neither Dissatisfied For early planning, the most important maps to me are ones that could slow down or force a change in the alignment— critical habitat needs to be a map on its own and specifically identify the species. Wetlands are important, and they did not show, and they are key as well. None I like the simplicity of Eco- Plan, but the two key fea- tures are not available for testing so not sure how it will ultimately work. I do not like combining critical habitat with “species at risk”— I do not think that species at risk is helpful at all because some are protected and some are not, so it does not really inform the planning process. A wetland map and a map of threatened and endangered species. The critical habitat could be combined with T and E species; however, since criti- cal habitat only applies to fed- erally funded projects, it may be good to be separate. To determine whether a proposed alignment may need to be shifted. To anticipate potential resource agency concerns. To plan ahead for potential mitigation. Tester 11 Chrome Satisfied Satisfied Satisfied They cover the main themes within our transportation plan. How do you know what the initial Basemap is? Eco-Plan Advanced Basemaps create shading that alters other data. Export a map to jpg or pdf. To identify environmental items in the planning process Can you export a map? Tester 12 IE9 Very Satisfied Very Satisfied Very Satisfied I think they are relevant, but I think they will be even more relevant if this website was fully functional and allowed users to upload shapefiles, etc. I think some of the layers that have very large attribute tables with a lot of abbreviations are hard to under- stand what they mean without going to the “Maps” and learning what they are. Easy access to all data sets that are important in one location and knowing they are the most recent version. Also, I like the idea that all users will have access to the same data from the most recent update and transpor- tation planners would be looking at the same data that resource agencies/ regulatory agencies would be looking at. This is great from a consistency stand- point. I also like the func- tionality and descriptions of the Get Started map and the gallery maps. Has the NEPAssist tool offered through EPA been considered to be added to this site some- how? That tool provides a lot of data; some similar or the same data but some additional data like soils, nonattainment areas, EPA facilities, demo- graphics, etc. We would use the website during creation of our Metropolitan Transportation Plan, individual project and corridor studies, and other transportation planning efforts such as bike/pedestrian planning. I think we would be interested in the Advanced ver- sion where we can load lots of local data such as our Regional Ecosystem Framework, parks, conservation areas, etc. I also think that local partners such as cities/counties would be inter- ested in this tool in addition to resource agencies and conserva- tion NGO partners. I would suggest having a few resource agencies/ regulatory agencies test this tool also. I think the benefit of a tool like this is that it is easily accessible by many people and could be added to very easily by users who know of other helpful data and information and improve function- ality for agencies that don’t have the exper- tise or money to have their own GIS staff or resources. (continued) (continued on next page)

144 145 Table I.4. Responses to Eco-Plan Questionnaire Tester # Browser Ease of use Usefulness of content Navigation Relevancy of the theme maps to your transportation planning Confusing content or terms What do you like most What do you NOT like Additional content or function you want How would you use this site Additional comments or suggestions Under the “Data” tab there is Natural Environment, Cultural/Social, and Built Environment, but I don’t see the Social and Built Environment data sets available through the “Get Started” or “Maps” tab. Is this just listing out data but not really to be integrated into the “Get Started” Map? It looks like the Get Started map is for very basic ecological/natural environment data, whereas the Data site will link you to other types of data that can be accessed through the Eco-Plan Advanced. Is that correct? What about the National Conser- vation Easement Database data being added? It’s impor- tant to know what conservation areas are already protected, and this is a national database that could be accessible by all and then locals could load their own data (parks, etc.) in the Advanced tool for a more holis- tic protected lands map. Also—What about data from RIBITS, the USACE new GIS tool for mitigation banks? This gets to the mitigation side of projects and planning, but these are also protected lands that are important to know when doing planning. Tester 13 IE10 Satisfied Very Satisfied Satisfied Very relevant Colors indicated in legends some- times difficult to differentiate. Ecosystem layers Watersheds Population density Yes Only to rethink the color palettes of individual data sets, so that they are not too similar. Easier to see data ini- tially on the light grey background. Tester 14 Chrome Very Satisfied Very Satisfied Very Satisfied Very relevant. As a Planner, our job is to help communities identify projects in their area or planning for the future projects and when they will be neces- sary. These maps will help us to identify if there are any potential impact areas at the start of the process. No, and if there was I would look it up online. The interactive parts of the maps. I also like being able to add in our own transpor- tation plan or project for impacts. So far, nothing. Well, if you can get everything up and running; as you have it laid out now I am not sure what else we would need. As explained above, I would use it at the start of projects or during transportation plan updates/ starts. I really enjoy the mas- sive amount of infor- mation you have on the various pages, such as the informa- tion provided under the interactive maps. I wonder if this would be confusing to nor- mal users? Is there a way to put some of the titles and informa- tion into a collapsible format, so you have to expand to see more? I am looking at the Data Services section specifically. Tester 15 Chrome Satisfied Neither Satisfied The wetlands, critical habitat, and protected areas are useful. Hazardous/contaminated sites, invasive species, anadromous streams, air quality non- attainment areas, mileposts As an environmental analyst for Alaska DOT, I analyze various resources, land uses, species, etc., when preparing an environ- mental document. Though I still go out and ground truth resources, it is helpful to have a mapping program that can give me an idea of what I should be looking for or what is in the vicin- ity of the project area. (continued) (continued on next page)

146 147 Table I.4. Responses to Eco-Plan Questionnaire Tester # Browser Ease of use Usefulness of content Navigation Relevancy of the theme maps to your transportation planning Confusing content or terms What do you like most What do you NOT like Additional content or function you want How would you use this site Additional comments or suggestions Tester 16 Satisfied Satisfied Satisfied The theme maps were good. However, testers noted some of the data were not up to date. That could be a result of state agencies not reporting updates in a timely manner. While tes- ters benefit from a work place with a dedicated GIS staff, other smaller planning organi- zations would benefit from real-time access to the data. The “Get Started” page was very clunky with limited map functional- ity. The difficulty of use led the tes- ters to confusion as to need of the tool. If “Get Started” is the first thing a new user encounters, it could turn them off. The testers had trouble understanding the need for the Get Started page. If the intent is to walk people through the steps of overlaying data, then be clear about that in the descrip- tion on the home page. On the home page, there seem to be a lot of options to choose from and dif- ficult to determine what is most relevant to the user’s needs. Per- haps “Get Started” needs a longer explanation or rephrased title. It seems to refer to getting started with the IEF, when a user might assume it means get started with the web tool. On the home page, there needs to be better narrative as to the content of the page. There should be a heading or explanation about the three icons. The “Learn more” link should not be in that location. Taking a new user to that link so early would lead to a great deal of confusion. Perhaps locate the AGO links off to the side with spe- cific description of what it is. If we are trying to engage low-level GIS users, make the main page as easy to move through as possible. The Eco-Plan explanation at the bottom should be at the top of the page as part of the introduction to the site. Moving all AGO related links and text to one location, perhaps bottom of page, would be good. The Map Gallery and the associated functionality of the theme maps and the narrative accompanying the theme maps. These maps are much more intuitive than maps under “Get Started.” The users can be sure they are getting federal data layer feeds straight from the source. The limited navigation features on the Get Started map. No scroll bar on Legend. The legend lists all fields, not just ones identified in the map. Severely limits leg- end usefulness. Option to make the maps full screen. Improved map navigation. The website could be a resource to staff without ARC GIS installed on their computers (the agency has limited licenses) who wish to take a cursory look at national- level data sets and protected areas in the region. Note: na = not applicable. (continued)

148Table I.5. Comments on Eco-Plan Advanced Test Scripts Tester # Task 6: Create a free ArcGIS Account Task 7: Create and save a map Task 8: Select a basemap Task 9: Find and add data Task 10: Change basemap Task 11: Change transparency Task 12: Change visibility range Task 13: Find an existing Eco-Plan theme map and save it locally Tester 1 Not applicable—have ArcGIS online account No save option on ribbon Tester 2 Received error message “the layer, USFS_ Ecological_ Subregion” cannot be added to the map. Tried twice at different scales and it did not work either time. See above comment about this layer not being added to the map. “Set Visibility Range” does not appear on the menu choices. Tester 3 (ArcGIS Sign In box) Not at top. Not an easy find. Tester 4 Some hiccups with the “tags” but ok otherwise Tester 5 Links as written did not sink to the ESRI site, but took me to Bing. Then, ARCGIS online signaled that the roads layer I was adding could not be accessed. Note that these could at least be found when using the “Add” button in ArcOnline, but the ARC site still then produced an error message. But, the instructions to the left are apparently not complete. Tester 6 (continued on next page)

149 Table I.5. Comments on Eco-Plan Advanced Test Scripts Tester # Task 6: Create a free ArcGIS Account Task 7: Create and save a map Task 8: Select a basemap Task 9: Find and add data Task 10: Change basemap Task 11: Change transparency Task 12: Change visibility range Task 13: Find an existing Eco-Plan theme map and save it locally Tester 7 Tester 8 Skipped and signed in with an existing user account. There are 5 options. Opened second link using hyperlink in this table. Tester 9 Tester 10 Tester 11 Put “public” in the create your “public” account Tester 12 Tester 13 No dropdowns for Language, Region, or Units Tester 14 The base- map loads much faster than expected. Zoom in or out was not required. All three worked. Tester 15 I already have an account so I did not create a new one. Tester 16 Testers logged into exist- ing free AGO account. Limited navi- gation similar to .net AGO Stores saved content, your maps you cre- ated on their server Case sensitive Selected Natural Geo- graphic Functional Did not change transpar- ently until we zoomed in/out No zoom extent When zoom into neigh- borhood level layer is not visible. Selected Critical habitat (continued)

150Table I.6. Responses to Eco-Plan Advanced Questionnaire Tester # What functions do you like most? What additional functions would you like to have? How would you use Eco-Plan Advanced web application in your planning efforts? Are the differences between Eco-Plan and Eco-Plan Advanced clear to you? Do you have any additional comments or suggestions? Tester 1 Ability to interact with the mapping data. Ability to link from an exist- ing ArcGIS online account to the Eco Plan advanced ArcGIS online content I see Eco-Plan as a useful tool for planning level exercises and a useful tool for conduct- ing initial project reviews. Yes The site looks like it’s a really useful tool for transportation planners. I would like to know more about the ability to print or save a map once you have made one for a project. Tester 2 Because of the inability to add a layer to the base- map, I am not sure what functions I like most. Cannot tell from this test. As a method to look for areas of concern or areas of opportunity. I think so, even though some of the features do not appear to work in my test. No, I look forward to seeing and using this tool in the future. Tester 3 Layering Sketch or draw on the maps so that you can put projects on the map. Cross-sectional analysis Basic corridor planning. Yes. EcoPlan has very little use besides tutorial. Would be helpful to clarify the differ- ences though between the two. How would upload regional data? Is this MAC compatible? What difference will a paid account do? Not sure if there is additional capability. Tester 4 The ability to save maps to a personal account and the ability to add layers avail- able through ArcGIS Online. Would like to be able to export layers to a GIS shapefile or raster I would use to browse layers in ArcGIS Online, share an exist- ing map with outside clients/ stakeholders or view premade maps in a presentation. Yes Tester 5 I like the idea of adding our own info, wonder what the projections and datum issues will be, if any. Not sure until we test with our own data. Yes Tester 6 Adding other local data sets will be very important Yes Tester 7 Access to more basemap layers and a more robust, customizable user experi- ence as well as the ability to save what your “project” with the data you want. na Yes, Eco-plan is a more pre- made approach for the novice users, and Eco-Plan Advanced is a tool suited to someone more comfortable with GIS and one who wants to tailor a project to their specific needs. I feel that the tools do what they are supposed to do and do it fairly well. I understand that the best available data is being used, but it is important for users to understand that many of these data sets are out of date and mapped at a fairly coarse resolution. Nothing can be done about this, but users must be aware of this and use the data appropriately. (continued on next page)

151 Table I.6. Responses to Eco-Plan Advanced Questionnaire Tester # What functions do you like most? What additional functions would you like to have? How would you use Eco-Plan Advanced web application in your planning efforts? Are the differences between Eco-Plan and Eco-Plan Advanced clear to you? Do you have any additional comments or suggestions? Tester 8 Since I signed in with my user account, I am not sure of what features are Eco Plan Advanced versus ArcGIS. I do like the feel of the plat- form as it’s fairly user friendly and has more utility than Google Earth. I like being able to upload my own data (a function of ArcGIS). We were able to upload our data from our pilot project suc- cessfully. I liked the base map features. We had envisioned creating a user interface for our TRB/ SHRP 2 pilot project to share our project data and results, and are likely to use ARCGIS to share our data and create maps. It would be helpful to be able to link or upload materials to the Eco-Plan portion of the pro- gram for planners and non- technical users. Yes. Based on my experience and needs, I would use Eco-Plan advanced. How- ever, for sharing informa- tion to steering committees and other groups, I would use Eco-Plan. Tester 9 I like being able to have some control over what’s avail- able on the map. I would like to see addi- tional symbology options (e.g., making railroad tracks not on the background look like railroad tracks). Being to add layer files from ArcGIS would solve this problem and be very useful. Also, being able to upload files that allow more features than cur- rently allowed would be beneficial. This would allow me to be able to post parcel maps or even a species map that allow me to identify spe- cific occurrences of criti- cal species. I would use it to create basic maps that I could share with clients and other people that don’t use ArcGIS. Yes, I believe it’s clear. In a nutshell, Eco-Plan just shows the maps in a con- trolled environment while Eco-Plan Advanced allows the user to turn on/off lay- ers (other than the trans- portation plan), add layers, label features, etc. I feel that when I use Eco-Plan Advanced that I’m just in ArcGIS Online. When I found a portal to Eco- Plan Advanced, I couldn’t sign in. Is there some actual difference between Eco-Plan Advanced and ArcGIS Online? Tester 10 I am a little confused—is this Eco-plan advanced or Arc- GIS. It is great to be able to impact and save maps. I am guessing it just depends on how much I want to pay for Arc-GIS—am I wrong? It would greatly depend on the cost. The available maps are very limited and not very useful. So, the main reason to use it is to upload my own maps and share—so it would depend. Yes I know we were not testing the bottom of the page, but I am assuming that the icons will be activated and the login for Eco-Plan advance will work. (continued on next page) (continued)

152Table I.6. Responses to Eco-Plan Advanced Questionnaire Tester # What functions do you like most? What additional functions would you like to have? How would you use Eco-Plan Advanced web application in your planning efforts? Are the differences between Eco-Plan and Eco-Plan Advanced clear to you? Do you have any additional comments or suggestions? Tester 11 Ease of making maps. Very quick response Link between map layer and data -> like attribute tables in ArcGIS I would use it in the develop- ment of long range plans Yes This will be a great resource for planning Tester 12 The functionality—it is very easy to use. I do have moderate GIS skills and the site is way faster than using GIS. I think it pro- vides the most critical information and looks to be fairly simple to add our own regional data to it. Well, it appears that there are fees to be able to load your own data. I really see this tool being most beneficial to use the information already available in it and then being able to add our own regional data. There is a limit on the free account of 1000 features in each Shapefile, but for parks and other layers (roadway system) for a region our size, that limit is too small. We would need to be able to load our regional data without huge fees associated with the use of the tool. The only comment I would have on function is hav- ing some sort of refer- ence built in for the web sites URLs that you have to load in to use them in the system such as the Critical Habitat. If I didn’t have the URL handy to add it in then I wouldn’t have known how to get that data layer. How do you know from the list of other data sets and maps available in the Gallery on the regular Eco-Plan what the URL is so that it can be added to the Advanced version? We would use it during devel- opment of the Metropolitan Transportation plan if we were able to load in our own shapefiles. We could load alternative alignments for corridor studies and many other planning efforts such as bike/pedestrian, etc. to ensure we are capturing all of the ecological data during the different phases and types of transportation plans. Yes, to me they are. (continued on next page) (continued)

153 Table I.6. Responses to Eco-Plan Advanced Questionnaire Tester # What functions do you like most? What additional functions would you like to have? How would you use Eco-Plan Advanced web application in your planning efforts? Are the differences between Eco-Plan and Eco-Plan Advanced clear to you? Do you have any additional comments or suggestions? Tester 13 High number of data sets Select colors in legends Add selected data sets to compare with “related” con- tent on Eco-Plan data sets I understand the ability to add layers from other sources; don’t know what else can be done, i.e., look at attri- bute tables, other GIS operations Other than the legend color scheme comments above, I feel this is a very useful tool for portraying the relation- ships of data sets graphically. Tester 14 Adding in your own data and pulling in from other sources. Allows you to make a unique custom map for any situation. Nothing at this time. As I use it more in the future I might have more sug- gestions. Without being able to use it in conjunc- tion with a project. I am unable to put something else here. Yes and I think they will be clear to other users who have not used ArcGIS Online. Tester 15 I like that I can add layers from many different sources. Seems much more useful for the type of work we do in the Environ- mental section of the Alaska DOT. Useful to be able to add layers from local agencies and our own agency when doing analysis. I would like maps to auto- matically default to my location. As an environmental analyst for Alaska DOT, I analyze various resources, land uses, species, etc. when preparing an environmental document. Though I still go out and ground truth resources, it is helpful to have a mapping program that can give me an idea of what I should be looking for or what is in the vicinity of the project area. Yes. I prefer the Eco-Plan Advanced. (continued on next page) (continued)

154Table I.6. Responses to Eco-Plan Advanced Questionnaire Tester # What functions do you like most? What additional functions would you like to have? How would you use Eco-Plan Advanced web application in your planning efforts? Are the differences between Eco-Plan and Eco-Plan Advanced clear to you? Do you have any additional comments or suggestions? Tester 16 Speed of data upload and display (could be time of day). Preloaded a good selection of basemaps and overlays. Testers tried the share links option. The link takes people directly to the user interface, cannot modify, just view, can change base map. This might be a function of AGO, but testers would have liked to see an exercise in how to create communities. Ideally the testers would like the ability to store local data on the tool with a clip and ship option for users. In theory, we could see creat- ing maps that display critical areas and our ecological data in the region and mak- ing those accessible to fel- low AGO users at regulatory and resource agencies. However the limitations associated with AGO costs and crediting system would hinder the execution of the idea. At this time, EWG is not planning to acquire a paid AGO account and take on the cost associated with storing data on the ESRI servers and covering costs for related downloads and report generation. EWG cur- rently uses the free version offered by ESRI as a con- tractual ARC GIS user. To get the full functionality of the data, one would need to be able to download the data directly to a PC, but the size constraints and real time access to data updates quickly become an issue, which leads back to the ini- tial need for the national data tool. Yes. Able to highlight areas of concern. Add map notes, mark up the maps, change color of markups, but clunky; an option only for people with AGO accounts. Smaller MPOs who do not have the technical capacity and do not have money to spend on ARC GIS would use it if it was at no cost. County level staffs in transportation and planning departments are potential users. (continued)