Performance Measures in Snow and Ice Control Operations (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1 Monitoring the performance of snow and ice control operations has increasingly become an important task for highway agencies and contractors because of increasing stakeholder expectations. Different indicators have been used to assess performance of snow and ice control operations, in both the United States and abroad, but with varying degrees of success. Furthermore, comparison of performance across and within agencies is limited since no single measure applicable to the different roadway classifications, storm charac- teristics, or traffic conditions is widely accepted. This report summarizes the research conducted under NCHRP Project 14-34 and the development of a guide that agencies can use to measure the performance of snow and ice control operations. To achieve the project objective, the research team followed five steps described here: • Review of available literature (Step 1), • Evaluation of existing performance measures (Step 2), • Identification of core outcome- and impact-related performance measures for snow and ice control operations (Step 3), • Development of a guide to support the adoption of the core performance measures (Step 4), and • Development of tools and materials to support agencies’ assessment of performance (Step 5). The result of this project encompasses performance indicators, along with methods and measures for assessing agencies’ performance in snow and ice control operations. The taxonomy extracted from readily available literature includes over 50 currently used measures and indicators. An assessment of their characteristics yielded commonalities between them, highlighting 10 categorical areas of performance assessment (Figure S-1): The review of the literature also highlighted common challenges, which in turn fuel the new research and approaches being considered by practitioners today. These challenges relate to the difficulty of obtaining reliable, accurate, and timely information about field conditions, as well as correctly estimating the severity of winter weather events. In this sense, temporal and spatial variability in weather, resource constraints, and expected levels of service, along with a general lack of quantifiable information on the results of maintenance activities, all make it exceedingly difficult to objectively measure performance. Furthermore, understanding and factoring storm and seasonal (winter) severity is an evolving field and continues to be a challenge to researchers and implementing agencies. As for research and implementation themes, these are directly linked to the continued interest in the integration of traffic operations data with maintenance data and the continued interest in methods to establish overall cost–benefit information and the relative impact of maintenance practices. S U M M A R Y Performance Measures in Snow and Ice Control Operations

2 Performance Measures in Snow and Ice Control Operations One product of this research is a guide that identifies a core set of performance measures related to snow and ice control operations. In identifying these measures, the research team looked at various input-output-outcome-impact categories and measures that fall under each category. Input and output measures are important to agencies for informing day-to-day tactics and decision making about event response. However, the guidance provided in this report focuses on the outcome and impact end of the spectrum and is geared toward enabling a greater consistency in collecting, analyzing, and reporting outcomes and impacts associated with snow and ice control operations. To that end, measures provided in the guidance can be generally considered lag measures and are primarily more suited for retrospective and strategic decision making. One purpose of the guide is to provide insight on developing a framework for an efficient and flexible performance measurement program for snow and ice operation management to help decision makers identify appropriate adjustments that could be made to manage resources effectively through the use of the proposed performance measures. The guide highlights some of the effective practices in winter maintenance and provides detailed procedures that highway agencies and contractors can follow to monitor their level of performance with a focus on safety, mobility, and sustainability. “Sustainability” in this context is defined from an agency’s perspective rather than a societal perspective. From an agency’s perspective, sustainable operations are defined by their environmental stewardship, efficiency of response, and the satisfaction of the traveling public. Defining performance measures is a collaborative activity that requires a careful look at the agency’s mission, goals, and operational objectives. It is likely that no two agencies will have the same set of performance measures to assess the success or effectiveness of their programs. As agencies seek to create a core set of performance measures in these areas, it is important to note the following: • No individual performance measure is a perfect representation of the complexity of snow and ice response. A more useful approach to performance measurement and reporting would be based on multiple measures that together provide a balanced report card of agency performance. • Not all performance measures that are important to an agency can be fully controlled by the agency’s response activity. For such performance measures, the linkage from the activity performed to the measure may be indistinct, but overall trends may still be valuable for the agency. • Starting the process of performance measurement is the first and often the most important step. The search for the perfect performance measure or analytical approach Storm Characteristics/Severity Material Management Labor Resource Allocations Level of Maintenance Response Maintenance Response Outcomes Level of Operational Responses Traveler Experience, Mobility, and Safety Cost, Budget, and Funding Transportation Resilience Economic Activity Figure S-1. Performance assessment categories for snow and ice control.

Summary 3 can sometimes seem to be a burden on an agency. However, by focusing on what can be done immediately and in some priority segments of their jurisdictions, agencies often find that there is a path toward continual improvement of the performance measures once they begin the process of measurement. • Performance measurement will likely be based on a subset of agency roadways. Agencies are unlikely to have the ability to collect data on every mile of their roadways, and performance measurement will likely be based on a sampling of observations from agency roadways. As such, it is important that the agency identify the priorities for where observations about performance are needed, noting that they may not be the same as the priorities for snow and ice control. In the future, agencies may be able to leverage emerging technologies in fixed and mobile sensing to collect much more data on their roadways. In the interim, working from a subset of roadways is a necessary first step. • Lastly, performance measures identified by an agency need to be simple and easily understood not only by the agency’s stakeholders but also its own staff. When creating performance measures, agencies need to be highly aware of the reporting requirements and the data needs. The more stringent the requirements, the more sophisticated the data needs; or the less automated the analysis approach, the more challenging the performance measure becomes for agency staff to assess. Performance measures are driven by the mission and goals of an agency. Based on these goals, operational and maintenance objectives can be developed. If there is an operational objective, it is necessary that there be a way to measure its achievement. An agency that sets a high level-of-service (LOS) performance standard during an event could incur more cost than an agency that has a lower or no LOS performance standard. Similarly, recovery objectives (e.g., how quickly roads/systems will be cleared) drive the level of response activity. In other words, the operational objectives define the performance measures. The objective is achieved through attainment of targets set for the performance measures. In Figure S-2, seven objectives are identified for agencies involved in snow and ice control. Seven performance measures were identified to support monitoring of these seven objec- tives. However, each performance measure includes many possible variations, depending on the agency’s current and desired capabilities. Table S-1 shows how the operational objectives relate to the identification of performance measures. Figure S-2. Categories of operational objectives.

4 Performance Measures in Snow and Ice Control Operations These measures were identified to support a wide variety of agency capabilities, types, and functions. Together, they provide a balanced view of outcomes and impacts associated with snow and ice control operations. The guide contains 10 key steps to develop and assess performance measures. The steps provide a systematic process that allows agencies to define, implement, use, and evaluate performance measures that satisfy their needs. Key recommendations include: • Define and use a “weather event” as the starting point for performance measurement. Transitioning from a seasonal approach to an event-based data collection and management approach allows for a greater flexibility in the definition of performance measures. • Develop both a storm severity and a seasonal severity index. The value of a performance measurement is greatly enhanced by pairing it with severity at the event and seasonal level. • Pick a consistent LOS and recovery criteria and how they are measured across the agency. A certain amount of subjectivity is unavoidable in snow and ice performance measurement, especially when defining LOS and recovery criteria. However, defining how these are measured can be made more consistent within the agency, depending on its existing capabilities. • Report performance information. Telling a story with performance measures is not only possible but essential for snow and ice programs. This type of report gathers data on winter weather to assess severity and how the agency responded to each event. These data help agencies identify good performance, areas that need improvement, and how they can use their resources more efficiently and achieve greater benefits. However, performance measures are one part of a larger story. An effective performance measurement program should be tailored to an agency’s particular needs, taking into account a wide array of circumstances and capabilities. Performance measures have the ability to support effective and timely decision making at multiple levels of an agency. The specific performance measures used by agencies are largely determined by the timing of the decisions. When used to influence decision making during a storm, a department of transportation might use measures of actual conditions, which could include storm speed, solid material application rate, pavement temperature, and air Objective Identified Performance Measures Maintain level of service during event Percent of time road segments meet agency-defined level-of-service thresholds during winter storms Meet recovery criteria set by agency Percent of segments meeting time to regain or recover to acceptable criteria for agency-defined segments after the end of event Meet reliability targets for specific routes Percent of trips within accepted difference between measured travel time index and additional expected travel time index for snow and ice events for selected routes Support safe operations of the roadway Five-year rolling average of fatalities and injuries (number, rate) during a winter season Meet customer satisfaction ratings Customer satisfaction ratings for snow and ice response Support efficient use of resources to meet operational objectives Cost of snow and ice control to meet established performance criteria for a given winter severity Support environmental stewardship goals by optimizing material use Agency within acceptable difference between expected and actual use of salt and other materials in a season Table S-1. Relationship between operational objectives and performance measures.

Summary 5 temperature. For post-storm assessments, measures such as a winter storm index, winter mobility index, amount of salt or other materials used, or lane miles plowed might be more suitable. When conducting annual reviews, performance measures that summarize an entire season’s activities can be beneficial, including the number of snow events, number of freezing rain events, total snow amount, and total number of incidents. Performance measures can help track investment decisions, strategies, and tactics to demonstrate a clear basis for action. Once the decisions are made, the same performance measures can provide an assessment of the decision and enable adjustments. Using perfor- mance measures for supporting decision making is dependent on the reliability and consis- tency of the measures. While reliability is self-evident, consistency is also necessary to prevent a whiplash effect in decisions where a rapidly varying measure can create uncertainty in the nature of the decision. Logical times to review the role of performance measures in decision making include during snow and ice strategic planning and budgeting, annual maintenance reviews and meetings, and after-action reviews.