4
Assessment of Other Metrics Potentially Applicable to Chemical Agent Disposal Facilities

To accomplish the last two items on its statement of task, the committee assessed the development and use of leading and lagging indicators by various government and private organizations, including any new initiatives that the committee learned about in the course of its fact finding. Although it conducted this assessment the committee developed no related recommendations for implementation by the chemical agent disposal facilities (CDFs). Each such facility is in a unique situation with respect to site-specific geography, agent and munitions to be disposed of, demographics, culture, management, and regulatory climate. As a result, each site needs to be able to determine the metrics that are most appropriate for it. A top-down prescription for a standard set of metrics to be used at all CDFs would be less than helpful in the committee’s view. Instead, the committee presents a general overview of the types of metrics that might provide general guidance to the CDFs for the continuing development of their safety and environmental programs and culture.

The committee assessed metrics that are currently used by the Department of the Army and the Federal Aviation Administration. They also assessed those of the Center for Chemical Process Safety of the American Institute of Chemical Engineers (AIChE), Corning, Dow Chemical, Motorola, and Praxair. The discussion of applicable metrics that follows is based on information compiled in Appendix B.

Data on environment, safety, and health (ESH) matters have historically been collected to provide management with quantifiable outcomes such as actual and hidden costs, lost time, and worker illness and injury. These data were often tied to regulatory compliance and/or worker compensation costs. The data used were almost exclusively lagging indicators—that is, they were collected after an incident, to determine strategies based on recorded outcomes to prevent future incidents. While lagging indicators serve useful purposes, they need to be supplemented with leading indicators to ensure continuous improvement of an ESH program. Also, lagging indicators are of minimal use for an organization such as the Chemical Materials Agency (CMA), whose mission is changing, moving from disposal of chemical agents to plant closure and disposal of hazardous and secondary waste.

Companies and organizations like the CMA that want to be the best in the field have recognized the limitations of lagging indicators and are seeking to use leading indicators to anticipate possible incidents within the ESH categories. While the literature is replete with information on leading indicators, their actual implementation to achieve continuous improvement is limited at best. Notwithstanding this, leading indicator models appear to have certain elements in common:

  • Identifying hazards through risk assessment,

  • Communication,

  • Training,

  • Documentation,

  • Periodic review by top management,

  • Follow-up on findings and corrective action,

  • Analysis of near misses,

  • Sharing of lessons learned,

  • Worker involvement, and

  • Audits and assessments.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 19
 assessment of other metrics Potentially applicable to chemical agent disposal Facilities To accomplish the last two items on its statement of and/or worker compensation costs. The data used were task, the committee assessed the development and use almost exclusively lagging indicators—that is, they of leading and lagging indicators by various government were collected after an incident, to determine strategies and private organizations, including any new initiatives based on recorded outcomes to prevent future incidents. that the committee learned about in the course of its While lagging indicators serve useful purposes, they fact finding. Although it conducted this assessment the need to be supplemented with leading indicators to committee developed no related recommendations for ensure continuous improvement of an ESH program. implementation by the chemical agent disposal facili- Also, lagging indicators are of minimal use for an ties (CDFs). Each such facility is in a unique situation organization such as the Chemical Materials Agency with respect to site-specific geography, agent and (CMA), whose mission is changing, moving from dis- munitions to be disposed of, demographics, culture, posal of chemical agents to plant closure and disposal management, and regulatory climate. As a result, each of hazardous and secondary waste. site needs to be able to determine the metrics that are Companies and organizations like the CMA that most appropriate for it. A top-down prescription for a want to be the best in the field have recognized the standard set of metrics to be used at all CDFs would limitations of lagging indicators and are seeking to be less than helpful in the committee’s view. Instead, use leading indicators to anticipate possible incidents the committee presents a general overview of the types within the ESH categories. While the literature is of metrics that might provide general guidance to the replete with information on leading indicators, their CDFs for the continuing development of their safety actual implementation to achieve continuous improve- and environmental programs and culture. ment is limited at best. Notwithstanding this, leading The committee assessed metrics that are currently indicator models appear to have certain elements in used by the Department of the Army and the Federal common: Aviation Administration. They also assessed those of • the Center for Chemical Process Safety of the American Identifying hazards through risk assessment, • Institute of Chemical Engineers (AIChE), Corning, Communication, • Dow Chemical, Motorola, and Praxair. The discussion Training, • of applicable metrics that follows is based on informa- Documentation, • tion compiled in Appendix B. Periodic review by top management, • Data on environment, safety, and health (ESH) mat- Follow-up on findings and corrective action, • ters have historically been collected to provide manage- Analysis of near misses, • ment with quantifiable outcomes such as actual and Sharing of lessons learned, • hidden costs, lost time, and worker illness and injury. Worker involvement, and • These data were often tied to regulatory compliance Audits and assessments. 

OCR for page 19
0 EVALuATiON OF SAFETy ANd ENViRONmENTAL mETRiCS While the definitions of lagging and leading indica- with the production line. The task processes are often tors varied from one organization to the next, when the evaluated by means of a job hazard analysis/process combined information from these organizations was analysis as required under the Occupational Safety and considered some common themes for developing a sys- Health Administration Voluntary Protection Programs tem for using leading indicators became apparent. The (VPPs). While preventive maintenance may not in itself definitions developed by this committee (see Appendix be an ESH direct indicator, the failure of a mechanical A) capture the intent of all of the organizations reviewed. system may lead to an event that can produce an injury, The entire committee agrees that measurement is the exposure, or environmental insult. To this end, keeping precursor to control, and that the usual lagging indica- to the preventive maintenance schedule and monitoring tors (e.g., those used by CMA and reviewed in Chapter the completion of the tasks in that schedule are leading 3) should not be neglected as they ultimately lead to indicators of ESH. improvement in safety performance outcomes. Analysis of measured data is vital to ensuring evi- Any measurements and derived indicators must be denced-based control of safety. Analyses could entail part of an overall system—an environmental policy or the more thorough investigation of incidents and/or near a commitment to continuous improvement—if they are misses to ensure that the causative factors have been to be effective in driving improvement. This system identified; application of standard methodologies such must possess a control philosophy (whether it is called as root-cause analysis; generation of indices in terms of risk management or safety management or is a business rates rather than absolute numbers; and application of strategy such as Six Sigma), the commitment of top quality control techniques for visualizing trends. Such management, and specific goals for each indicator. analyses provide a bridge between the raw data and Input measures are the precursors to good safety management action, so that management has a clearer performance. They include ensuring that safety is understanding of what needs to be changed and the designed into all equipment and procedures, timely and potential effects of its actions on safety performance. effective training for all personnel, setting and meeting Accountability at both the organizational and indi- individual safety goals, and completion of tasks set vidual levels is essential. Many organizations require (e.g., corrective actions, preventive actions, and permits the evaluation of support for and actual performance issued). These inputs do not in themselves guarantee a of ESH matters in employee and supervisor appraisals. safe organization, but they are a sign of how ready an In the organizations that excel in ESH, accountability organization is to achieve and improve safety. includes penalties for specifically defined substandard Process measures are indicators that the organiza- performance. tion and its workforce are performing their duties in Overall, much can be learned from the practices a safe manner. Again, they do not guarantee safety, of industrial and government organizations about the but without such indicators, reported levels of safety derivation and use of leading indicators that could be may reflect chance avoidance of rare events rather applied to chemical demilitarization operations. Again, than safety levels achieved as a result of design and because of the unique circumstances in which each control. Typical process measures of safety include chemical agent disposal facility finds itself, the com- the number of near misses or incidents, behavior-based mittee generalized its assessment to assist facilities in safety observations, rates of compliance with written furthering their safety and environmental programs and procedures, participation in pretask hazard assess - cultures. The International Civil Aviation Organization ments, and audits or assessments of task performance summarizes overall safety management as follows: and workplace factors. Note that assessments can be A safety management system . . . , as a minimum, identifies self-assessments, which provide useful training and safety hazards, ensures that remedial action necessary to involve the workforce, or independent assessments, maintain an acceptable level of safety is implemented . . . , provides for continuous monitoring and regular assessment which provide unbiased assurance of the state of pro- of the safety level achieved . . . , and aims to make continuous cesses. Ideally, all of these leading process measures improvement to the overall level of safety.1 should have been validated against outcome (lagging) indicators to ensure that they are indeed necessary conditions for safety performance, a step that is often 1Elwyn Jordan, Aviation Safety Inspector, Federal Aviation Ad - neglected. Process measures include the actual physical ministration, “Introduction to safety management systems (SMS),” processes of each task on a production line as well as presentation to the Federal Air Regulation 135 Seminar on April 19, 2007. completion of all preventive maintenance associated