4
Management Systems: Lessons Learned Process and the eRoom Tool

LESSONS LEARNED—A MANAGEMENT SYSTEM

An important management system in any manufacturing process with high hazard potential is one in which institutional knowledge and experiences are captured and shared with all affected personnel. Management systems can be composed of both information technology elements and processes whereby human work is performed. The chemical stockpile disposal program makes use of both techniques. There are frequent and focused (usually telephone) meetings on lessons learned. These meetings involve the lessons learned coordinator for each site plus appropriate experts to provide advice/information on any particular topic. The lessons learned database was set up in 2002 and has been continually updated. The current (2010) version enables personnel to query the database to obtain information about procedures and specific topics and to search site spill history and prior closure activities. The database covers multiple years of operation and multiple facilities and would appear to be serving the needs of the program. Many management personnel, both Army and civil service, have access to this database. While similar commercial systems do exist, the lessons learned database, which was developed internally by the Army and its contractors, is able to codify and catalogue, as well as search and retrieve, needed information.

In setting up the database, the program has successfully wrestled with most of the key questions surrounding use of the database; information collection, retrieval, and sharing; worker training; and systems support and management. The database has been in use since early 2002, but the design and operational details were revised in 2006. The newer version is more database, in review mechanisms to ensure the accuracy complete in controlling how information gets into the and usefulness of the data, and in delineating to whom and what type of access will be available. In a teleconference call with members of the committee, the Army asserted that the database program is substantially improved over that in use in 2002 particularly with regard to some shortcomings identified in the earlier National Research Council (NRC) report Evaluation of Chemical Events at Army Chemical Agent Disposal Facilities (NRC, 2002).1 The Army attributed the improvement to a unified ownership of all aspects of the database by its contractor, the URS Corporation, with support from the Army. This includes keeping it up to date. The Army also provided the committee access to the most recent version of the lessons learned program.


Finding 4-1. The current version of the lessons learned database is significantly improved over the 2002 version; it is much easier to access and use the search functions. The Army is to be commended on implementing the changes that made the database more usable.

1

Teleconference with participants Timothy Garrett, Site Project Manager, ANCDF; Amy Dean, Environmental Engineer, Project Manager for Elimination of Chemical Weapons, CMA; Peter Lederman, committee chair; and Leigh Short, committee member, February 11, 2010.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 28
4 management systems: lessons learned Process and the eroom Tool lessoNs learNed—a maNagemeNT sysTem support and management. The database has been in use since early 2002, but the design and operational An important management system in any manu - details were revised in 2006. The newer version is more facturing process with high hazard potential is one complete in controlling how information gets into the in which institutional knowledge and experiences database, in review mechanisms to ensure the accuracy are captured and shared with all affected personnel. and usefulness of the data, and in delineating to whom Management systems can be composed of both infor- and what type of access will be available. In a telecon- mation technology elements and processes whereby ference call with members of the committee, the Army human work is performed. The chemical stockpile asserted that the database program is substantially disposal program makes use of both techniques. There improved over that in use in 2002 particularly with are frequent and focused (usually telephone) meetings regard to some shortcomings identified in the earlier on lessons learned. These meetings involve the les- National Research Council (NRC) report Evaluation sons learned coordinator for each site plus appropriate of Chemical Events at Army Chemical Agent Disposal experts to provide advice/information on any particular Facilities (NRC, 2002).1 The Army attributed the topic. The lessons learned database was set up in 2002 improvement to a unified ownership of all aspects of and has been continually updated. The current (2010) the database by its contractor, the URS Corporation, version enables personnel to query the database to with support from the Army. This includes keeping obtain information about procedures and specific topics it up to date. The Army also provided the committee and to search site spill history and prior closure activi- access to the most recent version of the lessons learned ties. The database covers multiple years of operation program. and multiple facilities and would appear to be serving the needs of the program. Many management person- Finding 4-1. The current version of the lessons learned nel, both Army and civil service, have access to this database is significantly improved over the 2002 ver- database. While similar commercial systems do exist, sion; it is much easier to access and use the search func- the lessons learned database, which was developed tions. The Army is to be commended on implementing internally by the Army and its contractors, is able to the changes that made the database more usable. codify and catalogue, as well as search and retrieve, needed information. In setting up the database, the program has suc- 1 Teleconference with participants Timothy Garrett, Site Project cessfully wrestled with most of the key questions sur- Manager, ANCDF; Amy Dean, Environmental Engineer, Project Manager for Elimination of Chemical Weapons, CMA; Peter rounding use of the database; information collection, Lederman, committee chair; and Leigh Short, committee member, retrieval, and sharing; worker training; and systems February 11, 2010. 

OCR for page 28
 MANAGEMENT SYSTEMS: LESSONS LEARNED PROCESS AND THE eROOM TOOL It is noted that having a good lessons learned system learned process has and continues to significantly ben- is a widely used operations tool in industry. The devel- efit the conduct of chemical demilitarization closure opment of the current programmatic database took a activities. more bottom-up approach than the previous (2002) version discussed in NRC (2002). With continuing defining lessons learned and the lessons learned support by the Army, the current version of the data- Process base has gone through several iterations to arrive at its present form. The data in the system are almost entirely A lesson learned is derived from knowledge, expe- specific to operational issues arising during weapon rience, training, exercises, and actual incidents, and it destruction at their respective facilities. Nonetheless, reflects both positive and negative lessons. The lessons the data on closure, while currently limited to the three learned process can be divided into four discrete steps facilities that have closed thus far—Johnston Atoll (see Figure 4-1): Chemical Agent Disposal System, Aberdeen Chemical Agent Disposal Facility, and Newport Chemical Agent 1. Identify idea and articulate concepts. Disposal Facility—are likely to be of interest to demoli- 2. Codify, catalogue, approve, and store. tion contractors, for example. The committee strongly 3. Search and retrieve. supports the concept of continuous improvement of the 4. Integrate into current work activity. existing system for programmatic data. Only by completing all of the above steps is the value of the prior knowledge and experience able to The importance of lessons learned be fully assimilated and be useful in planning and In any endeavor, the experience that comes from executing a specific task or change. If any of the four doing offers opportunities for learning and gaining steps is not completed, the objective of having a func- wisdom. This is true whether the outcome of the activ- tioning lessons learned process is not fully realized. ity was good and as anticipated—or otherwise. That is Likewise, the continuous improvement process applies to say that we learn from our past, both good and bad. to all steps. It is also important that any staff member, Similarly, the chemical stockpile disposal program government, or contractor be able to easily access the has embraced a lessons learned approach. When deal- data and find any lesson learned that is applicable to ing with chemical agents and other toxic substances, a particular issue. Moreover, the committee believes protecting the safety and health of the workforce and that a mechanism should exist whereby proposed les- the surrounding communities becomes an essential sons learned that are initially rejected be independently priority. This is true for activities spanning construc- reviewed and potentially reconsidered for inclusion in tion and agent disposal operations, and it extends to the the database. closure and dismantlement of the disposal facilities. It The current lessons learned process flow (shown in becomes an even more critical priority as the majority Figure 4-2) was adapted by the committee from a more of the workforce shifts from workers trained in agent detailed flow sheet prepared by the Army. Many of the disposal operations to those trained in demolition but current documented lessons learned are pertinent to who are less familiar with chemical agent properties agent processing operations. Some of the information and safe practices in agent issues. concerning closure lessons learned from the Johnston Strong anecdotal evidence indicates that a lessons Atoll Chemical Agent Disposal System and associated learned approach has been helpful in the planning, procedures apparently exists only in hard copy and is construction, operation, closure, and deconstruction not able to be digitally searched. However, there is a of chemical agent disposal facilities over the course of growing body of electronically searchable knowledge the chemical stockpile disposal program.2 The com- and experience relevant to closure, deconstruction, mittee judges that a continued, formalized, lessons and dismantlement. This latter body of knowledge, the closure-based lessons learned, is the focus of the evaluation done by the committee. 2 Personal communication among Brad Tibbils, Project Manager, This committee has focused its attention on the pro- URS; Peter Lederman, committee chair; Leigh Short, committee grammatic lessons learned process that is managed by member; and Deborah Grubbe, committee member, March 3, the Army and its prime contractor. Useful data reside 2010.

OCR for page 28
0 REVIEW OF CLOSURE PLANS FOR THE BASELINE INCINERATION CHEMICAL AGENT DISPOSAL FACILITIES 4. Integrate into current work activity Time 3. Search database and retrieve 2. Codify, catalogue, approve, store 1. Identify idea; articulate concept(s) Continuous process FIGURE 4-1 Steps of the lessons learned process. Issue advisory for Figure 4-1 immediate action R01790 vector editable Proposed Subject Review lessons Initial site expert and Required completion learned to other upload to review actions web site reviews database Reject Uploaded to or database; rework no action FIGURE 4-2 Flow sequence for lessons learned. SOURCE: Closure community portal document 100218–LL Flow Chart–CMA.ppt. in the information system and are reasonably straight- ties as some new lessons learned may originate from forward to use, although a little training is needed. The other than Army or prime contractor sources. current programmatic lessons learned program is very Finding 4-2. Lessons learned over the course of con- much improved over the earlier (2002) versions with regard to searchability, accessibility, and a more for- ducting closure operations at chemical agent disposal malized process of entering the lessons learned. How- facilities will be helpful to completing without incident Figurefuture closure activities within the chemical stockpile 4-2 ever, it is incumbent on the Army and its contractors to remain good stewards and to continuously improve R01790 disposal program, and they will minimize costs by Editable veducing the time and effort needed for learning curves r ectors the process. Any staff person who has an account on the system and training. can submit a lesson learned. There is a lessons learned Recommendation 4-2. The Army should continue to submission form that is available electronically and in hard copy. Subcontractors who do not have accounts support the closure lessons learned processes and to must take the extra step of raising a potential lesson encourage the prime contractor for closure operations learned with a prime contractor representative. This to strengthen the timeliness and manner in which the limitation could become an issue during closure activi- lessons learned are shared. In this regard, it is important

OCR for page 28
 MANAGEMENT SYSTEMS: LESSONS LEARNED PROCESS AND THE eROOM TOOL that all contractors on-site have access to or knowledge There is no documented appeal process in place if of the lessons learned applicable to their specific site the submitter disagrees with the decision at the site level activities. by the lessons learned coordinator or site subject matter expert to reject the lesson learned. The individual(s) Each site has a lessons learned coordinator who who submitted the lesson learned may disagree with the receives the submitted lessons learned forms and logs disposition decision and should have an opportunity to them into the system for review. This coordinator document this position and make a case for inclusion. ensures that the data are complete and assigns each There does not appear to be an opportunity to question data set a lessons learned number. At this point, one this rejection. Rejected submissions should be reviewed of three outcomes is possible: the lesson learned can independently from the initial review to ascertain that be rejected; it can be accepted and entered into the in fact the lesson might not be useful to, for example, system; or it can be reworked at the initiating site and a demolition contractor. While the committee has not then forwarded. seen evidence of serious problems with this part of the Because each site has a lessons learned coordina- process, the inability to capture what information was tor, system variation is introduced by having differ- discarded at a given point in time is not optimal. ent people exercising judgment. However, terms and The search mechanism for the lessons learned data- categories for the system are preprogrammed into the base is significantly improved over the 2002 version. software, which reduces variability. The title for a les- However, the novice or inexperienced user may not son learned is at the discretion of the submitter and is have adequate ability to conduct a search without help free form. The submitter is also allowed to categorize or extra training. It was not apparent to the commit- suggested priority ratings for timing and safety. If site tee that novice users are sufficiently familiar with the managers determine that a lesson has imminent rel- search functions. This may become a more serious issue evance to safety, an email is generated and uploaded during closure when most of the activities are not typi- to the database. This determination of a high priority cal of the more standardized activities that take place requiring immediate notification of the other sites is during agent disposal operations. made by the lessons learned coordinator and subject Finding 4-3. There is no system in place to review expert at the site. The site coordinator and other appropriate personnel a determination to reject a proposed lesson learned. verify the uniqueness of the proposed lesson learned Rejection may become a more important issue during and, if appropriate, recommends inclusion in the data- closure than it is during operations because the review base. An email accompanies any lesson learned that is system is not geared to closure. The current system forwarded beyond its originating facility. The proposed depends in part upon knowledge of how the search lesson learned is subject to further review, and appro- mechanism is constructed and upon use of the appropri- priate actions (if any) are conducted. ate search words or terms. Once the lesson learned is forwarded from the site, it Recommendation 4-3a. The Army should require a is subject to external review by a program-wide subject matter expert who reviews and approves, or reviews and mechanism to validate the decision to reject a lesson issues, the lesson learned for information only. A lesson learned. learned issued only for information indicates that no spe- Recommendation 4-3b. The Army should require cific action is required by any facility. This review pro- cess at the program level allows for greater consistency implementation of a means to familiarize people with within the specified subject area. Coordination among those paper-based lessons learned from the experiences the subject matter experts is vital to ensure consistency at the Johnston Atoll Chemical Agent Disposal System in lessons learned treatment across subject areas. The that are not accessible through the electronic lessons subject matter expert is the person who can revise or alter learned database. a lesson learned entry during the review process, as well Recommendation 4-3c. The Army should consider as being qualified to make a determination that the lesson learned is only for informational purposes. developing a real-time user support tool to help novice users search the lessons learned database.

OCR for page 28
 REVIEW OF CLOSURE PLANS FOR THE BASELINE INCINERATION CHEMICAL AGENT DISPOSAL FACILITIES menting a more proactive system by which information In addition to the searchable electronic database, is immediately pushed out to users who are specifically the Army and its contractors participate in weekly notified when a lesson learned is approved in their area teleconferences and conduct quarterly meetings on or for their facility. This will become increasingly more lessons learned relevant to closure preparations. These critical as closure activities accelerate. Such a system activities serve to make the preparatory efforts directed could have subject matter experts taking a greater lead- toward closure lively and current, and attendance has ership role in the process, for example, by checking the been high. The ability to query an expert in real time type of data a user is seeking and ensuring that the user is another excellent way to ensure that lessons learned is able to find all pertinent information. One of the key deliver the benefits intended. aspects of lessons learned is their value as an appropri- ate “just in time” tool. A lesson learned too early can be access to the lessons learned database lost and forgotten, and a lesson learned too late may be disastrous for individuals and the program. There are two levels of access to the database, one of which is widely available, and another that in addi- Finding 4-4. Since the number and type of contractors tion to access allows data input/changes, but is not on-site will differ during closure and agent disposal widely available. It appears to the committee that the operations, the use of the lessons learned database and current system operates in a manner that could inhibit its applicability may be different during closure opera- a potential user (particularly during closure activities) tions from what has been the case previously. from correctly locating an applicable lesson (assuming it is present). The Army should consider how the clo- Finding 4-5. The lessons learned database is search- sure lessons learned information could be made avail- able, but the search mechanism is relatively difficult to able to all potential subcontractors during the bidding use by the novice user. phase for particular closure tasks. Such information may be pertinent to all bidders, only one of whom will Recommendation 4-5. Rather than relying completely be selected. on the current means of searching the lessons learned During actual closure operations, a different set of database system, the Army should develop a proactive contractors will be on-site and a very different set of mechanism that assists new or novice users, particular- problems may arise. There appears to be no current ly dismantling subcontractors to find, or be made aware means of ensuring that the lessons learned will be acces- of, the data in the lessons learned that would apply to a sible during closure and/or the knowledge contained particular problem. in the database will be made available to appropriate subcontractor personnel. The information in lessons learned documents can provide a firm foundation to The eroom facilitate safe, fast, and cost-effective closure opera- The contractor maintains an eRoom, an electronic tions, but the information must be readily available. repository of documents related to the chemical stock- Currently, although a detailed database does exist, pile disposal program that includes closure-related it may be somewhat difficult for an inexperienced user documents, permit-related documents, and documents (such as any closure contractors) to access or obtain the relating to operational matters (see Box 4-1). However, pertinent information contained therein. This has not there does not appear to be a very strong in-place train- seemed to be an issue during chemical agent destruc- ing system to familiarize all appropriate employees tion operations, but as indicated above, it might be (including those primarily involved with closure) with more problematic during closure. At present, manage- the use and benefits of the eRoom. It is potentially a ment holds periodic meetings and phone calls to share very strong management tool. Other companies that lessons; however, during closure, that approach could have a similar system typically find it necessary to suffer from unfortunate timing, and it may require par- devote a considerable amount of time and resources ticipants to have good memories. A user who is looking to ensuring that it is used to the fullest possible extent. for closure information to, for example, prepare a work The Army and its contractor might evaluate whether the plan for occluded space surveys might have difficulty current training is adequate, and whether the use of the finding any appropriate information in the database as eRoom could be strengthened to benefit closure activi- currently configured. The Army should consider imple-

OCR for page 28
 MANAGEMENT SYSTEMS: LESSONS LEARNED PROCESS AND THE eROOM TOOL Box 4-1 description of the cma eroom An eRoom is an electronic space established by project management to enable members of a team selected by project management to collaborate and share information pertaining to work-in-progress. This is accomplished by making project information available to the team members for reviewing, copying, commenting upon, and possible editing irrespective of organizational affiliation or geographic location. Project management selects a member of the team to establish, implement, and coordinate eRoom activities. This team member is referred to as the coordinator. There may be more than one coordinator for an eRoom. A coordinator, with IT support, adds other team members to the eRoom membership (list) at the request of project management and, with manage - ment guidance, assigns one of three possible roles to each member. Team member roles are that of observer, participant, and coordinator. Each role provides different levels of functionality within the eRoom. An observer may view and copy contents located within the eRoom; a participant may view, copy, add, and modify contents; a coordinator may view, copy, add, modify, and delete any content. A coordinator may also modify roles and access permissions to content for team members. The coordinator monitors the eRoom for usage and periodically consults with management as to whether access to the eRoom by any individual should be maintained or terminated. The eRoom content primarily consists of files, folders, and objects, including audio and video. A team member who has been assigned a role which would enable them to add content may either drag and drop or upload content from another location. The team member who adds the content becomes the “owner” and may specify at the time of content addition which other team members may view the content, are identified as co-owners of the content, and may edit the content. The team member may also send an email alert to individual or multiple team members to advise them of the content availability. The eRoom incorporates additional functionality which may be deployed by the implementing organization. The Closure eRoom was established to disseminate programmatic information to stakeholders and to exchange information from the various sites for enhancement in the development of site-specific documents. SOURCE: Rafael J. Gramatges, Specialty Group Manager, URS Corporation, March 29, 2010. ties. As an example, the wording of specific documents tions might need to be made available to subcontractors such as sampling strategies and permit language at dif- since they will be a key part of the closure process. ferent sites could be reviewed for internal consistency One eRoom screen shows the details of how each before submission to a regulatory agency. topic area within the eRoom is addressed. For example, An individual must ask for access to the eRoom. If a screen on the topic of coordination and collaboration an individual has not used the eRoom within 60 days outlines when the eRoom was created, who the main the access is canceled. An individual may be granted contact is, key news and information items for the access to portions of the eRoom (for example, closure- users, and a legal disclaimer about privileged informa- related topics only or monitoring-related topics only). tion. This is followed by a listing of all the files, docu- The committee was told that this security protocol is ments, and training materials deemed pertinent to the needed because the eRoom is URS company-wide and subject area of closure. not restricted to Chemical Materials Agency activities Another eRoom screen outlines the history of docu- only. For this reason access is limited; a change would ment retention and change. This is extremely helpful require action at a high URS corporate level. when determining the age and relevance of informa- Typical screens for the eRoom show who has access tion. The records retention page lists the document’s to the room, what role the individuals have in the orga- name, the date it was modified, the name of the docu- nization, and where their offices are physically located. ment owner, and the document’s size. From the screens the committee reviewed, it is apparent The eRoom is a potentially very powerful tool both that a relatively small number of individuals actively for coordination purposes during draft markups or for use the eRoom. In order for the eRoom closure lessons someone looking for information on a particular topic learned to be an effective management tool, the room (for example, a Resource Conservation and Recovery would typically need to be used by a wide variety of Act permit and its contents). The use of the eRoom people, and during closure. For example, selected por- appears to be relatively widespread, but relatively few

OCR for page 28
 REVIEW OF CLOSURE PLANS FOR THE BASELINE INCINERATION CHEMICAL AGENT DISPOSAL FACILITIES Finding 4-6. The eRoom is a very powerful informa- individuals are heavy users. The committee noted that the closure managers at their June meeting consistently tion sharing and management tool, both for develop - referred to the eRoom and often asked for certain spe- ing new documents and for allowing users to find cific documents to be uploaded so that all sites could information that is pertinent to a particular issue or have access to them. As with the lessons learned, use problem. of the eRoom is for the most part from the bottom up Recommendation 4-6a. The committee strongly sup- in terms of personnel. There appears to be less indica- ports the concept of the eRoom and encourages its use tion of a proactive use of the eRoom as a design tool as often as possible. or by the Chemical Materials Agency management as a means of promoting consistent sets of information Recommendation 4-6b. The committee suggests that among similar documents. The eRoom would perhaps the Army and its contractor examine current eRoom be more useful if the contractor had a system that was u sage and, if appropriate, develop procedures to more aggressive in “pushing” information in the docu- increase its usage, including the development of new ments to users. The concept of timely access to lessons documents and determining who should have access learned was described above, and this same concept during closure and dismantling activities. also needs to be considered in any use of the eRoom during closure, particularly by subcontractors active in closure activities. reFereNce NRC (National Research Council). 2002. Evaluation of Chemical Events at Army Chemical Agent Disposal Facilities. Washington, D.C.: National Academy Press.