National Academies Press: OpenBook

Guide for the Process of Managing Risk on Rapid Renewal Projects (2012)

Chapter: 10 IMPLEMENTING THIS GUIDE

« Previous: 9 IMPLEMENTING RISK MANAGEMENT PLAN
Page 112
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 112
Page 113
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 113
Page 114
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 114
Page 115
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 115
Page 116
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 116
Page 117
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 117
Page 118
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 118
Page 119
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 119
Page 120
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 120
Page 121
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 121
Page 122
Suggested Citation:"10 IMPLEMENTING THIS GUIDE." National Academies of Sciences, Engineering, and Medicine. 2012. Guide for the Process of Managing Risk on Rapid Renewal Projects. Washington, DC: The National Academies Press. doi: 10.17226/22665.
×
Page 122

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

110 INTRODUCTION This guide has outlined an efficient and effective process for managing risks on rapid renewal projects. However, adequate planning and logistical support are required for a DOT to successfully implement this process. This chapter summarizes key logistical issues to consider when planning, staffing, and conducting the risk management process. Adequate planning, appropriate resources, careful coordination, and integra- tion into continuous project management processes are the keys to successful risk management implementation. The DOT should initiate the risk management process early in the project’s life cycle, and then update as appropriate. The DOT also needs to engage the appropriate participants and pro- vide them with relevant information for each of the risk management process steps. The DOT ultimately needs to adequately plan and resource the meetings, workshops, and project management staff throughout the process to ensure an efficient and effective process. A good planner and a qualified facilitator are keys to successful implementation. PROCESS OF IMPLEMENTING THIS GUIDE When to Apply This Guide Risk management is beneficial in all phases of project development. In general, the earlier risk management is started, the more time the project team has to react to the identified risks and the easier the risks are to manage, and thus the more ben- efits the project will realize from risk management. However, there is such a thing as 10 IMPLEMENTING THIS GUIDE Adequately but efficiently plan and imple- ment the risk management process described in this guide.

111 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS IMPLEMENTING THIS GUIDE starting too early to conduct effective risk management for individual projects. This can be true when a program is just being established, but the purpose and overall scope for individual projects have not yet been established. Once a project’s purpose and overall scope have started to take shape, various elements of the risk management process can be applied to maximize benefits. The fol- lowing guidance applies to large and/or complex projects, or projects with significant specialty elements: • When a project is in the scoping or preliminary design phase (e.g., prior to ap- proximately 10% design) and the DOT has yet to select a preferred alternative, the process can be particularly useful for evaluating the risks of each alternative rela- tive to the other alternatives. The process applied at this point includes structuring (Chapter 4); risk identification (Chapter 5); risk assessment (Chapter 6); and con- sidering some elements of risk management (Chapter 8), especially proactive risk reduction for significant risks. This comparison can help the DOT make decisions among alternatives, such as design alternatives, funding alternatives, or project delivery alternatives. If cost and schedule estimates also exist for each alternative at this time, risk analysis (Chapter 7) can also be conducted to quantify uncer- tainty in the cost and schedule for each alternative, which can then be compared among alternatives to help make decisions. An example of this type of comparison for project cost, where one alternative (full build) is about $100 million (or 6%) less than the other (phased full build), is shown in Figure 10.1. The corresponding project schedule, disruption, and longevity can also be compared in a similar way. At this stage of project development, these elements of the risk management pro- cess can be conducted in less detail than would normally be done for a preferred alternative, especially if results are being used only to compare alternatives. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 6 0 0 7 0 0 8 0 0 9 0 0 1 ,0 0 0 1 ,1 0 0 1 ,2 0 0 1 ,3 0 0 1 ,4 0 0 1 ,5 0 0 1 ,6 0 0 1 ,7 0 0 1 ,8 0 0 1 ,9 0 0 2 ,0 0 0 2 ,1 0 0 2 ,2 0 0 C um ul at iv e Pr ob ab ili ty or P er ce nt ile Overall Project Cost (Inflated $ million) Full Build Alternative Phased Full Build Alternative Difference at 80th Percentile Figure 10.1. Comparison of Cost Uncertainty for Project Alternatives.

112 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS • After the DOT has selected a project alternative (e.g., after completion of environ- mental documentation, or near 30% design), the original structuring, risk identifi- cation, and risk assessment for the preferred alternative (if done previously) can be updated to reflect the greater level of project development. Additional detail can also be included at this stage to get a better picture of the preferred alternative’s risks and opportunities. The DOT can also conduct risk analysis (Chapter 7) in this phase if cost and schedule uncertainty and defensible development of con- tingency to adequately cover those uncertainties are of interest to the DOT. Risk management planning (Chapter 8) and implementation (Chapter 9) are now also appropriate and beneficial for the preferred alternative. Again, the earlier in project development that the risk management process can be started, the greater the benefits. • As the project progresses beyond preliminary design and the environmental process to final design, right-of-way acquisition, and utility relocations, the DOT should update the risk management process at key project milestones, at some predeter- mined time interval, or both. For example, the U.S. Federal Transit Administra- tion (FTA) historically has required risk management updates at key project mile- stones, such as entry to final design and application for FTA’s funding grant. Other agencies, such as the Washington State DOT (WSDOT), typically will conduct annual updates for their large, complex, or high-visibility projects. When appro- priate, risk management can be integrated with value engineering (VE), where ways to pro actively reduce significant risks or capitalize on VE opportunities can be explored. • When a project nears construction procurement, some agencies will update the risk management process to develop a validated engineer’s estimate (including contingency) and to guide risk allocation for contract document preparation. The agency could also conduct a more detailed assessment of construction risks (e.g., management of traffic or construction staging) and plan specific risk management actions for those risks (either individually or collectively), if not done previously. This could be particularly useful for rapid renewal projects, which often employ innovative construction technologies and materials. • Unless a project has particularly complex construction staging or specialty con- struction, the risk management process during construction usually focuses on continuing to manage previously identified risks (rather than identifying, evaluat- ing, and managing new risks) and on managing contingency. However, there are cases in which risk identification and subsequent steps might be conducted (or repeated) during construction. For example, when a major failure has occurred during construction, the owner might want to make sure that the contractor has identified and can effectively manage similar potential problems through project completion. The risk management process is easily “scalable” to match project type, size, com- plexity, and needs. For projects that are not as large or complex, the risk management process should be much simpler. For example, structuring, risk identification, risk

113 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS assessment, and risk management planning might be conducted only once, although risk management implementation would have to be carried through to the project’s completion to realize the maximum benefits. WSDOT has such a policy for any proj- ect with an estimated cost between $25 million and $100 million (Washington State Department of Transportation, 2006). How to Apply This Guide The keys to success for the risk management process include proper planning, alloca- tion of appropriate resources, careful coordination, and integration into continuous project management processes. Lack of preparation and focus can grind a group to a standstill, resulting in inefficiency, frustration, and wasted effort. To ensure that the risk management process fulfills its potential, the DOT must conduct an effective and efficient risk management process, as outlined below. Leadership and Facilitation of the Risk Management Process Project leadership should provide command emphasis for the risk management pro- cess. The project leadership has to establish and continually reinforce the need for risk management to ensure that project-team members participate appropriately. Project leadership should also communicate the need for risk management “up the chain” to ensure that the proper external resources (including independent subject-matter exper- tise) are provided. Effective facilitation is essential for efficient and effective meetings and workshops that are inevitably part of the risk management process. A weak or untrained facilita- tor can cause a meeting or workshop to lose focus and fail. The facilitator should be knowledgeable (in general, but not necessarily with the specifics) about the various phases of rapid renewal projects. The facilitator also needs to be adequately trained in the risk management process and the underlying principles and guidance and should have practical facilitation experience (preferably for risk management). A few key points on facilitation include • Maintain a positive, engaging presence. • Try to achieve consensus, as well as project-team buy-in. Be fair—let all qualified voices be heard equally and do not let strong personalities dominate (bias) the dis- cussion. Encourage participation and responsibility. As long as no adverse group dynamics are at work, follow a policy that “silence is acquiescence.” • Appropriately consider all available information. • As tactfully as possible, keep the group focused—stay on task and on time. If bogged down, stimulate the discussion by asking different questions or asking questions differently (i.e., from a different angle). • Always keep in mind the goals for the risk management process—adequate but efficient. Keep the level of detail and quality of the assessments appropriate and consistent with the purpose for the risk management process. • Try to remain neutral, but do not be a pushover. The facilitator must believe (be convinced) that the assessments are reasonable and bias-free.

114 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS Participation in the Risk Management Process Project leadership should actively participate in the risk management process. Without consistent engagement by the project leadership, the risk management process will falter. Consistent leadership will ensure that the risk management process is carried to its conclusion and that risk management objectives are met. For example, project leaders often must provide key input to the risk management process, as well as make risk-based decisions regarding the project’s development. Project staff often do not have the knowledge or the authority to make such decisions, which can slow project development and hobble risk management. Project staff often do, however, have infor- mation on potential risks and risk management options. Project leaders should invite and encourage the entire team’s input into the process. Participants should be adequately qualified in their respective areas of expertise. Expertise can come in the form of project expertise (project-team members are experts about the particular project) and subject-matter expertise (discipline experts). A given participant can fulfill more than one role in the risk management process, if qualified to do so. However, the facilitator should tactfully request that participants who are not knowledgeable on a particular topic refrain from offering opinions on that topic. Unqualified opinions degrade the quality of assessments, as well as reduce the effi- ciency of the effort. Participants should include key project-team members (including the cost estima- tor and scheduler) and independent subject-matter experts. Perhaps the easiest way to avoid bias in the risk management process is to include project experts as well as project-independent experts. The interactions of these two groups are extremely use- ful for highlighting potential project issues and for reaching potential solutions. The independent experts could be the same as used for VE, realizing some efficiency. Participants should be at least minimally trained on the risk management process, their roles within the process, and on how to perform those roles. Previous chapters in this guide and the companion Simplified Risk Management Training course pro- vide a good training basis for participants. Otherwise, the facilitator should provide minimal training at the beginning of the workshop (see Appendix C for an overview presentation that provides such training, which should be made at the beginning of a workshop). Planning the Risk Management Process Planning for the risk management process is important and nontrivial. A good check- list, as well as a good planner, can help immensely when planning for the risk manage- ment process. The typical planning tasks and logistical considerations for a project risk management process include the following steps: Step 1. Initiate the Risk Management Process. • Identify the need and scope, as well as commitment, for risk management. This includes (but is not limited to): — Coordinating with the project team;

115 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS — Possibly tying risk management and VE processes together at key milestones; and — Determining if qualitative or quantitative analyses are needed (e.g., to quantify project performance uncertainty, from which appropriate budget and contin- gency can be determined). • Identify the funding source and secure funding for risk management. Coordinate with DOT management and the project team, and complete funding administra- tive requests or actions. Step 2. Prepare for the Risk Management Meetings or Workshops. • Identify the risk management process steps to be covered in a meeting/workshop. The DOT might implement a number of risk management process steps in one meeting (e.g., structuring, risk identification, risk assessment, and risk manage- ment planning), or have separate meetings, to suit the needs of the DOT. The DOT might tie risk management and VE together, and/or conduct a separate preparatory session up front to plan subsequent workshops and meetings, including identifica- tion of participants. • Implement necessary contracts and task orders (DOT internal and for consultants). Give sufficient lead time to contracting personnel, and follow up as required. • Identify and confirm participants, including facilitator, independent subject- matter experts, and project-team members. Follow up as needed. Iterate when the study schedule changes or for project risk management updates. Identify key project issues for which experts are needed (e.g., independent cost estimator and scheduler). Communicate the workshop schedule or agenda, responsibilities, and logistics to all members. • Identify the schedule for risk management, including risk management meetings and workshops. Iterate when member participation or facilities change or for proj- ect risk management updates: — Select the format for the workshop (e.g., single, all-encompassing meeting ver- sus more linear format with extended schedule and several smaller workshops or even interviews); — Plan for schedule changes when conflicts occur with other major events involv- ing significant resources or personnel; and — Develop a meeting or workshop agenda and distribute to all participants. • Identify, schedule, and confirm facilities for risk management meetings or work- shops. Iterate when the study schedule changes. Visit the facilities prior to the workshop start date to meet the necessary contacts and to assess the facilities. Facilities include — Venue: location, buildings (including access, after-hours access, and visitors’ passes), quiet main meeting room to comfortably accommodate all participants and one to two smaller breakout rooms, and parking.

116 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS — Support services and materials: printing and copying; information technology (computer network, phone, e-mail); LCD projectors (×2); notebook computer (for technical documentation); projection screen; dry-erase board and markers; paper flipchart and markers; power extension cords (3-prong grounded); daily refreshments; “working” meals; and miscellaneous office supplies. • Send a risk management workshop requirements packet to the project team (i.e., instructions for project-team preparation), such as project description and cost and schedule estimates. Follow up as needed. • Review and modify the requirements packet as needed, and deliver to the project team as soon as possible. • Establish and communicate the deadline for project team’s response. • Send project information (with instructions) to independent experts to review beforehand—especially review relevant design and cost/schedule estimate infor- mation for subsequent structuring. Step 3. Conduct the Risk Management Meetings or Workshops (according to Chapters 4 through 8 of this guide). • Kick off the risk management meeting workshop. Ensure that participants’ travel schedules are consistent with their required workshop participation. The risk man- agement facilitator should arrive early to set up the facilities and provide an over- view of the process (see Appendix C) and develop common understanding of the project. • Develop consensus on all risk management inputs. Document assessments in real time (e.g., on a computer screen using the Microsoft Excel workbook template, on a whiteboard). Having a separate notetaker working with the facilitator helps im- mensely. Break out into smaller groups for specialized topics, for which a second facilitator will be needed. A second facilitator also provides redundancy in case something happens to the first facilitator, thereby protecting the large investment made for the workshop. Provide adequate time (e.g., after the workshop) to review and finalize risk management inputs as well as to develop and implement the risk model (if needed). • Prepare a workshop risk management results briefing (if results are to be briefed outside workshop participants). As early as possible, forecast the briefing schedule and communicate to briefing attendees (especially if not participating in a work- shop). For example, the briefing might precede a separate VE workshop. • Present and discuss risk management results. Step 4. Document the Risk Management Process and Results. • Prepare and submit a draft risk management report, including risk management plan (which includes the risk register).

117 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS • Finalize the risk management report based on feedback from the project team and other workshop participants. Step 5. Implement the Risk Management Plan (according to Chapter 9 of this guide). • Ensure DOT commitment and resources. • Establish responsibility and authority. • Plan for and conduct monitoring and updates as appropriate (as above), as well as manage contingency. A separate logistics planner, working in concert with the risk facilitator, can help accom plish the above steps. Companion Implementation and Training Materials As previously noted, a qualified facilitator, as well as DOT management and project- team commitment, planning, and participation of appropriate project team and inde- pendent experts are keys to successful implementation of the risk management process outlined in this guide. A companion Simplified Risk Management Training course for this guide has been developed especially to train DOT facilitators to conduct impor- tant parts of the risk management process described in this guide on relatively simple projects. Also, forms and a Microsoft Excel workbook template have been developed (and are included in the training) to help the facilitator conduct the important aspects of risk management on simple projects (see Appendix C and Figure 10.2). This training is also useful for DOT management and potential participants, including key project- team members and independent experts (e.g., from DOT headquarters), to help them better understand the process. However, this training is not required for everyone who participates in the risk management process. Typically, the facilitator will provide a short overview of the process at the start of a workshop to adequately explain the pro- cess for the participants, and it will be up to the facilitator to subsequently guide the participants through that process. Such an overview presentation has been developed and is provided in Appendix C. The training course is 2 days, in which a hypothetical (but realistic) DOT rapid renewal project is evaluated for illustration and concept reinforcement. The class con- sists of individual modules, generally one for each chapter in this guide. However, whereas this guide focuses on the concepts (the what), the class focuses on the imple- mentation (the how to) and includes simple exercises and examples to accomplish this. Notes, in the form of annotated versions of all the PowerPoint slides shown in the class, provide additional details to what is provided in this guide. The focus is on structuring, risk identification, risk assessment (including risk severity analysis and prioritization), risk management planning, and risk management implementation, especially for relatively simple projects that a DOT can evaluate in house, which will help to optimize the performance of those projects. The class does not include detailed training in full quantitative risk analysis (Chapter 7) to quantify the uncertainty in project performance, which can be used to defensibly establish budgets and milestones (and contingencies). Such analyses require

118 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS specialized skills that cannot be developed in a 2-day class. Instead, the training will allow a DOT to effectively supervise such analyses, as well as supervise the evaluation of more complex projects. To help the facilitator conduct selected parts of the risk management process on relatively simple projects, specific forms have been developed to guide and document information developed in the workshop. In addition to hard-copy forms (PDFs), these forms also have been replicated in a Microsoft Excel workbook template for data entry and subsequent automatic analysis of that information. Such analyses include determination of (a) the mean values of base and total (base + risk) performance measures; (b) the severity (in terms of combined change in total performance mea- sures) of each risk and opportunity, based on which they are prioritized; and (c) the cost-effectiveness of possible risk management actions, based on which such actions can be recommended and resulting revised mean values of total performance measures are determined. The training includes the use of these forms and template. Figure 10.2. Overview presentation of forms and template (Appendix C) and training materials.

119 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS CONCLUSIONS ON IMPLEMENTING THIS GUIDE The risk management process presented has the potential to greatly improve the ability of project leadership and team members to make critical decisions as well as improve project performance with respect to the rapid renewal objectives. However, the process must be adequately planned and resources made available, and then followed through to its completion, to obtain these benefits in an efficient way. The following are keys to success: • Prepared technical resources (project-team and project-independent experts); • A (preferably two) qualified facilitator/analyst (to ensure an accurate, defensible, and efficient process); • A good planner (for logistics); • Organizational leader and system to provide — Active organizational support; — Adequate resources and participation; and — Commitment to implement the process. This chapter has offered important guidance on the logistics of the risk manage- ment process, including when and how to apply the process, to help ensure that the DOT realizes the full benefits of risk management. Additional guidance is provided in companion materials, including training materials, workshop introductory overview presentation, and specific forms and a Microsoft Excel workbook template.

120 GUIDE FOR THE PROCESS OF MANAGING RISK ON RAPID RENEWAL PROJECTS Example The hypothetical QDOT case study (see Appendix D), which is used to adequately illustrate the various steps of the risk management process and includes a risk management plan (RMP; Appendix E), involves implemen- tation of the risk management process on this project (as described in Chapters 2 through 9), following the principles and process outlined in this chapter, as documented in the RMP and summarized below. QDOT did the following (as documented in the RMP): • Assembled relevant project information (i.e., regarding scope, strategy and status, conditions and as- sumptions, cost estimate, schedule); • Convened a group of key project team staff and independent subject-matter experts from the key project disciplines in a series of workshops facilitated by a qualified risk elicitor/analyst, to conduct risk assess- ment and risk management planning (consistent with the principles, processes, and guidance described throughout the guide), culminating in an RMP (including the risk register); and • Assigned a risk manager (with appropriate authority and resources) to implement the resulting RMP, including monitoring, updating, and recommending project risks, risk reduction plans, contingency, and recovery. This process was well planned, supported by management, and adequately resourced. Adequate support and resources (including an organizational structure) were then provided to implement that plan throughout project development. Construction of the QDOT project was successfully completed on January 31, 2013, at an inflated cost of $22.0 (with $2.0 million remaining as a cost contingency and 2.0 months remaining as a schedule contin- gency). There were few unanticipated problems and no recovery actions. Performance of QDOT US-555 and SH-111 Project Project Performance Base Base + Contingency Actual Unused Contingency Cost (YOE$M) $17.0 $24.0 $22.0 +$2.0 Schedule (months) 35.0 40.0 38.0 +2.0

Next: 11 CONCLUSIONS »
Guide for the Process of Managing Risk on Rapid Renewal Projects Get This Book
×
 Guide for the Process of Managing Risk on Rapid Renewal Projects
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s second Strategic Highway Research Program (SHRP 2) S2-R09-RW-2: Guide for the Process of Managing Risk on Rapid Renewal Projects describes a formal and structured risk management approach specifically for rapid renewal design and construction projects that is designed to help adequately and efficiently anticipate, evaluate, and address unexpected problems or “risks” before they occur.

In addition to the report, the project developed three electronic tools to assist with successfully implementing the guide:

• The rapid renewal risk management planning template will assist users with working through the overall risk management process.

• The hypothetical project using risk management planning template employs sample data to help provide an example to users about how to use the rapid renewal risk management template

• The user’s guide for risk management planning template will provide further instructions to users who use the rapid renewal risk management template

Renewal Project R09 also produced a PowerPoint presentation on risk management planning.

Disclaimer: This software is offered as is, without warranty or promise of support of any kind either expressed or implied. Under no circumstance will the National Academy of Sciences or the Transportation Research Board (collectively "TRB") be liable for any loss or damage caused by the installation or operation of this product. TRB makes no representation or warranty of any kind, expressed or implied, in fact or in law, including without limitation, the warranty of merchantability or the warranty of fitness for a particular purpose, and shall not in any case be liable for any consequential or special damages.

Errata: When this prepublication was released on February 14, 2013, the PDF did not include the appendices to the report. As of February 27, 2013, that error has been corrected.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!