National Academies Press: OpenBook
« Previous: Understanding KM
Page 17
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 17
Page 18
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 18
Page 19
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 19
Page 20
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 20
Page 21
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 21
Page 22
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 22
Page 23
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 23
Page 24
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 24
Page 25
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 25
Page 26
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 26
Page 27
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 27
Page 28
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 28
Page 29
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 29
Page 30
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 30
Page 31
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 31
Page 32
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 32
Page 33
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 33
Page 34
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 34
Page 35
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 35
Page 36
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 36
Page 37
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 37
Page 38
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 38
Page 39
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 39
Page 40
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 40
Page 41
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 41
Page 42
Suggested Citation:"Implementing KM." National Academies of Sciences, Engineering, and Medicine. 2015. A Guide to Agency-Wide Knowledge Management for State Departments of Transportation. Washington, DC: The National Academies Press. doi: 10.17226/22098.
×
Page 42

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

17 Guide to Agency-Wide Knowledge Management for State DOTs Implementing KM “If you don’t know where you are going, you will end up somewhere else." – Yogi Berra Step 1: Assess Risks and Opportunities The first step in implementing an agency-wide approach to KM is to identify focus areas based on an assessment of agency risks and opportunities. Purpose of a Knowledge Assessment Identify current areas of vulnerability – e.g., employees of retirement age with valuable and difficult to replace knowledge. Assess your organization’s current level of capability and bench strength in key skill areas that will be needed for future achievement of strategic goals – e.g., active traffic management or contract negotiation. Identify opportunities for expanding use of existing techniques for knowledge transfer that are working well in the organization. Options for Conducting a Knowledge Assessment A knowledge assessment can be approached with different levels of detail. Some agencies may want to use a thorough approach involving surveys and focus groups to understand how knowledge is currently Knowledge Assessment Options Quick Litmus Test Senior Leadership Workshop In-depth Knowledge Survey Knowledge Risk Assessment

18 Guide to Agency-Wide Knowledge Management for State DOTs being captured and shared – and where the specific gaps are. Other agencies may want to begin with a more “lightweight” assessment approach. Guidance for different assessment methods is provided below. KM Litmus Test. The following “KM Litmus Test” can be taken by senior managers to provide a quick indication of the need for (and likely payoff from) implementing an agency-wide approach to KM in the organization. This won’t yield the types of insights that are really needed to identify specific focus areas, but it can provide an initial indication of key risk areas and some motivation for moving forward. The KM Litmus Test for DOTs Please check which of the following apply: More than 20 percent of our most senior managers will be eligible to retire in the next 5 years. Many knowledgeable mid-career employees are likely to leave the organization over the next few years to pursue other opportunities. We haven’t identified specific skill or knowledge areas important to our organization. We don’t track how many employees we have with critical skills or experience levels. We don’t generally maintain up to date documentation of our core business processes and procedures. We don’t have standard ways of debriefing employees, contractors and consultants before they leave in order to capture lessons learned and other valuable knowledge. There are no mentoring programs or organized forums for sharing knowledge between experts and novices (newcomers). Employee training and development budgets are limited and shrinking. One part of our organization may not know what the other part is doing—even if working on a similar task or problem. It is difficult or time consuming to find current information that would help us to improve efficiency – templates, lessons-learned, checklists, etc. Employees don’t feel they have the time to chat with colleagues in the organization in an informal way. Our peer agencies are ahead of us in implementing knowledge management and retention strategies. Scoring: If you marked 7-12 items, then your organization would likely reap substantial benefits from an agency-wide approach to KM. If you checked 4-6 items, then your organization could be enhanced through additional knowledge management practices. If you checked 1-3 items, you can focus on fine-tuning and sustaining what you are already doing. A knowledge assessment provides an opportunity to understand current strengths, risks and weaknesses. The results should be used to shape the KM strategy and set priorities for what to do first.

19 Guide to Agency-Wide Knowledge Management for State DOTs Senior Leadership Workshop. This knowledge assessment approach involves convening the senior management team for a half-day workshop to identify and agree on the critical “at risk” knowledge areas that are important to the strategic mission of the organization. Key questions for each workshop participant are: What are the key challenges and opportunities facing your specific area in the next 2-3 years? What Key Performance Indicators (KPIs) do you use to measure success? How do you see these changing over the next 2-3 years? What are the critical types and areas of skill or know-how that will be needed for success in the next 2-3 years? Which of these areas are currently weak or at risk given existing bench strength and likely retirements or employee departures? What are you currently doing to address weaknesses and risks – e.g., succession planning, updating position descriptions, improving documentation, cross training, mentoring, communities of practice, etc.? What more should we be doing as an organization? After the workshop, the Human Resources department can be requested to develop a knowledge attrition profile that (1) identifies key employees in each business area – i.e., those with specialized or unique expertise and (2) estimates the likelihood of these individuals retiring within the next 2, 5 and 10 years. This profile is helpful for understanding potential future knowledge gaps in the organization. In-Depth Knowledge Survey. A more formal, in-depth knowledge assessment method is to circulate a Knowledge Assessment Survey to the employees in the agency. A sample survey is included as resource 1-1 in the “Additional Resources” section of this Guide. The survey is comprehensive, and identifies: How and where employees currently seek information and knowledge – from both people and codified sources; Current knowledge sharing behaviors; and Perceived gaps in available information or knowledge that impact job performance. Compiled survey results can help agencies to identify: The extent and nature of knowledge gaps that may be impacting organizational effectiveness; Key employees who are serving as the “go to” sources of information and knowledge; Isolated employees or workgroups that aren’t accessing available sources of information and knowledge; and Georgia DOT Loss of Institutional Knowledge The Georgia DOT (GDOT) conducted a workforce assessment that found that critical agency knowledge was at risk of being lost due to pending staff retirements: Forty-five percent of the senior leadership (Division Director and above) was within five years of retirement. Thirty-seven percent of the GDOT office heads was within five years of retirement. Forty-three senior staff positions were expected to retire within 15 months. To mitigate these risks, succession planning, mentoring, and KM education was introduced throughout the GDOT community.

20 Guide to Agency-Wide Knowledge Management for State DOTs Strengths and weaknesses in both formal and informal methods for knowledge sharing. Knowledge Risk Assessment. A quantitative assessment of the level of knowledge risk for each position in the agency (or within selected departments) can be performed to guide both workgroup-level and agency-wide KM activities. The Tennessee Valley Authority (TVA) developed a simple approach to calculating a knowledge risk score. This approach has been adapted by Constellation Energy/Exelon to examine knowledge risk for 160 program areas and allocate resources effectively to ensure future success. The knowledge risk score for a position can be calculated as the product of two factors: attrition risk and position risk. Based on the total risk score, positions can be prioritized and options can be identified to retain or mitigate knowledge loss. Using the Assessment Results Regardless of which assessment methods are used, it is important that the assessment findings be summarized in a manner that helps the agency determine priorities and strategies. Developing an agency-wide KM strategy is covered in Step 2. Knowledge Risk Assessment Attrition Risk is a number from 1-5 where: 5 Projected retirement or departure within current or next fiscal year 4 Projected retirement or departure in 3rd fiscal year 3 Projected retirement or departure in 4th fiscal year 2 Projected retirement or departure in 5th fiscal year 1 Projected retirement or departure in or after the 6th fiscal year Position Risk is a number from 1-5 where: 5 Mission critical knowledge (knowledge undocumented, unique, no duplication) 4 Critical knowledge and skills (limited duplication exists at other sites) 3 Important systematized knowledge and skills (documentation exists) 2 Proceduralized or non-mission critical knowledge and skills (training programs are current) 1 Common knowledge and skills Total Risk Score = Attrition Risk * Position Risk 20-25 High priority (immediate action needed) 16-19 Priority 10-15 High importance Under 10 Moderate to low importance There is no single “one- size-fits-all” KM strategy—use the knowledge risk assessment to target areas for improvement. Agencies can tailor the level of detail and limit the knowledge risk assessment to particular focus areas in order to keep the effort to a manageable size.

21 Guide to Agency-Wide Knowledge Management for State DOTs Step 2: Develop a KM Strategy What Is a KM Strategy? A KM Strategy is a written document that defines what the agency hopes to accomplish through KM techniques and how. The strategy should support the agency’s overall strategic plan and reflect what is learned in the assessment of risks and opportunities. The strategy need not be lengthy or elaborate – it can fit on a single page. Why Develop a KM Strategy? The process of developing the KM strategy allows key players in the organization to get engaged and aligned. It provides an opportunity for the agency leadership to set a clear direction and commit to holding staff accountable for moving forward. How to Develop an Agency KM Strategy A KM Strategy can be developed either as a follow-on activity from the agency’s broader strategic planning process, or through a stand-alone effort. A small working group of 3-5 individuals can be designated to develop the strategy. These individuals can be drawn from the following functions: central office business units (e.g., planning, maintenance, design, construction), field offices (e.g., district or regional maintenance, traffic operations, project delivery) and relevant support functions (e.g., human resources or organizational development, information technology, research/library, public affairs/ communications). Each working group member can be responsible for seeking input from a broader group as part of the process. A single individual should be designated to lead the effort and serve as the liaison to the executive team. The charge of the working group is to (1) establish KM goals based on the agency’s strategic plan and the priority focus areas identified in the assessment step; (2) identify appropriate KM strategies to pursue; (3) identify resources required for the first 6-12 months; and (4) seek executive team endorsement of the strategy – and commitment of resources. KM at NASA NASA’s KM strategy was shaped, in part, as a result of the Space Shuttle Columbia disaster. The agency recognized that while it had many written procedures and “knowledge databases,” improvements were needed in NASA’s culture of knowledge sharing. A Chief Knowledge Officer (CKO) position was established to address these concerns and develop a KM strategy. This strategy focused on creating stronger collaboration networks and online communities of practice, building a culture of openness and sharing, infusing lessons learned into the project development life cycle, and holding project teams accountable for capturing and leveraging lessons learned. NASA’s recognition and reward structure was revised to encourage learning and knowledge sharing proficiencies.

22 Guide to Agency-Wide Knowledge Management for State DOTs Linking to Agency Strategic Plans Many transportation agencies have developed strategic plans and/or business plans that include organizational values or goals related to KM. Here are some examples from current state DOT strategic plans: Innovation Excellence Organizational Strength Learning and Growth Collaboration Partnerships Customer Service The KM strategy can “hook into” these values or goals that have already been articulated. KM strategies can also consider more externally focused agency goals related to safety, mobility, infrastructure preservation, economic development and environmental stewardship. For example, pending loss of key individuals in the agency’s safety function may impact the agency’s ability to select and deliver the most effective set of safety improvements for the available dollars. Lead managers from key functional areas of the department can help to identify situations where KM strategies are needed to mitigate risks of employee departures or to build capabilities in emerging skill areas. KM Goals A KM strategy document should begin with a statement of goals for KM, which may include one or more of the following: Addressing priority risks of knowledge loss associated with pending retirements of experienced employees; Building the institutional memory of the agency before people leave or retire; Building bridges across “islands of knowledge” that may exist within functional silos or individual field offices; Stimulating innovation through the sharing of ideas and knowledge across disciplines; Strengthening the agency’s adaptability and agility; Increasing employee efficiency and effectiveness in specific functional areas; and Strengthening customer service. A KM Strategy must align with the mission of the organization. Establishing a clear linkage between the KM strategy and existing organizational goals puts the KM strategy on a solid foundation. KM at Caltrans Caltrans developed an enterprise risk profile, following Risk Management Standards from the International Standards Organization (ISO 31000). Through this process, workforce development was identified as one of the top agency risks. The Chief Risk and Ethics Officer led the development of Knowledge Transfer initiatives as a key risk management strategy – including development of a Knowledge Transfer Guidebook. See reference [6].

23 Guide to Agency-Wide Knowledge Management for State DOTs These examples are fairly general, but can be tailored as needed to the agency’s specific areas of concern. For example, one agency may choose to emphasize strengthening knowledge sharing across staff serving in similar roles in different field offices. A second agency may want to focus on fostering innovation in areas such as context- sensitive design, public-private partnerships or traffic management. KM Strategies The next step is to identify a set of strategies that will achieve the goals that have been established. For example, if the goal is to address priority risks associated with pending retirements, strategies might include a mix of analyses (to identify unique expertise), training and job shadowing (to bring other employees up to speed), documentation and targeted recruiting. KM strategies can be considered from each of the categories outlined above in the section on Key Elements of KM: KM Leadership & Direction strategies involve strengthening the agency’s understanding of KM, defining roles and responsibilities for KM, collecting information to help identify and diagnose gaps in skills, and tracking results of KM activities. Social Learning & Communities strategies involve facilitating person-to-person knowledge sharing and collaboration. Communities of practice are perhaps the most well-known KM technique in this category. Knowledge Codification & Dissemination strategies involve documentation of business processes and procedures, and capture of information (e.g., lessons learned) from project teams, individual employees and contractors through structured interviews. Results of these interviews may be produced in various forms – documents, slide presentations, videos, charts, etc., stored in various repositories for convenient retrieval. Succession & Talent Management strategies related to KM involve proactive steps to identify and manage risks associated with knowledge loss resulting from employee transitions. These include tracking and sharing information about the available skills within the organization, and activities to address current or emerging skill or experience deficits. As illustrated in Figure 4, a balanced set of KM strategies will target People, Process, and Information Management/Technology: Information Management/Technology elements include provision of electronic platforms for knowledge sharing as well as KM Is Not a Content Repository “Although IT is a wonderful facilitator of data and information transmission and distribution, it can never substitute for the rich interactivity, communication, and learning that is inherent in dialogue. Knowledge is primarily a function and consequence of the meeting and interaction of minds.” – Fahey & Prusak (1998) See reference [29]

24 Guide to Agency-Wide Knowledge Management for State DOTs practices to ensure that relevant data and content are captured, stored and documented for easy retrieval. People elements seek to foster knowledge sharing behaviors and build individual employee skills. Process elements seek to ensure that work processes are improved based on prior experience, and that lessons learned are distilled to inform future processes. These three elements combine to provide the motivation, the means, and the opportunity for knowledge sharing in an organization. Figure 4. KM Strategies One common pitfall is to have Technology be the centerpiece of the KM strategy. It should not be. Look first at People and Process elements, and then identify a limited number of technology strategies needed to support these other elements. Technology should not be the driver of KM.

25 Guide to Agency-Wide Knowledge Management for State DOTs Table 1 below displays elements of agency-wide KM, classified by People, Process and Information Management & Technology. Descriptions of each of these elements are provided in the next section. Note that many elements fall into more than one of the three categories – for example, Communities of Practice fit into all three: they reinforce knowledge sharing behaviors, provide opportunities for problem solving related to ongoing work processes, and may be supported by online collaboration tools. Table 1. Elements of Agency-Wide Knowledge Management AGENCY-WIDE KM ELEMENTS PEOPLE PROCESS INFO MGT & TECHNOLOGY KM LEADERSHIP & DIRECTION Strategic Planning & Policy Development • • • KM Education/Training • Knowledge Assessment • • • Knowledge Mapping/Social Network Analysis • • KM Metrics • • • SOCIAL LEARNING & COMMUNITIES Communities of Practice • • • Peer Reviews • • After Action Reviews • Social Networking and Collaboration Platforms • • Expertise Locator/Smart Org Charts • • • Recognition & Rewards for Collaboration • KNOWLEDGE CODIFICATION & DISSEMINATION Lessons Learned Repository • • Organizational Narratives/Storytelling • Knowledge Books/Continuity Books • • Business Process Documentation/Automation • Contractor Knowledge Transfer • Content Management/Portals/Wikis • Common Vocabulary/Content Classification • Personalization/Role-Based Subscriptions • SUCCESSION & TALENT MANAGEMENT Talent Tracking • • Desk-Side Reviews • • Mentoring, Shadowing and Job Rotation • Phased Retirements • • Leadership Training •

26 Guide to Agency-Wide Knowledge Management for State DOTs Moving the Strategy Forward Once strategies are identified, the working group needs to consider who will be responsible for KM implementation, and which resources will be required. A detailed implementation plan isn’t required at this stage – just enough information to enable the senior leadership of the agency to decide to endorse the strategy and approve moving it forward. Keep in mind that an agency-wide approach to KM may not require significant new investments, since most organizations are already doing elements of KM. Agencies can start with a modest effort and expand as appropriate once benefits are clearly demonstrated. Importance of a KM Lead In addition to strong top management support, an essential element of any agency-wide KM strategy is identifying a single KM lead person. While different elements of KM can be carried out by staff in multiple divisions, the KM lead is responsible and accountable for making sure the entire collection of activities is meeting the established goals. They are in a position to best understand what combination of strategies would work best to address identified gaps – and to adjust priorities as new needs arise or as availability of resources changes. In order to ensure that KM is sustainable in the organization, the KM lead should work closely with 1-2 others who would be in a position to take over KM lead responsibilities in their absence. It is important to recognize that the success of KM initiatives will depend on the capabilities of the designated KM lead, and the level of upper management support they are given. Ideally, the KM lead will have the necessary knowledge and personality to serve as a champion for KM techniques across the organization. Because KM involves working across organizational silos, the lead should be perceived as neutral and have good communication and negotiation skills. There is no single best organizational location for a KM lead. Possibilities include: Research, Training, Human Resources, Risk Management, Performance Management or Business Support. Because KM should not be a technology-centric initiative, the Information Technology (IT) unit is not generally the best choice for locating the KM lead. However, the KM lead will need to develop a close working relationship with IT. Remember to consider a holistic approach to KM—this is more sustainable and more effective than a set of disconnected initiatives. Designating a single KM lead with the right set of capabilities is a critical element of an agency- wide KM strategy. Use Technology as an enabler rather than as the centerpiece of the KM strategy.

27 Guide to Agency-Wide Knowledge Management for State DOTs Example KM Strategies An example outline of a “modest KM strategy” for a fictitious DOT is shown below. This strategy allocates 50% of a full-time equivalent (FTE) employee to serve as the KM lead and orchestrate implementation. Figure 5 shows a second example from a real organization. This example illustrates a useful format linking agency strategic goals, KM goals, KM strategies, and implementation steps. A Modest DOT KM Strategy 1. Business Case for KM a. Within the next 2 years, four of the six district administrators are likely to retire. The next level of management in the districts has less than 5 years of experience. b. There is a need to strengthen the support system for newer staff – encourage them to learn from the existing core of senior, experienced staff in other districts. c. There is a need for consistency in core work processes across districts. 2. Goals a. Increase opportunities for collaboration across districts. b. Capture critical institutional knowledge from senior district staff before they leave the agency. 3. Strategies a. Establish and support communities of practice for district construction engineers and district maintenance engineers. b. Map standard business processes and use as part of the orientation process for new employees. c. Conduct knowledge interviews with senior district administrators and document key lessons for their successors. 4. Implementation and Resource Needs a. Designate a KM lead – allocate 50% of an FTE for overseeing all KM activities. b. Train two individuals in business process mapping. c. Brief managers on KM strategy and gain buy-in for staff participation in Communities of Practice. d. Establish quarterly reporting on progress to executive team. Figure 5. An Example KM Strategy Source: Adapted from Reference [25]

28 Guide to Agency-Wide Knowledge Management for State DOTs Step 3: Create a KM Implementation Plan Why Develop a KM Implementation Plan? The KM Strategy defines the goals for KM, outlines the types of initiatives that the agency wants to pursue, and designates a lead person to move it forward. A KM Implementation Plan provides more detail on how the strategy will be carried out. It provides an opportunity to identify specific activities, make sure that the right people are involved, and ensure that the necessary resources are made available to maximize their effectiveness. How to Develop a KM Implementation Plan The following steps can be followed to develop a KM implementation plan: 1. Identify KM activities to carry out the strategy. 2. Identify metrics and evaluation methods. 3. Develop a detailed plan for first set of initiatives. 4. Develop a communication plan to build and sustain awareness and engagement. 5. Identify resource needs: personnel, IT infrastructure, consultant support, etc. 6. Develop a budget. 7. Establish a schedule of milestones. 8. Set up periodic review and reporting cycles to track milestone completion and metrics. Additional guidance for identifying and planning appropriate KM techniques is provided below. See the next section (Step 4) for guidance on defining metrics for KM. KM Techniques Capsule summaries of KM techniques that can be considered for implementation are provided below, organized by strategy type. Building a Sustainable KM Program One DOT tried to develop a KM strategy in conjunction with a quality initiative. KM pilots were launched with promising results. Unfortunately, even though there were good intentions, it was difficult to get traction because staff and management felt that they were already overburdened and couldn’t take on any new activities. Then, the executive champion for KM left the agency, and KM activities were discontinued. This program suffered from having a single point of failure. Building more broad- based buy-in at the start could have enabled the initiative to persist through a change in leadership. One strategy is to have “Knowledge Stewards” in each of the main business units who have the recognition of their peers so that they can then spearhead and infuse KM efforts in their respective units.

29 Guide to Agency-Wide Knowledge Management for State DOTs KM Leadership & Direction Applicability: Agencies wanting to pursue a holistic KM approach. Expected Results: Identify and address the agency’s priority risks and needs; target KM activities where they will have the greatest payoff. Implementation Tip: Designating a KM lead is essential for making progress, but educating and involving a larger set of individuals in planning and implementation will ensure that KM takes root in the organization and has sustained impact. Techniques: Strategic Planning and Policy Development. Periodic review and revision to the KM strategy and supporting agency policies. KM Education/Training. Education and training to build understanding of KM strategies and techniques. Knowledge Assessment. Information gathering to understand where critical knowledge resides, what gaps there are (areas where additional knowledge is needed), as well as how knowledge is currently shared within the organization. It may also include assessment of key risk areas associated with potential loss of critical expertise or institutional knowledge. The results of this assessment can be used to target and plan appropriate KM techniques. Knowledge Mapping/Social Network Analysis. Formal mapping and analysis of knowledge sources and flows; this may be part of a knowledge assessment or a separate effort. Knowledge maps may also be produced to facilitate access to knowledge resources – see, for example, NASA’s Knowledge Map at: http://km.nasa.gov/knowledge-map/ . KM Metrics. Developing metrics to measure results of KM initiatives and tracking and reporting these metrics. KM at Wisconsin DOT Wisconsin DOT developed a useful matrix that provides additional guidance on when to consider various KM techniques, and the approximate level of effort required. This matrix is included as Table 2 on the next page. WisDOT also produced a KM Guide emphasizing simple, low-cost techniques. See reference [10].

30 Guide to Agency-Wide Knowledge Management for State DOTs Table 2. Wisconsin DOT KM Tools Matrix TOPIC AND TASKS BRIEF DESCRIPTION MIGHT BE GOOD FOR… RESOURCES Documenting processes Writing down processes Incumbent writes down steps in key tasks. Stable, routine tasks; quick reference Low Videotaping processes Incumbent is videotaped performing key tasks. Quick capture; including context Low Formalizing process Formalizing process Manually require steps be completed in certain way. More complex tasks Low Automating process Automation requires steps be completed in certain way. Highly complex tasks with many players Med Expert decision system Incorporates expert judgment. Provides decision. Complex decisions that can be modeled High Experiencing together Double filling key positions New employee and retiring employee work together. Critical positions with sole complex knowledge Low Cross training Train employees to do a range of overlapping work. Positions with sole knowledge Med Communities of practice Employees with similar work regularly communicate. Positions scattered throughout agency Med Sharing experience Exit interviews HR or supervisor asks questions of departing employee. All departing employees Med Expert interviews Interviewer asks questions of knowledgeable employee. Employees with extensive specific knowledge. Med Last lectures Departing employee gives open-ended talk. Departing employees with extensive tacit knowledge. Med Storytelling Current employees share stories of challenges faced. Current employees with extensive tacit knowledge. High Developing leaders Rotation program Selected employees work in one or more new areas. Employees showing leadership promise. High Leadership program Selected employees receive agency exposure. Employees showing leadership promise. High Source: Wisconsin DOT

31 Guide to Agency-Wide Knowledge Management for State DOTs Social Learning & Communities Applicability: Agencies wanting to shift the culture of their organizations to foster innovation, collaboration and group problem solving. Expected Results: Increased innovation, faster ramp-up time for new employees, less chance of “rookie mistakes,” improved organizational resilience, improved employee retention, less dependence on any single individual. Implementation Tip: Collaboration and social networking tools have value, but face-to-face interaction is also essential for building the kinds of trust relationships required for productive collaboration. Techniques: Communities of Practice. Communities of individuals with similar roles in the organization that meet periodically to brainstorm, problem-solve and share their experiences. Peer Reviews. A peer review process prior to initiation of a major project phase or initiative to ensure that lessons from prior efforts are considered. After Action Reviews. Holding a team meeting following completion of an initiative or major project phase to identify and document what went well, what could have been improved, and how to fix things to be sure that they don’t go wrong again. Some organizations also use Before- and During-Action Reviews to document lessons learned as they go and make corrective actions as needed. Expertise Locator System/Smart Org Charts. Tracking specialized expertise or skills and providing an online “yellow pages” of expertise in order to locate appropriate individuals in the organization. Social Networking and Collaboration Platforms. Platforms for social networking and forums that allow people to share views on related topics of interest - either in-house or external. These may include specific areas for onboarding of new employees to allow them to connect to peers and “learn the ropes” together. Recognition and Rewards for Collaboration. Recognition of collaboration and knowledge sharing behaviors in employee performance reviews, providing opportunities to showcase and reward collaborative efforts. Knowledge Codification & Dissemination Applicability: Agencies wanting to ensure that all employees have ready access to documentation about the “who, what, how, when and Recognize that interpersonal trust and respect are prerequisites for knowledge sharing. Allow time for face-to- face communication and relationship building. Encourage learning and knowledge sharing behaviors through the agency’s recognition and reward system. Consider including learning and knowledge sharing proficiencies as part of the employee’s annual performance review.

32 Guide to Agency-Wide Knowledge Management for State DOTs why” of key business processes. Codification and dissemination activities focus on the kind of knowledge that lends itself to capture and documentation – such as procedural knowledge, key lessons and past events. Expected Results: Improved consistency of practice across the agency, lower chance of employees “reinventing the wheel,” greater awareness across the agency of effective techniques (or costly mistakes). Implementation Tips: It is best to be selective about what is captured, and make sure that the effort required will have sufficient payoff in the future. Information products should be designed to facilitate future use (e.g., checklists, presentation slides) and opportunities for using them should be actively pursued. Techniques: Lessons Learned Capture. Collecting, reviewing and publishing lessons that can be easily accessed and applied. These lessons can document both reasons for success as well as reasons for outcomes that varied from original expectations. Organizational Narratives/Storytelling. Capturing nuggets of wisdom through interviewing individuals before they retire and making them accessible/searchable via agency intranet sites or other information repositories. Continuity Books. Writing the “how to do my job manual,” or the do’s and don’ts of various organizational processes and making them available through a content management system. Knowledge Books. Conducting in-depth interviews with experts to distill and document unique knowledge that can’t be found in other available sources. Business Process Documentation/Automation. Systematically mapping current business processes to provide a common understanding of steps, inputs, outputs and roles. This can be helpful for complex processes, and those where multiple business units are involved. In some cases, it may be possible to take this a step further and automate processes via workflow management tools. Contractor Knowledge Transfer. Building in standard contract requirements for documentation of information sources, assumptions, products, lessons and other resources discovered as part of consultant or contractor projects. Content Management Systems And Knowledge Repositories. Providing electronic platforms for storing and retrieving content including procedure manuals, business process maps, training Knowledge Books at Kraft Foods Kraft Foods has developed a series of “Knowledge Books” to capture and retain specialized knowledge held by individuals with unique expertise – such as formulation of concentrated dairy products. These knowledge books are created through application of a standard methodology called MASK (Method for Analyzing and Structuring Knowledge). They focus on explaining why things are done a certain way and are rich with commentary and examples. See reference [1] Captured knowledge has no value if it is not used. Any knowledge capture activities should be accompanied by specific strategies for ensuring that the organization benefits.

33 Guide to Agency-Wide Knowledge Management for State DOTs videos, presentations, photographs, and lessons learned documents. Ensuring that new (and existing) employees are made aware of these resources and encouraged to use them. Common Vocabulary/Content Classification. Developing standard key words and content classification methods to facilitate discovery of available content. Personalization/Role-Based Subscriptions. Enabling employees to subscribe to newly added content based on their roles or specific areas of interest. Succession & Talent Management Applicability: Agencies looking to lessen the impacts of employee departures – targeting certain critical positions (e.g., a senior engineer within one year of retirement); or addressing more general needs (e.g., growing the next generation of leaders). Expected Results: Smoother transitions as employees retire or transition to new jobs; improved availability of employees with skills to take on challenging technical or leadership positions. Implementation Tip: It is best to be proactive about succession and talent management – don’t wait until key employees announce their retirement to take action. Techniques: Talent Tracking. Defining knowledge, skills, and abilities for each position and tracking employee skill sets over time as they complete training and gain experience. Desk-Side Reviews. A successor sits down with the upcoming retiree at the retiree’s desk and listens as he/she shows notes, templates, shortcuts, or memory aids for performing his/her job. Job Shadowing. A less experienced employee follows a more experienced employee in the organization throughout the day, learning through observation. Job Rotation. An individual is moved to different parts of the organization in order to learn the “system” perspective. Mentoring. A formal program is set up that links mentees (less experienced employees) with mentors (more experienced employees) in the organization. The program may include guidelines and coaching for mentors in order to maximize effectiveness. Reverse mentoring can also be done for certain technology-related skills to allow a more experienced employee to learn from a newer employee. Don’t wait until employees are close to retirement to begin succession management and knowledge capture activities. Knowledge retention should start from day 1 of the employee’s tenure.

34 Guide to Agency-Wide Knowledge Management for State DOTs Succession Management. Targeting positions for succession management strategies including training, shadowing, double- filling positions, etc. Formal Phased Retirement Programs. As individuals are nearing retirement in the organization, have them reduce their time by partially retiring and spending time in training/educating their successors. (Note: double-filling positions may be prohibited in some states by law.) Retirees and Alumni Callbacks. Bringing back an organization’s retirees in various roles to bridge certain knowledge and skill gaps. Leadership Training. Offering training and apprenticeship experiences to further development of future leaders of the organization. Defining Roles and Responsibilities for KM Implementation Roles and responsibilities for implementation will depend on the scope of planned KM activities. Some agencies may choose to focus their KM efforts on succession management, and have the HR department take the lead. Others may focus on capturing lessons learned as part of major projects, and have the Project Management Office lead. Agencies interested in potentially pursuing the full range of KM strategies will want to appoint a single individual (or small team) to handle the KM leadership and strategy activities. This individual or team can then initiate some KM techniques and play a support role on others. Table 3 illustrates what a logical mix of responsibilities might be, indicating lead or support involvement for the KM lead, HR, IT, Research/Library and managers of major divisions and their constituent business units. Additional Implementation Advice Some additional tips to ensure successful KM implementation are: Start small – use pilots to test different methods and expand using the benefit of your experience. For each initiative, recognize that there will be a life cycle: startup planning, pilot, evaluation, and evolution. Think about how you will evaluate and report on each activity before you begin. When estimating time requirements for the KM lead, anticipate a mix of planned activities and responsive services based on requests and opportunities for value-added interventions. Lessons Learned: Mentoring One DOT established a mentorship program. The goal was to capture the expertise and field knowledge of long-time employees, and to pass this knowledge on to the next generation. This initiative enjoyed strong top management support. However, the program had mixed success, and the agency learned some valuable lessons about what it takes to make a mentoring program work. More careful implementation planning and oversight would have ensured a greater level of success and sustainability.

35 Guide to Agency-Wide Knowledge Management for State DOTs Table 3. Responsibility Chart for KM Elements AGENCY-WIDE KM ELEMENTS KM LEAD HUMAN RESOURCES IT RESEARCH/ LIBRARY DIVISION/ OFFICE MANAGER KM LEADERSHIP & DIRECTION Strategic Planning and Policy Development KM Education/Training Knowledge Assessment Knowledge Mapping/ Social Network Analysis KM Metrics SOCIAL LEARNING & COMMUNITIES Communities of Practice Peer Reviews After Action Reviews Social Networking and Collaboration Platforms Expertise Locator/ Smart Org Charts Recognition & Rewards for Collaboration KNOWLEDGE CODIFICATION & DISSEMINATION Lessons Learned Repository Organizational Narratives/ Storytelling Knowledge Books/ Continuity Books Business Process Documentation/Automation Contractor Knowledge Transfer Content Management/ Portals/Wikis Common Vocabulary/ Content Classification Personalization/ Role-Based Subscriptions SUCCESSION & TALENT MANAGEMENT Talent Tracking Desk-Side Reviews Mentoring, Shadowing and Job Rotation Phased Retirements Leadership Training Key: Lead Role Support Role

36 Guide to Agency-Wide Knowledge Management for State DOTs Step 4: Monitor Results Purpose of Measurement Without a good understanding of return on investment or value being added, KM initiatives will be difficult to sustain. Beyond ensuring accountability for KM activities, measurement also provides important information to the KM lead about what is working well, what isn’t, and what adjustments need to be made. Measurement for an agency-wide KM effort seeks to answer four questions: Costs: What are we spending on KM support activities? How much time is it consuming? Outputs: What are we delivering or producing? Exposure/Use: What is the “market penetration” of the KM support activities? Who are we reaching? Outcomes: Is the effort having its intended impact? Are we accomplishing our goals? Approaches to Measurement Agencies can approach measurement at different levels of detail. Since measurement can be a time-consuming activity, it is best to begin by identifying a few key measures and then add new ones as needed as the KM effort evolves. It is important to strike a balance between the level of effort needed to track KM results and the value of information being produced. Costs: Costs of KM implementation can be estimated based on the time spent by the KM lead (and others) to plan, facilitate and support KM activities. IT development and content management cost elements may be included as well. Outputs: The KM lead can maintain a simple activities log to track products and services provided to support KM implementation. Products may include guidance documents or model policies; services may include briefings, trainings, or meeting facilitation. Exposure/Use: Employee participation in KM activities, and use of knowledge resources can be tracked via manual methods such as meeting sign-in sheets, or automated methods built into systems (e.g., tracking of web page hits or document downloads). A balanced approach to monitoring KM value and effectiveness should include tracking of costs, outputs, exposure or use, and outcomes. Documenting return on investment is important, but it is also valuable to capture success stories and qualitative descriptions of outcomes.

37 Guide to Agency-Wide Knowledge Management for State DOTs Outcomes: Tracking outcomes is challenging because of the intangible nature of the results being sought – e.g., employee knowledge, organizational effectiveness, and consistency in application of policies and procedures, and resilience. Typical approaches to measuring outcomes involve either surveys or interviews with employees, or studies of specific business process efficiency changes. While time or cost savings should be quantified wherever possible, it is helpful to supplement this with qualitative information that helps people to understand why and how a KM technique was beneficial. In some cases, employee stories about how they benefited from a KM activity can be translated into estimates of cost savings for the agency. For example, the VDOT KM office facilitated a business process streamlining effort that resulted in an estimated savings of $300,000 annually. Table 4 provides examples of output, exposure/use and outcome metrics for the four categories of KM techniques discussed above. How to Measure KM Results The US Navy has developed an excellent Metrics Guide to Knowledge Management Initiatives - see reference [27]. This guide defines the following steps for measuring and monitoring KM results: 1. Identify key stakeholders and what they want to know about KM results (e.g., agency leadership, project managers, KM lead). 2. Select a measurement framework (e.g., balanced scorecard, cause-and-effect diagram, matrix showing expected versus actual results). 3. Define how the monitoring results will be used by each stakeholder. 4. Identify measures – tailored to KM initiatives and their implementation phase, including both quantitative and qualitative measures. 5. Develop data gathering approach (surveys, interviews, automated usage logs, etc.). 6. Collect and analyze information. 7. Identify changes to be made based on the results. The Navy Guide is a useful reference for design of an approach to KM measurement, and includes an appendix with sample metrics. In order to mitigate risks, the KM outcome metrics should map back to the agency’s strategy, goals and objectives.

38 Guide to Agency-Wide Knowledge Management for State DOTs Table 4. Sample KM Metrics AGENCY-WIDE KM ELEMENTS KM OUTPUTS EXPOSURE/USE IMPACTS/OUTCOMES KM LEADERSHIP & DIRECTION Strategic Planning and Policy Development KM Education/Training Knowledge Assessment Knowledge Mapping/Social Network Analysis KM Metrics • Completion of KM strategy • Completion of KM implementation milestones • Number and percent of individuals participating in KM planning & implementation • Awareness of KM strategies and resources SOCIAL LEARNING & COMMUNITIES Communities of Practice (CoPs) Peer Reviews After Action Reviews (AARs) Social Networking and Collaboration Platforms Expertise Locator/Smart Org Charts Recognition & Rewards for Collaboration • Number of CoPs coordinated • Number of briefings conducted on AARs • Modifications to recognition programs • Number of posts on collaboration site • Number of participants in CoPs • Number of AARs completed • Employee satisfaction • Employee retention • Reported benefits from collaboration (stories) KNOWLEDGE CODIFICATION & DISSEMINATION Lessons Learned Repository Organizational Narratives/Storytelling Knowledge Books/Continuity Books Business Process Documentation/Automation Contractor Knowledge Transfer Content Management/Portals/Wikis Common Vocabulary/Content Classification Personalization/Role-Based Subscriptions • Number of processes documented • Number of lessons learned added to repository • Number of videos produced • Policy in place for contractor knowledge transfer • Number of lessons downloaded • Number of videos watched • Average number of monthly updates to Wiki page • Number of active subscriptions • Employee awareness and evaluation of available resources • Savings from process streamlining and automation (study) • Reported benefits from use of available resources (stories) • Process consistency (study) SUCCESSION & TALENT MANAGEMENT Talent Tracking Desk-Side Reviews Mentoring, Shadowing and Job Rotation Phased Retirements Leadership Training • Number and percent of position descriptions updated with knowledge, skills & abilities requirements • Number of training sessions for mentors • Number and percent of employees completing leadership training • Number and percent of employees participating in job rotation or mentoring • Bench strength (number of possible successors for given position) • Reported benefits from job rotation and mentoring (stories)

39 Guide to Agency-Wide Knowledge Management for State DOTs KM Outcomes Before outcome measures can be defined, it is helpful to define expectations for each KM activity or initiative. Key types of outcomes for KM implementation include: adaptability/agility, creativity, institutional memory building, organizational internal effectiveness, and organizational external effectiveness. Once expectations are set, then specific surveys, interviews, or studies can be designed to assess results. The following examples provide a starting point for thinking about expected results from selected KM activities. Expertise Locator System: Reduced training time or learning curve. Improved decision making due to knowing the right person to address the questions from the expertise locator system. Reduction of re-work. Improved service to external partners or stakeholders. After-Action Reviews (AARs): Application of existing know-how in future similar activities. Reduced training time or learning curve. Improve organizational decision making or quality of products resulting from application of best practices/lessons learned. Time reduction for document preparation process due to learning from others via the AAR. Improved agency image (as measured by customer satisfaction surveys) due to learning from previous successes and failures and reacting expeditiously. Lessons Learned (LL) Repository: Improved performance and decision making through use of the LL resources. Reduction in cycle time through use of the LL resources. Decreased learning curve for new employees. Increased consistency in performing job duties across similar roles in the agency. Collaboration Platforms: Decreased average problem-solving time. Reduced rework through learning from others. Improved employee retention due to higher satisfaction associated with collaborative environment. Organizational Narratives/Continuity Books/Knowledge Books: Reduced tendency for employees to “reinvent the wheel” or repeat past mistakes due to improved institutional memory. KM at Chevron “Of the initiatives we’ve undertaken at Chevron during the 1990s, few have been as important or as rewarding as our efforts to build a learning organization by sharing knowledge. In fact, I believe this priority was one of the keys to reducing our operating costs by more than $2 billion per year—from about $9.4 billion to $7.4 billion—over the last seven years.” – Derr, T.K. (1999) See reference [28]

40 Guide to Agency-Wide Knowledge Management for State DOTs Accelerated transfer and use of existing know-how for increased learning. Decreased learning curve for new employees. Communities of Practice: Increased collaboration due to stronger sense of belonging and trust. Innovations and creativity increased through knowledge sharing. Improved decision making and work quality resulting from collaborative efforts. Increased organizational resilience due to improved collaboration and networks. Table 5 and Figure 6 illustrate how the Virginia DOT has characterized outcomes from their KM initiatives. Table 5. Sample Outcomes from the Virginia DOT KM Initiatives Initiative Outcome Lessons Learned Database Increased knowledge base on successful construction practices, strong level of support and utilization from construction managers and inspectors, national recognition for agency. After Action Review: Winter Maintenance Led to statewide implementation of anti-icing program based on techniques developed by frontline managers; findings will be incorporated into staff training programs. Standard Operating Procedure Development: Emergency Response Task Force Definition and common understanding of VDOT’s response to different incident types: crashes, terrorist attacks, HAZMAT spills, and weather-related events. Facilitation: Interagency Coordination on Incident Management Improved working relationships, leading to shortened incidence clearance times. Organizational Network Analysis: for Successful Construction Project Evaluation of how team member communication network contributed to project delivery efficiency and effectiveness – lessons can be applied to future construction projects. Process Mapping: Environmental Review Process Annual cost savings of $300,000 from process streamlining, improved understanding of process and its intersection with other processes, model for other groups. CoP Support: Construction Statewide vertical and horizontal integration of construction expertise used to inform the state strategic plan. Development of an evaluation approach should be an integral part of the planning for each new initiative.

41 Guide to Agency-Wide Knowledge Management for State DOTs Figure 6. Virginia DOT Measurement Framework for KM Outcomes (ROI = return of investment)

42 Guide to Agency-Wide Knowledge Management for State DOTs

Next: Learning from Experience »
A Guide to Agency-Wide Knowledge Management for State Departments of Transportation Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB's National Cooperative Highway Research Program (NCHRP) Report 813: A Guide to Agency-Wide Knowledge Management for State Departments of Transportation presents guidance for state transportation agencies on adopting an explicit knowledge management (KM) strategy and the ways that organizations have implemented such strategies. KM is an umbrella term for a variety of techniques for preserving and enhancing the knowledge of an organization’s employees and effectively employing that knowledge as a productive asset.

A PowerPoint Presentation also accompanies the report.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!