National Academies Press: OpenBook
« Previous: 1 Introduction
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×

2

Looking Back and Moving Forward

Health professions education (HPE) has evolved over time and has moved through various models over the years, said Valerie Williams, vice provost for academic affairs and faculty development at the University of Oklahoma Health Sciences Center. The process of change has generally been gradual, as the status quo shifted in response to challenges and new circumstances. However, said Williams, COVID-19 has been a “remarkably

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×

different challenge” for HPE, and the pandemic rapidly affected every aspect of HPE and introduced disruption and chaos into the environment. Faculty and staff have risen to the challenge, said Williams, and demonstrated “an awesome level of resolve and commitment” to ensuring that the education of future health professionals could pivot and stay on track. Williams emphasized that acknowledging the compassion and dedication of health professions educators is critical for building the strength and resilience necessary to get through this challenge together.

The COVID-19 pandemic has forced an extraordinary amount of change, said Williams, and it can be difficult to see the end goal in the midst of disruption and chaos. Williams said there are two types of people who like change: those who have designed the change, and a “wet baby.” People who design change are willing to think through the steps for innovation and adaptation. The need for change is evident to them and the necessary next steps may be quite clear. Others, who do not see the need for change, may expect to be convinced, remain skeptical, or oppose any change from the status quo. In the situation with a “wet baby,” said Williams, the baby wants change—being a “wet baby” is uncomfortable. The “change” results in a much happier, more comfortable baby. Of course this metaphor can only be pushed so far, Williams noted with humor; however, she used it to make this point—the lessons participants are learning will include awareness and interest in HPE change along a continuum from not ready to very ready for HPE change, particularly given what we are learning in the era of COVID-19.

Williams shared one example of how things have changed during the pandemic. Early in the process, she said, health professionals and educators talked about “social distancing” as important to mitigating transmission risk. However, as time went on, the messaging shifted toward “physical distancing,” as understanding of the term “social distancing” grew. With the realization that social distancing could be contributing to loneliness, isolation, and loss of the human “social” contact, the importance of shifting the language to support mental health and a general sense of well-being became apparent.

Williams asked workshop participants questions about their top concerns within HPE at different points in the pandemic. Williams reported several of the respondents’ concerns at the beginning of the pandemic were within learner assessment, competence, and feedback. As time passed, additional concerns surfaced to include learner isolation, distress, disparities, and safety issues. These critical areas will be addressed by our next three presenters, said Williams, and they are noted in the following three sections of this proceedings. The first speaker focused on her experiences during the pandemic as an educator, a health professional, and a mother; the second speaker framed how to use a systems approach to learn from the disruption

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×

of the COVID-19 pandemic; and the third speaker discussed how to understand and assess the experiences and lessons learned from the pandemic so we can begin plotting a way forward.

THE IMPACT OF COVID-19 ON FACULTY

Skylar Stewart-Clark, Charleston Southern University

Skylar Stewart-Clark, assistant professor and diversity and inclusion liaison at the Charleston Southern University physician assistant program, presented on the “significant seismic shift in everyone’s cognitive load” during the COVID-19 pandemic. Whether a parent, spouse, practicing clinician, or a health professions educator, the pandemic required immediate and dramatic shifts in roles and responsibilities. Stewart-Clark is a mother to four children, one with significant needs, and she noted the significant challenges around the shift to working and teaching from home while managing all of her children and their school schedules. As a faculty member, Stewart-Clark teaches patient assessment and diagnostic methods. She shared her initial response of it being “jarring” to think about how to teach these hands-on skills online. As a clinician, Stewart-Clark practices hospital medicine, and she was routinely called in for extra shifts to assist with surge capacity or to fill in for colleagues diagnosed with COVID-19. Simultaneously balancing all of these roles left Stewart-Clark feeling “isolated at times, quite uncomfortable, and pretty unsure” as to how to manage the chaos (see Figure 2-1). She shared her reflections on these experiences in four different areas with workshop participants: balancing work and life, leadership, empathy, and felt needs.

Balancing Work and Life

Managing both work and home life simultaneously added a new load of stress and the potential for burnout, said Stewart-Clark. Stewart-Clark relied on her value system to help her decide how to respond to competing demands. For example, her son, who is on the autism spectrum, required significantly greater attention as he shifted to a virtual learning environment because of the COVID-19 educational disruption. One evening, Stewart-Clark was working “feverishly” on a presentation and her son asked her to watch him play with his LEGOs.

Stewart-Clark drew on a framework called 10-10-10, developed by Suzy Welch (2009), in order to decide what to do. The 10-10-10 framework considers what the consequences of a decision will be in 10 minutes, 10 months, and 10 years. Stewart-Clark decided that the near-term and mid-term consequences were minimal, but that in 10 years, her son would remember that she

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Image
FIGURE 2-1 A seismic shift in cognitive load during the COVID-19 pandemic.
SOURCE: Stewart-Clark presentation, December 3, 2020.

pulled away from her work to spend quality time with him. Being “your own personal change manager” and making decisions based on your values can help manage the stress of prioritizing competing demands. When confronted with stressful, overwhelming circumstances, she said, it is critical to take a step back, reflect on one’s values, and then make choices that are in line with those values and one’s desired vision of the future.

Leadership

The pandemic had a disproportionate effect on female faculty, said Stewart-Clark. Nearly everyone’s responsibilities increased during the pandemic, she said, but for female faculty this was often in the context of less support in one or more areas of responsibility. In response to this change, Stewart-Clark used her leadership position to push for greater flexibility for faculty. For example, when her physician assistant (PA) program was set to return to the classroom in a hybrid format, bringing in small cohorts of students at designated times, Stewart-Clark made sure that the hybrid experience was available to faculty as well. Faculty members—particularly those with caregiving responsibilities—needed flexibility to arrange their schedules and responsibilities according to their needs.

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×

While there was some pushback initially, she said, in the end the idea was well supported and proved to be equitable. Stewart-Clark continued to push for other policy changes that could benefit women and caregivers, noting that COVID-19 has presented an opportunity to move issues from the backburner to the forefront and make meaningful changes. She urged participants to “lead from a position of authenticity” when creating space for innovation and change.

Empathy

COVID-19 has been incredibly disruptive to the health care learning environment, said Stewart-Clark, and it threatens student well-being. Stewart-Clark reflected on her own experiences as a clinician in an effort to empathize with her students. At the peak of the pandemic, she was being called in to work additional shifts, and from one shift to the next, “something would change.” For example, there were changes to the setup of the hospital, the COVID-19 treatment protocols, or the way that practice administrators communicated the changing expectations. Eventually, Stewart-Clark “started to accept the fact that something was going to change,” and this helped her mentally prepare herself.

Likewise, students were dealing with significant and continual changes to the learning environment. Her response, said Stewart-Clark, was to be as transparent and empathetic as possible and to allow as much flexibility as possible. She conducted social and emotional check-ins during each class, and engaged her students in a multitude of ways in order to give them a sense of community and belonging. For example, one day she invited the students to bring their pets with them to virtual class. Stewart-Clark hoped that her approach of transparency, empathy, flexibility, and inclusivity would convey to students the importance of cultivating grace for themselves and others, something that in a broad context seems absent in the daily practice of providers who are charged with caring for and maintaining the health of patients with diverse lived experiences.

Perceived Needs

Stewart-Clark noted that one of the most powerful lessons she has learned during the pandemic has been the importance of observing the needs of a group while using one’s own experiences to inform a response. For example, she said that being “thrust into practicing telemedicine” in her clinical practice was a great source of insight into how her students were coping with the transition to virtual learning. She also observed that many students had a “heart of service” but nowhere to serve, and at the

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×

same time, there were many patients with chronic diseases who, because of socioeconomic barriers, lost access to care and ran out of medications.

Out of these observations, Project Learn to Serve was born in conjunction with a local safety net clinic. The socially distanced service-learning project, which began in August 2020, served more than 200 patients with a diagnosis of hypertension or diabetes who had been lost to followup. PA students connected with the patients via telephone or video, and they used a decision tree to triage patients into emergent, urgent, routine, or telemedicine disposition. This program, said Stewart-Clark, increased clinic operational capacity, increased primary care telemedicine visits, and strengthened the chart review process for high-risk patients, ultimately improving care delivery. Moreover, the students who were involved had “profound outcomes” in learning as well as the opportunity to experience health care from a public health perspective. Stewart-Clark relayed the experience of one of her students, who said:

The most valuable learning from this project was witnessing the importance of an interprofessional team in optimizing patient care, and being exposed to the need for facilities like the Dream Center in underserved areas and how we can contribute to this now as students and in the future as medical providers. Having the opportunity to speak with each patient directly and learning the obstacles they face to access care in America was eye-opening. I will take this experience with me for the rest of my life.

In her concluding remarks, Stewart-Clark described the pandemic as highlighting “what we already know to be true about meaningful growth during challenging times.” She challenged workshop participants “to prioritize with a values-based strategic approach, to lead with authenticity and question the system, to communicate with empathy and transparency, and to respond appropriately to your felt needs.”

A SYSTEMS APPROACH TO INFORMING THE FUTURE

Pinar Keskinocak, Georgia Institute of Technology

Pinar Keskinocak, chair and professor in the School of Industrial and Systems Engineering at the Georgia Institute of Technology, gave workshop participants an overview of a systems approach to health care and HPE, and how it might help inform a post-COVID-19 world. There are multiple complex and interacting systems that affect our health, said Keskinocak, including our physical bodies, behaviors, workplaces, schools, faith institutions, environment, regulations, policies, socioeconomic conditions, and culture. To design and run an effective health system, it is critical to

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×

understand the features and characteristics of each system as well as how they interact and fit together. HPE, said Keskinocak, is a “keystone in the system of systems,” and has a significant effect on the health and well-being of individuals and the population.

HPE and health care delivery have traditionally been focused on treating the sick. However, Keskinocak said, it is well established that medical care is not sufficient for ensuring better health. Medical care is responsible for roughly 10–20 percent of health outcomes, with the other 80–90 percent attributable to health-related behaviors and environmental factors (NAM, 2017). COVID-19 has underscored this fact, as preexisting conditions have been associated with significantly higher risk of complications. This highlights the importance of promoting healthy lifestyles, prevention, and nonpharmaceutical interventions, she said.

Keskinocak said that using a systems approach to examine the experiences with COVID-19 provides an opportunity to be intentional in rethinking and redesigning HPE, both to respond better to future crises and also to provide better care during normal times. Challenges that could be addressed using this approach include

  • How do we better enable team-based work and better integrate HPE with practice?
  • How do we build multisector partnerships?
  • How do we shift the conversation from the leading causes of death to the leading causes of life?
  • How do we better integrate physical and mental health?
  • How do we adopt a competence approach to HPE? Which core competencies should be included in our curriculum, considering both the current and future needs across the system?

A systems approach begins with understanding the characteristics of complex systems like HPE, said Keskinocak. Complex systems often involve multiple stakeholders, multiple perspectives, and multiple—and sometimes conflicting—objectives and incentives. Balancing these conflicts requires considering the different perspectives and making trade-offs; for example, the decision to provide in-person learning or virtual learning requires consideration of the health and safety of learners and educators, the opportunities for learning, and the needs of various stakeholders. There is variability within complex systems, as well as limitations in available information, inaccuracy, and asymmetry. Uncertainty and disruptions are common, said Keskinocak.

In a complex system, a course of action may be optimal from an individual or subgroup perspective but may have unintended negative consequences in other areas of the system. For example, she said, the educational

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×

goals of an educator in a clinical setting may have unintended effects on the efficient care and discharge of patients, resulting in crowding and delays in admission of more patients. Keskinocak emphasized there is a need for better visibility, communication, and proper incentives to ensure the choices of individuals or subgroups align with the overall system’s objectives. Particularly during a crisis like COVID-19, operating within a silo is inefficient and ineffective. Instead, there is a need for systems that are adaptive, responsive, agile, flexible, evidence-based, and integrated with other systems, Keskinocak said.

Next, Keskinocak described the systems engineering approach. Systems engineering, she said, follows a structured, unified approach to improve or redesign systems considering multiple decisions, stakeholders, objectives, and constraints. The first step is analysis of the current state, which involves identifying problem areas, measuring the magnitude of the problem, and detecting potential root causes of the problem or bottlenecks. The second step is building a road map toward an improved state. This requires designing interventions to make improvements to the system, or redesigning the system entirely, along with evaluating the potential effects of these proposed changes. Finally, the effects of the proposed changes must be assessed after implementation. Analytic methods such as statistics, machine learning, and simulation can allow researchers to predict the effects of interventions even before they are implemented.

Keskinocak illustrated the systems engineering approach using an example of childhood asthma. If there are a large number of children presenting to the emergency department with asthma attacks, and they are experiencing long wait times, a siloed approach would view the long wait times as the symptom and would work on optimizing care for the children and making it more efficient. A systems engineering approach, on the other hand, would identify the root causes of the asthma attacks, such as air quality, access to medication, and health management education. Focusing interventions in these areas could reduce the number of children who end up in the emergency department in the first place, said Keskinocak. Williams added that in the midst of a crisis such as the COVID-19 pandemic, it is easy to slip back into siloed thinking and siloed approaches; keeping the systems engineering framework in mind can help prevent this return to a “comfortable place.”

The systems engineering approach has been widely used in other industries to improve efficiency, reliability, quality, and safety, and HPE is poised to achieve similar benefits, she said. This approach can be used to address challenges including health disparities, access to care, and the needs of an aging population, which are complex issues beyond the traditional patient–provider relationship. A systems approach to HPE has the potential to change the education of the future generation of health professionals and

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×

ultimately to improve the health and well-being of individuals and populations, Keskinocak concluded.

FROM SURVIVE MODE TO THRIVE MODE: DEVELOPING ACTIONABLE EVIDENCE

David Daniel, James Madison University

At the beginning of the pandemic, said David Daniel, professor of psychology at James Madison University, health professions educators were in survival mode, with little time to prepare for rapid changes and new demands. Although participants were throwing suggestions for tools and technologies at educators, Daniel said it was a challenge to triage the suggestions, to decide which ones to pursue, and to assess whether they were useful. To assess these ideas for change, it is critical to first identify and understand the end goals, he said.

Daniel used the war on hunger as an illustration of how the goals of an intervention must be carefully identified and targeted. When the war on hunger began in the 1950s, said Daniel, the goals were to make food that was accessible, affordable, convenient, and engaging. Adequate food was being grown in the United States but it was challenging to get the healthy foods to people who needed it most. Those who were tasked with fighting the war on hunger leveraged tools such as the new interstate highway system and food engineering in order to create and disseminate food that was easy to transport, affordable, and convenient. Unfortunately, said Daniel, although we won the war on hunger by producing cheap and tasty food, the unintended consequences of this approach were high rates of obesity, diabetes, hypertension, and heart disease. The four goals of making food accessible, affordable, convenient, and engaging were important, but what was missing was the critical goal of ensuring the food was nutritious. The lesson, said Daniel, is to “Pick your goals wisely!”

When choosing goals and assessing approaches for reaching these goals, Daniel said there are three main considerations: efficiency, effect, and practical usefulness. Efficiency is often discussed in terms of economics; however, money should not be the “sole arbiter of what works and what does not work.” The time and effort of those involved is also a critical consideration. For example, is an HPE intervention worth the time and effort from both the educators and the students? Answering this question also requires looking at the effect: Did an intervention result in the kind of learning that was desired? Did it affect the correct target? Were there unintended consequences?

Teaching and learning are messy, complex, and dynamic systems, and even the best-intentioned approaches can potentially create more problems

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×

than they solve, Daniel said. For example, the use of an online learning platform could be beneficial for supplementing coursework; however, it could potentially interfere with the learning process if students are too busy interacting with the platform rather than with the material itself. Daniel emphasized the goal of HPE is developing “usable knowledge.” Finally, assessing practical usefulness requires answering three questions:

  1. Is the approach solving a problem we actually have?
  2. How much improvement will we get?
  3. Will it work in this context?

Taking lessons from the COVID-19 pandemic to build a better HPE system moving forward will require consideration of a number of issues, said Daniel. The first step is to identify the specific goal of an intervention or approach. Daniel noted that our goals often shift over time, so our approaches must also be able to shift. The second step is to examine whether the system was working before it was adapted, and for whom it was working or not working. For example, was the system of in-person HPE education and training working well for all stakeholders before it was forced to move online because of COVID-19? Therefore, the third step is to determine how well the new system (e.g., virtual education) works, and for whom it is working and not working. Once the crisis subsides, will the new system still be useful or beneficial? The fourth and final step, said Daniel, is to ask how we can take these lessons and prepare for future challenges in HPE.

Daniel added that in addition to assessing systems and setting goals, it is critical for educators to create a safe place where students can come together during times of anxiety and crisis. While educators are also affected by the pandemic, “We do not want our stress to be contagious and compound the issue.”

Daniel underscored that now—in the midst of the pandemic—is the time to begin assessing and planning ahead. “Now is the time to think about whether the adaptations created new problems or exacerbated old ones. Now is the time to plan ahead for the future by learning from the present.” This is the time, he said, to identify the ways in which educators adapted, both formally and informally, and to assess whether the adaptations worked.

REFERENCE

Magnan, S. 2017. Social determinants of health 101 for health care: Five plus five. NAM Perspectives discussion paper. National Academy of Medicine, Washington, DC. https://doi.org/10.31478/201710c (accessed April 14, 2021).

Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 5
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 6
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 7
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 8
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 9
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 10
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 11
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 12
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 13
Suggested Citation:"2 Looking Back and Moving Forward." National Academies of Sciences, Engineering, and Medicine. 2021. Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/26210.
×
Page 14
Next: 3 Innovation Brought by COVID-19 »
Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop Get This Book
×
 Lessons Learned in Health Professions Education During the COVID-19 Pandemic, Part 1: Proceedings of a Workshop
Buy Paperback | $35.00 Buy Ebook | $28.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

During the COVID-19 pandemic, health professionals adapted, innovated, and accelerated in order to meet the needs of students, patients, and the community. To examine and learn from these experiences, the National Academies of Sciences, Engineering, and Medicine's Global Forum on Innovation in Health Professional Education convened a series of workshops, the first of which was a one-day virtual workshop on December 3, 2020.

The first workshop explored lessons learned in the grand challenges facing health professions education (HPE) stemming from the COVID-19 pandemic and how those positive and negative experiences might inform development of sustainable improvements in the value, effectiveness, and impact of HPE. Educators, students, administrators, and health professionals shared ideas, stories, and data in an effort to discuss the future of HPE by learning from past experiences. Topics included: evaluation of online education; innovations in interprofessional education and learning opportunities within the social determinants of health and mental health; effects on preclinical and clinical education; regulatory and accreditation changes affecting HPE; and stress and workload on students and faculty. This publication summarizes the presentations and panel discussions from the workshop.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!