National Academies Press: OpenBook

International Benchmarking of US Immunology Research (1999)

Chapter: 2. Benchmarking Results

« Previous: 1. Introduction
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

2

BENCHMARKING RESULTS

2.1 Methods

The assessment of a country's relative standing in research performance is subject to multiple complexities. Many measures could be used to evaluate the position of US-based research in immunology, but each suffers from generic or specific limitations. Such limitations include the collaborative and international scope of immunological research, which makes the drawing of national lines somewhat arbitrary; the inequality inherent in of comparing a large, common enterprise with multiple smaller ones; and the difficulties of sifting information on the specific field of immunology from that on related research fields in large, aggregate databases. In addition, in the case of this panel's operations, budget and time constraints effectively ruled out any major undertakings to generate new data sets.

Within those constraints, the panel's strategy was to use three mainstream performance measures: reputation survey, citation analysis, and journal publication analysis it then relied on the convergence of findings to compensate for the specific shortcomings of each measure. Although these measures have some independence, the degree of independence could not be assessed because the panel could not compensate for all of the shortcomings of the methodology.

As shown in Figure 2.1 and Figure 2.2, the United States is clearly dominant in number of papers published in immunology. It produced 63% of the world's papers in immunology in 1981–1996; no other country produced 10%, and only one produced even 5%. Thus, comparisons are made in terms of the United States versus the rest of the world instead of country by country basis.

2.1.1 Reputation Survey

To estimate reputation, the panel conceived a virtual congress in which leading experts in immunology were asked to identify potential participants. Specifically, the field was divided into four major subfields–cellular immunology, molecular immunology, immunogenetics, and clinical aspects of immunology–and each was subdivided into four to 10 sub-subfields. The panel (composed of 11 US-based and three non-US-based scientists) then identified five to 15 respected leaders in each sub-subfield, based in the United States and in other countries, and polled each in person, by telephone, or by mail. The pollees were asked to imagine that they were about to organize an international congress session on their particular sub-subfield and to furnish a list of five to 20 potential speakers.

When the initial lists were considered, it became clear that there was a bias related to the laboratory location of the pollees: US-based investigators routinely named a higher percentage of Americans than did non-US-based investigators. The nationality of the poller also appeared to have an influence: the three non-US pollers often obtained a list moreenriched in non-US speakers. Thus, additional subfield leaders were polled to approximate a 50:50 US: non-US ratio.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

Figure 2.1 Contribution of United States and Other Nations to Immunology Papers in 1981-1996

Source: Calculated from data obtained from Institute for ScientificInformation on high-impact papers in immunology.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

Figure 2.2 Percentage of World's Papers in Immunolgy from 1981-1996, by Country

Source: Calculated from data obtained from the Institute for ScientificInformation database on High Impact papers in Immunology.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

An advantage of our approach is that it incorporates the opinions of a variety of respected members of the immunology community: up-and-coming leaders as well as established ones, investigators from all over the world, and leaders of all sub-subfields, both basic and clinical. The disadvantages are multiple: variable modes of polling, a variable sample size that was usually too small to allow statistical treatment of the data, lack of objective criteria, and nonproportionate sampling of researchers in countries outside the United States.

2.1.2 Citation Analysis

To identify the most frequently cited authors of immunologic research articles, a “high-impact” immunology database was commissioned from the Institute for Scientific Information (ISI). (See http://www.isnet.com/products/rsg/impact.html for more information on “high-impact” paper methodology.) The ISI database was scanned over the years 1990-1997. For each year, the 200 most-cited papers in journals relevant to the field of immunology were listed. The authors having more than five papers on the list were ranked according to average number of citations per paper (174 authors ranging from 70.2 to 638.5/paper), and the country of the laboratory of each was listed. In addition, panel staff determined whether each author was also cited on the virtual-congress lists.

The contribution of the United States and other nations to immunology citations from 1981-1997 is shown in Figure 2.3. Of the 174 authors identified in the scan of the ISI database, 72% including the top 112 authors identified were in US-based laboratories. The ISI database indicates that US laboratories produced 63% of the papers, which garnered 66% of the citations, in journals relevant to immunology during the period 1981-1997.

The major strengths of this mode of analysis are its relative objectivity and its providing a basis of comparison with the virtual-congress polling data. However, the analysis suffers from some flaws related to the organization of the ISI database. Data from several of the top-level general journals (such as Cell) and immunology journals (such as Journal of Experimental Medicine) were not included. The panel found that the most serious limitation caused by interpreting the results obtained with this type of database is that the cited-paper analysis lists only the first author of each paper. The traditional author order in immunology puts the principal investigator last, so a graduate student or technician could have a better chance of appearing on the list of the most frequently cited papers than a principal investigator. The manner of referencing also males it difficult to know which country the particular research was done in. In addition, the list of authors is truncated after the first 15 names, possibly excluding authors who participated in large clinical trials. An additional limitation imposed by the database was that there was no breakdown according to subfield and sub-subfield of immunology.

2.1.3 Journal Publication Analysis

The panel identified four leading general journals (Science, Cell, Nature, and Blood) and one of the top journals focused specifically on immunology (Immunity). Panel members scanned the tables of contents of each of the journals for 1995-1997, identifying immunology papers in the general journals and the laboratory nationality of the principal investigator and the subfield in all the journals. In addition, a small sample from the Journal of Experimental Medicine was analyzed.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

Figure 2.3 Contribution of United States and Other Nations to Immunology Citations in 1981-1997

Source: Calculated from data obtained from the Institute for ScientificInformation database on High Impact papers in Immunology.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

Additional journals–such as Nature Medicine, Lancet, Journal of Clinical Investigation, and New England Journal of Medicine–could have been analyzed as well but such an analysis was beyond the resources of this panel. Others–such as the European Journal of Immunology, Journal of Immunology (US), International Immunology (Japan), and Immunology (UK)–reflect mainly the country of research origin and so are not appropriate here.

This approach appears to be procedurally the least biased of the three. It does suffer from the usual weaknesses of any analysis based on publication data, including the dominance of English as the language of international discourse and the vagaries of the peer-review journal-acceptance process.

2.2 Results

Although the three criteria for evaluating US research efforts in immunology were quite distinct and had different strengths and flaws they led to basically the same conclusion: immunology research in the United States is pre-eminent in the world. The data supporting that conclusion are summarized below and in the following tables.

2.2.1 Reputation Survey

The data-collection measures for the different sub-subfields proved highly variable, including the number of pollees (3-17), the fraction of US-based pollees (41% - 83%), and the number of names cited per pollee (a few to more than 20). In addition, the database for the individual sub-subfields was far too small to permit any kind of statistical analysis. Those points are partially illustrated by some of the entries in Table 2.1. The panel decided to emphasize the data pooled by subfields in which there was generally much less variation in the data-collection measures, although attention was still paid to findings in the context of the sub-subfield groupings for any nuances of interest.

In all four subfields, a clear majority of the names cited were investigators directing US laboratories 60 - 70% in all cases. However, the position of leadership was not so evident in some domains, for example:

  • In cellular immunology, the sub-subfields of lymphocyte development and self-nonself recognition.

  • In molecular immunology, the sub-subfield of NK receptors.

  • In immunogenetics, the sub-subfield of inherited immunodeficiency.

  • In clinical aspects of immunology, the sub-subfields of tumor immunology and transplantation and immunosuppressive drugs.

In those domains, the proportion of US-based investigators was closer to 50%. The statistical significance of the differences could not be tested, but in general they correspond well to the collective opinion of the panel numbers.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

Table 2.1 Immunology International Reputation Survey Results

 

Pollees

Virtual-Congress Speakers

Subfield

Sub-subfield

No. of Polles

No. U.S.

% US

# Non-US

No. Non-US

No.

No. US

US

No. Non-US

% Non-US

Cellular immunology and immunobiology

1. Lymphocyte development

17

7

41.2

10

58.8

189

98

51.9%

89

47.1%

 

2. Cellular interactions

10

6

60

4

40

62

46

74.2%

16

25.8%

 

3. Self-nonself recognition

11

6

54.5

5

45.5

122

59

48.4%

62

50.8%

 

4. Cellular traffic adhesion

4

3

75

1

25

75

47

62.7%

28

37.3%

 

5. Host-parasite interactions

x

x

x

x

x

x

x

X

x

X

 

6. Innate immunity

5

3

60

2

40

67

49

73.1%

18

26.9.%

 

7. Mucosal immunity

4

3

75

1

25

99

70

70.7%

29

29.3%

Molecular aspects of Immunology

1. B- and T-cell genes

8

3

37.5

5

62.5

113

65

57.5%

47

41.6%

 

2. NK receptors

11

6

54.5

5

45.5

124

76

61.3%

48

38.7%

 

3. Receptors and tolerance

8

6

75

2

25

71

51

71.8%

20

28.2%

 

4. Transcription

8

4

50

4

50

88

72

81.8%

16

18.2%

 

5. Antigen uptake

5

4

80

1

20

79

46

58.2%

32

40.5%

 

6. Structural analysis

7

5

71.4

2

28.6

34

26

76.5%

8

23.5%

 

7. Apoptosis

4

3

75

1

25

28

22

78.6%

6

21.4%

Immunogenetics

1. Evolution of MHC

x

x

x

x

x

x

x

x

x

x

 

2. Autoimmune disease

4

2

50

2

50

63

41

65.1%

22

34.9%

 

3. Inherited immunodeficiency

10

6

60

4

40

145

74

51.0%

71

49.0%

 

4. Neoplastic immune disorders

6

5

83.3

1

16.7

61

41

67.2%

20

32.8%

Clinical aspects of immunology

1. Immunopathologies and immune interventions

13

8

61.5

5

38.5

258

157

60.9%

99

38.4%

 

2. Tum or immunology

4

2

50

2

50

34

26

76.5%

8

23.5%

 

3. Transplantation and immunosuppressive drugs

13

7

53.8

6

46.2

189

108

57.1%

81

42.9%

Source: Nonrandom polling data obtained by panel members interviewing equal numbers of US and non-US immunologists whose research the panel members felt had made significant contributions to specific subfields of immunology. The chosen immunologists (pollees) were instructed to select other prominent immunologists in their subfields that they would invite to talk at a virtual immunology conference. Invited researchers are referred to here as virtual congress speakers.

Both the polles and virtual congress speakers remain anonymous and are reported as frequencies and percentages of US and non-US researchers in table.

Percentage might not add to 100%, because of rounding errors or missing data.

X= data insufficient.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
2.2.2 Citation Analysis

The results of the panel's citation analyses are shown in Figure 2.4 and Table 2.2. As shown in Figure 2.4, the top 3 countries for the immunology field based on percentage of the world's citations from 1981-1997 in immunology are

  1. United States

  2. England

  3. Switzerland

Another measure that can be used is relative citation impact (RCI). RCI is the country's share of the world's citations in the field, divided by its share of world publications in the field. It can be thought of as a comparison of a country's citation rate for a particular field with the world 's citation rate for the field. A relative citation impact greater than 1 shows that the country's rate for the field is higher than the world's. Some believe RCI is a measure of both the influence and the visibility of a country's research (as disseminated through publications) and it gives some indication of the quality of the average paper. As shown in Table 2.2, the top 3 countries based on relative citation index for 1981-1997 are

  1. United States

  2. Belgium

  3. Australia

A bit disconcertingly, only 54% of the authors identified in the ISI Immunology the high impact citations were also cited in the survey conducted by the panel. The discrepancy can probably be attributed to flaws in the two approaches, in particular such defects in the ISI database as the exclusion of critical journals and the use of only the first author in the citations.

2.2.3 Journal Publication Analysis

The results of the journal publication analysis are shown in Table 2.3-Table 2.7. US-based investigators produced about three-fourths of the immunology papers published in the journals Cell, Science, and Immunity in 1995-1997. US-based researchers contributed about two-thirds of the immunology papers published in Nature in 1995-1997. In contrast, scientists in countries other than the United States published more immunology articles in Blood than US-based researchers in those years (54% and 46%, respectively). The immunological subfield in which a international presence was especially strong was immunogenetics for the papers published in Cell and Blood. However, in Nature, Science, and Immunity, US-based researchers published more papers in immunogenetics than non-US-based researchers. The United States had more papers published in all five journals in the subfield of molecular immunology. The subfield of cellular immunology and clinical aspects were dominated by US-based researchers in all journals examined except Blood. Because of its extensiveness, only 6 months of the Journal of Experimental Medicine was analyzed, as shown in Table 2.8.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

2.3 Summary

According to all three evaluation methods, US-based research in immunology plays a dominant role in the worldwide effort. Within the limitations of each measure and the limits of the panel's use of them, the data produce a strikingly consistent outcome: all three approaches assigned a 2:1 to 3:1 dominance vis-a-vis the rest of the world. Of course the dominance needs to be weighed in relation to the relative richness of the United States in numbers of investigators, institutions and resources.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

Table 2.2 Relative Citation Impact of High Impact Papers in Immunology, by Country, 1981-1997

Country

No. Citations

Fraction of World's Citations

No. Papers

Fraction of World's Papers

RCIa

UNITED STATES

632,326

66.03

2,360

63.39

1.04

BELGIUM

11,626

1.21

44

1.18

1.03

AUSTRALIA

16,701

1.74

65

1.75

1.00

SWITZERLAND

38,487

4.02

153

4.11

0.98

ENGLAND

69,645

7.27

278

7.47

0.97

JAPAN

31,689

3.31

127

3.41

0.97

FRANCE

33,877

3.54

139

3.73

0.95

SWEDEN

13,865

1.45

57

1.53

0.95

CANADA

23,329

2.44

101

2.71

0.90

ISRAEL

6,098

0.64

27

0.73

0.88

NETHERLANDS

21,378

2.23

95

2.55

0.87

DENMARK

3,910

0.41

18

0.48

0.84

GERMANYb

34,927

3.65

164

4.41

0.83

ITALY

15,746

1.64

75

2.01

0.82

SCOTLAND

4,009

0.42

20

0.54

0.78

Totals

957,613

--

3,723

--

--

Source: Calculated from data obtained from the Institute for ScientificInformation database on High Impact papers in Immunology.

Notes: a RCI is the relative citation impact, a country's percentage of world citations divided by its percentage of world papers. Threshold set at minimum of 17 papers.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

Figure 2.4 Percentage of World's Citations in Immunology in 1981-1997, by Country

Source: Calculated from data obtained from the Institute for ScientificInformation database on High Impact Papers in Immunology.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

Tables 2.3 Authorship of Immunology Papers in Blood, 1995-1997

Subfield

No.(%) US

No.(%) Non US

Total

Cellular immunology

58(46.4)

67(53.6)

125

Molecular immunology

47(54.7)

39(45.3)

86

Immunogenetics

10(37.0)

17(63.0)

27

Clinical aspects

158(44.5)

197(55.5)

355

TOTALS

273(46.0)

320(54.0)

593

Source: Original analysis conducted for this report. Panel membersreviewed tables of contents and decided subfields of immunology ofeach paper.

Tables 2.4 Authorship of Immunology Papers in Cell, 1995-1997

Subfield

No.(%) U.S.

No.(%) Non U.S.

Total

Cellular immunology

14(60.9)

9(39.1)

23

Molecular immunology

45(75.0)

15(25.0)

60

Immunogenetics

0(0)

1(100)

1

Clinical aspects

12(75.0)

4(25.0)

16

TOTALS

71(71.0)

29(29.0)

100

Source: Original analysis conducted for this report. Panel membersreviewed tables of contents and decided subfields of immunology ofeach paper.

Table 2.5 Authorship of Immunology Papers in Immunity, 1995-1997

Subfield

No.(%) U.S.

No.(%) Non U.S.

Total

Cellular immunology

113(72.9)

42(27.1)

155

Molecular immunology

145(78.3)

40(21.6)

185

Immunogenetics

32(72.7)

12(27.2)

44

Clinical aspects

26(81.2)

6(18.7)

32

TOTALS

316(76.0)

100(24.0)

416

Source: Original analysis conducted for this report. Panel membersreviewed tables of contents and decided subfields of immunology ofeach paper.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×

Table 2.6 Authorship of Immunology Papers in Nature, 1995-1997

Subfielda

No.(%) U.S.

No.(%) Non U.S.

Total

Cellular immunology

37(68.5)

17(31.5)

54

Molecular immunology

48(61.5)

30(38.5)

78

Immunogenetics

6(66.7)

3(33.3)

9

Clinical aspects

9(64.3)

5(35.7)

14

TOTALS

100(64.5)

55(35.5)

155

Source: Original analysis conducted for this report. Panel membersreviewed tables of contents and decided subfields of immunology ofeach paper.

Table 2.7 Authorship of Immunology Papers in Science, 1995-1997

Subfielda

No.(%) U.S.

No.(%) Non U.S.

Total

Cellular immunology

44(67.7)

21(32.3)

65

Molecular immunology

68(82.9)

14(17.1)

82

Immunogenetics

6(60.0)

4(40.0)

10

Clinical aspects

46(66.7)

23(33.3)

69

TOTALS

164(72.6)

62(27.4)

226

Source: Original analysis conducted for this report. Panel membersreviewed tables of contents and decided subfields of immunology ofeach paper.

Table 2.8 Authorship of Immunology Papers in the Journal of Experimental Medicine, February 1996-July 1996

Subfielda

No.(%) U.S.

No.(%) Non U.S.

Total

Cellular immunology

53(58.9)

37(41.1)

90

Molecular immunology

61(77.2)

18(22.8)

79

Immunogenetics

13(86.7)

2(13.3)

15

Clinical aspects

37(64.9)

20(35.1)

57

TOTALS

164(68.0)

77(32.0)

241

Source: Original analysis conducted for this report. Panel membersreviewed tables of contents and decided subfields of immunology ofeach paper.

Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
This page in the original is blank.
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 21
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 22
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 23
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 24
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 25
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 26
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 27
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 28
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 29
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 30
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 31
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 32
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 33
Suggested Citation:"2. Benchmarking Results." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 1999. International Benchmarking of US Immunology Research. Washington, DC: The National Academies Press. doi: 10.17226/9444.
×
Page 34
Next: 3. Key Factors »
International Benchmarking of US Immunology Research Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!