National Academies Press: OpenBook

Assessing the Value of Research in the Chemical Sciences (1998)

Chapter: 7 Research as a Critical Component of the Undergraduate Educational Experience

« Previous: 6 Patents and Publicly Funded Research
Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×

7
Research as a Critical Component of the Undergraduate Educational Experience

K. Barbara Schowen

University of Kansas

Introduction

I would like to consider the theme of this paper—the undergraduate research experience—in terms of four separate considerations: Does it make a difference? For whom? How do we know? What is the evidence of the last 10 years? I will begin with the answers to the first two questions. Yes, undergraduate research does make a difference—it makes a difference for students, for undergraduate curricula, for chemical science both as an academic discipline and as a profession and for the nation and society as a whole. But how do we know this? This question brings us to the main focus of this workshop—namely, assessment. I see my role in this forum as addressing not assessment of the value of research per se, but rather assessment of the value of the (undergraduate) research experience. Here, I will attempt to show how the first two questions posed have been answered (or can be answered), and by what means—namely, by a variety of assessment techniques applied during the past 10 years. These include data from the records of undergraduate research—site administrators in academia, survey responses, personal histories, and anecdotes, as well as the general impressions that most people involved with undergraduate education have formed over the years.

However, first of all, you may ask: "Why 10 years?" Suffice it to say that I chose that time frame because the National Science Foundation's REU (Research Experiences for Undergraduates) program got off the ground in 1986 with the first sites in place by 1987—hence this program, which specifically focuses on undergraduate research experiences, has been in operation for 10 years (see Box 7.1 for a short history). Now I do recognize that many undergraduates participate in research without REU funding. I picked REU because it represents a national recognition of the importance of undergraduate research and a commitment to provide research experience for undergraduates. In this sense, it marks a watershed or point of reference—and it started 10 years ago.

Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×

BOX 7.1 History Leading Up to the Research Experiences for Undergraduates Program

A bit of history may put the Research Experiences for Undergraduates(REU) program in perspective. After Sputnik made Americans focus again on what it might take to maintain the technological and scientific leadership enjoyed during and immediately after World War II, many projects were initiated to strengthen the quality of science education At the same time, the National Science Foundation established a program that would enable undergraduates to enhance their science education in the major. This was the URP or Undergraduate Research Program, which provided funding for students to spend a summer in active research laboratories, often away from the home campus. The locations where students went—usually at research universities—were termed project sites Typically to students were supported at each project site for 10 weeks. This program was very active throughout the 1960s and 1970s. For example, I remember that during the late 1702s when I administered our project in the University of Kansas chemistry department we also had projects in the biochemistry department, the chemical engineering department, and the medicinal chemistry department just on our one campus. It was highly organized at the NSF, with uniform application deadlines and starting and ending dates across the country it was generally thought that the program was a worthwhile way to introduce young scientists to the kind of practical experience that might allow them to make a rational career decision. Nevertheless, this program was one of the first casualties after the national elections of 1980.

By the mid-1990s them was much alarm at the decrease in the numbers of young people going into science careers. For example, data gathered by the American Chemical Society showed a 25 percent decline in graduation of B.S.-level chemists during the early 1980s, and the National Science Foundation was given the so-ahead from Congress to come up with plans to help counteract this trend. What emerged was a new program. Research Experiences for Undergraduates, or REU. Again, stipends were provided to students so that they could enjoy full-time commitment to research during the summer months at research sites. There were some noticeable differences from the older URP program. Sites were considerably more autonomous and given scope to tailor their programs to suit their calendar and likely participants. Deadline dates varied Some programs ran for 6 weeks, some for 10. Some programs focused on one area of chemistry only—for example, materials science or nuclear chemistry; some on particular target groups, such as women or Native Americans. In all cases, underrepresented minorities and women were clearly encouraged to apply and often actively recruited.

Undergraduate Research Does Make a Difference

Does undergraduate research make a difference? To whom does it make a difference? From what point of view? I will look at this question successively from the point of view of the discipline, the profession, the nation, and the student. Let us begin with the discipline of chemistry.

If I had asked, for example, "Does organic chemistry (in the undergraduate curriculum) make a difference, and what is the evidence for that in the last 100 years?," you would, of course, think the question ludicrous and the answer obvious. The answer, I think, is equally obvious with respect to undergraduate research. Chemistry as an undergraduate discipline clearly has two components—one of content and one of method. It is a body of knowledge (complete with concepts, symbols, and facts)

Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×

traditionally learned through the classroom lecture. It is also a collection of techniques (complete with observation, manipulation, data collection, and interpretation) traditionally learned in the laboratory. But chemistry is also about discover)' and application—a dynamic synthesis of content and methodology to find out something, or make something, or do something new. This, of course, is what chemistry is all about, and what we call research.

So one way research experience might be said to "make a difference" is that it enhances, completes, or rounds out a science major's education. It helps develop an understanding of the complexity and context of science, not merely the content and methods. It is a way to validate the previous years of learning (analogous to the years of French grammar, vocabulary, dictation, and the reading of literature that can lead to communicating with others in French and writing one's own poem) and of contributing at last, in however small a way, to a body of knowledge that we call chemistry. What we have concluded here, then, is something I really believe should apply to all disciplines—but above all to those in the natural sciences: Research is a necessary component of a bachelor's degree education.

This is not a new idea—at least not in chemistry. Many undergraduate chemistry programs require undergraduate research; many others strongly recommend it and help make the opportunity available. This was also the conclusion arrived at by a group of NSF-REU chemistry site directors at a workshop held here in Washington in 1990, namely:

Chemistry is a dynamic experimental science for which research is an inherent component. Such a discipline requires "learning by doing," an inquiry approach, and an apprenticeship experience. A student's education in chemistry is incomplete without research experience.

Similar statements appear in a report issued by an analogous biological sciences site directors workshop held in 1993:

Research not only serves as an important adjunct to the traditional classroom, but is absolutely essential in modem, quality, undergraduate science education.

They also are found in the American Chemical Society's Committee on Professional Training guidelines for professional certification, and from the Council of Undergraduate Research.

Conclusion 1. Undergraduate research experience makes a difference in the discipline of chemistry because it belongs in the curriculum, just as a lecture course in organic chemistry or a laboratory course in methods of analysis does.

Of course, the finding of Conclusion 1 is not what the general public, or even many chemists, often mean when contemplating the "does it make a difference?" question. Instead, what they are thinking is: Does exposure to undergraduate research end up producing more and better trained chemists who will contribute to the economic health of the nation and the quality of life for its citizens? This, I am certain, is what Congress had in mind when it reenacted funding for undergraduate research experience in the mid-1980s. Here the questions are: Are we capturing people—that is, are more people majoring in chemistry? And are we retaining them so that they go on to work as chemists, or go on to graduate school, and then opt for research careers?

Let us look at one source of evidence relating to how undergraduate research bears on these issues. This information comes from records maintained for the past 10 years of undergraduate research and REU-site activity in chemistry at the University of Kansas. We have had the good fortune to be funded as a Chemistry NSF-REU site for the past 10 summers, and each year we bring in 10 to 12 students from other colleges who join about an equal number of our own undergraduates for 10 weeks of immersion in intensive and genuine research, including formal presentation of results.

Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×

TABLE 7.1 Postbaccalaureate Activity of Participants in the Chemistry Summer Undergraduate Research Experience at the University of Kansas from 1989 to 1997

Activity

1988-1990

1991-1993

1994-1995

1996-1997

Graduate school (chemistry/biochemistry)

40 (65%)

52 (88%)

31 (70%)

162 (75%)

Medical school

15 (24%)

2 (3%)

8 (18%)

32 (15%)

Industry (B.S. degree)

4 (6%)

1 (3%)

2 (5%)

9 (4%)

Undecided/unknown/other

3 (5%)

4 (7%)

3(7%)

13 (6%)

Total

62 (100%)

59(100%)

44 (100%)

216 (100%)

NOTE: Data based on stated plans of college seniors, plus longitudinal tracking of information. National Science Foundation's Research Experiences for Undergraduates (NSF-REU) program plus University of Kansas undergraduates.

Polling these students upon leaving the program and tracking them for subsequent years has generated the information summarized in Table 7.1. It is clear that a very significant number (at least 75 percent of the 216 students who have participated in our research programs since 1988) go on to advanced study in the chemical sciences, and another 4 percent go into chemistry-related jobs. If we compare these students with our majors in general, including those who do not participate in research, the sum of those two numbers is noticeably lower: closer to 50 percent. Similar results are reported at other schools—for example, by Professor John Hogg at Texas A&M: ''I estimate that up to 75 percent of students who participate in our NSF-REU program attend graduate school"—and at meetings and conferences.

Conclusion 2. Undergraduate research experience makes a difference by increasing the numbers of individuals who choose careers in chemistry or who go to graduate school in the chemical sciences.

In addition to societal benefits accruing from a larger pool of talented people entering the chemical professions, we should also consider the various indirect benefits to society that might accrue from people educated in curricula involving research experience. For example, are more informed scientists and nonscientist citizens emerging as public school teachers, or as policy makers, communicators, journalists, science advocates and lobbyists, elected officials, critical thinkers, problem solvers, and simply citizens who are science literate? Although they certainly can be obtained, data bearing on these considerations are not as easy to come by. I have no special information in this regard to discuss today, as important as it might be for assessing the value of the undergraduate research experience.

Not unrelated to the concept of indirect benefits to society is the question of the inclusion of all of our citizenry. Does undergraduate research experience have an effect on the numbers of underrepresented people who become productive scientists or who join the ranks of the scientifically literate in this country? In other words, what about the impact of undergraduate research on minorities and women? My own data on the minority question are very limited, so I will give examples only as they relate to women. Our 10-year REU data show a real increase in the percentage of women applicants, such that it is now about 50 percent, and an increase in the percentage of women participants. Since research

Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×

experience has been shown to have a positive influence on later graduate school and career choices (influence that seems not to be particularly gender specific), it follows that more women doing undergraduate research will lead to more women doing science later. Since the numbers are relatively small each year (10 to 12 students), there is a fair amount of scatter. Nevertheless, the trend is clear.

Conclusion 3. The last decade has shown a clear increase in the numbers of women participating in undergraduate research and attending graduate school.

Also, one might ask, has undergraduate research made a difference from the perspective of the student? For example, has it increased his or her understanding and appreciation of the major and of the discipline of chemistry? Has it helped in making career decisions? Has it helped in the decision to go to graduate school? Has it helped define an area of interest for future study or work? And, if a student did go to graduate school, has it made adjustment easier? Did it increase retention?

In 1995, I conducted a survey of current chemistry graduate students with the help of individuals from two other institutions, John Hogg at Texas A&M University and Dale Hawley of Kansas State University. A number of questions relating to those just posed were on the survey. There were 129 respondents, with 110 (85 percent) of those having engaged in research during the undergraduate years. There were 41 respondents from the University of Kansas (about 45 percent of the graduate students), 20 from Kansas State, and 70 from Texas A&M (about 25 percent of the graduate students). Details of the characteristics of those completing the survey forms are given in Box 7.2.

BOX 7. 2 Characteristics of Students Completing the Three-School Survey

The students completing the three-school survey (the University of Kansas, Kansas State University, and Texas A&M University) were 61 percent male, 39 percent female, 15 percent international, and about 5 percent members of recognized U.S. ethnic minority groups. All of them had been chemistry majors as undergraduates, and 74 percent had earned B.S. degrees Slightly more were graduates of 4 year colleges (46 percent) than Ph.D. granting universities (39 percent). Of these students, 85 percent had engaged in research during their undergraduate years, and 12 percent had done so for 2 years of more.

Of those who had participated in undergraduate research, 95 percent had done so in the field of chemistry, 45 percent had worked in two or more different research groups, and 32 percent had been part of formal summer internship programs. About 23 percent had had their research experience only in the summer months, while almost 50 percent had done it mainly during the academic year at the home institution. More than half (58 percent) reported having received compensation for their work at one time or another. Most of the work was “wet" laboratory chemistry (87 percent) as opposed to computational or theoretical chemistry. Fifty-eight percent of the students had had at least one off-campus experience at a college or university, with an-approximately equal distribution among the two types of institutions.

Of the respondents as a whole, 93.5 percent were planning careers in chemical science: 76 percent in research (20 percent academic, 40 percent industrial), and 13 percent in teaching with no significant research component. The great majority (84 percent) were going on for the Ph.D. degree, with 16 percent planning to seek a postdoctoral appointment thereafter. The remaining students were divided between staying for the master’s degree only (5 percent) or transferring out of chemistry altogether (3 percent). (Not all students responded to all questions.)

Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×

Based on data obtained in the survey, where the data refer to those students who had participated in undergraduate research, the following can be observed:

  • Less than 2 percent reported an unsatisfactory undergraduate research experience. We have no data as to numbers of students with such negative experience who did not go on to graduate school, but the data indicate that of the 85 percent of students in graduate school in our survey who had prior research experience, at least 97 percent had had a positive exposure.
  •  For 77 percent of the students, undergraduate research was a contributing factor in the decision to go to graduate school. For 44 percent, it was the major factor.
  •  Some 60 percent found that research experience helped in deciding which graduate school(s) to apply to.
  •  An undergraduate research experience was seen as helpful in adjusting to graduate school (56 percent) and in getting started with graduate research (52 percent).
  •  Almost 90 percent considered the experience useful in helping them decide on a general area of research to pursue in graduate study.
  •  About two-thirds (67 percent) believed that prior research exposure helped them to "stay the course" and remain in graduate school.

Interestingly, the data showed no noticeable gender distinctions; that is, the responses from males and females were not significantly different. (The only instance where a difference was noticed was in the question about postdoctoral plans.)

Conclusion 4. Undergraduate research experience makes a difference for both men and women chemistry students by favorably disposing them toward graduate school, helping them select a school and area of research, better preparing them for graduate education, and influencing them to remain and complete a graduate degree.

We should also be concerned with questions having to do with other, more intangible benefits to the student resulting from undergraduate research experience. Such intangibles may relate to the establishment of mentor and peer relationships and of networks of advisors, co-workers, and friends. They may equally relate to the personal growth and increased self-confidence developing from an experience of common endeavor, and perhaps of real accomplishment and productivity. They surely should relate to the enhanced personal aspirations and broadened horizons that we believe may come from exposure to scholarly work, professionalism, dedicated scientists and motivated peers, and the joys and trials of discovery.

In this case, all the data we have are fairly anecdotal, coming from conversations or other communication with former students or their research advisors. I will illustrate by describing two individuals. Following his sophomore year at the University of Northern Iowa, the first student came as a 1988 summer REU participant to our department. He chose to work on an organic synthesis problem and, as he put it, "may have gotten a few drops of product." Nevertheless, he got a lot out of the program and still maintains close contact with most of the other participants who were with him back in 1988. After selecting a graduate school, because he remembered being captivated by one of our Friday seminar talks given by one of our faculty that described gas-phase mechanistic studies, the student made the important decision to go into gas-phase physical organic chemistry. He has just completed a postdoctoral stint, has developed methods that directly observe transition states, and has just started his academic career as an assistant professor at a major research university. This is an example of the profound influence of undergraduate research on a career path.

Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×

The second student is currently a junior at the University of Kansas and is a chemical engineering major, with an interest in biomedical engineering and a desire to attend medical school. She was a student in my organic chemistry class last year—the very top student out of a class that began with 450 others. She had never done research. Last summer, partly with my assistance, she located an internship working on a biochemistry project at the University of Cincinnati Medical School and returned, glowing with enthusiasm: "I know I want to do research." The director of the program there wrote me the following on July 31, 1997:

[She] has turned out to be exceptional, intellectually and technically. Considering that she had no background in gene transcription and had never held a pipette, [her summer research advisor] is amazed. On Thursday, she will present the results of her research at a minisymposium.

For both of these people, there can be little doubt as to the value of the undergraduate research experience. Further evidence comes from written comments provided by the students who completed the three-school survey described above, just a few of which I will take the time to read now.

I participated in research at a major pharmaceutical firm, at a different university for a summer, and worked in a lab at my university for 3 years. I consider the experience gained to be the most important part of my undergraduate experience.

Even though I did some research at my small college prior to doing research at a big university, I had no idea what "real" research was like. The research work at the big university influenced more than 95 percent of my decision.

My undergraduate research helped to integrate me with the chemistry faculty and resulted in 2 full papers which have since appeared in peer-reviewed journals.

(You will note that I have not addressed research output or productivity, although we seem to average about 0.5 papers per student in the program.) Another comment from a student:

I must stress the importance of the research experience in making students an integral part of the department and converting them from bench warmers in class to productive, thinking scientists.

A research experience is critical to the decision process of selecting a division of study, and helping to decide whether a final goal of academic or industrial [work], teaching, or research is desired.

Conclusion 5. Personal testimony, informal communication, written comments and responses to surveys, and the experience of mentors and advisors all indicate that undergraduate research experience makes a positive difference in the education and the professional and personal development of its participants.

Summary

A variety of assessment methods, some more quantitative than others, can be applied to the question of determining the value of the undergraduate research experience. Results obtained and discussed today indicate that such experience provides an essential component of science education, contributes to more informed career decisions, generates more students choosing careers in chemistry, promotes success in graduate programs and probably future work in chemistry, and results in increased personal confidence and a better appreciation for and understanding of science. Furthermore, it contributes to the nation's research enterprise by playing an important role in producing more, better prepared, and better qualified scientists for the future.

Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×

Discussion

Beverly K. Hartline, Office of Science and Technology Policy: I definitely share the enthusiasm for the value of undergraduate research experiences. But I am very curious to know if there are any data on the performance of these students in courses subsequent to having had that experience as compared with students who have not had the experience (or even to their own previous performance). Is information available on their actual performance in graduate school? In industry, if they take an industrial job? That is, are they really better prepared? Or has their intuition been improved as a result of the undergraduate research experience.

K. Barbara Schowen: Those are very important questions. I personally don't have any data on how students subsequently perform in industry, and so I can't address that point. Do they do better in their returning courses? I also have no hard data here. However, they return from the experience very enthusiastic about their discipline and are certainly going to be paying attention to their courses. But I can't answer the question, as to whether they earn more "A"s than they did before. I think it is possible to obtain these data. In fact, one of the points of this workshop is to determine what metrics should be used in assessing the value of the undergraduate research experience and how to obtain the needed data.

Michael P. Doyle, Research Corporation: Before 1982, the Associated Colleges of the Midwest (which included Carleton, Grinnell, St. Olaf, and Macalester) carried out a similar survey of their undergraduate research participants, and about 650 people responded. Curiously, the same number, about 70 percent, said it was an important determinant of their future career and their success in that career. This survey focused on people who had been out of the program for 5 or more years, and so they had been through school. The data seem to confirm what we have known for a long time.

In your particular case, participation in the Research Experiences for Undergraduates program has a special requirement—namely, that the students have already professed an interest in chemistry, are entering a chemistry program, and are predisposed to go on to graduate school. So, in fact, aren't you just measuring what you expect to find?

K. Barbara Schowen: That is a hard question to answer. The control experiment would be to poll all the people who had not participated in research and find out what percentage of them went to graduate school.

Raymond E. Fornes, North Carolina State University: Many of you are aware that there is a national conference on undergraduate research each year. If you look at the data on students who have participated in that conference (and I think there are on the order of 2,000 papers presented at the national meeting each year), you find that there is a much higher percentage coming from the small colleges—the Carletons, for example—than from the major research universities.

If I understood you correctly, in the REU program, you got five students per year from Kansas and five from other colleges.

K. Barbara Schowen: No, we had 10 to 12 from colleges other than our own. We have about 25 total in the program.

Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×

Raymond E. Fornes: But still, that is on the order of 10 percent of your undergraduate student population in chemistry. So the question is, how do you foster greater participation in research experiences, particularly in the large universities, where such a high percentage of students are in the sciences?

K. Barbara Schowen: That is one thing that I have spent a lot of time doing as an undergraduate advisor in our department. We strongly encourage our students to participate in research, and our department tries to make this happen. Professor Daryle Busch, who is also at the workshop, has a large number of undergraduates working in his group.

From the data that came from the three-school survey, about 45 percent of the students were from large public institutions, and the others were from small 4-year colleges. I believe that the information gathered from the two groups was similar.

Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×
Page 73
Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×
Page 74
Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×
Page 75
Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×
Page 76
Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×
Page 77
Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×
Page 78
Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×
Page 79
Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×
Page 80
Suggested Citation:"7 Research as a Critical Component of the Undergraduate Educational Experience." National Research Council. 1998. Assessing the Value of Research in the Chemical Sciences. Washington, DC: The National Academies Press. doi: 10.17226/6200.
×
Page 81
Next: 8 Scholarly Research: Oxymoron, Redundancy, or Necessity? »
Assessing the Value of Research in the Chemical Sciences Get This Book
×
Buy Paperback | $48.00 Buy Ebook | $38.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This book captures the messages from a workshop that brought together research managers from government, industry, and academia to review and discuss the mechanisms that have been proposed or used to assess the value of chemical research.

The workshop focused on the assessment procedures that have been or will be established within the various organizations that carry out or fund research activities, with particular attention to the Government Performance and Results Act (GPRA). The book presents approaches and ideas from leaders in each area that were intended to identify new and useful ways of assessing the value and potential impact of research activities.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!