National Academies Press: OpenBook
« Previous: Executive Summary
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 6
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 7
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 8
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 9
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 10
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 11
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 12
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 13
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 14
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 15
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 16
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 17
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 18
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 19
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 20
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 21
Suggested Citation:"1 Overview." National Research Council. 2007. Approaches for Evaluating the NRC Resident Research Associateship Program at NIST. Washington, DC: The National Academies Press. doi: 10.17226/12020.
×
Page 22

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1 Overview In Rising Above the Gathering Storm, the National Academies posited that “The prosperity the United States enjoys today is due in no small part to investments the nation has made in research and development at universities, corporations, and national laboratories over the last 50 years. Recently, however, corporate, government, and national scientific and technical leaders have expressed concern that pressures on the science and technology enterprise could seriously erode this past success and jeopardize future U.S. prosperity”(NAS/NAE/IOM, 2007). To address these challenges, the Gathering Storm report recommended action in four areas: K–12 education, higher education, science and engineering research, and economic and technology policy. Many of the report’s recommendations were later echoed in the American Competitiveness Initiative (ACI), introduced by President Bush in his State of the Union Address on January 31, 2006. The ACI was warmly received by legislators from both parties on Capitol Hill, generating bills designed to implement its key provisions. While considering these proposals, Congress took the immediate step of requiring an examination of existing federal education programs in science, technology, engineering, and mathematics (STEM) fields. As noted in a statement by Secretary Spellings on the new Academic Competitiveness Council (ACC) and its mission, “The Deficit Reduction Act, signed into law by the President on February 8, 2006, included an Academic Competitiveness Council chaired by the Secretary of Education, and consisting of members of the Federal Government whose agencies have education programs in science, technology, engineering, and mathematics. Its mission under law is to evaluate the effectiveness of each program, identifying areas of overlap and recommending ways to efficiently integrate and coordinate in the future.” The ACC’s efforts included assessing the success of these programs, identifying areas for improvement for current and future programs, and laying the groundwork for sustained collaboration among the programs (DOE, 2007). Postdoctoral programs3—of which there are several in the sciences and engineering (S&E)— are of critical importance in providing additional research training, knowledge, and opportunities to bridge higher education with the start of a career in S&E. While many Ph.D.s do go straight into full-time employment, a large number of doctorates view a few years as a postdoc to be a valuable opportunity to build skills and a reputation and a chance to experiment in the labor market. FEDERAL PROGRAMS TO SUPPORT POSTDOCS IN S&E According to the GAO (2005:3), there were 207 “education programs funded in fiscal year 2004 that were designed to increase the numbers of students and graduates pursuing STEM4 degrees and occupations or improve educational programs in STEM fields….” Included in these 3 A postdoctoral position or “postdoc” is a temporary position awarded in academia, industry, non-profits, or government primarily for gaining additional education and training in research. Individuals with postdoctoral appointments are called postdocs. 4 STEM is science, technology, engineering, and mathematics. 6

programs are several programs that provide and emphasize postdoctoral opportunities, such as: the National Aeronautics and Space Administration’s NASA Postdoctoral Program; the National Institute of Health’s Ruth L. Kirschstein National Research Service Awards (NRSA) for Individual Postdoctoral Fellows (F32), Career Transition Award (K22), Pathway to Independence (PI) Award (K99), Postdoctoral Intramural Research Training Award (IRTA), Cancer Research Training Award (CRTA), Postdoctoral Visiting Fellowship (VF), National Institute of General Medical Sciences Pharmacology Research Associate (PRAT) Program, and Women's Health Postdoctoral Fellowship; the National Science Foundation’s Postdoctoral Research Fellowships; and the U.S. Department of Agriculture’s Agricultural Research Service (ARS) Postdoctoral Research Associate Program.5 The National Research Council Fellowships Office of The National Academies manages several postdoctoral research programs on behalf of federal agencies. Table 1-1 lists several programs administered by National Research Council Fellowships Office. 5 While these programs focus on individual postdocs, other programs target academic institutions with funding to support groups of postdocs. 7

TABLE 1-1 Selected National Research Council Research Associateship Programs Agency Research Associateship Program Department of Defense Air Force Research Laboratory (AFRL) Department of Defense Armed Forces Radiobiology Research Institute (AFRRI) Food and Drug Administration/HHS Center for Devices and Radiological Health (CDRH) Defense Threat Reduction Agency/DOD Chemical and Biological Defense (CBD) Federal Aviation Administration/DOT FAA/Civil Aerospace Medical Institute (CAMI) Federal Highway Administration/DOT Federal Highway Administration/Turner-Fairbank Highway Research Center (FHWA) US Army Corps of Engineers/DOD Institute for Water Resources (IWR) Department of Energy National Energy Technology Laboratory (NETL) Department of Energy National Energy Technology Laboratory Methane Hydrates Fellowship Program (NETL/MHFP) Health & Human Services National Institute for Occupational Safety and Health (NIOSH) Department of Commerce National Institute of Standards and Technology (NIST) Health & Human Services National Institutes of Health (NIH) HHS and DOC National Institutes of Health (NIH)/National Institute of Standards and Technology (NIH/NIST) Department of Commerce National Oceanic and Atmospheric Administration (NOAA) Department of Defense Naval Medical Research Center/Naval Health Research Center (NMRC/NHRC) Department of Defense Naval Postgraduate School (NPS) Department of Defense Naval Research Laboratory (NRL) Department of Defense Navy Marine Mammal Program (MMP) Pacific Disaster Center Pacific Disaster Center (PDC) Department of Defense US Army Aviation and Missile Command (AMCOM) Department of Defense US Army Edgewood Chemical Biological Center (ECBC) Department of Defense US Army Medical Research and Materiel Command (AMRMC) Department of Defense US Army Natick Soldier Research, Development and Engineering Center- US Army Research, Development and Engineering Command (NSRDEC) Department of Defense US Army Research Laboratory (ARL) Department of Defense US Army Research Office (ARO) Department of Defense US Army Research, Development & Engineering Command, Night Vision & Electronic Sensors Directorate (NVESD) Department of Defense US Army Research, Development, and Engineering Command/Armament Research Development and Engineering Center (RDECOM/ARDEC) Environmental Protection Agency US Environmental Protection Agency (EPA) Department of the Interior US Geological Survey (USGS) US Marine Mammal Commission US Marine Mammal Commission (MMC) Department of Defense US Military Academy/US Army Research Laboratory (USMA/ARL) Note: Excluded were programs directed solely toward pre-doctoral students or faculty. Source: NRC Associateships Research Opportunity Directory. One concern, as noted by both the GAO (2005) and the Academic Competitiveness Council (ACC) (2007), was that not all programs have been evaluated, and even in cases of completed or ongoing evaluations, not much is known about the beneficial impact of individual programs.6 Such evaluation is important and can illuminate a number of impacts, including: 6 However, there have been several program evaluations of federal S&E programs and the authors of those evaluations might disagree that the evaluations have not produced information about the impact of those programs. 8

• The program’s success in fostering diversity across gender, race/ethnicity, and/or socioeconomic status; • The effectiveness of an agency’s information dissemination strategies; • Benefits to the recipients, the agency, and the nation as measured by the quality of the scholars’ experiences in the program; and • Benefits to recipients, the agency, and the nation as measured by the careers of postdocs. It was in this spirit that the National Institute of Standards and Technology (NIST) approached The National Academies to undertake a study of the NRC Resident Research Associateship Program at NIST (hereafter “NIST/NRC RAP”). THE NRC RESIDENT RESEARCH ASSOCIATESHIP PROGRAM AT NIST As noted above, the Fellowships Office of the National Research Council (NRC) of the National Academies administers a series of postdoctoral programs at several federal agencies. Collectively, these programs are known as the NRC Resident Research Associateship Programs. The Programs were established in 1954, modeled after the Rockefeller Foundation Fellowship program administered by the NRC from 1919 to 1955. NIST was the first sponsor: “The initial sponsorship of the RAP was through the National Bureau of Standards (now NIST) in 1954. NIST was joined by the Naval Research Laboratory in 1955 and other sponsors soon followed. The Research Associateship programs have continued to expand to the present day where over 30 federal agencies participate.”7 The NRC has made more than 1,000 awards in the NRC Resident Research Associateship Program at NIST. (Overall, the NRC has made 11,000 awards to postdoctoral and senior scientists and engineers to conduct research in federal laboratories.) A NRC Resident Research Associate at NIST is a researcher and a term (temporary) employee of NIST. Associateships are analogous to fellowships or similar temporary employment programs at the postdoctoral level in universities and other organizations. Awardees are offered two-year term civil service appointments. During their tenure, Associates conduct research in one of six areas of interest to NIST: chemistry, computer science, engineering, materials science, mathematics, and physics. Associates devote their full-time effort to the research program proposed in their applications and are in residence at the sponsoring laboratory at NIST during the entire period of the Associateship.8 According to the NRC, the objectives of the programs are “(1) to provide postdoctoral scientists and engineers of unusual promise and ability opportunities for research on problems, largely of their own choice that are compatible with the interests of the sponsoring laboratories and (2) to contribute thereby to the overall efforts of the federal laboratories. For recent doctoral graduates, the programs provide an opportunity for concentrated research in association with See for example: OSIRE (2003), McCullough and Thurgood (2004), Kimsey in NRC (2006a), Mantovani et al. (2006), Michie et al. (2007), and Finkelstein and Libarkin (undated). 7 This information is adapted from The National Academies’ RAP website, “Mission,” available at: http://www7.nationalacademies.org/rap/RAP_Mission_Statement.html. [Accessed June 24, 2007.] 8 However, during the expert panels held with NIST/NRC Research Associates (see description at the end of this chapter), it was noted that some Research Associates work on research projects other than what they proposed in their application packages. This fact is independent of whether such change is a positive or negative for the Research Associate. It is not clear how often this happens. 9

selected members of the permanent professional laboratory staff, often as a climax to formal career preparation. Participating laboratories receive a stimulus to their programs by the presence of bright, highly motivated, recent doctoral graduates with records of research productivity. New ideas, techniques, and approaches to problems contribute to the overall research climate of the laboratories. Indirectly, associateships also make available to the broader scientific and engineering communities the excellent and often unique research facilities that exist in federal laboratories.”9 Associates are encouraged to publish their research in refereed science and engineering journals, and many also present papers at U.S. and international conferences. The objectives for each agency participating in the Research Associateship Programs may go beyond the NRC’s goals, emphasize different aspects, or vary from the objectives of other agencies. The objectives of the NIST/NRC RAP, as stated by NIST, are: “The postdoctoral program brings research scientists and engineers of unusual promise and ability to perform advanced research related to the NIST mission, introduces the latest university research results and techniques to NIST scientific programs , strengthens mutual communication with university researchers, shares NIST unique research facilities with the U.S. scientific and engineering communities, and provides a valuable mechanism for the transfer of research results from NIST to the scientific and engineering communities.”10 Additional objectives for NIST might include attracting a sufficient number of qualified candidates, increasing the pool of potential employees, increasing the breadth or depth of research capability at NIST, or increasing the productivity of NIST,11 NIST may also be interested in whether employees who were research associates have better outcomes than employees who were not in the program and the effect of the program on NIST’s mission and reputation over the long-term. Finally, there are objectives for those who apply to the NIST/NRC RAP, including increased collaboration or networking, productivity, training, skills, knowledge, ability to work independently, or career choices.12 Many former NIST/NRC RAs have gone on to distinguished careers: “Ten high level managers who now work at NIST were past postdocs. 9 NRC, “NRC Associateships Research Opportunities Directory.” Available at: http://nrc58.nas.edu/pgasurvey/data/aobooks/rapbooks.asp?mode=frntmtr&progctr=50&seq=20. 10 Office of Academic Affairs, “Office of Academic Affairs.” Available at: http://www.nist.gov/oiaa/acdmaffr.htm. 11 During the expert panel with NIST advisors and managers, participants commented that the research associates were a way for NIST to cover a wider range of expertise. The postdoc program brings in staff to do research. It also brings in new ideas. Research Associates at NIST do some of the innovative research. The NRC notes that participating laboratories receive a stimulus to their programs by the presence of bright, highly motivated, recent doctoral graduates with records of research productivity. New ideas, techniques, and approaches to problems contribute to the overall research climate of the laboratories. Indirectly,associateships also make available to the broader scientific and engineering communities the excellent and often unique research facilities that exist in federal laboratories. 12 During the expert panel with former research associates, participants suggested benefits to the program included increased collaboration (including the range of people you work with) and networking. The postdoctoral position was seen as a helpful step to the next career choice. Participants mentioned that the program acclimated postdocs to working in the government. In some areas within NIST, the postdoctoral position seemed to the participants to be the primary way to get a full-time job there—former postdocs assumed this because in some areas almost everyone was a former research associate. During the expert panel with NIST advisors and managers, participants commented that the postdoc is a way for people to explore working at NIST—doctorates might not want to come in as a full-time regular appointment, but would be interested in trying the place out for 2 years. NIST advisors and managers also noted that the Program is a good way for NIST to “try out” postdocs who might later become employees. 10

Three members of the National Academy of Engineering and the National Academy of Sciences were past NIST postdocs. Four of the present 27 NIST Fellows are past postdocs (Curry, 2004).”13 Current application requirements include: • Research Associateships at NIST are open only to citizens of the United States; Permanent residency status is not sufficient; • Research Associateships at NIST are awarded only to persons who have held the doctorate less than five years at the time of application; • Awardees must hold the Ph.D., Sc.D., or other earned research doctoral degree recognized in U.S. academic circles as equivalent to the Ph.D. or must present acceptable evidence of having completed all the formal academic requirements for one of these degrees before tenure may begin; • Applicants must have demonstrated superior ability for creative research; and • A primary objective of the associateship programs is to provide a mechanism for new ideas and sources of stimulation to be brought to the sponsoring laboratory. Thus, persons with recent prior affiliation with a specific laboratory may not be eligible to apply for an associateship there.14 Reapplication is possible. Persons who have previously held an associateship may apply for another award only if a period of at least two years will have elapsed between termination of the first award and the proposed tenure of a second. Persons who have previously applied for an associateship, but who were not recommended for an award by the panels, may reapply after one year. Candidates who were recommended for an award by the panels, but who were not offered an award because of funding or other limitations, may reapply at any time without a mandatory waiting period. The program arose from humble beginnings: “In 1954, when the U.S. Civil Commission [now the Office of Personnel Management] granted the postdocs status as two-year term federal employees, 21 scientists applied for the first competition, and NIST made six awards, paying $5,940 each. By 2004, the number of postdocs who may be hired annually is capped at 60 (Curry, 2004).” To date there have been over 6,000 applications to the NIST/NRC RAP and over 1,000 Research Associateships have been awarded. THE ROLE OF THE NATIONAL ACADEMIES The National Research Council—via its Fellowships Office—conducts the Research Associateship Programs in cooperation with sponsoring federal laboratories and research organizations approved for participation.15 The Fellowships Office conducts a national competition to recommend and make awards (except NIST, which makes its own awards) to outstanding scientists and engineers at the postdoctoral level for tenure as guest researchers at participating laboratories. 13 See Curry (2004) for biographies of selected NIST postdocs. NIST Fellows are esteemed senior scientists. 14 Prior affiliation includes direct full-time employment relationships either with the laboratory or with a contractor whose work is performed there. A long-term consulting relationship usually makes an applicant ineligible. 15 Material in this section was adapted from the NRC RAP website, “Associateship Programs at NIST,” available at: http://qdev.boulder.nist.gov/817.03/jobs/nrc_local.htm. [Accessed June 24, 2007.] 11

The Associateship Programs office receives all application materials and supporting documents and conducts the competitive evaluation of applications. This evaluation for NIST/NRC Research Associateships (NIST/NRC RAs) is conducted by special panels convened for this purpose. Panelists are chosen to review applications on the basis of the applicants’ stature and experience in the fields of science and engineering, and their evaluation becomes the basis from which awards are offered by NIST. Applicants are recommended for awards only after this open, national competition in which the panels rank candidates on the basis of quality alone. Final ranking in order of quality and the recommendation of applicants for awards are the exclusive prerogatives of the panels. Candidates for awards are selected by NIST from the panel’s recommended list. NIST makes award offers throughout the spring and fall. THE COMMITTEE’S CHARGE The National Academies appointed an ad hoc committee to prepare this report (Appendix A). The committee’s charge: “The Academic Competitiveness Council, in furtherance of the Administration’s American Competitiveness Initiative, seeks to ensure that the nation invests wisely and effectively in educational programs to meet its science and technology goals. The ACC, therefore, requires evaluations of important STEM16 education programs, including the NIST/NRC Postdoctoral Research Associateships Program. An ad hoc committee under the auspices of the Board on Higher Education and Workforce will describe the pool of applicants for and recipients of the NIST/NRC Postdoctoral Research Associateships and carry out a descriptive analysis of career outcomes of NIST postdoctoral scholars based on available information. As possible given available data, the committee will also describe how the program addresses agency goals. The committee will also outline an approach to evaluating the program relative to national S&E goals, NIST agency goals, and the value of the program to participants, which could be undertaken in a future study.” APPROACH AND SCOPE To meet its charge, the committee focused on three objectives: • To describe characteristics of NIST/NRC RAP applicants compared to the general pool of new science and engineering doctorates; • To describe the experiences of NIST/NRC RAs compared to Research Associates in other programs; and • To offer suggestions for conducting a more in-depth assessment of the careers of NIST/NRC RAs, with a particular focus on quantifying the benefits of the appointment to the recipients as well as to NIST. The committee was guided by two principal questions: 16 STEM is science, technology, engineering, and mathematics. 12

• Is NIST attracting the “best and the brightest” to the NIST/NRC RAP? • What is the impact of the program on the Research Associates, NIST, and to the scientific and engineering communities in general? In addition, the committee determined to offer recommendations where appropriate regarding data collection on applicants to the program, on the experiences of current and former NIST/NRC RAs, and on the views of NIST/NRC RAs and NIST employees toward the value of the program to NIST/NRC RAs, NIST, and to the scientific and engineering communities in general. In assessing the NIST/NRC RAP, the committee sought to compare that program to the other RAPs administered by the NRC. First, the committee compared characteristics of applicants to the NIST/NRC RAP to other RAPs, and to a lesser degree to the general pool of recent science and engineering doctorates. Second, the committee compared characteristics and experiences of NIST/NRC RAs to Research Associates in other programs.17 Finally, the committee undertook a very limited comparison of the careers of former NIST/NRC RAs to other former research associates, but data limitations prevented all but simple descriptive analyses. The approach taken by the committee is limited in several ways. Most importantly, the analysis is not causal in nature. The committee cannot explain similarities or differences between NIST/NRC RAs and the comparison groups. However, description is an informative first step that reveals patterns and trends. It suggests potential causal hypotheses (e.g., the rise in the number of female applicants is due to the growing number of women receiving doctorates, better recruitment of women doctorates, or the changing nature of the profession, in which postdocs have become necessary in the career pathway) that can be tested in the future. Description also reveals gaps in data collection that might be addressed in the future. The analysis can also highlight similarities and differences between the NIST/NRC RAP and other RAPs, which can be useful in designing more rigorous evaluations to assess the impact of the award. Second, the comparison between the NIST/NRC RAP and the other RAPs combined is imperfect. Not all of the RAPs have the same eligibility requirements (e.g., limited to U.S. citizens), nor are the appointments necessarily similar. However, in practice, these two groups of postdocs and postdoctoral appointments are quite close and there is the advantage that individuals in both groups fill out the same application form, awardees fill out the same final report, and Research Associate advisors fill out the same evaluation form—thus providing similar data.18 Third, the committee was limited by data availability (for details see below). Data do not exist back to the beginning of the NIST program in 1954. Comparative data on postdoctoral appointments and postdocs is relatively new—dating back to the mid- or late-1970s—and the 17 Given the timeframe for completing the report, the committee was not able to also compare the applicants to the NIST/NRC RAs and those awarded appointments to comparable postdocs outside the RAPs, although such a comparison might be possible in the future with the cooperation of other agencies that maintain postdoctoral programs. 18 Due to privacy concerns over the small number of individuals in some of the RAPs (because they do not make many awards or are relatively newer programs), it is not recommended to conduct analysis on each of the RAPs identified in Table 1-1. 13

most in-depth quantitative analysis of postdocs has only been undertaken in the past several years. SOURCES OF INFORMATION To complete its charge, the committee relied primarily on three sources of information: (1) data on applicants to the NIST/NRC and other RAPs and the experiences of NIST/NRC and other RAs, which are collected by the NRC’s Fellowships Office; (2) data on the careers of former RAs collected by NIST and the Fellowships Office; and (3) data on S&E doctorates and postdoctorates collected by the National Science Foundation (NSF). In addition, the committee examined relevant literature on S&E postdoctorates, including surveys of postdoctorates, and examples of evaluations of other federal education programs. Finally, the committee conducted original research in the form of multiple expert panels with current and former NIST RAs and NIST employees. Table 1-2 summarizes the sources, which are further elaborated on below. 14

TABLE 1-2 Summary of Primary Data Sources Topic Data Source General Focus Coverage Used in Applications National Academies’ Data collected from applications 1965-2007 Chapter 2 and awards “DataRAP” (especially demographic) 19 database National Academies’ Question: how did you hear about the 1989-2007 Chapter 2 “DataRAP” database program? NSF, Survey of Demographic data and postgraduation 1958-2005 Chapter 2 Earned Doctorates plans of doctorates Experiences National Academies’ Demographic data (of applicants who 1965-2007 Chapter 3 of Research “DataRAP” database were accepted) Associates National Academies’ Post-tenure exit survey of postdocs 1965-2006 Chapter 3 during the “DataRAP” database asking them about their experiences program during the program. National Academies’ Evaluation forms filled out by postdoc 1965-2006 Chapter 3 “DataRAP” database advisors Experiences National Academies’ Post-tenure exit survey of postdocs 1965-2006 Chapter 4 of former “DataRAP” database asking them about immediate career Research plans Associates NIST List of former postdocs who stayed at 1996-2007 Chapter 4 NIST Fellowships Office List of former postdocs with current 1965-2002 Chapter 4 20 Directory employer Data on applicants comes from two sources: a database constructed by the NRC’s Fellowship Office to hold application data and the NSF’s Survey of Earned Doctorates. The Fellowship’s Office maintains a database on applicants, collecting information submitted by applicants during the application process. Until very recently, data were entered manually from an application form (see Appendix B); currently, some information is pulled from a web-based application. Data are available from 1959 (1966 for NIST/NRC RAs) through February 2007. Key variables are: • Demographic information, including: birth date, citizenship, gender, race/ethnicity, and marital status; • Educational background, including highest degree, degree field, and doctorate awarding institution; • Desired postdoctoral position, including program and lab applied to; • Outcome indicators, including the ranking of the applicants and the outcome of the applications; and • How the applicant heard about the program. 19 The DataRAP database coverage begins in 1959 for RAs other than NIST/NRC RAs, but as we are interested in comparing NIST to other federal RAs, it is appropriate to use the NIST/NRC parameters as the limiting factor. 20 The Fellowships Directory coverage also begins in 1959. 15

In comparing NIST/NRC RA applicants to others who have recently received doctorates, one source of information is the NSF’s Survey of Earned Doctorates (SED), which began in 1957- 1958 to collect data annually on the number and characteristics of individuals receiving research doctoral degrees from all accredited U.S. institutions.21 All individuals, as they receive their research doctorate, are asked to complete the survey. The SED collects information on the individual’s education, characteristics, and postgraduation plans. Selected variables include: • Academic institution of doctorate • Birth year • Citizenship status at graduation • Country of birth and citizenship • Field of degrees (N = 279) • Marital status, number/age of dependents • Postgraduation plans (work, postdoc, other study/training) • Primary and secondary work activities • Source and type of financial support for postdoctoral study/research • Type and location of employer • Race and Hispanic ethnicity (for selected sub-groups) • Sex22 The survey has a greater than 90 percent response rate. Trend data are available back to 1957-1958. Partial data are available for the period 1920-1956. Data on characteristics of those awarded research associates and their experiences during the postdoctoral appointment come primarily from two sources: the post-tenure survey given to research associates by the Fellowships Office and the NSF’s Survey of Doctoral Recipients. This information is supplemented by one-time surveys of postdocs conducted by The National Academies and by Sigma Xi.23 Finally, the NSF’s Survey of Graduate Students and Postdoctorates in Science and Engineering is also a potential resource. The Fellowships Office conducts a post-tenure exit survey with individual research associates as they complete their postdoctoral appointment. The survey consists of a written questionnaire, and while nominally required, many postdocs do not fill it out. In addition, an evaluation form is requested of the research associates’ advisors. Key variables from these two evaluations: • Assessment of the research associate on innovative thinking, knowledge of field, motivation/initiative, overall science ability, independent research, and research technique; 21 For more information, see: “SRS Survey of Earned Doctorates,” available at: http://www.nsf.gov/statistics/showsrvy.cfm?srvy_CatID=2&srvy_Seri=1. [Accessed June 24, 2007.] 22 The NSF uses the term “sex” instead of “gender.” In this report, the terms “sex” and “gender” are used synonymously. 23 Sigma Xi is “an international, multidisciplinary research society whose programs and activities promote the health of the scientific enterprise and honor scientific achievement. There are nearly 60,000 Sigma Xi members in more than 100 countries around the world. Sigma Xi chapters, more than 500 in all, can be found at colleges and universities, industrial research centers and government laboratories.” From “About Sigma Xi,” available at: http://www.sigmaxi.org/about/overview/index.shtml. 16

• Productivity during postdoctoral tenure, including: number of domestic and international presentations given, number of peer-reviewed journal articles, number of patents applied for, and number of awards received; • Research associate’ views of the program, including: quality of mentoring from the Lab NRC Advisor, quality of laboratory support (e.g., equipment), how the National Academies Associateship award affected your career to date, quality of administrative support from the Laboratory, development of knowledge, skills, and research productivity, and evaluation of advisor (open ended); and • Future plans, including: future plans of the research associate, level of post tenure position, organization with whom post-tenure associate will be affiliated, and title of post tenure position. The principal source for information on postdocs in general—particularly outside academia—is the NSF’s Survey of Doctoral Recipients (SDR).24 The SDR collects information from individuals who have obtained a doctoral degree in a science, engineering, or health field. The SDR is biennial, and data are available for 1973-2003.25 The SDR is a longitudinal survey that follows recipients of research doctorates from U.S. institutions, who were living in the United States during the survey reference week, and who are under age 76. Using postdoctoral status as a starting point, a number of potential relevant variables can be assessed. Selected variables include: • Citizenship status • Country of birth • Country of citizenship • Date of birth • Educational history (for each degree held: field, level, institution, when received) • Employment status (unemployed, employed part time, or employed full time) • Geographic place of employment • Marital status • Number of children • Occupation (current or past job) • Primary work activity (e.g., teaching, basic research, etc.) • Postdoctorate status • Publication and patent activities • Race/ethnicity • Salary • Satisfaction and importance of various aspects of job • Sector of employment (e.g., academia, industry, government, etc.) • Sex 24 For more information on the SDR, see: “SRS Survey of Doctoral Recipients” available at http://www.nsf.gov/statistics/showsrvy.cfm?srvy_CatID=3&srvy_Seri=5. [Accessed June 24, 2007.] 25 The next SDR will cover 2006 and future surveys are expected to switch to even-numbered years. 17

As noted by the NSF: “There have been a number of changes in the definition of the population surveyed over time. For example, prior to 1991, the survey included some individuals who had received doctoral degrees in fields outside of S&E or had received their degrees from non-U.S. universities. Because coverage of these individuals had declined over time, the decision was made to delete them beginning with the 1991 survey. Survey improvements made in 1993 are sufficiently great that SRS staff suggest that trend analyses between the data from the surveys after 1991 and the surveys in prior years must be performed very cautiously, if at all.”26 A more limiting factor for the committee was that the survey merely asks respondents if they are currently in a postdoctoral position—but does not specify how long. Therefore, some respondents could be just starting a postdoctoral appointment (probably their first, but not necessarily) while others could be in the first, second, or third year of a position. This makes it difficult to compare the experiences of postdocs identified in the survey with research associates. In the most recent wave of the SDR—collected in 2006, though not available at the time this report was prepared—a series of questions relating to postdocs was added. Questions include: whether the respondent is currently employed as a postdoc and the number of postdoctoral appointments the individual has had. For each appointment, respondents are asked to provide data on start and end dates, the primary reason for taking the position, employment sector (e.g., academic, industry, government), whether the employer provided health or retirement benefits, and to what extent the most recent or current postdoctoral appointment impacted on the respondent (e.g., increasing knowledge, research skills, career opportunities). If these questions continue to be asked in future SDRs, these data would be very helpful in getting a broad picture of postdocs’ experiences. A one-time survey of postdocs was conducted by Sigma Xi, in partnership with the National Postdoctoral Association, the American Association for the Advancement of Science, and the National Bureau of Economic Research, in 2003. Invitations to participate in the survey were sent to 1,432 provosts and vice provosts, deans, human resources personnel, and leaders of postdoc offices and associations at 174 institutions. Of these, 46 institutions, including 18 of the top 20 academic employers and the largest government employer of postdocs, the National Institutes of Health (NIH), agreed to participate (Davis, 2005). Email was used to contact the 22,000 postdocs at the participating institutions, roughly 40 percent of all postdocs working in the United States at the time. The survey’s response rate was 34 percent. The survey focused on the following characteristics of postdocs: • Demographic information (e.g., sex, race/ethnicity, age, citizenship); • Benefits and services (e.g., salaries, benefits, and the adequacy of various resources); • Institutional environment (written policies and procedures and about the availability of various types of training); • The postdoc’s advisor; • The postdoc’s position; • Outcomes (e.g., time allocation, papers); • Career plans; and • The postdoc’s satisfaction with the postdoctoral appointment. 26 “SRS Survey of Doctoral Recipients” available at http://www.nsf.gov/statistics/showsrvy.cfm?srvy_CatID=3&srvy_Seri=5. [Accessed June 24, 2007.] 18

A departmental survey by the NSF is the Survey of Graduate Students and Postdoctorates in Science and Engineering (GSS).27 The survey includes data on the number and characteristics of postdocs in S&E fields in U.S. institutions. The survey is conducted annually, and it is targeted to individual academic departments at graduate institutions. Data include count data on postdoctorates by source of support, sex, and citizenship. Data was collected beginning in 1966, and the most current data, as of this writing, is for 2005. However, the collection of data on postdocs has changed over time, making it difficult to examine long term trends.28 Finally, a one-time survey of institutions with postdocs was conducted by the Committee on Science, Engineering, and Public Policy (COSEPUP) at The National Academies. “COSEPUP decided to survey the top 25 academic institutions (in terms of the largest numbers of postdoctoral scholars) and five each of the following: smaller institutions (in terms of number of postdoctoral scholars), medical schools, historically black colleges and universities (HBCUs), industry, research institutions, and government laboratories. The survey was conducted from November 1999 to April 2000. The survey was conducted of 49 organizations who have postdoctoral scholars. Forty of the 49 organizations responded (82 percent response rate) (NAS/NAE/IOM, 2000:138).” Career data are more difficult to come by. One source of information is The National Academies’ Associateships Program Directory: “A Directory of former NRC Associates was published in book form in 1995. Since then, more than 2,000 additional associateships have been awarded. In this web Directory, we have updated current information on many of our former Associates. This information can be viewed by Associate name, program, or date of award, or by state or country of the Associate’s current employment. The Directory is fully searchable.”29 Data cover 1959-2002. Using these data, it is possible to see where former NIST and other research associates have ended up—with the caveat that the information is limited to their current position, when they were asked to respond. A second source of career data consists of monthly reports on NIST Research Associates collected and maintained by NIST. These data record changes in the employment status of NIST Research Associates. NIST Research Associates may resign or be terminated (although they may then work as contractors with NIST) or stay at NIST by being converted to a term position or by being converted to a career conditional position. Data are available in hardcopy for 1998- 2001 and in electronic format for 2002-2007. Since these data cover research associates at the end of their tenure; these data cover research associates who began their appointments in 1996 or later. A third source of information on the careers of postdocs which might be helpful comes from the NSF’s SDR. It is possible to examine respondents who completed multiple surveys, so one could identify survey respondents who were postdocs in the 1999 survey and examine changes in employment or other characteristics, for those respondents who also responded to the 2001 survey. The NSF is currently in the middle of a feasibility study designed to explore ways to fill 27 For more information on the survey, see: “SRS Survey of Graduate Students and Postdoctorates in Science and Engineering,” available at: http://www.nsf.gov/statistics/showsrvy.cfm?srvy_CatID=2&srvy_Seri=2. [Accessed on June 24, 2007.] 28 Of particular note is that prior to 1975, NSF did not seek to collect data on all doctorate-granting institutions. For a fuller description of changes in the survey over time, see “SRS Survey of Graduate Students and Postdoctorates in Science and Engineering,” available at: http://www.nsf.gov/statistics/showsrvy.cfm?srvy_CatID=2&srvy_Seri=2. [Accessed on June 24, 2007.] 29 NRC Associateships Program Directory, available at: http://nrc58.nas.edu/pgasurvey/data/aodir/gen_page.asp. [Accessed on July 13, 2007.] 19

in the gap in data on postdocs. One goal is to improve coverage in two ways: by obtaining information on more postdocs (e.g., foreign Ph.D.-degreed postdocs or M.D.-Ph.D. postdocs) than might be covered in the SDR; and by covering more sectors beyond academia (e.g., postdocs at non-profits or government agencies), which fall outside the scope of the GSS. The NSF’s study may lead to a future survey of postdocs. Finally, the committee undertook original data collection in the form of three expert group discussions with current NIST RAs (CRA), former NIST RAs still working at NIST (FRA), and NIST RA advisors and division chiefs (AD). Each of the panels lasted approximately one hour and occurred on July 25, 2007. Questions asked of these participants (with the person the questions were directed to in parentheses) are listed in Box 1-1. Nine current research associates participated in the first expert panel. Six were male. Three began their appointments in 2005, five began in 2006, and one began in 2007. The research associates represented seven different labs. Six former research associates still at NIST participated in the second expert panel. Four were male. One finished their postdoctoral appointment in 2005; four finished in 2006, and one had just finished in 2007. Three had been converted to career conditional status; the other three had converted to term appointments. Participants represented five labs. The third panel consisted of eight advisors and lab chiefs. Two had been former NIST RAs. Seven were male. They represented five labs. 20

Box 1-1 Expert Panel Questions • What do you see as the value of the NRC/NIST postdoctoral program? (AD) o To the postdocs o To the NIST mission? o To your science/engineering field? • Is there any downside to the program? (AD/FRA) • Are there areas where you think it could be improved? (AD/FRA) • In what ways have you benefitted from being an NRC/NIST postdoc? (FRA) • Do the postdocs differ in any way from other entry level NIST employees? (AD) • What effect do you think the postdoctoral program has on the careers of NIST postdocs? (AD) • What effect do you think the postdoctoral program will have on your career? (CRA/FRA?) • Has the postdoctoral experience been what you expected ?(If not, explain.) (CRA) • Would you recommend changing anything about the postdoctoral program? (AD /CRA/FRA) • Why did you apply for the NIST postdoctoral program? (CRA/FRA) • What other postdoc programs, jobs, and other things did you apply for? (CRA/FRA) • Has having a NIST postdoc affected where you might like to be employed? (CRA) • What are your career plans after the postdoc ends? (CRA) o Has this changed as a result of your experience in the NIST postdoc? • Do you think NIST postdocs differ from other postdocs in science and engineering? (AD) • Are you satisfied with the current selection process for awarding NIST postdocs? (everyone) • If this postdoc program did not exist, how would that affect your work and staff?AD) • Are there any NIST postdocs that stand out in your mind as having made a significant impact on the field? (AD) • Do you continue to have contact with former postdocs (in what ways)? (AD) • Is there anything else you would like to tell us about the program?(everyone) 21

OUTLINE OF REPORT The report is organized in a chronological way, following the trajectory of doctorates from application, to award, to their experiences in the Program, and concluding with a career assessment of awardees after their appointments have ended. Chapter Two examines applicants to the NIST/NRC RAP and compares them to applicants to other RAPs. It also examines applications and awards disaggregated across several dimensions, such as gender or doctoral- granting institution. Chapter Three examines the experiences of NIST RAs and RAs at other federal agencies, as well as research associates’ views on the value of the Program. Chapter Four examines the careers of former research associates. Chapter Five presents an overall summary of preliminary results and recommendations. 22

Next: 2 Recruitment and Selection »
Approaches for Evaluating the NRC Resident Research Associateship Program at NIST Get This Book
×
Buy Paperback | $54.00 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The NRC Resident Research Associateship Program at NIST provides two-year temporary appointments for outstanding scientists and engineers. This book describes program applicants and awardees and offers suggestions for an in-depth assessment of career outcomes. Preliminary investigation indicates that outreach efforts produce more qualified applicants than NIST has slots to fill, the pool of applicants is increasingly diverse, and many Research Associates go on to permanent positions at NIST. The agency should conduct a more thorough evaluation of the program, including an assessment of outreach to potential applicants, individuals who decline an award, the program's impact on the careers of awardees, and the benefits of the program to NIST and the broader scientific and engineering community.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!