National Academies Press: OpenBook
« Previous: 1 Introduction
Suggested Citation:"2 Sources of the Data." National Research Council. 2011. Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment. Washington, DC: The National Academies Press. doi: 10.17226/13213.
×

2

Sources of the Data

The data used in this study were collected as part of the National Research Council’s Data-Based Assessment of Research-Doctorate Programs, and the data collection procedures and caveats are described in detail in that report.1 The committee authoring the Assessment identified several sources of errors in the data that could not be eliminated, including classification errors and data collection errors. The omission of field-specific measures, such as books, patents, and articles presented at refereed conferences in some science and engineering fields, means that the data do not capture the full scope of a program’s research productivity (see Box 2-1).

The data on research productivity that were collected during the study were analyzed in specific ways in the Assessment report, but the full database available to researchers could extend this analysis to explore alternate measures of research productivity by the faculty. For example, less emphasis could be placed on a count of journal articles, which were not judged on the basis of their impact, and greater emphasis could be placed on the citation measure. Alternately, only articles with citations could be counted. These are only a few suggestions for further analysis.

Once the data were released, institutions and others identified additional problems, which led to the release of a corrected data table in April, 2011.2 It is important for the reader to understand some of the limitations of the data used to produce the correlations and other analysis in this report.

______________

1 see Chapter 3 of the Assessment, “Study Design.”

2 A summary of the changes made to the data table and a log of individual corrections are available at www.nap.edu/rdp.

Suggested Citation:"2 Sources of the Data." National Research Council. 2011. Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment. Washington, DC: The National Academies Press. doi: 10.17226/13213.
×

BOX 2-1 Sources of Data Errors in the Assessment of Research-Doctorate Programs

1) Classification errors. The taxonomy of fields may not adequately reflect distinctions that the field itself considers to be important. For example, in anthropology physical anthropology is a different scholarly undertaking from cultural anthropology, and each subfield has different patterns of publication. By lumping together these subfields into one overall field, the committee is implying comparability. Were they separate, different weights might be given to publications or citations. Anthropology is not alone in this problem. Other fields are public health, communications, psychology, and integrated biological science. Although this study presents ranges of rankings across these fields, the committee encourages users to choose comparable programs and use the data, but apply their own weights or examine ranges of rankings only within their peer group.

2) Data collection errors. The committee provided detailed definitions of important data elements used in the study, such as doctoral program faculty, but not every program that responded paid careful attention to these definitions. The committee carried out broad statistical tests, examined outliers, and got back to the institutions when it had questions, but that does not mean it caught every mistake. In fields outside the humanities it counted publications by matching faculty names to Thomson Reuter’s data and tried to limit mistaken attribution of publications to people with similar names. Despite these efforts, some errors may remain.

3) Omission of field-specific measures of scholarly productivity. The measures of scholarly productivity used were journal articles and, in the humanities, books and articles. Some fields have additional important measures of scholarly productivity. These were included in only one field, the computer sciences. In that field peer-reviewed conference papers are very important. A discussion of data from the computer sciences with its professional society led to further work on counting publications for the entire field. In the humanities the committee omitted curated exhibition volumes for art history. It also omitted books for the science fields and edited volumes and articles in edited volumes for all fields, since these were not indexed by Thomson-Reuters. All of these omissions result in an undercounting of scholarly productivity. The committee regrets them, but it was limited by the available sources. In the future it might be possible to obtain data on these kinds of publication from résumés, but that is expensive and time-consuming.

NOTE: The computer sciences count as publications articles that are presented at refereed conferences, but until recently few of these papers were indexed by Thomson Reuters. To deal with this practice, the committee compiled a list of such conferences that were not indexed and counted these publications from faculty résumés, as it did in the humanities.

SOURCE: A Data-Based Assessment of Research-Doctorate Programs in the United States, p. 7.

Suggested Citation:"2 Sources of the Data." National Research Council. 2011. Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment. Washington, DC: The National Academies Press. doi: 10.17226/13213.
×

In addition to data from the Assessment, data on training grants and training slots were collected from the NIH website.3 Using these two sources, the panel has identified correlations among many of the characteristics of doctoral programs in the biomedical sciences mentioned in the statement of task:

Average Publications per Faculty Member

Average Citations per Publication

Percent of Faculty with Grants

Percent of Non-Asian Minority Faculty

Percent of Female Faculty

Awards per Faculty Member

Average GRE Scores

Percent of Non-Asian Minority Students

Percent of Female Students

Average PhDs per Year, 2002-2006

Average Cohort Completion Rate

Median Time to Degree

Appendix D provides the correlations for these 12 variables for each field. With the exception of Awards per Faculty Member, all are discussed in Chapter 3.

In addition to the above list, other variables, such as the percent of first-year students with research assistantships or the percent with external fellowships, were used in analyses in later chapters (e.g., Chapter 4 ). Appendix C contains definitions of all of the relevant variables from the Assessment; data on these variables for each biomedical program are included in the Excel table available with this report. Appendix E contains the statistical summary of each variable by field.

Finally, the panel relied on other results from the Assessment surveys of doctoral programs, faculty, and students for more targeted analysis. Data on doctoral student satisfaction, productivity, and changes in career objectives in neuroscience and neurobiology (Chapter 6 ) came from the survey conducted of doctoral students in that and four other sample fields (chemical engineering, physics, economics, and English)4. Data on postdoctorates in Chapter 7 were drawn from primarily unpublished results of the program and faculty surveys. Although not all of these data are discussed in the Assessment report, they are available in the online Excel data table that accompanies this report or in the full database available for public use.

______________

3 See http://grants.nih.gov/training/outcomes.htm#fundedgrants; data are from the version posted in 2009. Using NIH data, we were unable to associate training grant funding with particular programs. We were, however, able to tie them to particular institutions, and this is the approach we take in the analysis in this report.

4 See “Data from Student Questionnaires” in Chapter 7 of the Assessment.

Suggested Citation:"2 Sources of the Data." National Research Council. 2011. Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment. Washington, DC: The National Academies Press. doi: 10.17226/13213.
×

This page intentionally left blank.

Suggested Citation:"2 Sources of the Data." National Research Council. 2011. Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment. Washington, DC: The National Academies Press. doi: 10.17226/13213.
×
Page 13
Suggested Citation:"2 Sources of the Data." National Research Council. 2011. Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment. Washington, DC: The National Academies Press. doi: 10.17226/13213.
×
Page 14
Suggested Citation:"2 Sources of the Data." National Research Council. 2011. Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment. Washington, DC: The National Academies Press. doi: 10.17226/13213.
×
Page 15
Suggested Citation:"2 Sources of the Data." National Research Council. 2011. Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment. Washington, DC: The National Academies Press. doi: 10.17226/13213.
×
Page 16
Next: 3 PAIRWISE CORRELATIONS »
Research-Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Research Doctorate Programs in the Biomedical Sciences: Selected Findings from the NRC Assessment examines data on the biomedical sciences programs to gather additional insight about the talent, training environment, outcomes, diversity, and international participation in the biomedical sciences workforce. This report supports an earlier publication, A Data-Based Assessment of Research-Doctorate Programs in the United States, which analyzes data and rankings from more than 5,000 doctoral programs, 982 of which were in the biomedical sciences. Research Doctorate Programs in the Biomedical Sciences explores questions about degrees and completion rates as they relate to GRE scores, student funding, program facilities, diversity among faculty members, and other variables. The report examines 11 biomedical science fields including cell and developmental biology, genetics and genomics, microbiology, nutrition, and physiology, among others.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!