to the R&D process, could be useful. Such individuals might make a valuable contribution to appropriately structuring the study, interpreting the data, and broadening the analysis.

A quantitative approach to studying a discipline also requires adequate funding to cover the requisite expertise and additional staff time. If the analysis relies solely on existing data sources, the costs will be significantly less than if it includes the collection of new data through a statistical sampling, a formal survey, or the purchase of customized data runs from commercial data sources. The costs associated with generating new data can in fact be prohibitive, and spending large amounts of money on data for assessments may not be either cost-effective or necessary.

As has already been noted, there are significant problems associated with the use of existing data sources in performing a credible quantitative analysis and drawing an accurate statistical portrait of a discipline. Although each of the three panels experienced some unique difficulties in obtaining and analyzing the various data, the problems they encountered can be categorized according to the availability, quality, and comparability of the data.

As might be expected, certain classes of data are maintained on a regular basis and are made more easily available than others. For instance, the biennial demographic statistics on Ph.D. degree recipients collected by the NRC's Office of Scientific and Engineering Personnel (OSEP) and NSF are readily accessible and can yield useful information. Some professional societies conduct yearly or periodic membership surveys, although these surveys are not necessarily representative. Other consistently maintained data of potential relevance are collected by the Bureau of Labor Statistics, the National Center for Education Statistics, and the Census Bureau.

Because the major sources of long-term demographic data on science and engineering personnel, the OSEP Survey of Doctorate Recipients and Survey of Earned Doctorates, as well as the NSF's Science and Engineering Indicators and the agency's more specialized demographic surveys, are 1 to 3 years old at any given time, these sources are not very useful for detecting current status and near-term trends. This time lag in demographic data series presents a significant limitation on the potential uses of such data either for analyzing the existing state of the field or for planning purposes. Nevertheless, for some small and well-defined fields such as astronomy, or for well-monitored fields such as the mathematical sciences, more up-to-date statistics can be found in other sources, for example, in the case of astronomy, the monthly listing of available positions in the American Astronomical Society's job register.

Most of the available demographic data cover only the United States. Those data that are available for other countries use different definitions, categorizations, and survey methodologies, making any meaningful international comparisons very difficult.

Moreover, except for the data on human resources, the available data are spotty at best. Detailed funding records from the agencies are often hard to obtain or are not available. Although NSF maintains data on funding for all agencies according to discipline, such data are sometimes difficult to interpret and are often too aggregated to use for subdisciplines. In addition, the sample sizes are typically too small to