Appendix A Surveying the ECSE Community In its deliberations, the Computer Science and Telecommunications Board (CSTB) Committee on Academic Careers for Experimental Computer Scientists made considerable use of several informal surveys of the experimental computer science and engineering (ECSE) community. One survey was sent to the approximately 180 chairs of departments on the Forsythe list (i.e., all chairs of Ph.D.-granting departments in the United States and Canada in computer science and engineering (CS&E)); this questionnaire is presented as Exhibit A. Seventy department heads responded to this survey, of whom about 40 were themselves experimentalists. A second survey was sent to experimental computer scientists and engineers in academia identified through the procedure described in Box A.1; this questionnaire is presented as Exhibit B. A third survey was sent to graduate students in ECSE; this questionnaire is presented as Exhibit C. These students were identified by asking the ECSE faculty mentioned above to pass along the survey to their Ph.D. graduate students who had passed comprehensive exams. About 200 graduate students responded to the survey. (Because the committee had no way of knowing the number of Ph.D. graduate students working for the ECSE faculty members who received the questionnaire, the total number of graduate students who received the survey is not known.) These surveys were developed and sent under the auspices of the Computing Research Association, a professional organization that represents the interests of the CS&E research community; however, answers were returned directly to committee staff. Of course, the analysis of data and the conclusions drawn from these surveys are entirely the responsibility of the committee. Ideally, the committee would have conducted an ethnographic study of all institutions in which ECSE research is pursued and an analysis to distill common themes. Lacking the resources and the expertise to conduct a project of such dimensions, committee members decided on an approach that would yield as much information as possible. In particular, the information gathered by surveys was used primarily as a reality check on the insights derived from the discussions of committee members with their colleagues. Thus, although the committee believes that the surveys returned cover a considerable portion of the academic community of experimental computer scientists and engineers,1 both tenure-track faculty and graduate students, it must point out that they do not constitute a complete or necessarily even a representative sample of the community, and biases are undoubtedly present in the responses. Perhaps the most serious bias is the fact that respondents may have been disproportionately more dissatisfied with their current status in academia than a true random sampling would reveal. At the same time, survey respondents included a number of the leading experimentalists in the field. It would be misleading to present these survey results as being uniformly true for the entire ECSE community in academia. However, the comments of the respondents do represent the views of those individuals, and because the number of respondents constituted a substantial portion of the relevant community, it is fair to assert that their comments cannot be taken as isolated or aberrational. As an inspection of the surveys indicates, questions that would result in both quantitative and qualitative data were asked. No clear interpretations emerged from a consideration of the quantitative data, and, in the end, the committee found that the richest data were found in the qualitative answers that respondents provided. NOTE 1 The total number of faculty in Ph.D.-granting computer science and engineering departments in 1989 was 2,660 (Computer Science and Telecommunications Board. 1992. Computing the Future, National Academy Press, Washington, D.C., p. 255), although the number of those conducting experimental computer science or engineering is unknown. It is the committee's qualitative impression that experimental computer scientists and engineers are, broadly speaking, in the minority across the nation. The total number of Ph.D. graduate students is also unknown, although it is known that some 800 graduate students receive Ph.D.s in computer science or engineering every year. BOX A.1 Identifying ECSE Faculty Members Through a "Friends-of-Friends" Procedure Approximately 900 individuals were identified by sending a short e-mail note to about a half-dozen individuals suggested by committee members, asking them to respond with the names and e-mail addresses of five other individuals in ECSE and employed in academic institutions. These names and addresses were recorded in a database, and the same note was sent to these individuals. The process was then iterated, with notes sent only to new individuals who were not already recorded in the database. By the time the process was terminated (on the basis of elapsed time--about 10 weeks), nearly 1,250 names had been generated. The figure of 900 given above is the number of individuals mentioned by two or more respondents. The electronic questionnaire was sent to the 900 individuals. Of these, about 220 responded, representing about 90 departments. The results of this procedure are presented more fully in Figure A.1. This method for obtaining names was not comprehensive, in that a number of individuals personally known by the committee to fit the search criteria were not identified; these individuals were located at both less well known and better-known institutions. However, the committee believes that this method was able to generate coverage of a relatively large part of the ECSE community in a very short time and at negligible cost. NOTE: Brian Reid, a member of the committee, brought this procedure to the committee's attention and was responsible for its implementation. In the time since this procedure was executed, the committee has learned that statisticians have known of this technique for a long time and refer to it as "snowball sampling."