• The concepts were shown to be "better" quantitatively.

  • The mouse has had a significant impact as witnessed by a variety of subsequent implementations, improvements, and applications, as well as widespread use.

Despite these qualities, the replies to the chair's informal survey correlated strongly with whether the respondent was an experimentalist (yes) or a theoretician (no). The question exposed fundamental differences of opinion concerning the nature of research accomplishments. It also emphasizes that the research of junior faculty members—either theoreticians or experimentalists—whose senior faculty are predominantly in the other area might not be fully appreciated at promotion time.

In this chapter two general questions of evaluation are considered. The first concerns how CS&E implements its quality standards for research. Treating this matter entails a careful review of publication forums and traditions in ECSE. The second concerns differences in experimental and theoretical research and how these differences affect a professor's evaluation.


The scholarly articulation of a contribution is a key characteristic of research, and all intellectual communities have mechanisms through which new knowledge and information are disseminated and explicated. In addition, certain communities place considerable value on establishing priority and claiming credit for new ideas and innovations. Not surprisingly, the particular mechanisms used by any given community depend on the efficacy with which those mechanisms facilitate the dissemination of information and the establishment of priority.

Communication with other researchers in ECSE has several aspects. As in all fields, the first goal is to convey the content of the work. Next in importance, the academic researcher in ECSE wishes to convince other researchers or developers to use an idea or implementation. This requires the researcher to demonstrate the worth of the idea. Such arguments can be made on a quantitative or qualitative basis, although the former is likely to be more easily conveyed. The idea must be reported in great enough detail to allow others to reproduce it, or the actual implementation that embodies the idea (i.e., the artifact) must be provided to the community. Reproduction of experimental data may also require the availability of a genuine implementation.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement