and acceptance. For example, in addition to choosing individual test subjects who truly constitute a representative sample of the anticipated population of users as test subjects, because many envisioned SE applications involve simultaneous multiple users communicating and working together on a common task, attention must be given to appropriate sampling of groups of users as well as to observations and measurements of relevant group processes.

Finally, because of the immersive character of SEs, special attention in SE evaluation must be given to possible negative long-term psychological and social effects. Illustrative questions in this category include the following: To what extent, if any, will individuals begin to confuse occurrences in SEs with occurrences in the real world? How will an individual's self-image be influenced by spending large amounts of time in SEs that seriously transform the individual's interactions with the environment? What impact will widespread use of networked SEs have on various types of social institutions? Fundamental psychosocial questions of this type are not likely to be addressed adequately (if at all) by the developers of SE technology. However, it is important that they be seriously addressed by some group.

In general and as a consequence of the many special features associated with SE evaluation, as well as the current tendency to ignore evaluation in the SE field, the committee believes that it would be extremely useful to develop a special evaluation tool kit for this field. Such a tool kit could serve to educate people in the field, to provide a more or less standardized set of evaluation tools for the field, and eventually to help provide a cumulative and shareable database that would constitute both a current snapshot of accomplishments in the field and a guide for future research and development.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement