there is also a need to develop an extensible protocol for virtual audio. Such a protocol will need to encompass all the acoustic models in use today and those expected to be developed in the near future. This protocol should allow developers and designers of sonification systems to utilize SE technology, even as that technology makes dramatic improvements in its capabilities.
Most of the needs associated with the auditory channel lie in the domain of perceptual studies. With one major exception, discussed below, the technology for the auditory channel is either adequate now or will be adequate in the near future.
Many of the perceptual issues in the auditory channel that require attention fall under the general heading of adaptation to alterations in sensorimotor loops. Some specific examples in this category, all of which relate to the spatialization problem, concern the extent to which (and rate at which) listeners adapt to various types of distortions, including those associated with (1) the use of simplified HRTFs or transformations designed to achieve supernormal localization performance in VE systems and (2) various mappings between telerobotic sensor arrays and the human auditory system in teleoperator systems that make use of nonanthropomorphic telerobots. Knowledge of how well and how fast individuals can adapt to distortions or transformations of these types under various training conditions is essential to the design of effective systems. Other closely related examples focus on the use of the auditory channel for sensory substitution purposes, e.g., the presentation of auditory signals to substitute for haptic feedback that is difficult to provide because of current equipment limitations.
Another major area in the perceptual domain that requires substantial work falls under the heading of auditory information displays. Current knowledge of how to encode information for the auditory channel in ways that produce relatively high information transfer rates with relatively small amounts of training is still rather meager. It is obvious that encoding all information into speech signals is inappropriate and the statement that the encoding should be natural is simply not adequate to guide specific design choices. This general encoding and display problem is judged to be both important and difficult to solve. Also, it is believed that progress in this area will depend, at least in part, on an improved understanding of auditory scene analysis.