Box 6.1 Computer Graphics Since 1980
or a road map for the surgery itself. This is a planning system for neurosurgery that, of course, starts with computed tomography (CT) or magnetic resonance imaging (MRI) scans. The novel aspect is showing the results registered on top of the patient. This guides the surgeon in deciding where to cut the hole in the skull and, once inside, how much tissue to remove.
Now we have systems that combine graphics with something in the outside world. My theme today is that we could push that capability forward for the rest of us. The next video provides an example of how to do this for applications in which the medical imagery is coming to you in real time. For guidance, you could use not only the old data, but real-time data as well. This is similar to the previous application, except now you are using the real-time imagery as part of the display in order to guide what you are doing.
In the next 10 years, we will find that the use of 3D graphics is increasingly commonplace. There will be a merging of the live video with the display. You might well ask what we need live video for in our everyday lives. I hesitate to tell you, but I think it is going to be the year-2000 version of the picture phone. I say this despite the fact that some of you might laugh, because picture phones have come around about every 10 years and so far have been disappointing to just about everybody. I expect that people are not going to be disappointed 10 years from now. There will be enough bandwidth, display capability, and computing to make teleconferencing a compelling shared presence, which the current generation of teleconferencing hardware cannot do. My personal view is that the crucial aspect of shared presence is going to be the capability to extract a sense of 3D from one place and show it at another place in a way that makes participants feel they are together, even if they are thousands of miles apart.
We are going to have to do what you might call desktop VR for modeling and 3D teleconferencing. In 10 years, I think we will have the equivalent of, say, a dozen laptops packaged within a single workstation, with about a dozen times the power and display capability of a laptop (more pixels (perhaps 10 million), higher resolution, and a wider field of view). Think about a picture-window kind of display made up of, for example, 10 or 12 laptop-size screens and 10 or 12 cameras. These cameras will use the workstation's computational resources to extract a model of the small environment you are in—a little office cubicle, for instance—and transmit that model to another place. In this way, together with head-tracked stereo, you could share the sense of presence with the person you are talking to or the two or three people you might be having a teleconference with. I believe the principal problem remaining is how to display these remote environments, because their data representation is probably not going to be polygonal. This is going to be the most exciting frontier: modeling from imagery and warping the images in such a way as to make the output image seem more realistic. A lot of people are working on this. The 1996 SIGGRAPH conference, for example, featured many papers on how to take imagery, warp it, and correct it from another perspective, making the output image more like a photograph than a picture of a set of polygons.
BOX 6.2 Virtual Reality in Medicine: Examples