|
Head-Mounted Display Project
(Contributions of the Department of Computer Science der University of North Carolina at Chapel Hill)
'Warren Robinett
Warren Robinett
OVERVIEW: Wearing a Head-Mounted Display (HMD) puts a human user inside of a three-dimensional computer-simulated world. Because the positions of the head and hand are tracked by the system, the user can move through this virtual world, turn to look in any direction, and use gestures of the hand to interact with simulated objects. The image of the virtual world seen by the user is adjusted up to 30 times per second to correspond with the position and orientation of the user's head. The full-color, wide-angle, stereoscopic headgear provides a slightly different image to each eye so that the computer-generated scene appears fully three-dimensional to the user, and the wide-angle field of view gives him or her the feeling of being immersed in a computer-generated artificial reality. The user also hears sounds triggered by events in the virtual world, such as two simulated objects bumping into each other.
Two new capabilities are currently being added to the system. We have a force-feedback subsystem, the Argonne Remote Manipulator (ARM), and integrating this with the HMD will allow the user to touch objects in the virtual world using the ARM's handgrip. Adding a second HMD to the system will allow two people wearing HMDs to enter into a shared virtual world, seeing and interacting with each other.GOAL The goal of this project is to develop a multi-sensory HMD system, investigate methods of manual control suite to a HMD, and demonstrate its usefulness in real applications.APPLICATIONS. A HMD is most appropriate for applications which need to display three-dimensional data to the user, and which can benefit from the user's active exploration of the simulated three-dimensional space. As the three-dimensional spaces become more complex, the ability to explore interactively becomes more important.
The HMD is well-suited to medical imaging because anatomical structures are complex three-dimensional objects. Using a HMD for planning where to place radiation beams to irradiate a tumor offers the possibility of more effective cancer treatment by more clearly showing the radiation oncologist the three-dimensional situation of the tumor and nearby radiation-sensitive organs. Currently, radiation oncologists rarely use beams at odd angles, which might be the most effective, simply because the flat displays currently used don't show them enough information to be sure the odd-angle beam is placed properly. Using our HMD and very fast computers, the user will be able to visualize in 3D the tumor, surrounding anatomy, radiation beams and radiation dose.
A second planned use of the HMD for medical imaging goes by the nickname "X-ray vision." If the image data from a realtime ultrasound scanner is transformed to the viewpoint of a doctor wearing the HMD, then the doctor will have the impression of seeing inside the tissue being scanned, with the ultrasound image optically superimposed onto his or her direct view and touch of the tissue. The 3D ultrasound scanner and see-through HMD are currently under development.
Biochemists use computer graphics to understand the three-dimensional shapes of complex molecules such as proteins. The design of some classes of new drugs depends on finding drug molecules which fit into receptor proteins, like a key fits into a lock. The HMD and hand-held manipulator may help the biochemist more effectively look at, move, bend and twist simulated molecules and see how they fit together.
Another application area is architectural "walk-through." Before a building is constructed, an architect and client can walk from room to room through a three-dimensional computer model of the building being planned.HARDWARE. The HMD System Diagram (above) shows the interconnections between hardware subsystems. We are currently using two models of head-mounted viewer: the commercially-available VPL EyePhone and the AFIT HMD, developed by the Air Force Institute of Technology with our collaboration. Both viewers use a pair of color LCD displays stripped out of pocket television sets. The EyePhone's resolution is approximately 210 x 140 color pixels spread over an 80 degree field of view. The AFIT HMD has about the same number of pixels spread over a 50 degree field of view. To attach to the head, the EyePhone uses a soft rubber diving mask, with adjustable soft fabric straps running over the top and sides of the head to a counterweight behind. The AFIT HMD uses a bicycle helmet. The EyePhone uses the LEEP wide-angle stereoscopic optics, whereas the ART HMD uses very strong reading glasses.
The graphics are generated by PixelPlanes 4, the processor-per-pixel real time graphics engine developed here at UNC. At a rate of 10 frames per second, it can support a scene complexity of 1800 arbitrarily large interpenetrating shaded polygons or 1200 arbitrarily large interpenetrating shaded spheres for each of 2 independent viewpoints. We expect to get a factor of 20 increase in scene complexity from Pixel-Planes 5, the successor to Pixel-Planes 4, scheduled to come up later this year.
The position and orientation of the head and hand are tracked with a Polhemus 3 Space magnetic tracker. A Macintosh computer is used as a dedicated sound server, playing digitized sounds on command. The ARM provides force-feedback through a handgrip with all 6 degrees-of-freedom in translation and rotation. Our standard manual input device is a billiard ball with a Polhemus sensor inside of it and 2 switches mounted on it. However, we also have several other manual input devices which are not shown in the system diagram: joysticks, sliders, a Space-Ball, a VPL Data-Glove and a treadmill.NEW TECHNOLOGY. Because of speed and range inadequacies in existing commercial trackers, a high-speed, room-sized-working-volume tracker is being developed here. A head-mounted camera is aimed upward through a 3-way beam-splitter toward light-emitting-diode beacons on the ceiling, and calculates its own position and orientation from the known positions of these beacons. The ceiling can be covered with hundreds of beacons to achieve a large working volume.
To enable the medical X-ray vison application, a HMD with see-through capability is being designed and built here. A custom optical design will allow the user to see through to the outside world, with the computer graphics optically superimposed so that the real world and virtual world are seen simultaneously.
| 
