Augmented Reality: A New Vision

Abstract

Augmented Reality (AR) is a technology that combines computer generated and real world data by integrating real objects or space with virtual contextual information. As a technology, AR has been around for over 30 years. According to Scientific American, the term Augmented Reality was first coined in 1990 by a scientist at Boeing where they developed a prototype solution to help workers construct wiring harnesses. Applications are being developed in the medical, military, manufacturing, and entertainment industries that will improve productivity and processes via high-tech innovation. As production costs fall and technology improves, the adoption of AR will have a profound effect on our society; the way we interact with computers, one another, and the way we learn and work.

Introduction

Augmented Reality (AR) is a specific example of what Fred Brooks called Intelligence Amplification (IA): using the computer as a tool to make a task easier for a human to perform (Brooks, 1996). An AR user such as a technician or mechanic might wear a Head Mounted Display (HMD) through which he could see the real world, as well as computer-generated schematics and repair information projected on top of what he is seeing in real time (see Figure 1.). An augmented reality system is one that combines real and virtual, is interactive in real time, and is registered in 3D. AR attempts not only to superimpose graphics over a real environment in real-time, but also change those graphics to accommodate a user's head- and eye- movements, so that the graphics always fit the perspective. Three components needed to make an augmented-reality system work are head-mounted displays, tracking systems, and mobile computing power (Bonsor, n.d.). Over the past decade, there has been a boom in AR research as hardware costs have fallen enough to make the necessary lab equipment affordable.

Figure 1- Optical see-through HMD conceptual diagram (Azuma, 1992).

Image Guided Surgery (IGS) is the typical application area where virtual objects and real must be merged into a single unified scene, calling for Augmented Reality (AR) techniques. As minimally invasive surgeries gradually phase out open invasive surgeries, IGS systems will experience greater uptake among surgeons. Pre-operative imaging studies, such as CT or MRI scans, provide the surgeon a view of the internal anatomy to help better plan the surgery. Augmented reality can be applied so that the surgical team sees the CT or MRI data correctly registered on the patient in the operating room while the procedure is progressing. From these images the surgery is planned by creating a 3D model from the multiple views and slices in the pre-operative study (see Figure 2.). AR can be applied so that the surgical team can see the CT or MRI data displayed on the patient in the operating room while the procedure is ongoing (Vallino, 1998). These devices provide surgeons an unbeatable advantage by aiding real-time navigation and offering a 3D virtual representation.

Figure 2 - Virtual fetus inside womb of pregnant patient.

(Courtesy UNC Chapel Hill Dept. of Computer Science.) (Azuma, 1992).

Being able to accurately register the images at this point will enhance the performance of the surgical team and may eliminate the need for the painful and cumbersome stereotactic frames currently used for registration (Mellor, 1995).

How it Works

IGS systems are comprised of several features: a tracking device or camera; the hardware and software components; and a registration device, which is there to align the patient’s image with preoperative images. The most important step is registration. A special probe fitted with infrared-light-emitting diodes (LEDs) is used to teach the computer the exact location of the patient. The camera tracks the LEDs emitted by the probe as the surgeon touches different points on the patient's spinal anatomy. This builds a one-to-one relationship between the computer and patient. During surgery, the patient's anatomy moves, such as during breathing. The connection between patient and computer established during registration to automatically update images each time the anatomy moves. Throughout the surgery, the computer can generate images of the patient's anatomy as it relates to the surgeon's instrument.

Benefits

There are many significant benefits to implementing an IGS system. Image-guided procedures are substantially less invasive than traditional surgery. Because the image-guided