Radiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

DOI: 10.1148/radiol.2382041441
This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Wacker, F. K.
Right arrow Articles by Lewin, J. S.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Wacker, F. K.
Right arrow Articles by Lewin, J. S.
(Radiology 2006;238:497-504.)
© RSNA, 2006

Experimental Studies

An Augmented Reality System for MR Image–guided Needle Biopsy: Initial Results in a Swine Model1

Frank K. Wacker, MD, Sebastian Vogt, MS, Ali Khamene, PhD, John A. Jesberger, Sherif G. Nour, MD, Daniel R. Elgort, PhD, Frank Sauer, PhD, Jeffrey L. Duerk, PhD and Jonathan S. Lewin, MD

1 From the Department of Radiology, University Hospitals of Cleveland, Case Western Reserve University, Cleveland, Ohio (F.K.W., J.A.J., S.G.N., D.R.E., J.L.D.); Department of Radiology, Charité-Campus Benjamin Franklin, Hindenburgdamm 30, Berlin 12200, Germany (F.K.W); Department of Imaging and Visualization, Siemens Corporate Research, Princeton, NJ (S.V., A.K., F.S.); and Russell H. Morgan Department of Radiology and Radiological Sciences, Johns Hopkins School of Medicine, Baltimore, Md (J.S.L.). From the 2004 RSNA Annual Meeting. Received August 19, 2004; revision requested October 28; revision received December 28; accepted February 1, 2005; final version accepted May 20. Supported in part by Siemens Medical Solutions research grant and NCI grants R33CA88144 and R01CA81431. Address correspondence to F.K.W. (e-mail: frank.wacker{at}charite.de).


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 
Purpose: To evaluate an augmented reality (AR) system in combination with a 1.5-T closed-bore magnetic resonance (MR) imager as a navigation tool for needle biopsies.

Materials and Methods: The experimental protocol had institutional animal care and use committee approval. Seventy biopsies were performed in phantoms by using 20 tube targets, each with a diameter of 6 mm, and 50 virtual targets. The position of the needle tip in AR and MR space was compared in multiple imaging planes, and virtual and real needle tip localization errors were calculated. Ten AR-guided biopsies were performed in three pigs, and the duration of each procedure was determined. After successful puncture, the distance to the target was measured on MR images. The confidence limits for the achieved in-plane hit rate and for lateral deviation were calculated. A repeated measures analysis of variance was used to determine whether the placement error in a particular dimension (x, y, or z) differed from the others.

Results: For the 50 virtual targets, a mean error of 1.1 mm ± 0.5 (standard deviation) was calculated. A repeated measures analysis of variance indicated no statistically significant difference (P > .99) in the errors in any particular orientation. For the real targets, all punctures were inside the 6-mm-diameter tube in the transverse plane. The needle depth was within the target plane in 11 biopsy procedures; the mean distance to the center of the target was 2.55 mm (95% confidence interval: 1.77 mm, 3.34 mm). For nine biopsy procedures, the needle tip was outside the target plane, with a mean distance to the edge of the target plane of 1.5 mm (range, 0.07–3.46 mm). In the animal experiments, the puncture was successful in all 10 cases, with a mean target-needle distance of 9.6 mm ± 4.85. The average procedure time was 18 minutes per puncture.

Conclusion: Biopsy procedures performed with a combination of a closed-bore MR system and an AR system are feasible and accurate.

© RSNA, 2006


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 
Magnetic resonance (MR) imaging guidance for interventional procedures receives continually increasing attention. Open MR imagers designed in a C-arm or a double-doughnut fashion allow the physician patient access during imaging inside the magnet. In this way, biopsy needles or applicators for thermal ablation can be inserted with immediate real-time MR imaging feedback (1–5). However, such imagers are not widely available, and most of the open MR imagers operate at low field strength, which can hamper overall image quality and create substantial challenges for applications such as thermomonitoring (5–7), fast imaging, or functional imaging. Conventional closed-bore MR imagers operate at high field strength, provide excellent image quality for a multitude of imaging techniques, and are available in virtually every hospital in the Western world. However, the performance of MR imaging–guided procedures, such as biopsy and thermal ablation, is difficult with a closed-bore MR imager because the patient has to be moved back and forth between a position outside of the magnet bore, where a needle can be manipulated, and an imaging position inside the magnet bore, where images can be acquired. Therefore, the procedure must be performed in a "stop-and-go" fashion; in complicated cases, the number of repetitions of the patient's entrance into and exit from the imaging system can become substantial and thereby increase the procedure time (1–5).

In contrast to conventional image guidance, in which the instrument and anatomy are seen on a screen separate from the patient, augmented reality (AR) image guidance maps the medical data onto the patient's body. The physician can see beyond the surface, and the patient's body becomes transparent. Anatomic structures visualized by means of MR images acquired immediately before the start of the procedure are displayed at their actual location. This is a very direct and intuitive way of presenting image information that can be used for image-guided procedures, as well as for other commercial and research applications (8). In recent years, medical AR systems have been investigated for neurosurgery (9–11), laparoscopic surgery (12), cardiac interventions (13), oral implantation (14,15), ultrasonography (US)-guided procedures (16), and interventional MR imaging (17,18).

Thus, the goal of this study was to evaluate an AR system in combination with a 1.5-T closed-bore MR imager as a navigation tool for needle biopsy.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
 DISCUSSION
 References
 
System Hardware
The AR system was provided by Siemens Corporate Research (Princeton, NJ). Three authors (F.S., S.V., and A.K.) are employees of Siemens Corporate Research. The authors who are not Siemens employees had full control of inclusion of any data and information that might present a conflict of interest for those authors who are Siemens employees.

The AR system consists of a custom-made video-see-through head-mounted display (HMD) (Proview XL35; Kaiser Electro-Optics, Carlsbad, Calif) with extended graphics array resolution and a 35° diagonal field of view. Two color video cameras (GP-KS1000; Panasonic, Osaka, Japan) attached to the HMD provide a stereoscopic view of the scene; these cameras have a 15-mm lens and 30° field of view. A third head-mounted video camera (XC-77RR; Sony, Tokyo, Japan), rigidly attached to the two other cameras, is added onto the HMD for tracking (Fig 1a); it has a 4.8-mm lens and 95° field of view. The tracking camera is responsible for measuring the viewer's position and orientation in relation to an optically linked reference frame attached to the MR imaging table and surrounding the desired workspace (Fig 1). The position and orientation information facilitates rendering of the medical imaging graphics from exactly the observer's vantage point, making it appear firmly anchored with respect to the real scene. A second set of optical markers is attached to the biopsy needle (Fig 1b). Frame and needle marker sets provide the position and orientation of both the observer's viewpoint and the biopsy needle, respectively. The system runs on a single personal computer (Precision 530; Dell, Round Rock, Tex) and achieves real-time performance—that is, 30 frames per second, with a latency of about 0.1 second compared with the real scene—thereby generating a stable augmentation with no apparent jitter visible in the composite images. All procedures were performed by using a 1.5-T closed-bore magnet system (Magnetom Sonata; Siemens Medical Solutions, Erlangen, Germany). MR and AR computers were linked through a standard 100 megabits/sec Ethernet connection.


Figure 1
View larger version (128K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 1a: AR system close to MR imager during biopsy in a swine. (a) White reference frame (arrowhead) is attached to table, needle tip is inside the body, and clip-on marker set (arrow) is attached at the proximal end. The interventionalist is wearing a video-see-through HMD (*) that gives a stereoscopic view of the scene. (b) View of the real scene observed from behind the interventionalist shows reference frame (arrowhead) and clip-on needle marker set (arrow).

 

Figure 1
View larger version (136K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 1b: AR system close to MR imager during biopsy in a swine. (a) White reference frame (arrowhead) is attached to table, needle tip is inside the body, and clip-on marker set (arrow) is attached at the proximal end. The interventionalist is wearing a video-see-through HMD (*) that gives a stereoscopic view of the scene. (b) View of the real scene observed from behind the interventionalist shows reference frame (arrowhead) and clip-on needle marker set (arrow).

 
Calibration
To calibrate the MR and AR frame coordinate system, a calibration phantom with four MR fiducial markers and eight retroreflective optical markers was used. A high-spatial-resolution MR data set (three-dimensional fast imaging with steady-state precession sequence [repetition time msec/echo time msec, 9/5; flip angle, 25°; in-plane resolution, 0.9 x 0.9 x 0.9 mm; one signal acquired; bandwidth, 260 Hz/pixel]) was acquired with the calibration phantom at the center of the workspace surrounded by the frame marker set. A homogeneous transformation matrix was computed that could map any point in the MR coordinate system to the AR frame coordinate system once the relationship between the optical markers of the phantom and the AR frame coordinate system was established. This transformation matrix allowed augmentation of the MR sections with any generated computer graphics. The entire calibration procedure lasts 12–15 minutes. The matrix can be used without repeated calibration as long as the reference frame stays rigidly attached to the MR imaging table. In a second step, the needle is automatically calibrated to the marker cluster. This must be done prior to each biopsy to allow use of a new sterile needle for every puncture. After the clip-on marker cluster is firmly and rigidly attached to the needle, five to 10 poses of the marker cluster–needle complex are recorded with the stereo cameras of the HMD while the needle is rotated around its tip on a sturdy surface. We then apply a minimization scheme that results in the coordinates of the needle tip with respect to the attached marker cluster. On the basis of these data, needle position and orientation can then be computed during the intervention from the position and orientation of the marker cluster. In the AR view, the needle is augmented with a virtual cylindric tube. The needle calibration procedure takes 8–12 seconds.

Phantom Experiments and Statistical Analysis
The biopsy phantoms consisted of round buckets filled with hydroxyethylcellulose (500 g hydroxyethylcellulose in 2000 mL H2O; Natrosol gel, Hercules, Wilmington, Del). To mimic lesions within the phantom, 20 hollow plastic tubes, 6 mm in diameter and height, were embedded in the gel phantom. The tubes were placed, with their longitudinal axis parallel to that of the round bucket, at different spatial locations and levels inside the bucket and deep enough that they could not be seen with the naked eye. The gel was stiff enough that internal deformations caused by inserting a needle into the phantom were not substantial. For the virtual target experiments, a round bucket filled with Natrosol gel without real targets was used. The needle used for the simulated biopsy was a 20-cm-long MR-compatible 18-gauge needle.

MR images of the gel phantom with the tube targets were acquired by using a transverse T1-weighted spin-echo sequence (200/4.4, 90° flip angle, 5-mm section thickness, 256 x 256 matrix, in-plane resolution of 0.7 mm/pixel) and were then transferred to the AR system computer. By using these MR images, we segmented the targets and generated three-dimensional graphical models of each target by using a separate interactive custom-made segmentation tool. For the virtual target experiments, the targets were generated arbitrarily throughout the phantom.

To perform the needle placement for the biopsy experiments, the MR table containing the phantoms was moved to a position outside the imager. By using the HMD, the radiologist was shown a stereoscopic video view of the phantom overlaid with a graphical representation of the targets. Furthermore, the real biopsy needle was graphically enhanced with a thin virtual cylinder. In addition, a 7-cm-long thin virtual cylinder of a different color was used to represent the linear extension of the needle from its tip. This virtual extension makes it easy to aim the needle at the target while the needle is still completely outside the phantom.

Twenty biopsy experiments with tube targets and 50 experiments with virtual targets were performed by one of the authors (F.K.W.). The sample size for the study was not calculated a priori. The number of trials was predominantly determined as the most reasonable within the time and resource budget restrictions. The key personnel and equipment had to travel to the site with the required compatible MR imager and animal housing and testing facilities to conduct the tests. Prior to every experiment, the object of interest was augmented into the physician's HMD view. After the physician completed the AR-guided needle insertion, the location of the needle tip was recorded in the augmented reality coordinate system. Hydroxyethylcellulose gel droplets doped with gadopentetate dimeglumine (concentration, 0.5 mmol/L; Magnevist, Berlex Laboratories, Wayne NJ) were inserted through the needle and left in place inside the gel phantom at the needle tip (F.S., D.R.E.). The total puncture time from displaying the target on the HMD to reaching the final needle position was recorded. Transverse MR control images were then acquired to assess the hit rate, the distance of the markers to the center, and the minimum distance to the inner edge of the tube target in plane (lateral deviation). To assess the depth deviation of the marker, the distance of the marker to the center of the MR image that was used for preprocedure planning was measured by two authors in consensus (S.G.N., S.V.). The confidence limits of the achieved in-plane hit rate were estimated (19).

Two different placement errors were calculated (J.A.J.). The virtual placement error (or user error) is the Euclidian distance between the virtual needle tip recorded in the AR space at the end of the puncture and the virtual target center in all three dimensions. This error depicts how well the physician follows augmented guidance. A repeated measures analysis of variance was used to determine whether the placement error in a particular dimension, x, y, or z, differed from the others. To test for possible interactions between errors in x, y, and z dimensions, the R2 values and associated P values were obtained from calculations of the Pearson product-moment correlation coefficients. R2 value was selected for analysis to illustrate the very low proportion of variance accounted for by a linear relationship between x, y, and z error components (Table 1). The correlations between the errors in the x, y, or z direction are illustrated in Figure 2.


View this table:
[in this window]
[in a new window]

  		Table 1. Correlation between Placement Errors in x, y, and z Dimensions for Virtual Target Punctures

 

Figure 2
View larger version (47K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 2: Scatterplots illustrate overall absence of correlation between errors in x, y, and z dimensions for the needle placements with the virtual targets. R2 and associated P values are shown in Table 1.

 
The real placement error is the distance between the tube target center and the gadolinium-enhanced marker position, as computed from the MR control images. This error shows the end result, which is the position of the real needle tip with respect to the center of the lesion target. The distance from the nearest target boundary in the x-y plane was calculated to determine how close each placement came to missing the target or, in the case of a miss, how close the placement came to reaching the target. The distance from the center of the target z coordinate (puncture depth) was also measured by two authors in consensus (F.K.W., S.V.) to determine operator variability and the mean and range of how far from the target boundary misplacements were located. The confidence interval for the margin of error of our puncture technique was calculated for a lateral deviation on the basis of the minimum distance of the marker from the inner edge of the tube target. To test the puncture results for a temporal learning curve, the mean and variance of the first half of the data were compared with those of the second half of the data.

Animal Experiments and Data Analysis
Animal experiments were conducted on three living pigs that weighed 22–40 kg each. The experimental protocol was approved by our institutional animal care and use committee. The animals were anesthetized with an intramuscular injection of 6–10 mg tiletamine hydrochloride and zolazepam hydrochloride (Telazol; Fort Dodge Animal Health, Fort Dodge, Iowa) per kilogram of body weight. For maintenance of anesthesia, ketamine hydrochloride (Ketaject; Phoenix Scientific, St Joseph, Mo) and xylazine (Xyla-Ject; Phoenix Scientific) were infused and 1 mg of tiletamine hydrochloride and zolazepam hydrochloride (Telazol) per kilogram of body weight was added intramuscularly every 45–60 minutes. The animals were positioned in the MR imager in either a supine (n = 1) or a prone (n = 2) position. After the acquisition of scout images, transverse and coronal two-dimensional half-Fourier rapid acquisition with relaxation enhancement MR images (1100/118, 8-mm section thickness, 120° flip angle) and transverse true fast imaging with steady-state precession MR images (3.03/1.52, 5–8-mm section thickness, 70° flip angle) were used to visualize the abdominal anatomy of the animals (Fig 3a).


Figure 3
View larger version (186K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 3a: AR-guided MR image–based biopsy. (a) Baseline transverse true fast imaging with steady-state precession MR image (3.03/1.52, 70° flip angle, 6-mm section thickness) used for target selection. For this experiment, pancreas tail (arrow) is selected. The same image is used to create the augmented view. (b) Augmented view of needle placement as shown by the AR guidance system. The needle (blue) and its forward extension (yellow) are seen as a thin cylinder. The target structure is represented by the central green disk; the outer rings merge as the operator advances the needle. In contrast to this two-dimensional impression, the user is provided with a stereoscopic view of the augmented scene. (c) MR control image (true fast imaging with steady-state precession) after AR-guided puncture, with the needle (arrow) still in position.

 
The images were then transferred to the AR computer and, on the basis of these images, targets were selected and marked in the augmented space by using small circles with a diameter of 5 mm. Ten puncture attempts were performed by one author (F.K.W.) with AR guidance by using the pancreatic tail (n = 3), the gallbladder (n = 3), a renal calyx (n = 2), and a central bile duct (n = 2) as targets. For the actual puncture, transparent MR sections containing the target and the skin entry site were augmented into the HMD to enhance the location perception of the targets and to allow the interventionalist to avoid crossing important structures (Fig 3b).


Figure 3
View larger version (150K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 3b: AR-guided MR image–based biopsy. (a) Baseline transverse true fast imaging with steady-state precession MR image (3.03/1.52, 70° flip angle, 6-mm section thickness) used for target selection. For this experiment, pancreas tail (arrow) is selected. The same image is used to create the augmented view. (b) Augmented view of needle placement as shown by the AR guidance system. The needle (blue) and its forward extension (yellow) are seen as a thin cylinder. The target structure is represented by the central green disk; the outer rings merge as the operator advances the needle. In contrast to this two-dimensional impression, the user is provided with a stereoscopic view of the augmented scene. (c) MR control image (true fast imaging with steady-state precession) after AR-guided puncture, with the needle (arrow) still in position.

 
Immediately after the AR-guided puncture outside the magnet, the clip-on marker was removed from the needle and the animals were brought back into the imager with the biopsy needle still in position to verify the needle position on MR images (Fig 3c). The distance of the needle tip from the actual target was measured in three dimensions by two authors in consensus (F.K.W., S.G.N.), and the Pythagorean distance, D, was calculated as follows: D = {surd}(a2 + b2 + c2), where a, b, and c are the distances from needle tip to target in each dimension x, y, and z, respectively.


Figure 3
View larger version (194K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 3c: AR-guided MR image–based biopsy. (a) Baseline transverse true fast imaging with steady-state precession MR image (3.03/1.52, 70° flip angle, 6-mm section thickness) used for target selection. For this experiment, pancreas tail (arrow) is selected. The same image is used to create the augmented view. (b) Augmented view of needle placement as shown by the AR guidance system. The needle (blue) and its forward extension (yellow) are seen as a thin cylinder. The target structure is represented by the central green disk; the outer rings merge as the operator advances the needle. In contrast to this two-dimensional impression, the user is provided with a stereoscopic view of the augmented scene. (c) MR control image (true fast imaging with steady-state precession) after AR-guided puncture, with the needle (arrow) still in position.

 
The duration of the preprocedure imaging, image transfer, skin-to-lesion time, target display on the HMD–to-lesion time, and puncture time, which ranged from the selection of the target on the MR planning images to the confirmation of the final needle position on an MR image, were all recorded. The procedure time for each biopsy was calculated as starting with the beginning of the preprocedure imaging and ending with the end of the acquisition for the MR images that confirmed the final needle position after each biopsy. The procedure time per animal was the cumulative puncture time of multiple biopsies performed in one animal. This would allow assessment on multiple puncture trials in a single patient if multiple needle passes were necessary or if the target was missed and repositioning was necessary.

Binomial probability confidence limits were calculated with Java applets (J. A. Veeh, Auburn University, Auburn, Ala; available at: http://javeeh.net/statapps/Binn.html). All other statistical calculations were performed with SPSS software (SPSS for Windows, version 11; SPSS, Chicago, Ill).


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 References
 
Phantom Experiments
By using AR guidance, the interventionalist was able to quickly locate an appropriate point of entry into the phantom, as well as an appropriate angle of insertion. Before the needle was actually entering the phantom, the virtual extension of the biopsy needle body helped to plan the exact needle path and to make sure that the needle extension intersected the center of the virtual lesion target. The physician then inserted the needle with a firm and constant motion along the desired straight needle path all the way to the lesion target. The mean puncture time from the display of the targets to the final needle position was 4.2 seconds (range, 1.8–9.3 seconds).

The evaluation between the three-dimensional coordinates of the needle tip position in the AR space and the virtual target position with known three-dimensional coordinates in 50 biopsy procedures showed a mean error of 1.13 mm ± 0.5 (standard deviation) in all directions, with a maximum error of 2.6 mm. This result represents the precision with which the radiologist is able to perform the AR-guided needle placement procedure assuming that the AR system itself is error-free. The mean error for the first 35 punctures was 1.18 mm and that for the last 35 punctures was 1.08 mm, with an increase in variance for the second half (0.33 vs 0.14), which makes a strong learning curve bias unlikely. There was no statistically significant difference between errors in the x, y, or z directions (repeated measures analysis of variance: F = 1.48; df = 2, 48; P > .99).

In all 20 real target punctures, the radiologist correctly placed the tip of the needle inside the lesion targets. As recorded on the MR images of the phantom after the puncture, all gadolinium-doped droplets were inside the 6-mm-diameter tube in the transverse plane. Given a 100% hit rate in all experiments, the point estimate of the presumption of a correct placement was 1.00, with a 95% confidence level that the true presumption of a correct placement was greater than 0.83. The average minimum in-plane distance to the inner edge of the tube, beyond which placement would have been counted as a miss, was 1.44 mm (95% confidence interval: 1.01 mm, 1.87 mm). In four cases there was a minimum distance of zero, which corresponds to a droplet placement just on the inner target edge. The needle depth was within the 5-mm-thick target plane in 11 biopsy procedures, which corresponds to a likelihood of a correct placement of 0.55 (95% confidence interval: 0.32, 0.77). The mean distance from the center of the target plane was 2.55 mm (95% confidence interval: 1.77 mm, 3.34 mm). In nine biopsy procedures, the needle tip was outside the 5-mm-thick target plane, with a mean distance to the edge of the target plane of 1.5 mm (range, 0.07–3.46 mm).

Animal Experiments
The interaction of the virtual needle extension with the virtually enhanced target and the augmented semitransparent MR images (Fig 3) provided abundant information and guidance, leading to a straightforward puncture with an average skin-to-target time of 11 seconds (range, 3–18 seconds). The average time from displaying the target on the HMD to reaching the target was 30 seconds. Preprocedure imaging took less than 5 minutes for all experiments, and the mean image transfer time was less than 35 seconds. All procedure times are given in Table 2. The targeted puncture outside the MR imager was successful in all 10 cases, with a maximum error of 10 mm in a particular dimension, x, y, or z, and a mean Pythagorean distance of 9.6 mm (standard deviation, 4.85 mm) from the biopsy needle to the target.


View this table:
[in this window]
[in a new window]

  		Table 2. Procedure Time in the Animal Experiments

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
References
 
The MR image-based AR system presented in this study has proved to effectively facilitate needle biopsy outside the MR imager. The AR system uses MR images acquired immediately before the procedure and maps them into patient space, thus providing additional information that augments the real world. A three-dimensional display enables the physician to perceive the MR images in situ. In contrast to virtual reality, where reality is replaced with a completely virtual environment, AR incorporates the computer graphics into the reality of the user. Because of the stereoscopic view of the augmented scene, a tumor on the image appears in the location of the actual tumor, below the patient's skin. Mapping a biopsy needle into the same space enables the physician to monitor how the graphic representations of the needle path and the needle merge with the tumor. Augmented MR images reconstructed along the needle path help to avoid crossing vital structures. The AR system makes hand-eye coordination very intuitive. Since the preinterventional MR data set can be acquired and reformatted in any plane, there are no restrictions for choosing the entry point of the puncture. In addition, bore diameter and needle length are not issues because the actual procedure is performed outside the magnet bore without any spatial restrictions. This is also beneficial for patient comfort and access. An MR image-based AR navigation system has the potential to make the usually tedious and time-consuming MR-guided interventions in a closed-bore imager more efficient, with targeting and puncture times similar to those of computed tomography (CT)-guided punctures (20,21).

A limitation of the current AR navigation system is that the patient must be immobilized during the intervention. Although it might be feasible to induce anesthesia for complicated procedures such as thermal ablation, it is not acceptable for biopsy. Possible solutions for detection of patient movement would be the implementation of external landmarks to the patient's body. In relation to the frame of reference of our system, such markers could then be used to detect and correct for patient movement outside the magnet bore (16,22). Internal organ movement is another challenge with the AR system, which is currently limited to targeting retroperitoneal, musculoskeletal, and pelvic lesions without movement caused by breathing. To target liver lesions that move during the breathing cycle, a breathing motion correction must be implemented. There are simple solutions that have already been implemented for CT-guided punctures (23). More complex solutions could be based on methods used in radiation therapy, as well as on those used in positron emission tomography–CT image fusion (12,24–27). However, neither patient nor breathing motion correction will account for the internal target displacement or deformation during needle advancement. This can be overcome by performing control imaging during the course of an intervention, similar to what is normally done with CT guidance or, technically more challenging, by using a deformable finite element model that predicts deformations of the target organ (28,29).

Another drawback of the current prototype system is the weight of the HMD. During our experiments the HMD became uncomfortable to wear after approximately 30 minutes. This might not be an issue for clinical punctures with a single target because the average procedure time in our animal experiments was well below this time frame, and wearing the HMD is required only for a fraction of this time frame. However, implementation of lighter displays and smaller and lighter cameras, which are already commercially available (16,30), will help to increase the comfort of future HMDs. Another solution might be to replace the HMD with a translucent display mounted on a swivel arm.

A limitation of the current study is that multiple measurements were obtained in the same animal. However, the anatomic locations of the different targets were distributed throughout the bodies of the animals. Between the puncture procedures, the animal was brought back into the MR imager for control imaging. When aiming for a new target, the interventionalist had to access the animal from a different entry site, cross different structures, and use a new set of MR images for puncture control on the AR display; therefore, a learning curve bias seems to be unlikely in the animal experiments.

There are certain topics that should be addressed before using an AR-guided system. It is important to note that the reference marker frame needs to be firmly attached to the MR imaging table to avoid repeated calibration between patients. Along the same line, rigid attachment of the needle marker cluster, a stiff needle, and precise needle calibration are crucial to correctly augment the needle, as this is required for a successful biopsy.

In contrast to CT, for which a multitude of puncture-guiding devices are commercially available (31), there are only a few reports on such systems in MR imaging. The AR system provides an alternative to remotely guided robotic biopsy devices that were developed for breast biopsies with MR guidance in a closed-bore imager (32,33). In contrast to the "in-bore" biopsy with a robotic system, the AR-guided biopsy is performed outside the narrow magnet bore, thus enabling direct patient-physician contact, full tactile control of the procedure, and the use of devices such as radiofrequency electrodes and drainage catheters that do not fit well in a magnet bore because of their length. The AR-guided biopsy procedure used in this pilot study seems to be faster than what is reported in the literature about robotic devices (33,34). The puncture time measured with our AR system is in the range of that of conventional CT fluoroscopy–guided procedures (20,21).

In conclusion, AR guidance based on MR images for percutaneous biopsies is feasible and allows intuitive and accurate needle localization in phantoms as well as in animals. It has the potential to facilitate percutaneous interventions in the "procedure-hostile" MR imaging environment. It remains to be seen what procedures benefit most from the dynamic association of patient and image data. For simple biopsies, an experienced interventionalist will not ask for such a guidance tool, although augmented real-time visualization of important landmarks and critical structures might be helpful for less experienced users. However, given the cost and availability, US and CT guidance will remain the "workhorses" for biopsy procedures, and MR imaging is only mandatory if a lesion can be seen only on MR images or if radiation exposure should be avoided, such as in children. For more complex procedures, such as thermal tumor ablations that require positioning of multiple applicators and puncture of multiple lesions, AR guidance might be of help not only to reduce puncture risk and procedure time and to allow for more complete and radical therapy but also to seamlessly utilize MR imaging for monitoring thermal lesions and assessing end-organ function. The exceptional value of MR imaging for thermal ablation monitoring (7) encourages some groups to use either an open MR imaging approach (5,6,35) or a dual imaging modality approach, in which MR imaging is combined with either CT or US (36,37). Under these circumstances, a clinical AR system could be a powerful tool, and the cost of AR guidance would be negligible compared with reimbursement for thermal ablation procedures and potential patient benefit.

Practical application: An AR system based on MR images provides a fast means to perform radiation exposure–free percutaneous interventions with a closed-bore MR imager. While results reported here are preliminary and of limited scope, we believe that they suggest the potential of AR visualization for patient care.


   ACKNOWLEDGMENTS
 
The authors thank Bonnie Hami, MA, for her invaluable editorial assistance.


   FOOTNOTES
 

Abbreviations: AR = augmented reality • HMD = head-mounted display

See Materials and Methods for pertinent disclosures.

Author contributions: Guarantors of integrity of entire study, F.K.W., F.S.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; approval of final version of submitted manuscript, all authors; literature research, F.K.W., A.K., J.S.L.; experimental studies, F.K.W., S.V., A.K., S.G.N., D.R.E., F.S., J.L.D.; statistical analysis, F.K.W., S.V., J.A.J.; and manuscript editing, F.K.W., J.A.J., S.G.N., D.R.E., F.S., J.L.D., J.S.L.


   References
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 References
 

  1. Genant JW, Vandevenne JE, Bergman AG, et al. Interventional musculoskeletal procedures performed by using MR imaging guidance with a vertically open MR unit: assessment of techniques and applicability. Radiology 2002;223:127–136.[Abstract/Free Full Text]
  2. Lewin JS, Petersilge CA, Hatem SF, et al. Interactive MR imaging-guided biopsy and aspiration with a modified clinical C-arm system. AJR Am J Roentgenol 1998;170:1593–1601.[Abstract/Free Full Text]
  3. Schulz T, Bennek J, Schneider JP, et al. MRI-guided pediatric interventions [in German]. Rofo 2003;175:1673–1681. [Published correction appears in Rofo 2004;176:266.][Medline]
  4. Silverman SG, Collick BD, Figueira MR, et al. Interactive MR-guided biopsy in an open-configuration MR imaging system. Radiology 1995;197:175–181.[Abstract/Free Full Text]
  5. Wacker FK, Reither K, Ritz JP, Roggan A, Germer CT, Wolf KJ. MR-guided interstitial laser-induced thermotherapy of hepatic metastasis combined with arterial blood flow reduction: technique and first clinical results in an open MR system. J Magn Reson Imaging 2001;13:31–36.[CrossRef][Medline]
  6. Dick EA, Joarder R, de Jode M, et al. MR-guided laser thermal ablation of primary and secondary liver tumours. Clin Radiol 2003;58:112–120.[CrossRef][Medline]
  7. Jolesz FA, Hynynen K. Magnetic resonance image-guided focused ultrasound surgery. Cancer J 2002;8(suppl 1):S100–S112.
  8. Azuma R, Baillot Y, Behringer R, Feiner S, Julier S, MacIntyre B. Recent advances in augmented reality. IEEE Comput Graph Appl 2001;21:34–47.
  9. Kawamata T, Iseki H, Shibasaki T, Hori T. Endoscopic augmented reality navigation system for endonasal transsphenoidal surgery to treat pituitary tumors: technical note. Neurosurgery 2002;50:1393–1397.[CrossRef][Medline]
  10. King AP, Edwards PJ, Maurer CR Jr, et al. A system for microscope-assisted guided interventions. Stereotact Funct Neurosurg 1999;72:107–111.[CrossRef][Medline]
  11. Sauer F, Khamene A, Bascle B, Rubino GJ. A head-mounted display system for augmented reality image guidance: toward clinical evaluation for iMRI-guided neurosurgery. Proceedings of Medical Image Computing and Computer Assisted Intervention (MICCAI). Berlin, New York: Springer Heidelberg, 2001; 707–716.
  12. Herline A, Stefansic JD, Debelak J, Galloway RL, Chapman WC. Technical advances toward interactive image-guided laparoscopic surgery. Surg Endosc 2000;14:675–679.[CrossRef][Medline]
  13. Sorensen TS, Therkildsen SV, Makowski P, Knudsen JL, Pedersen EM. A new virtual reality approach for planning of cardiac interventions. Artif Intell Med 2001;22:193–214.[CrossRef][Medline]
  14. Ewers R, Schicho K, Truppe M, et al. Computer-aided navigation in dental implantology: 7 years of clinical experience. J Oral Maxillofac Surg 2004;62:329–334.[CrossRef][Medline]
  15. Ploder O, Wagner A, Enislidis G, Ewers R. Computer-assisted intraoperative visualization of dental implants: augmented reality in medicine. Radiologe 1995;35:569–572.[Medline]
  16. Rosenthal M, State A, Lee J, et al. Augmented reality guidance for needle biopsies: an initial randomized, controlled trial in phantoms. Med Image Anal 2002;6:313–320.[CrossRef][Medline]
  17. Gering DT, Nabavi A, Kikinis R, et al. An integrated visualization system for surgical planning and guidance using image fusion and an open MR. J Magn Reson Imaging 2001;13:967–975.[CrossRef][Medline]
  18. Wendt M, Sauer F, Khamene A, Bascle B, Vogt S, Wacker FK. A head-mounted display system for augmented reality: initial evaluation for interventional MRI [in German]. Rofo 2003;175:418–421.[Medline]
  19. Finkelstein M, Tucker HG, Veeh JA. Conservative confidence intervals for a single parameter. Commun Stat Theory Methods 2000;29:1911–1928.
  20. Gianfelice D, Lepanto L, Perreault P, Chartrand-Lefebvre C, Milette PC. Value of CT fluoroscopy for percutaneous biopsy procedures. J Vasc Interv Radiol 2000;11:879–884.[Medline]
  21. Silverman SG, Tuncali K, Adams DF, Nawfel RD, Zou KH, Judy PF. CT fluoroscopy-guided abdominal interventions: techniques, results, and radiation exposure. Radiology 1999;212:673–681.[Abstract/Free Full Text]
  22. Herline AJ, Herring JL, Stefansic JD, Chapman WC, Galloway RL Jr, Dawant BM. Surface registration for use in interactive, image-guided liver surgery. Comput Aided Surg 2000;5:11–17.[CrossRef][Medline]
  23. Holzknecht N, Helmberger T, Schoepf UJ, et al. Evaluation of an electromagnetic virtual target system (CT-guide) for CT-guided interventions [in German]. Rofo 2001;173:612–618.[Medline]
  24. Giraud P, Reboul F, Clippe S, et al. Respiration-gated radiotherapy: current techniques and potential benefits. Cancer Radiother 2003;7(suppl 1):15S–25S.
  25. Goerres GW, Burger C, Schwitter MR, Heidelberg TN, Seifert B, von Schulthess GK. PET/CT of the abdomen: optimizing the patient breathing pattern. Eur Radiol 2003;13:734–739.[Medline]
  26. Nehmeh SA, Erdi YE, Pan T, et al. Quantitation of respiratory motion during 4D-PET/CT acquisition. Med Phys 2004;31:1333–1338.[CrossRef][Medline]
  27. Shimizu S, Shirato H, Aoyama H, et al. High-speed magnetic resonance imaging for four-dimensional treatment planning of conformal radiotherapy of moving body tumors. Int J Radiat Oncol Biol Phys 2000;48:471–474.[CrossRef][Medline]
  28. Clifford MA, Banovac F, Levy E, Cleary K. Assessment of hepatic motion secondary to respiration for computer assisted interventions. Comput Aided Surg 2002;7:291–299.[CrossRef][Medline]
  29. Azar FS, Metaxas DN, Schnall MD. Methods for modeling and predicting mechanical deformations of the breast under external perturbations. Med Image Anal 2002;6:1–27.[CrossRef][Medline]
  30. Birkfellner W, Figl M, Huber K, et al. A head-mounted operating binocular for augmented reality visualization in medicine: design and initial evaluation. IEEE Trans Med Imaging 2002;21:991–997.[CrossRef][Medline]
  31. Cleary K, Nguyen C. State of the art in surgical robotics: clinical applications and technology challenges. Comput Aided Surg 2001;6:312–328.[CrossRef][Medline]
  32. Kaiser WA, Fischer H, Vagner J, Selig M. Robotic system for biopsy and therapy of breast lesions in a high-field whole-body magnetic resonance tomography unit. Invest Radiol 2000;35:513–519.[CrossRef][Medline]
  33. Pfleiderer SO, Reichenbach JR, Azhari T, et al. A manipulator system for 14-gauge large core breast biopsies inside a high-field whole-body MR scanner. J Magn Reson Imaging 2003;17:493–498.[CrossRef][Medline]
  34. Solomon SB, Patriciu A, Bohlman ME, Kavoussi LR, Stoianovici D. Robotically driven interventions: a method of using CT fluoroscopy without radiation exposure to the physician. Radiology 2002;225:277–282.[Abstract/Free Full Text]
  35. Gianfelice D, Khiat A, Amara M, Belblidia A, Boulanger Y. MR imaging-guided focused US ablation of breast cancer: histopathologic assessment of effectiveness—initial experience. Radiology 2003;227:849–855.[Abstract/Free Full Text]
  36. Gewiese B, Beuthan J, Fobbe F, et al. Magnetic resonance imaging-controlled laser-induced interstitial thermotherapy. Invest Radiol 1994;29:345–351.[CrossRef][Medline]
  37. Vogl TJ, Straub R, Eichler K, Sollner O, Mack MG. Colorectal carcinoma metastases in liver: laser-induced interstitial thermotherapy—local tumor control rate and survival data. Radiology 2004;230:450–458.[Abstract/Free Full Text]




This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Wacker, F. K.
Right arrow Articles by Lewin, J. S.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Wacker, F. K.
Right arrow Articles by Lewin, J. S.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
RADIOLOGY RADIOGRAPHICS RSNA JOURNALS ONLINE