Published online before print May 9, 2006, 10.1148/radiol.2401050743
(Radiology 2006;240:26-32.)
© RSNA, 2006
Differentiating Benign from Malignant Solid Breast Masses: Comparison of Two-dimensional and Three-dimensional US1
Nariya Cho, MD,
Woo Kyung Moon, MD,
Joo Hee Cha, MD,
Sun Mi Kim, MD,
Boo-Kyung Han, MD,
Eun-Kyung Kim, MD,
Mi Hye Kim, MD,
Soo Young Chung, MD,
Hye-Young Choi, MD and
Jung-Gi Im, MD
1 From the Department of Radiology and Clinical Research Institute, Seoul National University Hospital and the Institute of Radiation Medicine, Seoul National University Medical Research Center, 28 Yongon-dong, Chongno-gu, Seoul 100-744, Korea (N.C., W.K.M., J.-G.I.); Department of Radiology, Boramae Municipal Hospital, Seoul, Korea (J.H.C.); Department of Radiology, Bundang Seoul National University Hospital, Bundang, Korea (S.M.K.); Department of Radiology, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea (B.-K.H.); Department of Diagnostic Radiology, Yonsei University College of Medicine, Seoul, Korea (E.-K.K.); Kim Mi Hye Breast Clinic, Seoul, Korea (M.H.K.); Department of Radiology, Hallym University College of Medicine, Seoul, Korea (S.Y.C.); and Department of Diagnostic Radiology, College of Medicine, Ewha Womans University, Seoul, Korea (H.-Y.C.). From the 2004 RSNA Annual Meeting. Received May 1, 2005; revision requested June 30; revision received July 21; final version accepted August 25. Supported by Ministry of Science and Technology, Korea.
Address correspondence to W.K.M. (e-mail: moonwk{at}radcom.snu.ac.kr).
 |
ABSTRACT
|
|---|
Purpose: To compare prospectively obtained static two-dimensional (2D) and three-dimensional (3D) ultrasonographic (US) images in the diagnostic performance of radiologists with respect to the differentiation of benign from malignant solid breast masses with histopathologic examination as the reference standard.
Materials and Methods: This study had institutional review board approval, and patient informed consent was obtained. Conventional 2D and 3D US images were obtained from 141 patients (age range, 25–71 years; mean age, 46 years) with 150 solid breast masses (60 cancers and 90 benign lesions) before excisonal or needle biopsy. Four radiologists who had not performed the examinations independently reviewed 2D US images and stored 3D US data and provided a level of suspicion concerning probability of malignancy. The sensitivity, specificity, and negative predictive values of 2D images were compared with those of 3D US images.
Results: For all readers, 3D US images were the same as or better than 2D US images in terms of sensitivity (100% vs 100% for reader 1; 100% vs 98% for reader 2; 98% vs 93% for reader 3; 93% vs 92% for reader 4), specificity (58% vs 56% for reader 1; 51% vs 46% for reader 2; 83% vs 72% for reader 3; 86% vs 84% for reader 4), and negative predictive values (100% vs 100% for reader 1; 100% vs 98% for reader 2; 99% vs 94% for reader 3; 95% vs 94% for reader 4). These differences, however, were not statistically significant (P > .05).
Conclusion: The performance of the radiologists with respect to the characterization of solid breast masses with static 2D US images was similar to that with 3D US data.
© RSNA, 2006
|
INTRODUCTION
|
|---|
In addition to distinguishing cysts and solid breast tumors, ultrasonography (US) can be used to help differentiate benign from malignant solid lesions. Stavros et al (1) described a classification model with a reported 98.4% (123 of 125) sensitivity, 67.8% (424 of 625) specificity, and 99.5% (424 of 426) negative predictive value. This negative predictive value is similar to the negative predictive value of 98%–99% for the probably benign lesions reported at mammography (2) and suggests that follow-up is a reasonable alternative to biopsy for lesions that are considered probably benign at US. These results, however, were not reproduced by other investigators, particularly when static US images were used (3). Well-circumscribed cancers were misclassified as probably benign, and notable interobserver variabilities in terms of radiologists' descriptions and the assessments of breast lesions have been reported in the literature (3,4). Two-dimensional (2D) US is operator dependent for image acquisition and interpretation.
Three-dimensional (3D) US has shown promise in the obstetric, gynecologic, prostate, and cardiovascular fields (5–9). Three-dimensional US images are reconstructed from data obtained from a single sweep of the ultrasound beam across the lesion of interest. Three-dimensional US can display anatomic features and pathologic features in planes not possible with conventional 2D US (6). For the evaluation of breast masses, 3D US and volume data can be more complete than 2D US image data because the entire surface of the mass can be evaluated with dynamic and multisectional capabilities (5). Recent studies have shown that 3D US improves the analysis of breast lesions when compared with conventional US (10,11). By using 3D US, complex growth patterns and their margins can be better appreciated in three orthogonal planes in both benign and malignant tumors. Although the potential benefits of 3D US for the differentiation of solid breast masses have been emphasized, to our knowledge, previous studies have not addressed whether these benefits affect the final diagnostic assessment of lesions and therefore allow for better patient selection for biopsy.
Thus, the purpose of our study was to compare prospectively obtained static 2D US images with 3D US images in the diagnostic performance of radiologists with respect to the differentiation of benign from malignant solid breast masses with histopathologic examination as the reference standard.
|
MATERIALS AND METHODS
|
|---|
Patients
Between January and March 2003, 167 consecutive patients who had been scheduled to undergo excisional or percutaneous needle biopsy on the basis of suspicious mammographic or physical findings were examined with US. A total of 150 solid breast masses in 141 patients (age range, 25–71 years; mean age, 46 years) were depicted by US and were included in this study. Twenty-one additional patients with clustered microcalcifications detected at mammography were excluded from the study because the mass associated with microcalcifications was not seen at US. Another five patients with a breast mass larger than 3.8 cm were excluded because more than one 3D volume acquisition was necessary to include the whole lesion. The study was approved by the institutional review board of Seoul National University Hospital, and informed consent was obtained from all patients.
Lesions and Follow-up
Lesions manifested as a clinically occult lesion at mammography in 76 instances, a palpable mass in 58, nipple discharge in eight, and an incidental lesion at US in eight. Mammograms were available from 136 patients with 142 lesions. Lesions were observed as a mass in 69 instances, a mass with microcalcifications in 42, asymmetric density in nine, and architectural distortion in two. No mammographic abnormality was found for 20 (14%) lesions. According to the American College of Radiology Breast Imaging Reporting and Data System (BI-RADS) (12), the final assessment of 150 solid breast masses on the basis of both mammographic and US findings before biopsy was category 3, probably benign lesions, for 21 masses (14%); category 4, suspicious lesions, for 97 (65%); and category 5, highly suggestive of malignancy, for 32 (21%). Biopsy was performed in the 21 probably benign lesions because of patient or referring clinician preference on clinical grounds.
The nature of all masses seen at US was confirmed with histopathologic examination after surgical excision (n = 78) or with US-guided percutaneous core-needle biopsy (n = 72) within 24 hours of the US examinations. Surgical excision was performed in 78 masses because of suspicious or malignant findings at a previous percutaneous fine-needle aspiration (n = 38) or core-needle biopsy (n = 16) and referring clinician preference on clinical grounds (n = 24). Of the 150 masses, 60 (40%) were cancerous and 90 (60%) were benign. The malignant lesions were infiltrating ductal carcinoma not otherwise specified in 46 cases, infiltrating lobular carcinoma in three cases, medullary carcinoma in one case, mucinous carcinoma in one case, and ductal carcinoma in situ in nine cases. The benign lesions consisted of 39 cases of fibrocystic change, five papillomas, one radial scar, and 45 fibroadenomas. Surgery was performed in all 60 lesions with malignant findings. Follow-up information was available for 88 of 90 benign masses. The mean follow-up time was 20 months (range, 12–26 months). All benign lesions except 10 fibroadenomas and four fibrocystic changes were followed for more than 2 years, and lesion stability was confirmed. The diameters of lesions at histopathologic examination were 4–31 mm (mean, 12.6 mm) for invasive cancer, 6–36 mm (mean, 17.9 mm) for ductal carcinoma in situ, and 4–32 mm (mean, 12.8 mm) for benign lesions. Mass size at US was 4–9 mm for 43 lesions, 10–19 mm for 76, 20–29 mm for 21, and 30–35 mm for 10.
US Examinations
All US images were obtained with one type of scanner (Voluson 530D; GE Kretz, Zipf, Austria) by one breast radiologist (W.K.M.) with 7 years of experience in breast US. This radiologist had 10 months of experience in 3D US. A 5–10-MHz linear transducer was used for 2D US, and a 5–10-MHz dedicated volume transducer was used for 3D US. The order of the US examinations (ie, either 2D US first or 3D US first) was determined randomly. For 2D US, the scanning protocol included transverse, oblique, and longitudinal real-time imaging of the solid masses. For 3D US, the 2D images were first obtained in an identical plane with the same depth, focus position, and gain settings. A volume box size was then chosen to include the lesion and the maximum amount of normal surrounding tissue. The size of the volume box was usually 3.7 x 4.0 cm, and the sweep angle was 25°. The volume scan was automatically performed by using a slow-tilt movement of the sectorial mechanical transducer. Immediately after data acquisition, volume data were reconstructed and displayed in three orthogonal planes—transverse, sagittal, and coronal. Representative 2D US transverse, oblique, and longitudinal images and 3D US data were selected by the radiologist who performed the real-time imaging and were saved as image files on a magneto-optical disc for later review. For 3D US, the time required to acquire data ranged from 3 to 6 seconds, and the total examination took 8–17 minutes (mean, 12.1 minutes), which included data storage. For 2D US, the total examination took 10–35 minutes (mean, 19.2 minutes). The size of the three 2D US image files that were saved as bitmap format was 0.9 megabyte per case and that of 3D US image files ranged from 6 to 8 megabyte per case. Image files were masked and randomized.
Imaging Review
Four radiologists (B.K.H., E.K.K., M.H.K., S.Y.C.) who specialize in breast imaging and who had not performed the US examinations independently analyzed 2D transverse, oblique, and longitudinal US images and stored 3D US data without knowledge of the physical examination or mammographic results. PC-based software (ACDSee Classic, ACD Systems, Victoria, Canada; PC 3D-View 2000, GE Kretz, Zipf, Austria) for 2D US images and for 3D US data and a 21-inch video monitor with 2048 x 2560 8-bit pixels (model DR110; Dataray, Denver, Colo) were used in a darkened room. The radiologists were from four academic centers other than the institution where US examinations were performed, and their levels of experience with breast US varied (range, 6–23 years; mean, 12.2 years). For 3D US, two radiologists had 6 months of experience, and two had 2 years of experience. Before participating in the study, all four radiologists were trained with 10 cases that were not included in this study until they were accustomed to using the program to review stored 3D US data. When 3D data were opened, images were displayed in three orthogonal planes and reviewed by using only a resectioning function, although several other review modes were available in the 3D US viewer program (eg, surface and volume rendering, niche mode, and a rotating function). The principal advantages of the review mode used in this study were its speed and simplicity. Three hundred sets of US image files (150 2D US and 150 3D US) were divided in two, and the resulting two groups of 150 were reviewed within 1 month; 2D and 3D US images were read independently. During the review process, the radiologists did not adjust the brightness or contrast of 2D or 3D US images. No time limitation was set for review.
For 2D and 3D US images, the radiologists described the shape (round or oval, lobulated, irregular), orientation (wider-than-tall, taller-than-wide), margin (circumscribed, microlobulated, ill defined, spiculated), echogenicity (hyperechoic, isoechoic, hypoechoic), echotexture (homogeneous, intermediate, heterogeneous), posterior acoustic transmission (unaffected, enhanced, shadowing), boundary echo (absent, thin pseudocapsule, thick echogenic halo), and calcifications within the mass (absent, present) (1); they then provided a BI-RADS final assessment category to indicate the probability of malignancy (12). The category of benign or malignant was based on the overall assessment of the findings the radiologists evaluated. Each lesion was further categorized as benign (ie, benign or probably benign) or malignant (ie, suspicious or highly suggestive of malignancy). In terms of management, follow-up was recommended for benign or probably benign lesions, and biopsy was recommended for suspicious or highly suggestive of malignancy lesions.
Statistical Analysis
The reader agreements between the four radiologists for descriptions of shape, orientation, margin, echogenicity, echotexture, posterior acoustic transmission, boundary echo, and the presence of calcifications within masses were calculated for the 2D US and 3D US images by using 
statistics. Variabilities in reader BI-RADS final assessment categories were also calculated for 2D US and 3D US images with statistics. A value of 0.20 or less was considered slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; and 0.81–1.00, almost perfect (13).
The sensitivities, specificities, and positive and negative predictive values for 2D US and 3D US images were calculated for each reader with histopathologic examination as the reference standard, and the values were compared with the McNemar test. Two-tailed P values less than .05 were considered to indicate a statistically significant difference. In addition, receiver operating characteristic (ROC) analysis was performed to assess and compare radiologists' performances for the characterization of solid breast masses with 2D US and 3D US images because sensitivity and specificity depend on how a radiologist defines positive diagnoses. Parametric estimates of the areas under the ROC curves (Az) were calculated and compared for reader performance with the two techniques by using LABMRMC (14,15). The statistical significance of the results was reported at 95% confidence intervals for the mean differences in Az values for reader performance with use of the two techniques. Mean differences were regarded as statistically significant at the 5% level when the corresponding confidence interval did not encompass zero. Statistical analyses other than ROC analysis were performed with statistical software (SPSS version 10 for Windows; SPSS, Chicago, Ill).
|
RESULTS
|
|---|
Reader Agreement
The radiologists showed fair to moderate agreement for US findings between 2D US and 3D US images (Figs 1, 2). For the two techniques, their descriptions of boundary echo were the most discordant ( = 0.304 ± 0.065 [standard error]), which was followed by posterior acoustic transmission ( = 0.340 ± 0.062), echotexture ( = 0.343 ± 0.186), margin ( = 0.412 ± 0.131), calcifications within masses ( = 0.441 ± 0.065), echogenicity ( = 0.445 ± 0.075), shape ( = 0.450 ± 0.124), and orientation ( = 0.534 ± 0.112). For the final assessment of solid breast masses, fair to substantial agreement was found between the 2D US and 3D US images: values were 0.665 ± 0.063 for reader 1, 0.270 ± 0.064 for reader 2, 0.329 ± 0.055 for reader 3, and 0.667 ± 0.047 for reader 4. Interobserver agreements between the four radiologists for final assessment of solid breast masses were similar for 2D US images ( = 0.218 ± 0.109) and for 3D US images ( = 0.243 ± 0.103) (P > .05).
View larger version (84K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1: Fibroadenoma in a 31-year-old woman. A–C, Three-dimensional US multiplanar images show 17-mm ovoid hypoechoic mass (arrow) with circumscribed margin. A, Transverse image. B, Sagittal image. C, Coronal image. D, Schematic representation of imaging plane in A in 3D volume. Final assessment by readers was probably benign with 2D and 3D US images.
|
|
View larger version (88K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 2: Breast cancer in a 54-year-old woman. A–C, Three-dimensional US multiplanar images show 10-mm hypoechoic mass (arrow in A) with spiculated margin. A, Transverse image. B, Sagittal image. C, Coronal image. D, Schematic representation of imaging plane in C in 3D volume. Final assessments by readers were suspicious or highly suggestive of malignancy with both 2D and 3D US images.
|
|
Sensitivity, Specificity, and ROC Analysis
For all four readers, 3D US images were the same or better than 2D US images in terms of sensitivity, specificity, and positive and negative predictive values (Table 1). Differences, however, were not statistically significant (P > .05).
The Az values of the four readers were 0.75 for 2D images and 0.78 for 3D images for reader 1; 0.85 for 2D images and 0.86 for 3D images for reader 2; 0.92 for 2D images and 0.93 for 3D images for reader 3; and 0.93 for 2D images and 0.93 for 3D images for reader 4 (Table 1). These differences were not statistically significant for all readers (P > .05).
Management
Management decisions (ie, follow-up or biopsy recommendation) were discordant for 2D US and 3D US images: 18 of 150 (12%) lesions for reader 1, 40 of 150 (27%) lesions for reader 2, 23 of 150 (15%) lesions for reader 3, and 14 of 150 (9%) lesions for reader 4 (Table 2). One of the four readers with 23 years of experience with breast US recommended follow-up, rather than biopsy, for four malignant lesions with both 2D US and 3D US images. Three of these lesions were infiltrating ductal carcinomas not otherwise specified, and one lesion was an infiltrating mucinous carcinoma. These lesions were all visualized as circumscribed hypoechoic masses and misclassified as probably benign.
Five cancers categorized as benign from 2D US images were correctly categorized as malignant from 3D US images by at least one of the four readers—four cancers by one reader and one cancer by two readers. All five cancers were described differently from 2D and 3D US images for at least one of the eight US findings: the shape (n = 2), orientation (n = 2), margin (n = 4), echogenicity (n = 1), echotexture (n = 2), posterior acoustic transmission (n = 1), boundary echo (n = 1), and calcifications within the mass (n = 1). One cancer categorized as benign with 3D US images was correctly categorized as malignant with 2D US images by one reader. The shape and echogenicity of this lesion were described differently with 2D and 3D US images.
|
DISCUSSION
|
|---|
Our results show that the performance of the radiologists with respect to the characterization of solid breast masses with static 2D US images was similar to that with 3D US data. For all four radiologists, 3D US images were the same or better than 2D US images in terms of sensitivity, specificity, and positive and negative predictive values for breast cancer diagnosis. Five cancers categorized as benign with 2D US images were correctly categorized as malignant with 3D US images by at least one of the four readers. A negative predictive value over 98% was obtained by three readers with 3D US images and without mammographic information, whereas a negative predictive value over 98% was obtained by only one reader with 2D US images. In this study, volume US data were automatically acquired with commercially available dedicated volume transducers, stored within a minute, and reviewed at a remote site by the breast radiologists.
In many breast imaging clinics, US examination is performed by a US technologist. Our study results and other study results suggest that 3D breast US with interactive visualization software offers the physician the ability to review the images after patients have left the US scanning suite and to reevaluate a diagnosis with specialists at remote locations (16). The reconstruction of arbitrary spatial planes from the stored image volume offers the advantages of image documentation and interpretation. The short time needed for the initial acquisition and immediate multidimensional reconstruction, as well as unlimited off-site review of images, may, in the future, increase efficiency by better utilizing the patient's, sonographer's, and the radiologist's time (17).
Rotten et al (10) described typical findings of breast cancer and fibroadenoma at 3D US on the basis of 186 cases. Benign tumors were often found to be surrounded by a continuous hyperechoic rim, whereas breast cancers had a discontinuous hyperechoic rim on coronal images. The spiculated margin of breast cancer was best appreciated on coronal images. US findings used to characterize a solid breast mass include its shape, orientation, margin, echogenicity, echotexture, posterior acoustic transmission, boundary echo, and presence of calcifications (1,18). Three-dimensional US can be used to evaluate these features more completely for the characterization of focal breast lesions. In addition, compared with 2D US, 3D US better allows comparisons of full data sets over time, thereby improving the accuracy of the evaluation of breast masses during follow-up or the monitoring performed to determine the effects of therapy (5). Advantages of 3D US over 2D US have been recently discovered from studies on computer-aided classification (19,20), breast biopsy (21,22), and power Doppler US (23).
The main limitation of the use of 3D US is its inferior imaging quality compared with that of conventional 2D US. In a study of 100 patients by Nelson et al (16), 3D US image quality was significantly lower than that of 2D US for a variety of organ systems. Image quality, however, did not affect the ability of the reviewers to identify a structure or to make a diagnosis with 3D US. As the resolution in volume data are not the same in all directions, extracted planes from volume data generally have poorer resolution compared with that of native 2D US images. Involuntary patient motion or transducer movement during data acquisition can introduce artifacts in 3D US images (24). Another limitation is related to volume display. Interfaces must be devised to allow larger volumes of data to be viewed and interpreted in a fast, intuitive manner.
Our study has some limitations. We did not compare real-time assessment of 2D US and 3D US for the differentiation of benign from malignant solid breast masses. We compared the diagnostic performances of radiologists by using static 2D US images and 3D US data. Although 3D US may obviate sonographer quality to a great extent, 3D US may not be currently better than a skilled breast imager performing real-time 2D US. We used a soft-copy reading for 2D US images instead of a hard-copy reading because a picture archiving and communication system monitor has been routinely used for US reading in our institutions. A previous study has shown that interpretative accuracy is similar whether sonograms are interpreted on a monitor or on film (25). In our study, 3D US data were reviewed by using only three orthogonal planes and a resectioning function; this probably lowered radiologist performance. Two of the four readers had only 6 months of experience of 3D US prior to the study and were not familiar with the findings of benign and malignant breast tumors at 3D US. This could also be a limitation. Some of the US images were obtained following percutaneous biopsy, which could have possibly altered the findings. The physician who acquired the static 2D images and the 3D volume data was aware of the study purpose, and this could have influenced the selection of representative 2D images included in this study. The US lexicon described and defined by Stavros et al (1) was used in this study, and the use of the new BI-RADS US lexicon by the American College of Radiology (26) may have increased the consistency and reproducibility of the US evaluation of solid breast masses.
In conclusion, the performance of the radiologists with respect to the characterization of solid breast masses with static 2D US images was similar to that with 3D US data. Our results suggest that stored 3D US data can be used to characterize solid breast masses.
|
ADVANCE IN KNOWLEDGE
|
|---|
- The performance of the radiologists with respect to the characterization of solid breast masses with static 2D US images was similar to that with 3D US data.
|
FOOTNOTES
|
|---|
Abbreviations: Az = area under ROC curve • BI-RADS = Breast Imaging Reporting and Data System • ROC = receiver operating characteristic • 3D = three dimensional • 2D = two dimensional
Authors stated no financial relationship to disclose.
Author contributions: Guarantor of integrity of entire study, W.K.M.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; approval of final version of submitted manuscript, all authors; literature research, N.C., W.K.M., S.M.K.; clinical studies, all authors; statistical analysis, J.H.C.; and manuscript editing, all authors
|
References
|
|---|
- Stavros AT, Thickman D, Rapp CL, Dennis MA, Parker SH, Sisney GA. Solid breast nodules: use of sonography to distinguish between benign and malignant lesions. Radiology 1995;196:123–134.[Abstract/Free Full Text]
- Sickles EA. Periodic mammographic follow-up of probably benign lesions: results in 3,184 consecutive cases. Radiology 1991;179:463–468.[Abstract/Free Full Text]
- Rahbar G, Sie AC, Hansen GC, et al. Benign versus malignant solid breast masses: US differentiation. Radiology 1999;213:889–894.[Abstract/Free Full Text]
- Baker JA, Kornguth PJ, Soo MS, Walsh R, Mengoni P. Sonography of solid breast lesions: observer variability of lesion description and assessment. AJR Am J Roentgenol 1999;172:1621–1625.[Abstract/Free Full Text]
- Downey DB, Fenster A, Williams JC. Clinical utility of three-dimensional US. RadioGraphics 2000;20:559–571.[Abstract/Free Full Text]
- Hamper UM, Trapanotto V, Sheth S, DeJong MR, Caskey CI. Three-dimensional US: preliminary clinical experience. Radiology 1994;191:397–401.[Abstract/Free Full Text]
- Johnson DD, Pretorius DH, Budorick NE, et al. Fetal lip and primary palate: three-dimensional versus two-dimensional US. Radiology 2000;217:236–239.[Abstract/Free Full Text]
- Hamper UM, Trapanotto V, DeJong MR, Sheth S, Caskey CI. Three-dimensional US of the prostate: early experience. Radiology 1999;212:719–723.[Abstract/Free Full Text]
- Downey DB, Fenster A. Vascular imaging with a three-dimensional power Doppler system. AJR Am J Roentgenol 1995;165:665–668.[Free Full Text]
- Rotten D, Levaillant JM, Zerat L. Analysis of normal breast tissue and of solid breast masses using three-dimensional ultrasound mammography. Ultrasound Obstet Gynecol 1999;14:114–124.[CrossRef][Medline]
- Meyberg-Solomayer GC, Kraemer B, Bergmann A. Does 3-D sonography bring any advantage to noninvasive breast diagnostics? Ultrasound Med Biol 2004;30:583–589.[CrossRef][Medline]
- American College of Radiology. Breast imaging reporting and data system (BI-RADS). 3rd ed. Reston, Va: American College of Radiology, 1998.
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–174.[CrossRef][Medline]
- Metz CE. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol 1989;24:234–245.[Medline]
- Kurt Rossmann Laboratories for Radiologic Image Research. Department of Radiology, University of Chicago. http://www-radiology.uchicago.edu/krl/KRL_ROC/software_index.htm. Updated August 3, 2004. Accessed March 2, 2005.
- Nelson TR, Pretorius DH, Lev-Toaff A, et al. Feasibility of performing a virtual patient examination using three-dimensional ultrasonographic data acquired at remote locations. J Ultrasound Med 2001;20:941–952.[Abstract]
- Benacerraf BR, Shipp TD, Bromley B. How sonographic tomography will change the face of obstetric sonography: a pilot study. J Ultrasound Med 2005;24:371–378.[Abstract/Free Full Text]
- Moon WK, Myung JS, Lee YJ, Park IA, Noh DY, Im JG. US of ductal carcinoma in situ. RadioGraphics 2002;22:269–280.[Abstract/Free Full Text]
- Sahiner B, Chan HP, Roubidoux MA, et al. Computerized characterization of breast masses on three-dimensional ultrasound volumes. Med Phys 2004;31:744–754.[CrossRef][Medline]
- Chen DR, Chang RF, Chen WM, Moon WK. Computer-aided diagnosis for 3-dimensional breast ultrasonography. Arch Surg 2003;138:296–302.[Abstract/Free Full Text]
- Weismann CF, Forstner R, Prokop E, Rettenbacher T. Three-dimensional targeting: a new three-dimensional ultrasound technique to evaluate needle position during breast biopsy. Ultrasound Obstet Gynecol 2000;16:359–364.[CrossRef][Medline]
- Fenster A, Surry KJ, Mills GR, Downey DB. 3D ultrasound guided breast biopsy system. Ultrasonics 2004;42:769–774.[CrossRef][Medline]
- Hochmuth A, Boehm T, Bitzer C, Fleck M, Schneider A, Kaiser WA. Differentiation of breast masses using 3-D sonographic and echo-enhancer-based evaluation of the vascular pattern: initial experiences. Ultrasound Med Biol 2002;28:845–851.[CrossRef][Medline]
- Nelson TR, Pretorius DH, Hull A, Riccabona M, Sklansky MS, James G. Sources and impact of artifacts on clinical three-dimensional ultrasound imaging. Ultrasound Obstet Gynecol 2000;16:374–383.[CrossRef][Medline]
- Hertzberg BS, Kliewer MA, Paulson EK, et al. PACS in sonography: accuracy of interpretation using film compared with monitor display—picture archiving and communication systems. AJR Am J Roentgenol 1999;173:1175–1179.[Abstract/Free Full Text]
- American College of Radiology. Breast imaging reporting and data system (BI-RADS): ultrasound. Reston, Va: American College of Radiology, 2003.
This article has been cited by other articles:
TOP
ABSTRACT
INTRODUCTION
