Radiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Published online before print January 13, 2005, 10.1148/radiol.2343031580
This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
2343031580v1
234/3/693    most recent
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Deurloo, E. E.
Right arrow Articles by Gilhuijs, K. G. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Deurloo, E. E.
Right arrow Articles by Gilhuijs, K. G. A.
(Radiology 2005;234:693-701.)
© RSNA, 2005

Breast Imaging

Clinically and Mammographically Occult Breast Lesions on MR Images: Potential Effect of Computerized Assessment on Clinical Reading1

Eline E. Deurloo, MD, Sara H. Muller, PhD, Johannes L. Peterse, MD, Albert P. E. Besnard, MD and Kenneth G. A. Gilhuijs, PhD

1 From the Departments of Radiology (E.E.D., S.H.M., A.P.E.B., K.G.A.G.) and Pathology (J.L.P.), Netherlands Cancer Institute/Antoni van Leeuwenhoek Hospital, Plesmanlaan 121, 1066 CX Amsterdam, the Netherlands. From the 2002 RSNA Annual Meeting. Received September 29, 2003; revision requested December 10; final revision received June 1, 2004; accepted June 18. Supported in part by Dutch Cancer Society grant NKI 99–2035. Address correspondence to K.G.A.G.


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 REFERENCES
 
PURPOSE: To investigate if and how computerized analysis complements characterization of breast lesions with clinical reading at magnetic resonance imaging.

MATERIALS AND METHODS: The institutional review board approved the use of data obtained prospectively and analyzed either prospectively with informed patient consent or retrospectively with waiver of consent. An existing computerized analysis system was retrained with 100 breast lesions (in 78 patients with mean age of 46.5 years) and tested with 136 other lesions (in 113 patients with mean age of 48.9 years; P = .15 for age difference between groups). Seventy-five lesions in the training set were previously rated by one of three radiologists in daily clinical practice. Lesion rating (as benign, probably benign, indeterminate, suspicious, or highly suggestive of malignancy) and probability of malignancy calculated with computerized analysis were included as covariates in logistic regression analysis to obtain a combined model. The performance of the model was compared with that of clinical reading alone in a set of 72 clinically and mammographically occult lesions not used to train the computerized analysis system (in 60 patients with mean age of 43.5 years; P = .09 for age difference between training and testing groups). Receiver operating characteristic (ROC) curves were plotted, and areas under the ROC curves were calculated and compared.

RESULTS: Performance of reading in the clinical setting, as indicated by area under the ROC curve (Az = 0.86), was similar to that of computerized analysis (Az = 0.85; P = .99). Significant overall improvement in performance was obtained with the combined model (Az = 0.91; P = .03). Improvement was accomplished mostly in characterization of lesions rated indeterminate or suspicious by radiologists.

CONCLUSION: Computerized analysis complements clinical reading and makes computer-aided diagnosis feasible. The complementary information has the potential to increase overall performance for clinically and mammographically occult lesions.

© RSNA, 2005


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS
DISCUSSION
 REFERENCES
 
The main advantage of contrast material–enhanced magnetic resonance (MR) imaging of the breast is its high sensitivity—currently well above 90%—for depiction of invasive carcinoma (1,2). A drawback is its lower specificity, at 37%–90% (1,2). Several methods have been investigated to improve the discrimination between benign and malignant lesions. Lexicons were designed to standardize the rating and reporting of lesions depicted on MR images and to reduce inter- and intraobserver variability (3,4). Pharmacokinetic models were developed with a similar goal (5,6). Further improvement in the objectivity of reading has led to methods for the automated classification of lesions into benign or malignant groups on the basis of features rated by radiologists (7,8). Other investigators have described the use of computerized methods not only to rate lesion features such as the degree of margin irregularity but also to classify lesions automatically into benign or malignant groups (9–11). These investigators reported promising results with regard to specificity and variability between readers. Most studies, however, were performed by using a set of lesions detected at clinical breast examination and/or mammography and also seen at MR imaging. In the majority of cases, such lesions can be characterized with histologic analysis of specimens obtained at needle biopsy.

Lesions detected at MR imaging but not at clinical examination or mammography are more difficult to assess, and they are more frequently detected with the increasing use of breast MR imaging. These lesions are typically referred to as incidental enhancing lesions. They may be additional findings in symptomatic patients (eg, in patients with symptomatic breast cancer or with equivocal mammographic findings) (12–18) or in asymptomatic women who have an increased lifetime risk for breast cancer and are evaluated with MR imaging (19–22). Additional findings at MR imaging are reported in approximately 29% of patients, particularly in young premenopausal women with dense breast parenchyma (2,12,23,24). To obtain a diagnosis of these lesions, two different approaches are typically pursued. First, fine-needle aspiration (FNA) or core biopsy may be performed with ultrasonography (US) used for guidance and with the location of findings at MR imaging taken into account (18,25). If the lesion is not visible at US, core-needle biopsy or needle localization for excisional biopsy may be performed with MR imaging for guidance, which requires the use of a dedicated breast biopsy coil (16–18,26–28). A drawback of these invasive procedures is that a minority of the additional findings—as little as 3% (12)—are malignant. Moreover, lesions detected at MR imaging in asymptomatic women are less often malignant than are additional findings detected in patients with symptomatic breast cancer (2,12,17). This is a point of concern in ongoing investigations of the use of MR imaging for screening of women who have an increased lifetime risk for breast cancer (19–22). In this group of women the cost of biopsy in benign lesions is high (in terms of increased stress to the patient, scarring, and decreased cost-effectiveness), which may be a drawback for the advocacy of MR imaging as a screening technique.

In a previous article, we reported the effectiveness of a fully computerized system for the characterization of breast lesions on MR images without taking into account the radiologist’s reading (11). Other investigators reported that, at x-ray–based mammography, a computerized calculation of the probability of malignancy of mammographic lesions has been offered to radiologists to serve as a second opinion. The decision to accept the interpretation of the computerized analysis system has always been at the discretion of the radiologist. This approach—computer-aided diagnosis—resulted in significant improvement of performance with regard to distinguishing between benign and malignant lesions (29–31), as well as in consistent reduction of reader variability (32).

For computer-aided diagnosis to be useful, it is essential that the computerized analysis provide information that is complementary to that provided by the radiologist’s reading (32–34). Moreover, the radiologist can benefit optimally from computerized analysis only with an understanding of how the system complements clinical reading. The aim of the current study was to investigate if and how computerized analysis complements clinical reading of breast lesions detected at MR imaging.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
DISCUSSION
 REFERENCES
 
The potential effect of computerized analysis on the results of clinical reading of clinically and mammographically occult breast lesions at MR imaging was assessed in four steps. First, an existing system for computerized analysis of breast lesions was updated by means of training with an additional set of lesions and testing with a different set of lesions that was larger than the training set. Second, the performance of clinical reading of MR images was evaluated for characterization of breast lesions. Third, a model was constructed that combined the results of computerized analysis with those of clinical reading. Finally, the performance of the model was assessed with an independent set of 72 clinically and mammographically occult lesions and compared with the performance of clinical reading without computer aid.

Inclusion Criteria
The analyses were performed by using data that were either (a) prospectively obtained after approval of the institutional review board and informed patient consent or (b) acquired for accepted clinical indications and retrospectively analyzed after approval of the institutional review board and waiver of informed consent. All lesions depicted at MR imaging in our clinic between November 9, 2000, and July 5, 2002, were eligible for inclusion. In addition to focal masses, areas of non-mass-related enhancement (eg, linear and segmental enhancement) were eligible. Lesions were consecutively included if they were pathologically proved (with FNA, core biopsy, or excisional biopsy) or showed transient enhancement (ie, contrast enhancement at the initial MR examination but no enhancement at follow-up MR imaging). Lesions that were not pathologically proved were included only if they were areas of transient enhancement. The volume of the lesion had to be smaller than 4 cm3. Lesions in patients who underwent core biopsy prior to MR imaging were excluded.

Lesions in the set of 72 incidental lesions had to be clinically and mammographically occult, in addition to satisfying the previously described criteria, and were consecutively added. Lesions visible at US were included only if US was a correlative examination for a lesion detected at MR imaging.

MR Imaging Technique
MR imaging was performed with a 1.5-T imager (Magnetom; Siemens, Erlangen, Germany). A dedicated phased-array bilateral breast coil (Siemens, Erlangen, Germany) was used. Our standard clinical protocol for MR imaging of the breast (a coronal three-dimensional fast low-angle shot sequence) was used. Images were acquired with the patient in the prone position and with both breasts imaged simultaneously. One of two standardized protocols was used: The first protocol includes an isotropic in-plane resolution of 1.35 x 1.35 mm and a section thickness of 1.35 mm. After acquisition of unenhanced images, four contrast-enhanced image series (120 seconds per series) were obtained with this protocol. The second protocol includes an isotropic in-plane resolution of 1.21 x 1.21 mm and a section thickness of 1.69 mm. After acquisition of unenhanced images, five contrast-enhanced image series (90 seconds per series) were obtained with this protocol. With both protocols, the contrast-enhanced images are obtained after the intravenous administration of gadoteridol (ProHance; Bracco–Byk Gulden, Konstanz, Germany) at a dose of 0.1 mmol/kg and a rate of 2–4 mL/sec by using a power injector (Spectris; Medrad, Indianola, Pa). The following parameters are used for both protocols: T1 weighting, 8.1/4.0 (repetition time msec/echo time msec), reconstructed in-plane matrix of 256 x 256 pixels, and no fat suppression. Subtraction images were created on a pixel-by-pixel basis for evaluation of the early and late enhancement phases in lesions.

Computerized Analysis of Breast Lesions on MR Images
A previously reported system (11,35) was used to perform the computerized analysis. In short, a point in the lesion detected by the radiologist is first designated manually on the image. The system then automatically shifts the designated point to the center of the contrast-enhanced area and automatically segments (ie, delineates) the lesion in three dimensions. The resulting segmentation is visually verified. When necessary, the location of the indicated point can be adjusted for better coverage. After segmentation, six morphologic and three temporal features in and around the segmented lesion are automatically rated. The computerized analysis system was originally trained with 80 lesions and validated with the same set of lesions by using cross validation (11). A subset of four features was found optimal for calculation of the probability of malignancy (11): washout, smoothness of contrast material uptake (irregularity of uptake pattern), mean margin sharpness, and variation in margin sharpness. Typically, low values for smoothness of uptake are found in lesions with inhomogeneous uptake and spiculated boundaries. Lesions with well-defined margins have high mean values of margin sharpness, and lesions with partially well-defined margins have high values of variation in margin sharpness.

In the current study, the system was updated with training and test data that included only lesions that were subjected to biopsy and pathologic analysis and that corresponded to areas of transient contrast enhancement on MR images. In addition, the system was tested with a data set separate from, and independent of, the training data set. The lesions were detected during clinical work-up and were consecutively added to the data set for updating of the computerized analysis system. The segmentation of the included lesions was performed retrospectively by one of the authors (E.E.D.) and was based on the radiologists’ reports.

Two hundred thirty-six consecutive lesions (symptomatic or incidental) in 189 patients were available for the update. The system was trained with the first 50 benign lesions and the first 50 malignant lesions (44 symptomatic and 56 incidental), which occurred in 78 patients (mean age, 46.5 years; range, 20–84 years). For this purpose, logistic regression analysis was performed with backward selection of features (threshold probability of feature removal [F] = 0.10) (36). All nine features calculated by the computerized analysis system were included in the backward feature selection. The system was tested with the remaining 136 lesions (52 benign and 84 malignant; 64 symptomatic and 72 incidental). These lesions occurred in 113 patients (mean age, 48.9 years; range, 26–86 years). There was no statistically significant difference in age distribution between the group of patients with lesions in the training set and the group of patients with lesions in the test set (Student t test, P = .15).

The performance of the system also was evaluated with regard to characterization of ductal carcinoma in situ (DCIS) in the test set. This was done because a good performance is required with regard to characterization not only of invasive lesions but also of in situ cancers, if the system is to be useful in a screening setting.

Clinical Reading of Breast MR Images
In our hospital, interpretation of breast MR images is done with hard copy or by using a viewing station that permits simultaneous viewing of two linked series in three orthogonal directions (11). The viewing station monitor displays all acquired image series (unenhanced and contrast-enhanced), as well as all subtraction images in each series. Clinical information such as age, clinical history, and indication for MR imaging is available at the time of reading. The lesions are rated according to a five-point scale by using morphologic and temporal descriptors (Tables 1, 2). The descriptive ratings include "benign," "probably benign," "indeterminate," "suspicious," and "highly suggestive of malignancy" (37,38). According to clinical guidelines employed in our clinic, patients with lesions rated indeterminate, suspicious, or highly suggestive of malignancy are always referred for US-guided FNA or core biopsy. US for lesions rated benign or probably benign is performed at the discretion of the radiologist. If the lesion is not visible at US, the patient typically undergoes short-term follow-up MR imaging 3–6 months later.


View this table:
[in this window]
[in a new window]

  		TABLE 1. Morphologic and Temporal Features Used in Clinical Interpretation of Breast MR Images

 

View this table:
[in this window]
[in a new window]

  		TABLE 2. Rating Categories Based on Morphologic and Temporal Features from Table 1

 
The clinical reading described in the current study was part of the clinical work-up of patients. Each lesion was read by one of three radiologists, one of whom is an author (A.P.E.B.). Each radiologist used the same rating guidelines. Two radiologists had each read more than 500 breast MR image series in the previous 5 years; the third radiologist had read more than 200 breast MR image series in the previous 2 years. The results of clinical reading were obtained retrospectively from the reports.

Combined Model of Computerized Analysis and Clinical Reading
Clinical reading results were available for 75 of the 100 lesions in the training set used to update the computerized analysis system. For the remaining 25 lesions no rating was available, because these lesions were cancers that were proved at pathologic analysis prior to MR imaging. The 75 rated lesions (50 benign and 25 malignant, 19 symptomatic and 56 incidental) were verified at pathologic analysis or were areas of transient enhancement. In this subset of lesions, logistic regression analysis with backward selection of features (F = 0.10) was used to investigate if, how, and by how much computerized analysis complemented clinical reading. The covariates in this regression analysis were the rating category to which the lesion was assigned by the radiologists in clinical reading and the probability of malignancy calculated in the computerized analysis. The output of the model was an updated probability of malignancy of the lesion. We refer to this model as the combined model. The effect of the model was visualized by plotting a response graph that shows the combined effect of the rating by the radiologists and the output of the computerized analysis system on the final result of the combined model.

Assessment of the Combined Model
The performance of the combined model was compared with the performance of the radiologists alone for all lesions and for DCIS only. For this purpose, a set of consecutive incidental lesions was used that was not employed either in the training of the computerized system or in the construction of the combined model. Seventy-two lesions in 60 patients were available for this assessment. Of the 72 lesions, 20 were malignant (all proved at pathologic analysis) and 52 were benign (either proved at pathologic analysis [n = 21] or exhibiting transient enhancement [n = 31]) (Table 3). Fifty patients had one lesion each, eight patients had two lesions each, and two patients had three lesions each. Indications for MR imaging of the breast included the following: screening in women with increased lifetime risk (>15%) for breast cancer (45 patients, 51 lesions), assessment of tumor extent prior to breast-conserving therapy (13 patients, 18 additional lesions), problem solving (one patient, two additional lesions), and planning of neoadjuvant chemotherapy (one patient, one additional lesion). The mean age of patients was 43.5 years (range, 26–64 years). No significant difference was found between the mean age of the patients with the 72 incidental lesions and the mean age of those with lesions in the training set for the computerized analysis system, although there was a trend toward lower age in patients with incidental lesions (Student t test, P = .09).


View this table:
[in this window]
[in a new window]

  		TABLE 3. Findings Characterized as Benign or Malignant

 
All clinical decisions were based on the readings by the radiologists; the results of the computerized analysis and of analysis with the combined model had no influence on clinical decision making. For the group of lesions detected in patients referred for correlative US (n = 59), the performances of clinical reading, computerized analysis, and the combined model were compared separately for lesions that were visible at US after MR imaging (ie, retrospectively visible; n = 34) and for those that were not (n = 25). This analysis was carried out because computerized analysis is expected to be most useful in clinical application when the lesion was not visible at US performed for correlation with MR imaging.

Statistical Analysis
Logistic regression analysis was performed by using statistical software (SPSS, version 10.0; SPSS, Chicago, Ill). Multiple lesions in the same breast were considered as independent lesions. Performance at the initial reading in the clinical setting, with computerized analysis, and with the combined model was quantitatively evaluated by using receiver operating characteristic (ROC) analysis (39). For the ROC analysis of clinical reading, the lesion ratings by all readers were pooled to obtain a summary performance curve for our clinic; this method of analysis is similar to the study design used by Stoutjesdijk et al (20) and is comparable with the hybrid study design proposed by Obuchowski (40). Note that in the ROC analysis, sensitivity and specificity were calculated on the basis of the rating and irrespective of referral for FNA or core biopsy. Therefore, the ROC curve was obtained independently of the operating points of the different radiologists: All radiologists used the same rating guidelines. Other software programs were used for the continuous data input (probability of malignancy calculated with the computerized analysis and with the combined model) (PROPROC and LABROC1; Kurt Rossmann Laboratories for Radiologic Image Research, University of Chicago, Chicago, Ill) and for the categorical data input (lesion ratings by the radiologists) (ROCFIT; Kurt Rossmann Laboratories for Radiologic Image Research, University of Chicago). A comparison of areas under the ROC curves was performed by using the method described by DeLong et al (41). The Student t test was used to compare patient age between the three groups of patients with lesions in the different sets (training set, test set, and set of incidental lesions). A P value of less than .05 was considered to indicate a statistically significant difference.


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 REFERENCES
 
Computerized Analysis of Breast Lesions on MR Images
Backward feature selection with the computerized analysis system yielded the same four features identified in the previous study as contributors to probability of malignancy (11): washout, smoothness of uptake (maximum across subtractions), mean margin sharpness (first subtraction), and variation in margin sharpness (maximum across subtractions). In the training set, the area under the ROC curve (Az) was 0.93 ± 0.03 (standard deviation). In the test set, the Az value was 0.91 ± 0.02. No significant difference was found in the performance of the system in both sets (P = .17), which indicates good ability of the system to generalize from the training cases. Seven cases of DCIS were in the test set of the computerized analysis system. All seven were correctly classified by the system.

Clinical Reading of Breast MR Images
Most lesions that were rated benign or probably benign were indeed benign (Fig 1). Only one (3%) of 29 lesions in these two rating categories was found to be malignant. Most lesions that were rated highly suggestive of malignancy were indeed malignant. Eight (89%) of nine lesions in this rating category were found to be malignant. These results show that lesions with a high probability of being benign and those that were highly suggestive of malignancy were accurately identified in clinical reading. Conversely, only six (26%) of 23 lesions rated indeterminate were malignant, and only five (45%) of 11 lesions rated suspicious were malignant. Moreover, nearly half (34 [47%] of 72) of all lesions were assigned to one of these two rating categories. Consequently, FNA or biopsy was performed in many benign lesions. These results confirm the hypothesis that clinical reading needs help with regard to lesions rated indeterminate and suspicious.



View larger version (21K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 1. Bar graph shows distribution of 72 clinically and mammographically occult lesions among the five rating categories as total number of lesions (white bars) and number of malignant lesions (black bars).

 
Combined Model of Computerized Analysis and Clinical Reading
Both the rating assigned by the radiologists in clinical reading and the probability of malignancy calculated with computerized analysis were selected by means of logistic regression analysis as significant covariates for inclusion in the combined model. The combined model is weighted toward the radiologist’s interpretation with regard to lesions rated benign, probably benign, or highly suggestive of malignancy but leans toward the result of computerized analysis for lesions rated indeterminate or suspicious (Fig 2). These results indicate that computerized analysis complements clinical reading in the characterization of lesions rated indeterminate or suspicious by the radiologist. A lesion rated probably benign will have a low probability of malignancy according to the result of analysis with the combined model (Fig 2, vertical axis), irrespective of the result of computerized analysis (Fig 2, horizontal axis). A lesion rated indeterminate, by contrast, will have a high probability of malignancy if the result of computerized analysis indicates a high probability, and a low probability of malignancy if the result of computerized analysis indicates a low probability.



View larger version (42K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 2. Graph shows effect and weight of covariates in logistic regression analysis. Probability of malignancy of lesions in each of five rating categories, according to results of computerized analysis, is shown on the horizontal axis; probability of lesion malignancy according to results of analysis with the combined model is shown on the vertical axis. Each curve represents a rating category.

 
Assessment of the Combined Model
The Az value for computerized analysis performed in the set of 72 incidental enhancing lesions was 0.85 ± 0.05 (Fig 3). The Az value for clinical reading in this set of lesions was 0.86 ± 0.05 (Fig 3). No significant difference was found between the performance of clinical reading and the performance of the computerized analysis system (P = .99). The Az value for the combined model used in the set of 72 incidental lesions was 0.91 ± 0.03 (Fig 3). The use of the combined model, thus, resulted in an overall performance that was significantly better than that achieved with clinical reading without computerized analysis (P = .03). Given the limited number of cases (72 lesions, 20 malignancies), it was possible to show significant improvement in performance only over a range of sensitivity and specificity values (indicated by area under the ROC curve). At 95% sensitivity, there is potential for a 20% increase in specificity (Fig 3).



View larger version (24K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 3. ROC curves show performance with clinical reading (dashed-dotted line; Az = 0.86), computerized analysis (dotted line; Az = 0.85), and combined model (solid line; Az = 0.91) in the set of 72 clinically and mammographically occult (52 benign, 20 malignant) lesions. TPF = true-positive fraction, FPF = false-positive fraction.

 
There was one case of DCIS in the set of 72 incidental enhancing lesions. This lesion was correctly classified by means of the combined model.

Figure 4 shows an example of an incidental lesion detected at MR imaging for screening in a woman with an increased lifetime risk for breast cancer.



View larger version (123K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 4a. Transverse MR images of the right breast of a woman who underwent MR imaging for screening because of an increased lifetime risk. (a) Unenhanced, (b) early contrast-enhanced, (c) wash-in (subtraction), and (d) washout (subtraction) images show incidental enhancing lesion of 12 mm, rated indeterminate by the radiologist. Images at US performed for correlation showed no abnormalities. At follow-up MR imaging, the lesion was no longer visible. The combined model calculated a 9.7% probability of malignancy.

 


View larger version (123K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 4b. Transverse MR images of the right breast of a woman who underwent MR imaging for screening because of an increased lifetime risk. (a) Unenhanced, (b) early contrast-enhanced, (c) wash-in (subtraction), and (d) washout (subtraction) images show incidental enhancing lesion of 12 mm, rated indeterminate by the radiologist. Images at US performed for correlation showed no abnormalities. At follow-up MR imaging, the lesion was no longer visible. The combined model calculated a 9.7% probability of malignancy.

 


View larger version (96K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 4c. Transverse MR images of the right breast of a woman who underwent MR imaging for screening because of an increased lifetime risk. (a) Unenhanced, (b) early contrast-enhanced, (c) wash-in (subtraction), and (d) washout (subtraction) images show incidental enhancing lesion of 12 mm, rated indeterminate by the radiologist. Images at US performed for correlation showed no abnormalities. At follow-up MR imaging, the lesion was no longer visible. The combined model calculated a 9.7% probability of malignancy.

 


View larger version (95K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 4d. Transverse MR images of the right breast of a woman who underwent MR imaging for screening because of an increased lifetime risk. (a) Unenhanced, (b) early contrast-enhanced, (c) wash-in (subtraction), and (d) washout (subtraction) images show incidental enhancing lesion of 12 mm, rated indeterminate by the radiologist. Images at US performed for correlation showed no abnormalities. At follow-up MR imaging, the lesion was no longer visible. The combined model calculated a 9.7% probability of malignancy.

 
In each rating category, approximately half (34 [58%] of 59) of the lesions in patients referred for US were retrospectively visible at US. Among lesions rated indeterminate or suspicious, eight (40%) of 20 that were visible at US were malignant. The results of clinical reading, computerized analysis, and analysis with the combined model showed a trend toward better performance for all three modes of analysis among lesions that were not visible at US (Az = 0.89, 0.93, and 0.95, respectively), compared with performance among lesions that were retrospectively visible (Az = 0.77, 0.83, and 0.87, respectively). The statistical significance of these differences, however, was not demonstrated (P = .08, .06, and .07, respectively).


   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
REFERENCES
 
The results of this study show that computerized analysis complements clinical reading and that the combination of the two has the potential to improve overall performance in characterization of incidental breast lesions at MR imaging.

Inclusion Criteria
Our assessment of the combined model was focused on clinically and mammographically occult breast lesions at MR imaging because we expected computerized analysis to have the greatest effect on performance for this group of lesions. Lesions had to be incidental (not palpable and not mammographically visible) prior to MR imaging, but lesions were not excluded if they were retrospectively visible at US (guided by MR images). On the one hand, application of computerized analysis will be most useful if the lesion is also retrospectively occult at US. On the other hand, retrospective visibility at US and subsequent FNA or core biopsy provided the proof of malignancy for most malignant tumors in our study, proof that was necessary for accurate training and testing of the computerized analysis system. Moreover, no significant difference in performance could be demonstrated for clinical reading, computerized analysis, or the combined model between the group of lesions that were retrospectively visible at US and the group of lesions that were not. A trend toward better performance was observed, however, among lesions that were not visible at US. These are the lesions that currently are most difficult to diagnose at radiologic imaging in the clinical setting, and MR imaging–guided biopsy and pathologic analysis are the only option for diagnosing these lesions.

Computerized Analysis of Breast Lesions on MR Images
A previously described computerized analysis system was updated in this study to include only pathologically proved lesions and areas of transient enhancement in the analysis (11). The area under the ROC curve, an indicator of the performance of the previously described system, was 0.95. Application of the previous system to the current test set yielded an area under the ROC curve of 0.91, identical to that of the current system for this test set. Moreover, the same four features were selected in both studies. Consequently, the performance of the computerized analysis system did not change substantially after updating with only pathologically proved lesions and areas of transient enhancement. At present, all lesions that are detected by our radiologists in daily clinical practice are entered into the database of the computerized analysis system. Consequently, updating of the system is an ongoing process.

The performance level of the computerized analysis system for the set of incidental lesions was somewhat lower (Az = 0.85) than that for the test set, the combined set of symptomatic and incidental lesions (Az = 0.91). Perhaps slightly different features in symptomatic and incidental lesions contributed to this difference in performance. Training explicitly with incidental enhancing lesions alone was not attempted because their number was insufficient for training and testing of the system. Training of the system exclusively with these lesions is a subject for further research.

For the screening setting, a system is required that is able to correctly diagnose DCIS. DCIS was included in the current study, but only a limited number of DCIS lesions were found. No difference could be demonstrated in the performance of the computerized analysis system for the seven cases of DCIS that were in the test set compared with the cases of invasive tumors. In fact, all seven DCIS cases were correctly classified by the system. The performance of the system in a larger set of DCIS cases remains a subject for further research.

Clinical Reading of Breast MR Images
Only 26% of incidental lesions rated indeterminate and 45% of those rated suspicious at clinical reading were found to be malignant in this study. Other investigators (16,18) also reported that less than half (29% and 19%, respectively) of lesions that were rated suspicious actually were malignant. The difficulties in distinguishing between benign and malignant lesions in these rating categories may be caused by an overlap between features, such as that reported for signal intensity (16,42,43) and that reported for morphologic characteristics (44,45). It appears, however, that the overlap in features with evaluation by radiologists at clinical reading differs from that at computerized analysis. The performance of clinical reading and of computerized analysis for the set of 72 incidental lesions was identical (Az = 0.86 and 0.85, respectively), but that of their combination (the combined model) was significantly better.

Indistinct lesions with conflicting characteristics were rated indeterminate in the current study. It is preferred that the number of lesions in this rating category be small, especially in a screening setting, because such lesions require additional work-up although only a minority of them are malignant (26% in the current study). The combined model may reduce the uncertainty of indeterminate findings, thus reducing the number of patients with benign lesions who are referred for US. If the indeterminate rating category is eliminated, then a decision will have to be made between the probably benign and suspicious rating categories, which will result in a redistribution of indeterminate lesions. As a consequence, the combined model would have to be retuned to the adjusted rating guidelines. It is likely that the computerized analysis component will then have a larger effect on the characterization of lesions rated probably benign than is currently the case.

Recently, a new Breast Imaging Reporting and Data System (BI-RADS) lexicon was introduced for standardized reporting of the results of MR imaging of the breast (46). During the study period, this lexicon was not yet available. Therefore, non–BI-RADS descriptors were used in the current study. The rating system used does, however, closely follow the BI-RADS classification system; our rating categories "benign," "probably benign," "suspicious," and "highly suggestive of malignancy" correspond to BI-RADS categories 2, 3, 4, and 5, respectively. The BI-RADS system does not include the category "indeterminate," but it does include a category of 0 ("needs additional imaging evaluation"). If the official BI-RADS lexicon had been available during our study period, it seems likely that many of the cases rated indeterminate would have been classified in BI-RADS category 0 (more information required for interpretation). To the best of our knowledge, none of the other investigators who used BI-RADS-like descriptors reported the frequency of lesions rated as BI-RADS category 0 (18,20,22). We deliberately included lesions that required more information for interpretation, because we believe that especially in this group of lesions a reduction in additional work-up is achievable. The results of our analysis show that the use of the combined model may enable the achievement of a reduction in the number of false-positive findings—and especially in the false-positive rates among indeterminate and suspicious findings—because the results of computerized analysis complement those of clinical reading.

Combined Model of Computerized Analysis and Clinical Reading
The typical definition of computer-aided diagnosis is diagnosis performed by a radiologist who uses the results of computerized analysis as a second opinion. Consequently, the results of computerized analysis may or may not change the judgment of the radiologist. The aim of this study was not to determine how often radiologists would be influenced by the results of the computerized analysis but, rather, to provide guidelines to radiologists for assessing the relevance of the results of computerized analysis and to enable the achievement of optimal benefit from computer-aided diagnosis in the future. In other words, we investigated whether the radiologist has any chance of improving his or her performance on the basis of the results of the computerized analysis, and if so, then how and by how much. The results of our test of the combined model provide this information: If radiologists always adhere to their initial judgment (Az = 0.86) or always rely on computerized analysis (Az = 0.85), their overall performance will be comparable but inferior to that possible if they favor only the results of computerized analysis for lesions considered indeterminate or suspicious (Az = 0.91). Although these results are promising, the specificity of radiologic interpretation with the combined model is not as high as that of pathologic analysis of specimens obtained at FNA or biopsy. Clinical application of computerized analysis can therefore not be expected to replace FNA or biopsy. In some situations, however, FNA or biopsy is not possible, is very difficult to perform (eg, for small lesions visible only at MR imaging), or is not desirable as part of the work-up (eg, for lesions detected in women at screening performed because of an increased lifetime risk of breast cancer). In these situations, application of computerized analysis may be of use to increase specificity without compromising current high sensitivity. These situations may be indications for future clinical applications of computerized analysis.

Study Limitations
In the current study, to determine the performance of clinical reading, a hybrid design was used instead of the classic multiple-reader–multiple-case study design, in which multiple readers each read the same sets of images in laboratory conditions. A point of concern in studies with this classic design is that the results may not be easily generalized to the clinical setting; radiologists may act differently in a study environment than in a clinical environment (47). For example, in a study by Rutter and Taplin (48), no evidence of correlation was found between the performance of mammographers in a laboratory experiment and their performance in their own clinical practice. Conversely, a limitation of the hybrid design used in our current study—where multiple readers each read a different set of images under clinical conditions—is that inter- and intraobserver variability cannot be easily assessed. In addition, the results will be meaningful to other centers and radiologists only if standardized reading guidelines are in use and explicitly described.

A medical researcher performed the manual part of the semiautomated segmentation for the computerized analysis, rather than the reading radiologist. In clinical use of the computerized analysis system, the reading radiologist eventually will perform the segmentation. The effect of variations in selection of a point in the lesion on the performance of the system is, however, expected to be small. In a previous study, two operators independently performed the segmentation (11). No significant difference in probability of malignancy was shown between operators in a set of 80 lesions (P > .6). Furthermore, at the selected operating point, no significant difference was found in true-positive fraction between the two operators.

Although computerized analysis improves the performance of the clinical reading for nearly half of all clinically and mammographically occult lesions, the interpretation by the radiologists may add subjectivity to the system. The interobserver variability is expected to be highest for lesions that are rated indeterminate or suspicious. It is in these categories, however, that computerized analysis contributes the most to the combined model. Because the interobserver variability of computerized analysis is small (11), it is expected that interobserver variability resulting from use of the combined model will be smaller than that with traditional reading by radiologists without computer aid. This assumption, however, needs to be validated in future studies.

Two different MR imaging protocols were used in the current study because both were in use in clinical practice in our hospital; the difference in acquisition time between the two protocols is small. If reading guidelines, patient populations, and MR imaging techniques differ substantially at other centers, the combined model may have to be retuned to allow computerized analysis to best complement the radiologist’s reading. Such adjustments may be made automatically in the software. Analyses are currently in progress to test the robustness of the system with such variations in acquisition parameters.

It should be noted that the current assessment of the combined model involved testing with a limited number of incidental enhancing lesions (52 benign, 20 malignant). Nevertheless, a significant difference in overall performance was demonstrated. Analyses with larger numbers of incidental lesions are necessary, however, to show significant improvement in specificity at any given sensitivity and to further support our findings.


   ACKNOWLEDGMENTS
 
The authors thank radiologists Elisabeth Joekes, MD, and Wim Koops, MD, for participating in this study; Guus Hart, MSc, for statistical assistance; Angelique Schlief for assistance with data extraction; and Marja van Vliet for assistance with the preparation of illustrations.


   FOOTNOTES
 
Abbreviations: Az = area under ROC curve, BI-RADS = Breast Imaging Reporting and Data System, DCIS = ductal carcinoma in situ, FNA = fine-needle aspiration, ROC = receiver operating characteristic

Authors stated no financial relationship to disclose.

Author contributions: Guarantors of integrity of entire study, E.E.D., K.G.A.G.; study concepts and design, E.E.D., S.H.M., K.G.A.G.; literature research, E.E.D., K.G.A.G.; clinical studies, E.E.D., K.G.A.G.; experimental studies, S.H.M., K.G.A.G.; data acquisition, E.E.D., K.G.A.G.; data analysis/interpretation, E.E.D., J.L.P., A.P.E.B., K.G.A.G.; statistical analysis, E.E.D., K.G.A.G.; manuscript preparation, E.E.D., K.G.A.G.; manuscript editing and revision/review, E.E.D., S.H.M., K.G.A.G.; manuscript definition of intellectual content and final version approval, all authors


   REFERENCES
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 REFERENCES
 

  1. Heywang-Kobrunner SH, Viehweg P, Heinig A, Kuchler C. Contrast-enhanced MRI of the breast: accuracy, value, controversies, solutions. Eur J Radiol 1997; 24:94-108.[CrossRef][Medline]
  2. Viehweg P, Paprosch I, Strassinopoulou M, Heywang-Kobrunner SH. Contrast-enhanced magnetic resonance imaging of the breast: interpretation guidelines. Top Magn Reson Imaging 1998; 9:17-43.[Medline]
  3. Ikeda DM, Hylton NM, Kinkel K, et al. Development, standardization, and testing of a lexicon for reporting contrast-enhanced breast magnetic resonance imaging studies. J Magn Reson Imaging 2001; 13:889-895.[CrossRef][Medline]
  4. Fischer U, Kopka L, Grabbe E. Breast carcinoma: effect of preoperative contrast-enhanced MR imaging on the therapeutic approach. Radiology 1999; 213:881-888.[Abstract/Free Full Text]
  5. Taylor JS, Tofts PS, Port R, et al. MR imaging of tumor microcirculation: promise for the new millennium. J Magn Reson Imaging 1999; 10:903-907.[CrossRef][Medline]
  6. den Boer JA, Hoenderop RK, Smink J, et al. Pharmacokinetic analysis of Gd-DTPA enhancement in dynamic three-dimensional MRI of breast lesions. J Magn Reson Imaging 1997; 7:702-715.[Medline]
  7. Nunes LW, Schnall MD, Orel SG. Update of breast MR imaging architectural interpretation model. Radiology 2001; 219:484-494.[Abstract/Free Full Text]
  8. Kinkel K, Helbich TH, Esserman LJ, et al. Dynamic high-spatial-resolution MR imaging of suspicious breast lesions: diagnostic criteria and interobserver variability. AJR Am J Roentgenol 2000; 175:35-43.[Abstract/Free Full Text]
  9. Penn AI, Bolinger L, Schnall MD, Loew MH. Discrimination of MR images of breast masses with fractal-interpolation function models. Acad Radiol 1999; 6:156-163.[CrossRef][Medline]
  10. Sinha S, Lucas-Quesada FA, DeBruhl ND, et al. Multifeature analysis of Gd-enhanced MR images of breast lesions. J Magn Reson Imaging 1997; 7:1016-1026.[Medline]
  11. Gilhuijs KG, Deurloo EE, Muller SH, Peterse JL, Schultze Kool LJ. Breast MR imaging in women at increased lifetime risk of breast cancer: clinical system for computerized assessment of breast lesions—initial results. Radiology 2002; 225:907-916.[Abstract/Free Full Text]
  12. Brown J, Smith RC, Lee CH. Incidental enhancing lesions found on MR imaging of the breast. AJR Am J Roentgenol 2001; 176:1249-1254.[Abstract/Free Full Text]
  13. Liberman L, Morris EA, Dershaw DD, Abramson AF, Tan LK. MR imaging of the ipsilateral breast in women with percutaneously proven breast cancer. AJR Am J Roentgenol 2003; 180:901-910.[Abstract/Free Full Text]
  14. Liberman L, Morris EA, Kim CM, et al. MR imaging findings in the contralateral breast of women with recently diagnosed breast cancer. AJR Am J Roentgenol 2003; 180:333-341.[Abstract/Free Full Text]
  15. Lee SG, Orel SG, Woo IJ, et al. MR imaging screening of the contralateral breast in patients with newly diagnosed breast cancer: preliminary results. Radiology 2003; 226:773-778.[Abstract/Free Full Text]
  16. Siegmann KC, Muller-Schimpfle M, Schick F, et al. MR imaging-detected breast lesions: histopathologic correlation of lesion characteristics and signal intensity data. AJR Am J Roentgenol 2002; 178:1403-1409.[Abstract/Free Full Text]
  17. Morris EA, Liberman L, Dershaw DD, et al. Preoperative MR imaging-guided needle localization of breast lesions. AJR Am J Roentgenol 2002; 178:1211-1220.[Abstract/Free Full Text]
  18. Liberman L, Morris EA, Lee MJ, et al. Breast lesions detected on MR imaging: features and positive predictive value. AJR Am J Roentgenol 2002; 179:171-178.[Abstract/Free Full Text]
  19. Warner E, Plewes DB, Shumak RS, et al. Comparison of breast magnetic resonance imaging, mammography, and ultrasound for surveillance of women at high risk for hereditary breast cancer. J Clin Oncol 2001; 19:3524-3531.[Abstract/Free Full Text]
  20. Stoutjesdijk MJ, Boetes C, Jager GJ, et al. Magnetic resonance imaging and mammography in women with a hereditary risk of breast cancer. J Natl Cancer Inst 2001; 93:1095-1102.[Abstract/Free Full Text]
  21. Kuhl CK, Schmutzler RK, Leutner CC, et al. Breast MR imaging screening in 192 women proved or suspected to be carriers of a breast cancer susceptibility gene: preliminary results. Radiology 2000; 215:267-279.[Abstract/Free Full Text]
  22. Morris EA, Liberman L, Ballon DJ, et al. MRI of occult breast carcinoma in a high-risk population. AJR Am J Roentgenol 2003; 181:619-626.[Abstract/Free Full Text]
  23. Kuhl CK, Bieling HB, Gieseke J, et al. Healthy premenopausal breast parenchyma in dynamic contrast-enhanced MR imaging of the breast: normal contrast medium enhancement and cyclical-phase dependency. Radiology 1997; 203:137-144.[Abstract/Free Full Text]
  24. Muller-Schimpfle M, Ohmenhauser K, Stoll P, Dietz K, Claussen CD. Menstrual cycle and age: influence on parenchymal contrast medium enhancement in MR imaging of the breast. Radiology 1997; 203:145-149.[Abstract/Free Full Text]
  25. LaTrenta LR, Menell JH, Morris EA, Abramson AF, Dershaw DD, Liberman L. Breast lesions detected with MR imaging: utility and histopathologic importance of identification with US. Radiology 2003; 227:856-861.[Abstract/Free Full Text]
  26. Smith LF, Henry-Tillman R, Mancino AT, et al. Magnetic resonance imaging-guided core needle biopsy and needle localized excision of occult breast lesions. Am J Surg 2001; 182:414-418.[CrossRef][Medline]
  27. Kuhl CK, Morakkabati N, Leutner CC, Schmiedel A, Wardelmann E, Schild HH. MR imaging-guided large-core (14-gauge) needle biopsy of small lesions visible at breast MR imaging alone. Radiology 2001; 220:31-39.[Abstract/Free Full Text]
  28. Bedrosian I, Schlencker J, Spitz FR, et al. Magnetic resonance imaging-guided biopsy of mammographically and clinically occult breast lesions. Ann Surg Oncol 2002; 9:457-461.[Abstract/Free Full Text]
  29. Huo Z, Giger ML, Vyborny CJ, Metz CE. Breast cancer: effectiveness of computer-aided diagnosis observer study with independent database of mammograms. Radiology 2002; 224:560-568.[Abstract/Free Full Text]
  30. Chan HP, Sahiner B, Helvie MA, et al. Improvement of radiologists’ characterization of mammographic masses by using computer-aided diagnosis: an ROC study. Radiology 1999; 212:817-827.[Abstract/Free Full Text]
  31. Jiang Y, Nishikawa RM, Schmidt RA, Metz CE, Giger ML, Doi K. Improving breast cancer diagnosis with computer-aided diagnosis. Acad Radiol 1999; 6:22-33.[CrossRef][Medline]
  32. Jiang Y, Nishikawa RM, Schmidt RA, Toledano AY, Doi K. Potential of computer-aided diagnosis to reduce variability in radiologists’ interpretations of mammograms depicting microcalcifications. Radiology 2001; 220:787-794.[Abstract/Free Full Text]
  33. Doi K, MacMahon H, Katsuragawa S, Nishikawa RM, Jiang Y. Computer-aided diagnosis in radiology: potential and pitfalls. Eur J Radiol 1999; 31:97-109.[CrossRef][Medline]
  34. Leichter I, Buchbinder S, Bamberger P, Novak B, Fields S, Lederman R. Quantitative characterization of mass lesions on digitized mammograms for computer-assisted diagnosis. Invest Radiol 2000; 35:366-372.[CrossRef][Medline]
  35. Gilhuijs KG, Giger ML, Bick U. Computerized analysis of breast lesions in three dimensions using dynamic magnetic-resonance imaging. Med Phys 1998; 25:1647-1654.[CrossRef][Medline]
  36. Altman DG. Relation between several variables. Practical statistics for medical research Boca Raton, Fla: Chapman & Hall/CRC, 1990; 325-364.
  37. Kriege M, Brekelmans CT, Boetes C, et al. MRI screening for breast cancer in women with high familial risk (abstr). Eur J Cancer 2002; 38(suppl 3):S51.
  38. Schnall MD, Ikeda DM. Lesion Diagnosis Working Group report. J Magn Reson Imaging 1999; 10:982-990.[CrossRef][Medline]
  39. Metz CE. ROC methodology in radiologic imaging. Invest Radiol 1986; 21:720-733.[Medline]
  40. Obuchowski NA. Multireader receiver operating characteristic studies: a comparison of study designs. Acad Radiol 1995; 2:709-716.[CrossRef][Medline]
  41. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44:837-845.[CrossRef][Medline]
  42. Stomper PC, Herman S, Klippenstein DL, et al. Suspect breast lesions: findings at dynamic gadolinium-enhanced MR imaging correlated with mammographic and pathologic features. Radiology 1995; 197:387-395.[Abstract/Free Full Text]
  43. Orel SG, Schnall MD, LiVolsi VA, Troupin RH. Suspicious breast lesions: MR imaging with radiologic-pathologic correlation. Radiology 1994; 190:485-493.[Abstract/Free Full Text]
  44. Heywang-Kobrunner SH, Bick U, Bradley WG, Jr, et al. International investigation of breast MRI: results of a multicentre study (11 sites) concerning diagnostic parameters for contrast-enhanced MRI based on 519 histopathologically correlated lesions. Eur Radiol 2001; 11:531-546.[CrossRef][Medline]
  45. Nunes LW, Schnall MD, Orel SG, et al. Breast MR imaging: interpretation model. Radiology 1997; 202:833-841.[Abstract/Free Full Text]
  46. American College of Radiology. Breast Imaging Reporting and Data System Atlas (BI-RADS Atlas) Reston, Va: American College of Radiology, 2003.
  47. Gur D. ROC-type assessments of medical imaging and CAD technologies: a perspective. Acad Radiol 2003; 10:402-403.[CrossRef][Medline]
  48. Rutter CM, Taplin S. Asssessing mammographers’ accuracy: a comparison of clinical and test performance. J Clin Epidemiol 2000; 53:443-450.[CrossRef][Medline]



This article has been cited by other articles:


Home page
 radtechHome page
T. G. ODLE
Breast MR
Radiol. Technol., September 1, 2006; 78(1): 45M - 66M.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
2343031580v1
234/3/693    most recent
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Deurloo, E. E.
Right arrow Articles by Gilhuijs, K. G. A.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Deurloo, E. E.
Right arrow Articles by Gilhuijs, K. G. A.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
RADIOLOGY RADIOGRAPHICS RSNA JOURNALS ONLINE