| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Breast Imaging |
1 From the Department of Radiology, Albert Einstein College of Medicine, Montefiore Medical Center, Bronx, NY (S.S.B., G.Y.); School of Engineering, Jerusalem College of Technology, Israel (I.S.L., B.N., P.N.B.); and Department of Radiology, Hadassah University Hospital, Jerusalem, Israel (R.B.L., M.S.L., S.I.F.). From the 2002 RSNA scientific assembly. Received January 16, 2003; revision requested March 25; revision received July 29; accepted September 5. Address correspondence to S.S.B., Staten Island University Hospital, 475 Seaview Ave, Staten Island, NY 10305-3498 (e-mail: sbuchbinder@siuh.edu).
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONSTATISTICAL CONSULTANT...REFERENCES |
|---|
MATERIALS AND METHODS: A CAC system was used to analyze 106 cases of lesions (42 malignant) that at blinded retrospective interpretation were assigned to BI-RADS category 3 by at least two of four radiologists. The CAC system automatically extracted from the digitized mammograms quantitative features that characterized the lesions. The system then used a classification scheme to score the lesions by the likelihood of their malignancy on the basis of these features. The classification scheme was trained with 646 pathologically proved cases (323 malignant), and the results were tested with receiver operating characteristic (ROC) analysis by using the jackknife method. Sensitivity, specificity, positive predictive value, and accuracy were calculated. Category 3 lesions were stratified among BI-RADS categories 2–5 according to CAC-assigned lesion score, and this classification was compared with the results of pathologic analysis.
RESULTS: Jackknife analysis of CAC results in the training data set yielded a sensitivity of 94%, specificity of 78%, positive predictive value of 81%, and area under the ROC curve of 0.90. Of the 42 malignant lesions that had been classified at conventional interpretation as probably benign, nine were assigned by the CAC system to BI-RADS category 4, and 29 were assigned to category 5. The CAC system correctly upgraded the BI-RADS classification of these 38 lesions (sensitivity, 90%) and incorrectly upgraded the classification of only 20 benign lesions (specificity, 69%).
CONCLUSION: The CAC system scored 38 of the 42 malignant lesions initially assigned to BI-RADS category 3 as BI-RADS category 4 or 5, and thus correctly upgraded the category in 90% of these lesions.
© RSNA, 2004
Index terms: Breast neoplasms, 00.30 • Breast neoplasms, diagnosis, 00.11 • Computers, diagnostic aid
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONSTATISTICAL CONSULTANT...REFERENCES |
|---|
Although the use of this assessment category appears widespread among radiologists, some prefer not to recommend follow-up examinations at 6-month intervals because of fear of provoking unnecessary anxiety in the patient, increasing costs, and "potentially fanning the flames of screening skeptics" (5). The generally accepted practice nevertheless appears to be monitoring of category 3 lesions with follow-up examinations every 6 months for 1–2 years, depending on the case (6). If the lesion appearance on follow-up mammograms is unchanged, a return to routine annual screening is thought reasonable. This approach is useful for avoiding unnecessary biopsies in a large number of cases, but it also requires the cooperation of the patient. The patient incurs higher radiation exposures and may undergo a protracted period of increased anxiety. In addition, diagnosis of the small number of lesions that eventually are found malignant will have been delayed. It is therefore desirable to find ways of minimizing the use of this category while avoiding an increase in the number of unnecessary biopsies.
It has been demonstrated that malignant lesions initially considered to be probably benign on the basis of rigorous diagnostic criteria are typically at an early stage at the time of definitive diagnosis and that a postponement of biopsy in favor of monitoring has a minimal effect on the prognosis (6). This approach has many inherent advantages over biopsy at the time of initial evaluation: It allows for overall decreases both in the number of unnecessary biopsies (presumably those with low positive predictive value) and in biopsy-associated morbidity and substantial associated costs (10,11). We believe, however, that it is possible to further refine the use of this assessment category by using computer-aided classification (CAC). Although earlier studies showed an improvement in mammographic interpretation with use of CAC (12–14), to our knowledge none have evaluated the performance of a CAC system specifically in classification of BI-RADS category 3 lesions. The use of CAC for this purpose, we expected, might result in negative screening results or definitive diagnoses of benign lesion in many patients, whereas other patients might be identified as requiring immediate biopsy. The purpose of our study, therefore, was to evaluate the performance of a CAC system in identifying malignancies among lesions assigned to BI-RADS category 3 at retrospective conventional mammographic interpretation.
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONSTATISTICAL CONSULTANT...REFERENCES |
|---|
Of these 752 cases, 547 underwent conventional retrospective interpretation. The study protocol specified that each case should be interpreted retrospectively four times, once each by four different radiologists in the study. In actuality, 529 cases were interpreted according to this protocol; 18 cases, however, were interpreted by only three radiologists. Each radiologist was blinded to the results of pathologic analysis, and each radiologist independently classified the lesions by using the BI-RADS categories. For this purpose, the radiologists were provided with the four standard mammographic views on which the initially reported findings that indicated the need for biopsy were clearly demarcated. Additional mammographic views available from the same examination and prior examinations also were provided at the request of the radiologist. In 106 cases (42 malignant lesions), the lesion was classified as BI-RADS category 3 by at least two radiologists, who suggested short-interval follow-up to monitor the lesion.
CAC Analysis
After the retrospective conventional interpretation, all of the mammograms were digitized at high resolution (600 dpi, 12 bits) by using a prototype CAC system described elsewhere (15), and the digital images were displayed on the computer monitor. An ellipse that encompassed the lesion but that did not necessarily correspond to its border was interactively defined on the digital image by one of the radiologists (R.B.L.), who was familiar with the CAC system and was also blinded to the results of pathologic analysis. Then the CAC system extracted quantitative features that characterized the lesion. Neither the findings at pathologic analysis nor the results of conventional mammographic interpretation were available to the radiologist during use of the CAC system for automated extraction of lesion features.
During this stage of classification, the CAC system extracted 50 features that characterized the findings according to spiculation, lesion shape, and definition of the mass margins. Spiculation was considered to be indicated by lines radiating from a centroid instead of by a saw-toothed border with a distinct margin. This analysis therefore also could be applied to areas of architectural distortion, to focal asymmetries, to masses that appeared smoothly marginated, and to masses in which the margins were partly obscured. For each mass, all extracted features and all findings at pathologic analysis were used as inputs for a stepwise discriminant analysis in which the power of each feature to discriminate benign lesions from malignant lesions was assessed. Features that contributed substantially to discrimination were selected by the stepwise discriminant analysis procedure for incorporation into a pattern recognition scheme that was developed to classify each lesion according to a score based on the extracted lesion features.
The pattern recognition scheme, which was based on the discriminant analysis method (16), classified each lesion by means of a single score derived from a combination of the features extracted by the CAC system and selected by the stepwise discriminant analysis procedure. To construct the classification scheme, the CAC system used a training procedure on a database of cases for which the extracted features were provided, along with the pathologic result, for each lesion. The training database was composed of the 646 cases (323 malignant and 323 benign lesions) remaining after the selection of 106 test cases from among the 752 cases culled from archives. After training was completed, the classification scheme, which was embedded in the software, assigned each lesion in the 106 test cases a single classifier or score on a continuous scale such that the higher the score, the higher the probability of malignancy.
After the lesions were scored according to the likelihood of their malignancy, limit values for score were calculated to stratify the lesions into score groups. The first limit was the score value below which no malignant lesions in the training set were found, the second limit was the score value below which 5% of the lesions were malignant (any small proportion of malignant lesions—eg, 2%—might have been chosen for this criterion), and the third limit was the score value above which 90% of the malignant lesions in the training set were found. Using these three limit values, the CAC system automatically stratified the lesions according to score into four groups that corresponded to BI-RADS categories 2–5.
Statistical Analysis
The jackknife technique (17) was applied to evaluate the performance of the CAC system in classifying the training set of 646 lesions (323 malignant), which did not include the 106 lesions that were classified as category 3 lesions at conventional retrospective interpretation. The jackknife or "leave-one-out" technique consisted of 646 rounds. In each round, the features extracted by the CAC system, as well as the findings at pathologic analysis, were provided for 645 cases, while the findings at pathologic analysis were withheld for the case being analyzed. The classification scheme assigned the lesion in that case a single score based exclusively on a combination of extracted features. Using the three limit values described earlier (see CAC Analysis), the CAC system then automatically stratified the cases by score into four groups signifying BI-RADS categories 2–5. In the statistical analysis, categories 2 and 3 were considered to indicate negative findings, and categories 4 and 5 were considered to indicate positive findings.
The results were evaluated with receiver operating characteristic (ROC) analysis (18). To calculate the specificity, positive predictive value, and accuracy of classification by the CAC system, a specific cut point value was defined that allowed discrimination between benign lesions and malignant lesions with an acceptable level of sensitivity. Because mammography is primarily a screening examination, a high level of sensitivity is required, even at the expense of specificity.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONSTATISTICAL CONSULTANT...REFERENCES |
|---|
|
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONSTATISTICAL CONSULTANT...REFERENCES |
|---|
Because the study design was retrospective, the radiologists knew that each lesion had been pathologically proved. This knowledge may have influenced their assessments and introduced a bias that would not have been present in a prospective clinical evaluation. In addition, although the number of actual category 3 lesions was small, the limited use of this category required that a large number of cases be accumulated to obtain the database. Although the results of this study show that cases currently assigned to category 3 could correctly be assigned to higher- or lower-numbered categories, they do not justify the elimination of this BI-RADS category.
Our study results, however, do indicate that a documented assessment of lesion benignity based on CAC methods may help mammographers to appropriately limit their use of the "probably benign" category. Although not all cases assigned to category 3 would be definitively classified either as negative or benign (BI-RADS categories 1 and 2, respectively) or as suspicious for or highly suggestive of malignancy (categories 4 and 5, respectively), a substantial shift away from the ambiguous category 3 to other, definitive assessment categories would streamline screening mammography and reduce the number of close interval follow-up examinations. This may further increase the use and acceptance of screening mammography and ultimately improve overall patient care.
|
STATISTICAL CONSULTANT COMMENTARY |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONSTATISTICAL CONSULTANT...REFERENCES |
|---|
Suppose that we have a sample of n values given by X1, X2, ... , Xn and that the sample mean
|
|
By combining the formulas for 
and -j, we can show that the missing data value Xj can be expressed as
|
Thus, the sample value Xj can be determined from the overall mean and the mean with Xj removed. This construct can be extended to other general parameters like those discussed in the preceding article. It is important that as the sample size n grows large, the jackknife estimators become unbiased.
|
FOOTNOTES |
|---|
Abbreviations: BI-RADS = Breast Imaging Reporting and Data System, CAC = computer-aided classification, ROC = receiver operating characteristic
Author contributions: Guarantors of integrity of entire study, all authors; study concepts and design, S.S.B., I.S.L., R.B.L.; literature research, R.B.L., G.Y., I.S.L.; clinical studies, M.S.L., R.B.L., S.I.F., S.S.B.; data acquisition, S.S.B., G.Y., S.I.F., M.S.L.; data analysis/interpretation, B.N., R.B.L., I.S.L.; statistical analysis, B.N., I.S.L., R.B.L.; manuscript definition of intellectual content, P.N.B., S.S.B., I.S.L., B.N.; manuscript editing, S.S.B., I.S.L., R.B.L.; manuscript preparation, revision/review, and final version approval, all authors
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONSTATISTICAL CONSULTANT...REFERENCES |
|---|
This article has been cited by other articles:
|
|
 |
|
  E. S. Burnside, J. Davis, J. Chhatwal, O. Alagoz, M. J. Lindstrom, B. M. Geller, B. Littenberg, K. A. Shaffer, C. E. Kahn Jr, and C. D. Page Probabilistic Computer Model Developed from Clinical Data in National Mammography Database Format to Classify Mammographic Findings Radiology, June 1, 2009; 251(3): 663 - 672. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
