Radiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     


Published online before print March 28, 2006, 10.1148/radiol.2392042127
This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
2392042127v1
239/2/385    most recent
Right arrow Submit a response
Right arrow View responses
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Lazarus, E.
Right arrow Articles by Livingston, L. S.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Lazarus, E.
Right arrow Articles by Livingston, L. S.
(Radiology 2006;239:385-391.)
© RSNA, 2006


Breast Imaging

BI-RADS Lexicon for US and Mammography: Interobserver Variability and Positive Predictive Value1

Elizabeth Lazarus, MD, Martha B. Mainiero, MD, Barbara Schepps, MD, Susan L. Koelliker, MD and Linda S. Livingston, MD

1 From the Department of Radiology, Rhode Island Hospital, Brown Medical School, 593 Eddy St, Providence, RI 02903. From the 2004 RSNA Annual Meeting. Received December 15, 2004; revision requested February 15, 2005; revision received April 17; accepted May 31; final version accepted July 18. Address correspondence to E.L. (e-mail: elazarus{at}lifespan.org).


    ABSTRACT
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
Purpose: To retrospectively evaluate interobserver variability between breast radiologists by using terminology of the fourth edition of the Breast Imaging Reporting and Data System (BI-RADS) to categorize lesions on mammograms and sonograms and to retrospectively determine the positive predictive value (PPV) of BI-RADS categories 4a, 4b, and 4c.

Materials and Methods: Institutional review board approval was obtained; informed consent was not required. This study was HIPAA compliant. Ninety-four consecutive lesions in 91 women who underwent image-guided biopsy comprised 59 masses, 32 calcifications, and three masses with calcification. Five radiologists retrospectively reviewed these lesions. Each observer described each lesion with BI-RADS terminology and assigned a final BI-RADS category. Interobserver variability was assessed with the Cohen {kappa} statistic. A pathologic diagnosis was available for all 94 lesions; 30 (32%) were malignant and 64 (68%) were benign. Pathologic analysis of benign lesions was performed on tissue obtained with image-guided core-needle biopsy. In cases referred for excisional biopsy after needle biopsy because of atypia or discordance, final surgical pathologic analysis was used for correlation with imaging findings. PPV for category 4 or 5 lesions was determined for all readers combined.

Results: For ultrasonographic (US) descriptors, substantial agreement was obtained for lesion orientation, shape, and boundary ({kappa} = 0.61, 0.66, and 0.69, respectively). Moderate agreement was obtained for lesion margin and posterior acoustic features ({kappa} = 0.40 for both). Fair agreement was obtained for lesion echo pattern ({kappa} = 0.29). For mammographic descriptors, moderate agreement was obtained for mass shape, mass margin, and calcification distribution ({kappa} = 0.48, 0.48, and 0.50, respectively). Fair agreement was obtained for calcification description ({kappa} = 0.32). Slight agreement was obtained for mass density ({kappa} = 0.18). Fair agreement was obtained for final assessment category ({kappa} = 0.28). PPVs of BI-RADS category 4 and 5 assignments were as follows: category 4a, six (6%) of 102; category 4b, 17 (15%) of 110; category 4c, 48 (53%) of 91; and category 5, 71 (91%) of 78.

Conclusion: Interobserver agreement with the new BI-RADS terminology is good and validates the US lexicon. Subcategories 4a, 4b, and 4c are useful in predicting the likelihood of malignancy.

© RSNA, 2006


    INTRODUCTION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
The Breast Imaging Reporting and Data System (BI-RADS) developed by the American College of Radiology (1) has standardized the assessment and reporting of breast lesions identified on mammograms. Both lesion description and management recommendations have become more consistent with use of BI-RADS. By making auditing easier, BI-RADS categorization of reports has also facilitated quality assurance practices (2,3).

Breast ultrasonography (US) has proved to be useful in the evaluation of masses detected with mammography or clinical examination, as US is used to distinguish cystic lesions from solid lesions and to further differentiate benign solid masses from malignant solid masses (46). However, standard terminology for describing lesions on breast sonograms has been lacking. Use of BI-RADS for breast US should standardize the reporting and classification of lesions detected on sonograms, thereby improving the utility of US in the work-up of breast masses. By identifying lesion descriptors that emphasize the distinctions between benign and malignant lesions on breast sonograms, the BI-RADS lexicon for US clarifies the indication for biopsy of particular lesions.

One other major addition in the fourth edition of BI-RADS involves the subcategorization of category 4 lesions. While classification of a lesion as BI-RADS category 4 indicates that a lesion has been recommended for biopsy, it provides no frame of reference for either the referring physician or the patient as to the prebiopsy risk for malignancy. Dividing category 4 lesions into those with a small (category 4a), moderate (category 4b), or substantial (category 4c) (1) likelihood of malignancy better informs the physician and patient as to the level of concern regarding the lesion and prepares both the physician and the patient for the likely biopsy findings and the potential need for follow-up.

Our study had two purposes: to retrospectively evaluate interobserver variability between breast radiologists who used the new BI-RADS terminology to characterize lesions identified on both mammograms and sonograms and to retrospectively determine the positive predictive value of the new BI-RADS categories (4a, 4b, and 4c) as they are used by radiologists performing breast imaging.


    MATERIALS AND METHODS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
Institutional review board approval was obtained, and the informed consent requirement was waived; this study was compliant with the Health Insurance Portability and Accountability Act.

Patient Lesions and Interpretation
The authors (all radiologists with subspecialty expertise in breast imaging) retrospectively evaluated 94 consecutive lesions in 91 women (mean age, 55 years; age range, 28–85 years) who underwent image-guided biopsy between August 1, 2002 and October 4, 2002. Four of the radiologists (E.L., M.B.M., L.S.L., and S.L.K.) underwent fellowship training in breast imaging and have practiced as faculty in an academic breast imaging section for 2–10 years. The other radiologist (B.S.) has been interpreting breast images for more than 30 years in her position as chief of the same breast imaging section. All five radiologists practice within the same group, and all met the standards of the Mammography Quality Standards Act as qualified interpreting physicians. The 94 lesions comprised 59 masses, 32 calcifications, and three masses with calcification; 52 lesions were evaluated by reviewing both mammograms and sonograms, 32 were evaluated by reviewing mammograms alone, and 10 were evaluated by reviewing sonograms alone. The lesions were evaluated on original mammograms and sonograms marked to indicate lesions, without the benefit of prior mammograms and sonograms for comparison. The lesions were marked so that all observers would examine the same biopsy-proved lesion, as some images showed more than one finding. Mammographic magnification views were obtained for all calcifications analyzed. All 62 masses were evaluated with US; 52 (84%) masses were evaluated with both mammography and US. Of the 94 lesions, 30 (32%) were malignant and 64 (68%) were benign.

Pathologic diagnosis was available for all lesions. Pathologic findings in benign lesions were evaluated with results from image-guided core-needle biopsy; a 14-gauge needle was used for US-guided biopsy, and a 9-gauge needle was used for vacuum-assisted stereotactic biopsy. In cases referred for excisional biopsy after needle biopsy because of atypia, positivity for malignancy, or discordance, final surgical pathologic analysis was used for correlation with imaging findings.

Each observer described each lesion by using the terminology of the fourth edition of the BI-RADS lexicon and assigned a final BI-RADS category, including the new subcategories of BI-RADS category 4. Each observer was provided a sheet containing the BI-RADS categories and descriptors for lesions seen on both mammograms and sonograms and instructed to select the most appropriate descriptors for each lesion.

While the evaluating radiologists were familiar with the "Guidance Chapter" (1) criteria for BI-RADS subcategorization, no formal training in the newest lexicon was provided. Thus, the criteria used by each evaluating radiologist were subjective and based on prior knowledge and experience. Table 1 lists the BI-RADS terminology used in this study.


View this table:
[in this window]
[in a new window]

 
Table 1. BI-RADS Fourth Edition Terminology

 
Statistical Analysis
The Cohen {kappa} statistic was used to assess interreader agreement for all descriptor variables. The guidelines of Landis and Koch were followed in interpreting {kappa} values: 0.00–0.20, slight agreement; 0.21–0.40, fair agreement; 0.41–0.60, moderate agreement; 0.61–0.80, substantial agreement; and 0.80–1.00, almost perfect agreement (7). All {kappa} statistics were calculated with statistical software (Stata, version 8; Stata, College Station, Tex). The positive predictive value for category 4 and 5 lesions was determined by using data from the assessments of all readers combined.

To determine if the results were skewed by any readers, we assessed the reliability between pairs of readers. Data from the final BI-RADS categorization of each lesion were used to determine the Pearson product-moment correlation coefficients between each pair of readers. These statistics were calculated with the SAS system (SAS Institute, Cary, NC). Intraclass correlation was also determined for final BI-RADS categorization by using (a) all results from each reader and (b) the mean of the results from each reader (Appendix).


    RESULTS
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
Mammographic Assessment of Masses
In describing the mass shape, overall agreement was moderate ({kappa} = 0.48). Substantial agreement was seen when the mass shape was characterized as irregular ({kappa} = 0.68), moderate agreement was seen when the mass shape was characterized as oval ({kappa} = 0.46), and fair agreement was seen when the mass shape was characterized as lobular ({kappa} = 0.24) or round ({kappa} = 0.31) (Table 2).


View this table:
[in this window]
[in a new window]

 
Table 2. Interobserver Variability in Description of Mammographic Lesions

 
In assessing the margins of the mass, overall agreement was moderate ({kappa} = 0.48). The greatest agreement was seen when the margins were characterized as circumscribed ({kappa} = 0.60) or spiculated ({kappa} = 0.69). Poor agreement was seen when the margins were characterized as microlobulated ({kappa} = 0.08), while fair agreement was seen when margins were characterized as indistinct ({kappa} = 0.38) or obscured ({kappa} = 0.27).

Overall agreement for mass density was slight ({kappa} = 0.18). Agreement was slight when mass density was described as fat containing ({kappa} = 0.11), equal to breast tissue ({kappa} = 0.15), or high ({kappa} = 0.20).

Mammographic Assessment of Calcifications
Agreement was nearly perfect when assessing the presence of calcifications ({kappa} = 0.94) (Table 2). The five observers demonstrated overall fair agreement when they described the calcifications ({kappa} = 0.32). Use of the terms amorphous and fine branching resulted in moderate agreement ({kappa} = 0.45 and 0.49, respectively). Agreement was fair for use of the terms coarse heterogeneous ({kappa} = 0.27), pleomorphic ({kappa} = 0.21), and vascular ({kappa} = 0.24). Use of the terms coarse (n = 3), dystrophic (n = 1), milk of calcium (n = 2), and punctuate (n = 2) was uncommon in this series of lesions and led to low {kappa} values ({kappa} = –0.01 for these terms). Figure 1 shows an image for which reviewers agreed on the number and distribution of calcifications but disagreed on the description of calcifications.


Figure 1
View larger version (123K):
[in this window]
[in a new window]
[Download PPT slide]
 
Figure 1: Mammogram shows calcifications in a 70-year-old woman. The calcifications occupy a total area of 6 x 3 mm and were photographically enlarged on this craniocaudal view. All reviewers agreed that the calcifications were grouped in distribution and numbered more than 10; however, the reviewers disagreed concerning calcification description, as they used the terms coarse (n = 1), coarse heterogeneous (n = 2), and pleomorphic (n = 2). Pathologic evaluation after vacuum-assisted stereotactic breast biopsy revealed a degenerating fibroadenoma.

 
Moderate agreement was achieved in the evaluation of the distribution ({kappa} = 0.50) and number ({kappa} = 0.48) of calcifications.

In evaluating the presence of architectural distortion, agreement was fair ({kappa} = 0.26). Agreement between readers for the presence of associated findings and special cases could not be assessed secondarily because the readers found few cases in which associated findings or special cases were present in the lesions.

Sonographic Assessment
For sonographic descriptors, substantial agreement was obtained for assessment of lesion orientation ({kappa} = 0.61) that was described as parallel or not parallel (Table 3).


View this table:
[in this window]
[in a new window]

 
Table 3. Interobserver Variability in Description of US Lesions

 
Substantial agreement was also found for evaluation of lesion shape ({kappa} = 0.66). The greatest agreement was achieved when lesion shape was described as irregular ({kappa} = 0.70) or oval ({kappa} = 0.71). The term round was used infrequently, and less agreement was found with its use ({kappa} = 0.29).

Evaluation of the lesion boundary, which was described as abrupt or having an echogenic halo, yielded similarly substantial agreement ({kappa} = 0.69).

Fair agreement was achieved for evaluation of the lesion margin ({kappa} = 0.40). Excellent agreement was seen with lesions that were considered circumscribed ({kappa} = 0.71). Fair agreement was seen with lesions that were considered angular ({kappa} = 0.22), indistinct ({kappa} = 0.22), microlobulated ({kappa} = 0.25), or spiculated ({kappa} = 0.26).

The many terms available to describe the US echo pattern yielded fair agreement between observers ({kappa} = 0.29). The terms complex ({kappa} = 0.40) and hypoechoic ({kappa} = 0.29) were used most commonly and demonstrated the most agreement in their use. The terms anechoic ({kappa} = –0.01) and isoechoic ({kappa} = 0.05) were used rarely, which likely accounted for poor agreement in their use. The term hyperechoic also was used rarely and demonstrated only slight agreement ({kappa} = 0.16).

Overall fair agreement was achieved in describing posterior acoustic features ({kappa} = 0.40). The greatest agreement ({kappa} = 0.66) was achieved when lesions were described as having posterior acoustic shadowing. When lesions were described as having either posterior enhancement or no change in the echo pattern, agreement was fair ({kappa} = 0.39 and {kappa} = 0.31, respectively). Lesions were rarely described as having combined posterior enhancement and shadowing, and agreement between observers who used this description was poor ({kappa} = 0.09). Figure 2 shows a lesion with good agreement for lesion orientation, shape, boundary, and echogenicity but disagreement for assessment of the lesion margin and posterior acoustic features.


Figure 2
View larger version (143K):
[in this window]
[in a new window]
[Download PPT slide]
 
Figure 2: Transverse sonogram in a 46-year-old woman with a solid breast mass (arrow). Four reviewers agreed on lesion orientation (parallel), shape (oval), boundary (abrupt), and echogenicity (hypoechoic). Reviewers did not agree on lesion margin and posterior acoustic features. Pathologic evaluation of core and excisional biopsy samples revealed infiltrating ductal carcinoma.

 
No {kappa} statistics could be calculated for assessment of alterations in the surrounding tissue, presence of calcifications, lesions determined to be special cases, or lesion vascularity because the observers believed these findings were present in the lesions only on rare occasions.

Final Assessment Category
Fair agreement was achieved for the final assessment category ({kappa} = 0.28) with all final categories (2, 3, 4a, 4b, 4c, and 5) (Table 4). The greatest agreement was found with lesions categorized as highly suspicious for malignancy (category 5) ({kappa} = 0.56). Although few lesions were rated as benign (category 2), agreement with this category was fair ({kappa} = 0.27). Fair agreement was obtained for category 3 and 4c lesions ({kappa} = 0.32 and 0.26, respectively); however, there was poor agreement between observers for category 4a and 4b lesions ({kappa} = 0.14 and 0.16, respectively). When the categories were grouped in terms of whether biopsy was required (categories 2 and 3 and categories 4a, 4b, and 5), moderate agreement was obtained ({kappa} = 0.45). Figure 3 shows a classic malignancy on a sonogram, with all reviewers classifying the malignancy as BI-RADS category 5.


View this table:
[in this window]
[in a new window]

 
Table 4. Interobserver Variability in Assigning Final BI-RADS Assessment Category

 

Figure 3
View larger version (154K):
[in this window]
[in a new window]
[Download PPT slide]
 
Figure 3: Transverse sonogram in a 56-year-old woman with a solid breast mass (arrow). All reviewers categorized this lesion as BI-RADS category 5. Pathologic evaluation of core and excisional biopsy samples revealed infiltrating ductal carcinoma.

 
The results of the Pearson product-moment correlation demonstrate fair agreement between all pairs of readers (Table 5). This indicates that the results of no one reader skewed the analysis. Intraclass correlation with all final BI-RADS assessments showed moderate reliability of 0.676. Use of each reader's mean value of BI-RADS assessment produced improved reliability, with an intraclass correlation of 0.912.


View this table:
[in this window]
[in a new window]

 
Table 5. Pearson Product Moment Correlation Coefficients between All Pairs of Readers

 
Positive Predictive Value
Pathologic findings in all lesions were available for review. Forty-eight (51%) of the lesions were diagnosed with core-needle biopsy only. In 46 (49%) lesions, pathologic analysis was based on the findings of excisional biopsy. With the results of all readers combined, positive predictive values of lesions categorized as BI-RADS 4 or 5 were as follows: category 4a, six (6%) of 102; category 4b, 17 (15%) of 110; category 4c, 48 (53%) of 91; and category 5, 71 (91%) of 78.


    DISCUSSION
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
Breast US is now considered integral in the assessment of a mass detected with clinical examination and mammography (5). Stavros et al (5) identified individual sonographic characteristics—including shape, margin, lesion boundary, lesion orientation, and echotexture—that indicate whether a lesion is malignant, benign, or indeterminate. With the use of these features, observers have successfully used US criteria to predict malignancy in a breast mass (4). Until recently, however, there has been a lack of uniformity in descriptive terms for US lesions, which can result in inconsistent diagnoses. Baker et al (10) examined observer variability in the description and assessment of solid breast masses on sonograms and identified the need to standardize terminology for breast US, which has been provided in the fourth edition of BI-RADS.

Our results show a high degree of agreement in describing lesions on sonograms, thus demonstrating the appropriateness of the terms chosen in the newest iteration of BI-RADS. The terminology was familiar to radiologists experienced in breast imaging, and their use was generally concordant.

Agreement for lesion orientation, shape, and boundary on sonograms was slightly better than agreement for lesion margin on sonograms and mammograms and lesion shape on mammograms, likely because of the greater number of choices for describing the second set of qualifiers. The lower rate of agreement for lesion echo pattern on sonograms and mass density on mammograms suggests that observers had difficulty in making these categorizations. However, results from Stavros et al (5) and Baker et al (10) indicate that these qualifiers are not very useful in the differentiation of benign and malignant masses.

Agreement for calcification description on mammograms was somewhat lower in our study than in prior studies (11,12). The agreement may be lower because of bias introduced by the method chosen to select the cases. To have a pathologic correlation available for assessment of positive predictive value, we chose to evaluate lesions referred for biopsy in this study. As a consequence, there was rare use of typical benign descriptors, which may have better observer agreement.

The fair agreement for overall BI-RADS category reported in our study ({kappa} = 0.28) was not much different from that reported by Berg et al (11) ({kappa} = 0.37) in a prior study in which only mammography was used. The lack of agreement between the {kappa} value obtained in our study and that obtained in the study of Berg et al (11) can be at least partially explained by the greater number of categories offered with inclusion of the 4a, 4b, and 4c subcategories.

Orel et al (13) showed that placement of mammographic lesions into BI-RADS categories is useful for predicting malignancy, with a positive predictive value of 30% for category 4 lesions and 97% for category 5 lesions. We obtained a similar positive predictive value of 91% for category 5 lesions.

While BI-RADS category 5 has always been used to identify lesions that are almost certainly malignant, BI-RADS category 4 historically has comprised a more heterogeneous population of lesions. Our results demonstrate that the optional subcategories of 4a, 4b, and 4c are useful in stratifying the likelihood of malignancy among the large heterogeneous group of category 4 lesions. This stratification is helpful in communicating the level of suspicion to referring physicians and patients, who may choose to use this information in management decisions (ie, which patients to refer to a breast specialist prior to biopsy). In our practice, some referring internists and gynecologists are willing to discuss the results of benign breast biopsies with their patients, but others prefer that patients receive the results of malignant breast biopsies from a breast surgeon who can counsel the patient on further intervention. The medical expenditure of an additional referral may not be necessary if the preprocedural risk assessment is low and can be clearly communicated.

For radiologists who use the subcategories, a medical audit of the positive predictive value of each category can provide additional feedback on interpretive performance.

There were several limitations to our study. Because we used only those lesions that were referred for biopsy, we had few descriptors typically associated with benign disease and few lesions that could be characterized as special cases. The small number of these cases may have decreased interobserver agreement in some areas, as Taplin et al (2) demonstrated that BI-RADS evaluation of negative findings and benign lesions is more consistent than BI-RADS evaluation of abnormalities.

Second, Berg et al (14) demonstrated that even for experienced breast imagers, BI-RADS training results in improved agreement for feature analysis and final assessment. While the guidance chapter in the fourth edition of the BI-RADS breast imaging atlas offers useful examples of lesions appropriate for each subcategory (1), the evaluating radiologists were not specifically asked to review these criteria. We did not provide specific training to the radiologists involved in this study to more accurately represent the usage of the majority of radiologists being introduced to this new version, who would likely have no formal training prior to its implementation. Additionally, this study was based on the performance of experienced breast imaging radiologists. Inconsistencies and errors in using the BI-RADS lexicon and categories among radiologists with different levels of experience may vary and should be studied.

Another limitation is that the cases had been evaluated by a radiologist prior to this study. We used cases that were assessed more than a year before the study was begun to minimize any recollection of a case that may have influenced categorization of the lesion.

In conclusion, the addition of the BI-RADS lexicon for US is helpful and can be used with good agreement among radiologists, even those without specific training in the new terminology. Additionally, use of the new optional subcategories 4a, 4b, and 4c is beneficial in stratifying the likelihood of malignancy in lesions recommended for biopsy.


    APPENDIX
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 
Under conditions similar to ours (where reasonable numeric values may be assigned to ordinal categories and there are more than three such categories), Hripcsak and Heitjan (8) recommended treating ordinal data as continuous data and examining agreement and reliability by using intraclass and product-moment correlation coefficients. Thus, the 10 orthogonal Pearson product-moment correlation coefficients between readers' judgments were presented, and reliability was estimated with intraclass correlation coefficients for interreader reliability and the reliability of the mean of the readers' judgments by using the methods suggested by Shrout and Fleiss (9).


    FOOTNOTES
 

Abbreviations: BI-RADS = Breast Imaging Reporting and Data System

Author contributions: Guarantors of integrity of entire study, E.L., B.S., L.S.L.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; manuscript final version approval, all authors; literature research, E.L., B.S., L.S.L.; clinical studies, all authors; statistical analysis, E.L., B.S., L.S.L.; and manuscript editing, all authors

Authors stated no financial relationship to disclose.


    References
 TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 APPENDIX
 References
 

  1. American College of Radiology. Breast imaging reporting and data system, breast imaging atlas. 4th ed. Reston, Va: American College of Radiology, 2003.
  2. Taplin SH, Ichikawa LE, Kerlikowske K, et al. Concordance of breast imaging reporting and data system assessments and management recommendations in screening mammography. Radiology 2002;222:529–535.[Abstract/Free Full Text]
  3. Geller BM, Barlow WE, Ballard-Barbash R, et al. Use of the American College of Radiology BI-RADS to report on the mammographic evaluation of women with signs and symptoms of breast disease. Radiology 2002;222:536–542.[Abstract/Free Full Text]
  4. Rahbar G, Sie AC, Hansen GC, et al. Benign versus malignant solid breast masses: US differentiation. Radiology 1999;213:889–894.[Abstract/Free Full Text]
  5. Stavros AT, Thickman D, Rapp CL, Dennis MA, Parker SH, Sisney GA. Solid breast nodules: use of sonography to distinguish between benign and malignant lesions. Radiology 1995;196:123–134.[Abstract/Free Full Text]
  6. Skaane P, Engedal K. Analysis of sonographic features in the differentiation of fibroadenoma and invasive ductal carcinoma. AJR Am J Roentgenol 1998;170:109–114.[Abstract/Free Full Text]
  7. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–174.[CrossRef][Medline]
  8. Hripcsak G, Heitjan DF. Measuring agreement in medical informatics reliability studies. J Biomed Inform 2002;35:99–110.[CrossRef][Medline]
  9. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86:420–428.[CrossRef][Medline]
  10. Baker JA, Kornguth PJ, Soo MS, Walsh R, Mengoni P. Sonography of solid breast lesions: observer variability of lesion description and assessment. AJR Am J Roentgenol 1999;172:1621–1625.[Abstract/Free Full Text]
  11. Berg WA, Campassi C, Langenberg P, Sexton MJ. Breast imaging reporting and data system: inter- and intraobserver variability in feature analysis and final assessment. AJR Am J Roentgenol 2000;174:1769–1777.[Abstract/Free Full Text]
  12. Baker JA, Kornguth PJ, Floyd CE. Breast imaging reporting and data system standardized mammography lexicon: observer variability in lesion description. AJR Am J Roentgenol 1996;166:773–778.[Abstract/Free Full Text]
  13. Orel SG, Kay N, Reynolds C, Sullivan DC. BI-RADS categorization as a predictor of malignancy. Radiology 1999;211:845–850.[Abstract/Free Full Text]
  14. Berg WA, D'Orsi CJ, Jackson VP, et al. Does training in the breast imaging reporting and data system (BI-RADS) improve biopsy recommendations or feature analysis agreement with experienced breast imagers at mammography? Radiology 2002;224:871–880.[Medline]



This article has been cited by other articles:


Home page
RadiologyHome page
N. Abdullah, B. Mesurolle, M. El-Khoury, and E. Kao
Breast Imaging Reporting and Data System Lexicon for US: Interobserver Agreement for Assessment of Breast Masses
Radiology, June 30, 2009; (2009) 2523080670.
[Abstract] [Full Text]


Home page
Am. J. Roentgenol.Home page
J. H. Shin, B.-K. Han, E. Y. Ko, Y. H. Choe, and S.-J. Nam
Probably Benign Breast Masses Diagnosed by Sonography: Is There a Difference in the Cancer Rate According to Palpability?
Am. J. Roentgenol., April 1, 2009; 192(4): W187 - W191.
[Abstract] [Full Text] [PDF]


Home page
RadiologyHome page
D. L. Weiss and C. P. Langlotz
Structured Reporting: Patient Care Enhancement or Productivity Nightmare?
Radiology, December 1, 2008; 249(3): 739 - 747.
[Full Text] [PDF]


Home page
Am. J. Roentgenol.Home page
S. J. Kim, E. Y. Ko, J. H. Shin, S. S. Kang, S. H. Mun, B.-K. Han, and E. Y. Cho
Application of Sonographic BI-RADS to Synchronous Breast Nodules Detected in Patients with Breast Cancer
Am. J. Roentgenol., September 1, 2008; 191(3): 653 - 658.
[Abstract] [Full Text] [PDF]


Home page
Am. J. Roentgenol.Home page
E.-K. Kim, K. H. Ko, K. K. Oh, J. Y. Kwak, J. K. You, M. J. Kim, and B.-W. Park
Clinical Application of the BI-RADS Final Assessment to Breast Sonography in Conjunction with Mammography
Am. J. Roentgenol., May 1, 2008; 190(5): 1209 - 1215.
[Abstract] [Full Text] [PDF]


Home page
JNCI J Natl Cancer InstHome page
D. L. Miglioretti, R. Smith-Bindman, L. Abraham, R. J. Brenner, P. A. Carney, E. J. A. Bowles, D. S. M. Buist, and J. G. Elmore
Radiologist Characteristics Associated With Interpretive Performance of Diagnostic Mammography
J Natl Cancer Inst, December 19, 2007; 99(24): 1854 - 1863.
[Abstract] [Full Text] [PDF]


Home page
J Ultrasound MedHome page
B. Mesurolle, L. Kadoch, M. El-Khoury, A. Lisbona, N. Dendukuri, and W. D. Foulkes
Sonographic Features of Breast Carcinoma Presenting as Masses in BRCA Gene Mutation Carriers
J. Ultrasound Med., June 1, 2007; 26(6): 817 - 824.
[Abstract] [Full Text] [PDF]


Home page
radtechHome page
T. G. ODLE
Breast Ultrasound
Radiol. Technol., January 1, 2007; 78(3): 222M - 242M.
[Abstract] [Full Text] [PDF]

eLetters:

Read all eLetters

Is BI-RADS reproducible ?
Stefano Ciatto
Radiology Online, 2 Apr 2007 [Full text]

This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
2392042127v1
239/2/385    most recent
Right arrow Submit a response
Right arrow View responses
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Lazarus, E.
Right arrow Articles by Livingston, L. S.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Lazarus, E.
Right arrow Articles by Livingston, L. S.


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
RADIOLOGY RADIOGRAPHICS RSNA JOURNALS ONLINE