| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Computer Applications |
1 From the Dept of Radiology, Univ of British Columbia, Vancouver, Canada (L.J.W.B.); R2 Technology, 325 Distel Circle, Los Altos, CA 94022 (S.A.W., K.F.O., R.A.C.); Dept of Radiology, Univ of Massachusetts Medical Center, North Worcester (C.J.D.); Breast Imaging Center, Thomas Jefferson Univ Hosp, Philadelphia, Pa (S.A.F.); Dept of Radiology, Massachusetts General Hosp, Boston (D.B.K.); UCSF Medical Center, San Francisco, Calif (E.A.S.); Dept of Mammography, Central Hosp, Falun, Sweden (L.T.); and Dept of Radiology, Univ of Chicago, Ill (C.J.V.). From the 1998 RSNA scientific assembly. Received Jul 19, 1999; revision requested Aug 26; revision received Sep 21; accepted Oct 21. Address correspondence to R.A.C. (e-mail: rcastell@r2tech.com).
 |
Abstract |
|---|
 TOP Abstract IntroductionMATERIALS AND METHODSRESULTSDISCUSSIONAPPENDIXReferences |
|---|
MATERIALS AND METHODS: All available screening mammograms that led to the detection of biopsy-proved cancer (n = 1,083) and the most recent corresponding prior mammograms (n = 427) were collected from 13 facilities. Panels of radiologists evaluated the retrospectively visible prior mammograms by means of blinded review. All mammograms were analyzed by a CAD system that marks features associated with cancer. The recall rates of 14 radiologists were prospectively measured before and after installation of the CAD system.
RESULTS: At retrospective review, 67% (286 of 427) of screening mammography–detected breast cancers were visible on the prior mammograms. At independent, blinded review by panels of radiologists, 27% (115 of 427) were interpreted as warranting recall on the basis of a statistical evaluation index; and the CAD system correctly marked 77% (89 of 115) of these cases. The original attending radiologists' sensitivity was 79% (427 of [427 + 115]). There was no statistically significant increase in the radiologists' recall rate when comparing the values before (8.3%) with those after (7.6%) installation of the CAD system.
CONCLUSION: The original attending radiologists had a false-negative rate of 21% (115 of [427 + 115]). CAD prompting could have potentially helped reduce this false-negative rate by 77% (89 of 115) without an increase in the recall rate.
Index terms: Breast neoplasms, 00.32 • Breast neoplasms, diagnosis, 00.32 • Cancer screening, 00.11, 00.1299 • Computers, diagnostic aid
|
Introduction |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONAPPENDIXReferences |
|---|
There is compelling evidence that many breast cancers detected at screening mammography are, in retrospect, visible on the previously obtained mammograms but have been missed by the interpreting radiologist in the prior year (7–11). Although there is no clear consensus as to the actual sensitivity of radiologists in the interpretation of screening mammograms, delayed detection of cancer is a substantial and potentially costly problem because of the associated increased mortality and increased cost of care. Although prior studies (12–18) with limited patient sampling and older mammographic techniques have addressed this issue, to our knowledge there has been no published study with sufficient patient sampling and statistical power in which the false-negative rate of the radiologists was reported. One of our main goals was to perform a more comprehensive study with sufficient statistical power to determine the false-negative rate of radiologists reading screening mammograms.
To overcome the known limitations of human observers, second (or double) reading of screening mammograms by another radiologist has been implemented at many sites. The results of studies (19–24) indicate a potential 4%–15% increase in the number of cancers detected with double reading. In a radiology practice that performs 10,000 screening examinations per year, generally between 30 and 100 cancers per year will be detected; thus, double reading in this practice could contribute to the diagnosis of 1–15 additional cancers per year.
Rapid and continuing advances in computer technology, as well as the ready adaptation of radiologic images to digital formats, have increased the interest in computer prompting to enable the attending radiologist to act as his or her own second reader (25). One very promising adaptation of computer-prompting technology is computer-aided detection (CAD) in screening mammography (26–28). The current CAD systems demonstrate a high rate of detecting cancerous features on mammograms.
This study of screening mammography was divided into three parts. The first part was a statistical evaluation of the radiologists' sensitivity in detecting action-warranting (ie, actionable), asymptomatic breast cancer on screening mammograms in a large patient population. The assessment of radiologists' sensitivity was determined with community practice–based radiologists in a blinded panel review of consecutive, asymptomatic, previously obtained mammograms with which biopsy-proved cancer was diagnosed at the subsequent mammographic screening. By means of independent quintuple reading of these prior mammograms, which was done without CAD, we determined the number of radiologists who recommended recall of the patient on the basis of sufficiently recognizable mammographic features of cancer. We therefore estimated the number of missed cancers on the basis of these radiologists' determination of what were actionable cancers on these prior mammograms. In the second part of the study, we determined the sensitivity of CAD in identifying cancerous lesions on these same prior screening mammograms.
The results of these two sections of the study led to a determination of the potential benefit of CAD to the radiologist. In defining this potential benefit, we assumed that all cases that would be recalled were those in which imaging depicted sufficiently recognizable mammographic features for cancer that, if prompted, would help the radiologist with his or her assessment. In the third part of the study, we addressed the concern that CAD prompting might lead to unnecessary recalls. Therefore, the change in radiologists' recall rate with CAD was measured. These studies were performed in part as clinical trials for the premarket approval application for a CAD system (ImageChecker M1000, version 1.2; R2 Technology, Los Altos, Calif) that was approved by the U.S. Food and Drug Administration in June 1998.
|
MATERIALS AND METHODS |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONAPPENDIXReferences |
|---|
Case Material
All of the available screening mammographic studies obtained from the 13 facilities from 1994 through 1996 from which asymptomatic, biopsy-proved cancer was diagnosed (hereinafter referred to as "current mammograms")—a total of 1,083 cases—were collected (Fig 1). All cases were in women with a mean age of 62.6 years (age range, 34.0–94.0 years); 14% (156 of 1,083) of these patients were younger than 50 years. All of the available corresponding screening mammograms that were obtained 9–24 months (mean, 14 months) before the current mammograms—a total of 493 cases—also were collected. If more than one prior screening mammogram was available, the most recently obtained one was used. These 493 studies were obtained in women with a mean age of 62.5 years (age range, 34.0–86.0 years); 9% (45 of 493) of these patients were younger than 50 years. Because corresponding prior mammograms were not available for more than half (590 of 1,083) of the current mammograms, the distribution of cancer types (ie, microcalcifications and masses), patient ages, and CAD performance on the current mammograms with (n = 493) and without (n = 590) corresponding prior mammograms were compared to evaluate whether the two data sets were similar (Fig 1).
|
Three community practice–based radiologists (hereinafter referred to as "designated radiologists") who met MQSA qualification standards were recruited to review the corresponding prior mammograms. These designated radiologists were not otherwise involved in the study. The 493 prior mammograms were divided into three case sets, and each designated radiologist independently reviewed one case set. By using the current mammograms with the overlays as a guide, the designated radiologist determined whether the subsequently diagnosed lesion was visible retrospectively on the prior mammogram. If so, the designated radiologist created a second standard-of-reference overlay, marked the location of the lesion on the prior mammogram, and indicated the lesion characteristics and BI-RADS assessment.
Of the 493 prior mammograms reviewed, 62 cases in which the cancer was found to be visible retrospectively were excluded from the subsequent study because mammographic evidence of previous breast surgery was thought to unduly influence the interpreter. Four other cases were excluded: For this study, the radiologists reviewed the original mammograms because copied images contain less diagnostic information. In these four cases, the original mammograms had to be returned to the originating facility before the last panel review. Thus, a total of 427 prior mammograms were used in this study (hereinafter referred to as "prior mammograms"), 286 (67%) of which had evidence of the subsequently diagnosed cancer (hereinafter referred to as "visible prior mammograms") (Fig 1).
Blinded Assessment by Panel Radiologists
Four panels, each consisting of five community practice–based radiologists who met MQSA qualification standards (hereinafter referred to as "panel radiologists"), also were recruited to participate in the study. The 20 panel radiologists had a mean of 17 years (range, 3–35 years) of experience practicing mammography and read a mean of 300 (range, 40–1,000) screening mammograms per month. The original films from the 286 visible prior mammograms were divided into four sets, with 64, 70, 71, and 81 cases in each set (Fig 2). Also included in each case set were an additional 45 mammograms—five nonvisible prior mammograms, 20 current mammograms (with biopsy-proved cancers) randomly chosen from those without obvious cancerous features (ie, no current mammograms with a BI-RADS assessment of 5 were included), and 20 mammograms randomly chosen from 100 screening mammograms that were obtained in other patients and confirmed to be normal by having at least one subsequent normal examination. Each panel radiologist reviewed only one case set.
|
Analysis of Mammograms by the CAD System
The original films of all 1,083 current and 286 visible prior mammograms collected for the clinical study were analyzed by a CAD system (ImageChecker M1000) that digitized the mammograms at 50 µm with 12-bit resolution by using a film digitizer. The digital images were analyzed by using software that highlights regions of interest with the following characteristics: (a) clusters of bright spots (ie, regions suggestive of microcalcification clusters), which are marked by a triangle-shaped marker; and (b) dense regions, with or without radiating lines, and parenchymal distortions (ie, regions suggestive of spiculated masses, densities, or architectural distortion), which are marked by an asterisk-shaped marker. These dense regions and distortions were all termed "masses" for the analysis described in this article.
The output of the CAD system was compared with the standard-of-reference location (or locations) of the cancer on the current and visible prior mammograms. A CAD mark on either the craniocaudal or mediolateral view in the correct area was scored as correctly marked. If the case had two biopsy-proved cancers, a mark on either cancer was considered to be correct. For lesions with both mass and calcification features that were documented by the radiologist, a marker of either type was counted as correct. The sensitivity of the CAD system was calculated overall for all current and visible prior mammograms.
Prospective Assessment of Change in Recall Rate with the CAD System
The change in radiologist recall rate before and after the installation of the CAD system was measured at five institutions (listed at the end of the article). The sites and participating radiologists all met MQSA qualification standards. The minimum acceptable site volume was 5,000 screening mammography cases per year, and the minimum radiologist reading volume that was acceptable for inclusion in the study was 100 screening mammograms per month. All mammograms included in the study were obtained in women seen for a screening examination.
A historical data review was conducted for a minimum of 4 months at each site before the installation of the CAD system, which tabulated the overall number of screening mammograms and the recall rate for the individual radiologists. The recall rate was defined as the percentage of patients who underwent screening mammography and were recalled immediately for additional imaging or biopsy.
The prospective study was initiated approximately 1 month after the installation of the CAD system. During this time, the medical and technical staffs were trained to operate the device, the system was integrated into the clinic's workflow, and study documentation procedures were implemented. The postinstallation phase of the study lasted a minimum of 4 months. All screening mammograms at a given site were processed on the CAD system, and the radiologist used the CAD information during his or her routine reading sessions before making a final diagnosis. The monthly screening volumes and recall rates were then calculated for each of 14 radiologists. All cases analyzed by the CAD system were recorded in CAD system logs, including the number of regions of interest found, which was used to determine the average number of marks per case. Pre- and postinstallation recall rates were measured. The pre- and postinstallation recall rates and 99% CIs were calculated on the basis of Clopper-Pearson exact likelihood limits, with equal probability in each tail. The 99% level was selected to partially adjust for the use of simultaneous inference involved in the multiple comparisons being performed and to avoid spurious significances that can result from unadjusted levels. In addition, data were analyzed by using the 
2 test, with continuity corrections for the two-by-two contingency tables for the pre- and postinstallation recall rates, to test the null hypothesis that there was no difference between these rates. Equivalently, the statistical analysis was performed to test for the null hypothesis of independence of the pre- and postinstallation recall rates observed in each of these periods. To adjust for the multiple tests performed, a Bonferroni correction was used for the set of 14 radiologists, not including the nonindependent test on the aggregate numbers. A Bonferroni significance level of .00366 was used for each of the individual hypotheses.
|
RESULTS |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONAPPENDIXReferences |
|---|
|
|
|
|
|
In this study, the CAD system placed an average of 1 mark per film on mammograms that did not depict cancerous lesions (4.1 marks per standard two-view bilateral screening mammogram).
Effect of CAD on Radiologists' Recall Rate
Fourteen radiologists from five facilities were involved in the study to measure radiologists' recall rate with the CAD system. According to historical data review, these 14 radiologists interpreted a total of 23,682 screening studies, with an overall recall rate of 8.3% (1,961 of 23,682). In the prospective portion of the study, these same radiologists, with the aid of CAD prompting, interpreted 14,817 screening mammograms and had an overall recall rate of 7.6% (1,126 of 14,817) (Table 5). There was no statistically significant increase in the radiologists' recall rate as a group or individually with the use of CAD at the threshold level used in this study. (See text regarding pre- and postinstallation rates at end of Materials and Methods section.)
|
|
DISCUSSION |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONAPPENDIXReferences |
|---|
The topic of retrospectively visible screening mammography–detected breast cancers has been addressed extensively in the literature (Table 6). The data from the study by Harvey et al (11) most closely parallel those in the present study. They found that 55 (75%) of 73 mammograms depicted cancers that were retrospectively visible on the most recently obtained prior mammograms by at least one of three radiologists who knew the cancer's eventual location, whereas 26 (36%) of 73 of the prior mammograms were found to be actionable by two different radiologists at true prospective blinded readings. In the current study, the case sets analyzed by the panel radiologists were enriched with cases in which 9–24 months later (mean, 14 months) the patients were given a diagnosis of cancer, unlike the case distribution in a normal clinical environment. Yet our findings of retrospectively visible (on 286 [67%] of 427 mammograms) and actionable (on 115 [27%] of 427 mammograms) cancers were similar to those in the Harvey et al study (Table 6).
|
In our study, similar to that of Harvey et al (11), we found that although a majority of cancers could be seen in retrospect, a much smaller percentage of the retrospectively visible cancers were considered to be actionable at blinded review (Table 6). For this reason, we did not estimate the CAD benefit by comparing the CAD results for all visible prior mammograms, but rather we estimated its benefit by comparing the CAD results for only the smaller subset of actionable prior mammograms. Furthermore, the derived benefit of CAD was scaled to the actionability of the panel radiologists to produce a more conservative estimate. For example, if none of the five panel radiologists (score 0/5) identified the lesion as being suspicious, then none of the 28 (34%) of 83 correct CAD markings were considered in the calculation of CAD benefit. Similarly, if only one of the panel radiologists correctly identified the lesion as cancer, then only 20% (1/5) of the correct CAD markings were considered in the calculation of CAD benefit, and so forth. It was only when the panel radiologists had a 100% consensus (5/5) interpretation of the lesion that we assumed the original radiologist would have acted on the CAD mark, if present.
We also determined that 112 of the visible prior mammograms were judged to be actionable by a majority (at least three of five) of the panel radiologists. Our statistically derived calculation of 115 actionable prior mammograms was very similar to these 112 cases. In summary, the CAD benefit was derived from only those cases that would have been worked up, if seen—not from the larger subset of retrospectively visible cases.
The reasons these retrospectively visible and actionable cancers did not result in a work-up by the original attending radiologists are unknown, but some have categorized them as oversights or errors in interpretation (31). Not surprisingly, such error rates have been shown to decrease with multiple readings (19–24). Another way to decrease these errors is to use CAD to prompt the radiologist to reassess those features exhibiting strong characteristics of cancer.
By collecting all the available prior mammograms obtained before the current mammograms, we were able to calculate the sensitivity of the original attending radiologists' interpretation by finding a subset of prior mammograms on which the panel radiologists found actionable cancer. In previous studies (12–18,32–40) involving smaller patient populations and older mammographic techniques with less controlled conditions, radiologists' sensitivity in reading mammograms has been estimated to be 83%–95% for first mammography screenings and 56%–86% for subsequent screenings.
The total number of mammographic examinations performed at the participating institutions during our study was calculated to be 290,000 cases, extrapolated from the 1,083 screening-detected current mammograms, by using a ratio for subsequent screening of 3.8 cancers detected per 1,000 screening mammograms (39), which is the expected incidence in a screened population. (Note: With three to 10 cancers detected per 1,000 screening mammograms, the original number of screening mammograms on which the 1,083 cancers were detected on the current mammograms ranged from roughly 110,000 to 360,000.) The 79% sensitivity of the original attending radiologists in interpreting mammograms in our study was calculated with a much larger study population and without case selection. It is likely that the majority of the current mammograms were not from the first screening examination, because we found no perceptible difference in the distribution of lesion characteristics, patient age, or CAD sensitivity in the subset of current mammograms with and the subset without corresponding prior mammograms (Table 4). First screening mammography studies should have a higher radiologist sensitivity, because the number of obvious cancers is likely to be greater in an initially screened population. In addition, only 14% of the current mammograms were obtained in women younger than 50 years.
We were concerned that a potential increase in false-positive interpretations might result from the use of CAD and lead to unnecessary recalls. Therefore, we measured the effect of the CAD system on radiologists' recall rate in the prospective clinical portion of the study (Table 5). The results of this analysis showed no significant increase in the recall rates of the 14 radiologists, who met MQSA qualification standards and were from a variety of clinical practices, before or after using the CAD system. This was not surprising, because the described CAD system does not detect cancer-compatible findings that are not visible to the radiologist on a mammogram. Instead, the CAD system simply marks findings that are at times overlooked during a reading session and that, once brought to the attention of the radiologist, will prompt reevaluation of that area on the mammogram. Thus, the false-positive marks by the CAD system are readily dismissed by the radiologist as being representative of an unimportant finding (ie, vascular or other benign calcification) or an area that, at repeated review, will show no evidence of anything that would prompt a recall.
The results of CAD analysis of the 1,083 current mammograms and 286 visible prior mammograms were very encouraging. The algorithms used by the CAD system were particularly sensitive in detecting microcalcifications that represented biopsy-proved cancer (99% sensitivity for the 406 current mammograms depicting microcalcifications), whereas the sensitivity for the detection of masses was lower (75% sensitivity for the 677 current mammograms depicting masses) (Table 3). The CAD system correctly marked 171 (60%) of the 286 visible prior mammograms (Table 1), and, importantly, it correctly marked 81% of the visible prior mammograms that were found to be actionable by the majority (at least three of five) of the panel radiologists (Table 2). It is important to reemphasize that all visible (and nonvisible) prior mammograms were the most recent screening examinations performed in patients who had a diagnosis of cancer on the basis of subsequent screening findings.
Not surprisingly, the CAD system marked an increasing number of lesions on the visible prior mammograms as the score of the five panel radiologists increased (ie, 34% of lesions that 0/5 radiologists detected to 92% lesions that 5/5 radiologists detected) (Table 1); these findings indicate that the more obvious the lesion characteristics were, the more likely that the lesion would be detected by the CAD system and the radiologists. However, on 177 (16%) of the 1,083 current mammograms (20 cases with no marks and 157 cases with marks in the wrong location), CAD did not detect cancers that were diagnosed by the original radiologists (Table 3). Thus, this technology currently must be viewed as a prompting aid to the radiologist to initiate a second review and not as a primary screening method.
Of importance in our evaluation was that the 115 cancers that were considered to be actionable by the panel radiologists were detected without CAD prompting, which indicates that there was sufficient evidence of cancer that was either not identified or, if seen, not acted on when the visible prior mammograms were first interpreted by the original radiologist. These results also indicate that not all retrospectively visible cancers are deemed actionable at blinded review. However, if sufficient attention is directed to the visible and actionable lesions in true clinical settings, then these lesions might be addressed and more cancers diagnosed earlier.
In this study we showed that the described CAD system, by selecting mammographic findings that warrant immediate repeated reading by the radiologist, offers the potential for improved cancer detection. However, double reading of mammograms by two different radiologists or by the same radiologist in two separate reading sessions also has been shown to result in increased sensitivity compared with one-pass interpretation by a single radiologist (19–24). Therefore, an important follow-up to this study will be a comparison between a single-radiologist reading with CAD prompting and a double-radiologist reading, in which the sensitivity, specificity, performance time, and costs of these two alternative approaches to enhanced mammographic interpretation are measured. The approach that leads to superior clinical results or that does so at lower cost is likely to prevail.
We conducted what is, to the best of our knowledge, the largest retrospective review of prior mammograms in a consecutive population of patients who had no symptoms and subsequently had a diagnosis of breast cancer on the basis of screening mammography findings. The results indicate that on 286 (67%) of 427 prior mammograms, breast cancers were visible retrospectively and that by using multiple readers (ie, a panel of radiologists), the cancers on 115 (27%) of these 427 prior mammograms could have been detected. The original attending radiologists' sensitivity for the detection of actionable lesions was 79%, with a false-negative rate of 21%. It is cases such as these 115 detectable cancers that should be targeted to increase the yield of screening mammography with appropriate prompting techniques, such as CAD (25) or multiple readings (19). The described CAD system successfully identified a large portion (77%) of these detectable and actionable cancers. Importantly, the use of a CAD system does not increase the work-up rate of the radiologists. The contributions of CAD, which is in the early stages of implementation, to complex image interpretation in mammography and to all fields of medicine is likely to be substantial.
|
APPENDIX |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONAPPENDIXReferences |
|---|
Current mammograms (n = 1,083).—Screening mammograms that were obtained between 1994 through 1996 from 13 facilities on which asymptomatic, biopsy-proved cancer was diagnosed.
Prior mammograms (n = 427).—The most recently obtained mammograms that corresponded to the current mammograms and were obtained 9–24 months before the current mammograms.
Visible prior mammograms (n = 286).—Prior mammograms on which a finding compatible with cancer was visible when retrospectively compared with the corresponding current mammogram on which the location of the cancer was known.
Actionable mammograms.—Mammograms interpreted by a radiologist to have a BI-RADS assessment score of 0, 4, or 5 (requiring further imaging evaluation or biopsy).
Actionable prior mammograms (n = 115).—Prior mammograms that were interpreted by the panel radiologists at blinded review to be actionable.
Original radiologists.—The radiologists who initially interpreted the prior mammograms as having no evidence of cancer. The sensitivity calculation of 79% was based on these radiologists' original interpretation.
Site radiologists.—The radiologists who reviewed the current mammograms to determine the location of the biopsy-proved lesion on an overlay, documented the lesion's characteristics, and provided a BI-RADS assessment of the lesion.
Designated radiologists.—The radiologists who compared the prior mammograms with the current mammograms, with knowledge of the biopsy-proved cancer's location as defined on the overlay, to assess the visibility of the cancer retrospectively on the prior mammogram.
Panel radiologists.—The 20 radiologists (five each who independently reviewed four case sets) who reviewed the visible prior, normal, and current mammograms to assess the actionability of the mammogram.
Computation of Radiologists' Sensitivity
The 115 actionable prior mammograms represented 27% (115/427) of all prior mammograms. Recall that all 427 prior mammograms had corresponding subsequent current mammograms that were obtained 9–24 months later (mean, 14 months) and on which the cancer was detected in the current year. If we assume constancy in the demographics of the population they served and in the number of patients screened between the prior and current examinations, then we can reasonably assume that these radiologists detected 427 other cancers in the prior year. Because we know that these radiologists also missed cancers on the 115 actionable prior mammograms in the prior year, we can calculate the total number of detectable cancers in the prior year as 427 detected cancers plus 115 missed but actionable cancers, which equals 542 detectable cancers. Therefore, the sensitivity and false-negative rate of these radiologists, Srad and FNrad, respectively, in a normal clinical environment can be calculated from this data set as follows: Srad = 427 detected cancers/542 detectable cancers = 79% and FNrad = 1 - Srad = 21%.
Facilities that provided case material:
Alta Breast Center, Oakland, Calif; David Grant Medical Center, Travis Air Force Base, Calif; El Camino Hospital, Mountain View, Calif; Kaiser Permanente Medical Center, Sacramento, Calif; Kaiser Permanente Medical Center, South San Francisco, Calif; Park Nicollet Clinic, Minneapolis, Minn; Sequoia Hospital, Redwood City, Calif; Walnut Creek Radiology, Walnut Creek, Calif. The following facilities also provided case material and participated in the prospective assessment of radiologists' recall rate: Kaiser Permanente Medical Center, Redwood City, Calif; Kaiser Permanente Medical Center, San Francisco, Calif; Ochsner Clinic, New Orleans, La; Susan G. Komen Breast Center, Dallas, Tex; and Vanderbilt University Medical Center, Nashville, Tenn.
|
Acknowledgments |
|---|
|
Footnotes |
|---|
Abbreviations: BI-RADS = Breast Imaging Reporting and Data System CAD = computer-aided detection MQSA = Mammography Quality Standards Act
Author contributions: Guarantors of integrity of entire study, all authors; study concepts and design, L.J.W.B., C.J.D., S.A.F., D.B.K., E.A.S., L.T., C.J.V.; definition of intellectual content, L.J.W.B., C.J.D., S.A.F., D.B.K., E.A.S., L.T., C.J.V., R.A.C.; literature research, L.J.W.B., S.A.F., E.A.S., C.J.V.; clinical studies, K.F.O.; data acquisition, K.F.O.; data analysis, S.A.F., E.A.S., C.J.V., S.A.W., R.A.C., K.F.O.; manuscript preparation and review, all authors; manuscript editing, S.A.F., E.A.S., C.J.V., S.A.W., R.A.C.
|
References |
|---|
|
TOPAbstractIntroductionMATERIALS AND METHODSRESULTSDISCUSSIONAPPENDIXReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  M. F. Chiang, R. Gelman, S. L. Williams, J.-Y. Lee, D. S. Casper, M. E. Martinez-Perez, and J. T. Flynn Plus Disease in Retinopathy of Prematurity: Development of Composite Images by Quantification of Expert Opinion Invest. Ophthalmol. Vis. Sci., September 1, 2008; 49(9): 4064 - 4070. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. Gur Imaging Technology and Practice Assessment Studies: Importance of the Baseline or Reference Performance Level Radiology, April 1, 2008; 247(1): 8 - 11. [Full Text] [PDF]
|
|
|
|
 |
|
 M. Gromet Comparison of Computer-Aided Detection to Double Reading of Screening Mammograms: Review of 231,221 Mammograms Am. J. Roentgenol., April 1, 2008; 190(4): 854 - 859. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. A. Taylor, R. Greenhalgh, R. Ilangovan, E. Tam, V. A. Sahni, D. Burling, J. Zhang, P. Bassett, P. J. Pickhardt, and S. Halligan CT Colonography and Computer-aided Detection: Effect of False-Positive Results on Reader Specificity and Reading Efficiency in a Low-Prevalence Screening Population Radiology, April 1, 2008; 247(1): 133 - 140. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Robinson, S. Halligan, S. A. Taylor, S. Mallett, and D. G. Altman CT Colonography: A Systematic Review of Standard of Reporting for Studies of Computer-aided Detection Radiology, February 1, 2008; 246(2): 426 - 433. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. J. Kim, W. K. Moon, N. Cho, J. H. Cha, S. M. Kim, and J.-G. Im Computer-aided Detection in Full-Field Digital Mammography: Sensitivity and Reproducibility in Serial Examinations Radiology, December 1, 2007; 246(1): 71 - 80. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. F. Brem Blinded Comparison of Computer-Aided Detection with Human Second Reading in Screening Mammography: The Importance of the Question and the Critical Numbers Game Am. J. Roentgenol., November 1, 2007; 189(5): 1142 - 1144. [Full Text] [PDF]
|
|
|
|
|
|
R. L. Ellis, A. A. Meade, M. A. Mathiason, K. M. Willison, and W. Logan-Young Evaluation of Computer-aided Detection Systems in the Detection of Small Invasive Breast Carcinoma Radiology, October 1, 2007; 245(1): 88 - 94. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. G. Elmore and R. J. Brenner The More Eyes, the Better to See? From Double to Quadruple Reading of Screening Mammograms J Natl Cancer Inst, August 1, 2007; 99(15): 1141 - 1143. [Full Text] [PDF]
|
|
|
|
|
|
J. L. Jesneck, J. Y. Lo, and J. A. Baker Breast Mass Lesions: Computer-aided Diagnosis Models with Mammographic and Sonographic Descriptors Radiology, August 1, 2007; 244(2): 390 - 398. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. K. Yang, W. K. Moon, N. Cho, J. S. Park, J. H. Cha, S. M. Kim, S. J. Kim, and J.-G. Im Screening Mammography-detected Cancers: Sensitivity of a Computer-aided Detection System Applied to Full-Field Digital Mammograms Radiology, July 1, 2007; 244(1): 104 - 111. [Abstract] |
= values for the 20 panel radiologists. * = values for additional three radiologists who were eliminated from the study because of low sensitivity or low specificity (see text). ¤ = the average sensitivity (84%) and specificity (82%) of the 20 remaining radiologists.