| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Letters to the Editor |
Department of Radiology, Imaging Research, University of Pittsburgh and Magee-Womens Hospital of the University of Pittsburgh Medical Center, Suite 4200, 300 Halket Street, Pittsburgh, PA 15213-3180 e-mail: gurd{at}upmc.edu
The article by Dr Birdwell and colleagues (1) in the August 2005 issue of Radiology highlights the one important observation we can state about published prospective studies attempting to define the benefits of computer-aided detection (CAD) in the clinical environment to date (1–3). All three studies—and in one of those studies, even the "low" and "high" volume readers—demonstrated consistently that changes in detection rates "track" changes in recall rates (1–3). When recall rates increase so do detection rates and with a similar order of magnitude. This suggests that the operating point of the observers (threshold in deciding whether to recall a woman) changes in a manner that is similar to becoming more (or less) "conservative"—namely, recalling more women or recommending recall for "less suspicious" findings, on average. This observation is reported consistently in all three studies and for all subsets of readers (ie, for academic and nonacademic practices, as well as for high- and low-volume readers). Since ultimately all of the abnormalities recalled for additional procedures are actually depicted on the mammograms in question (otherwise these would not be detected either by the observer and/or with CAD), the observed improvement in detection rate does not necessarily mean an actual improvement in performance.
In the study reported by Dr Birdwell and colleagues (1), it is likely that if the radiologists had simply been told or trained to be or had naturally migrated to being more "conservative" and, as a result, had increased their recall rates by 8% (or generated 73 additional recalls), one additional cancer would have been found. On the basis of their own average detection per recall, it is possible that the two additional cancers would have been found, and it is plausible that more than two could have been found. Hence, improvement in detection (improvement by only two findings of cancer) with an accompanying increase in recall rate could very well be equivalent to a change in the operating point of the observers when CAD was used. This "effect" could be achieved in ways other than the use of CAD (4).
Another minor comment: It is not clear at all how, with only two additional cancers, the reported lower bound of the confidence limit is positive. Last, in the case of clustered microcalcifications, omission is a primary reason for not recalling a woman, and since the cluster is actually visible on the mammogram, vigilance is an important issue. When a cluster is missed, the likelihood is high that it is related to vigilance. In the study by Freer and Ulissey (2), the most plausible explanation why seven of the eight additional cancers missed by radiologists alone (without CAD) were associated with microcalcifications is lower vigilance in searching for these clusters when the images were read prior to the presentation of CAD results.
 |
References |
|---|
 TOP References References |
|---|
and
Debra M. Ikeda, MD*
Department of Radiology, Stanford University Medical Center, Stanford, Calif* Division of Breast Imaging, Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115
e-mail: rbirdwell{at}partners.org
In their letter, Drs Sumkin and Gur acknowledge that published prospective studies of CAD in screening mammography, including a subset of their own study (1), have shown (a) an increase in the cancer detection rate, which is accompanied by (b) a similar magnitude increase in the recall rate. We agree with this statement as was reported in our article (2). However, Drs Sumkin and Gur then suggest that this improvement in cancer detection might simply be related to factors other than the use of CAD, and, if "the radiologists had simply been told or trained to be or had naturally migrated to being more ‘conservative,’" similar results might have been observed. We believe this is improbable.
In order for a radiologist to diagnose a cancer, the cancer must first (a) be detected (perception) and then (b) be correctly interpreted (cognition) as warranting recall. Simply being "more conservative" does not address observational oversights. Rather, being more conservative suggests that the radiologist would recall more detected lesions but with less compelling features for breast cancer. Being told or trained to be or becoming more conservative is primarily a function of interpreting a perceived abnormality—but the abnormality must first be perceived, either by the observer or with the aid of CAD.
An important benefit of CAD is marking cancers that are clearly present (ie, conspicuous) on the mammogram but that are simply "overlooked" by the radiologist in his or her review of large numbers of cases, the majority of which are normal. Further, CAD often correctly marks cancers that are overlooked because they are "subtle," at times prompting the radiologist to (correctly) recall the woman for further work-up. Conversely, radiologists will often (appropriately) dismiss such CAD prompts because the imaging features are not sufficiently compelling to warrant recall (3). By simply becoming more conservative, radiologists will certainly increase their recall rates but will not necessarily increase their cancer detection rates to the same extent, because the additional overcalling may not include those "obvious" cases that were overlooked at the initial reading nor those that are overlooked because of their subtle appearance.
Since the goal of screening mammography is to detect cancers before they become clinically apparent, strategies to maximize the screening outcome are important to incorporate into clinical practice. For example, double reading has clearly been shown to increase cancer detection—and CAD is simply another method to attain this goal. In a historical controlled study similar to that of Dr Gur and colleagues (1), Cupples et al (4) reported that their use of CAD (v2.0; R2 Technology, Los Altos, Calif) increased their cancer detection rate by 16.1%, whereas their recall rate increased by only 8.1%. They also found that the detection rate of invasive cancers 1.0 cm or smaller was increased 164%. This latter result deserves emphasis, in that finding cancers at an earlier stage is as important as finding more cancers.
There is general agreement that more women should be recalled than eventually turn out to have cancer—that is, the cancer detection rate is a fraction of the recall rate. The recall rate is, to some extent, a "surrogate" for the level of conservatism in interpretation by each radiologist. This interpretive bias should be similar for those suspicious findings detected by the radiologist unaided by CAD, as well as for those detected with the aid of CAD. Thus, we believe that in order to diagnose more cancers, more women will need to be recalled. It is reasonable that the increase in cancer detection with CAD will be associated with a comparable increase in the recall rate.
Drs Sumkin and Gur theorize that the reason Freer and Ulissey (5) reported more cancers with associated microcalcifications detected with CAD is lower vigilance on their part in searching for microcalcification clusters when the images were read prior to the presentation of CAD results, as is done in a sequential-reading prospective study such as ours and that of Freer and Ulissey. As stated in our article, our participating radiologists may have been "likely to be overly vigilant in their initial (pre-CAD) reading" since they "knew that any ‘misses’ on their part would be recorded" (1). This emphasizes the point that the radiologists might have been more (not less) vigilant in looking at the screening mammograms prior to employing CAD marks, but this also is only conjecture. Drs Sumkin and Gur questioned how the lower bound of the confidence limit could be positive with only two additional cancers detected. In the study by Freer and Ulissey (5), and as discussed in our article (see reference 12 in our article), the projected increase in cancers and corresponding confidence intervals reflect the intended use of CAD as an adjunct to the initial reading without CAD. In a letter to the editor (6) regarding that study, Miller stated that "the authors' study was appropriately designed and analyzed to reflect the additive—rather than comparative—value of computer-aided detection." The same can be said of this study, and thus the presence of even a single new cancer detected with CAD is sufficient to demonstrate that the benefit is nonzero.
Finally, the argument put forth that asking radiologists to be more conservative may be as effective as using CAD is rather similar to the idea that asking drivers to drive more cautiously may be as effective as driving cars with seat belts, antilock braking systems, and airbags in reducing car injuries. In both instances, these life-saving technologies serve as "safety nets" to assist in decreasing adverse events due to fatigue, distraction, and other factors leading to perception errors.
|
References |
|---|
|
TOPReferencesReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  R. M. Nishikawa and L. L. Pesce Computer-aided Detection Evaluation Methods Are Not Created Equal Radiology, June 1, 2009; 251(3): 634 - 636. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
