| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Letters to the Editor |
Departments of Surgery* and Radiology, Royal Free and University College Medical School, 67-73 Riding House Street, London W1P 7LD, United Kingdom
Editor:
We read with interest the article by Dr Huber and colleagues (1) in the August 1998 issue of Radiology that elegantly describes a computer-assisted method for quantifying contrast material–enhanced color Doppler ultrasound signals from both malignant and benign breast lesions.
To assess the sensitivity and specificity (diagnostic accuracy) of contrast-enhanced color Doppler ultrasonography (US), Dr Huber and colleagues' patient grouping was divided by using the median value of the color pixel density as a cutoff point. A 
2 test was then applied to determine a statistical difference in color pixel density between benign and malignant lesions.
Our concerns with this aspect of the method are twofold. First, one is unable to anticipate a clinically useful cutoff point prior to analysis; to do so is assumptive and therefore completely arbitrary. Second, this method ignores an important outcome gained in part from the use of paired control subjects (2,3), that is, whether contrast-enhanced examination can be shown to have a true advantage over conventional color Doppler US.
We have also been working with the US contrast agent SH U 508A (Levovist; Schering, Berlin, Germany) (4) along similar lines, and our results with 58 patients (16 recurrences) demonstrated a diagnostic accuracy of 90% for this agent in differentiating breast cancer recurrence from scar tissue—a significant and highly useful improvement over conventional color Doppler US (diagnostic accuracy, 80%; P = .038). We in effect compared peak (enhanced) color pixel densities with baseline (nonenhanced) color pixel densities with the use of receiver operating characteristic (ROC) curves (Figure). This method also uses the paired nature of the study group (enhanced vs nonenhanced) and therefore more accurately demonstrates any difference that may be produced by the addition of a contrast agent. In addition, useful cutoff points along the ROC curves can be easily defined for both contrast-enhanced and nonenhanced baseline scans. The method we used has been fully described in previous issues of Radiology (2,3).
|
References
Department of Radiology, Lainz Hospital, Wolkersbergenstrasse 1, 1130 Vienna, Austria
We are pleased by the interest in our article (1). Dr Winehouse and colleagues remark that using the median as a cutoff point is arbitrary and that an ROC analysis would help to determine an appropriate cutoff level. This is a frequent comment, and we have already discussed this point with one of the reviewers of the article. We perform ROC analyses frequently in our studies, but we intentionally refrained from doing so in this particular instance. We call to mind that, as the title indicates, this is very preliminary work. Here, we intended to point out where current methodologic limitations lie rather than to determine the ultimate diagnostic value of this method. Our sample consisted of 31 patients with malignant and 16 with benign lesions.
We agree that there is a paired nature in the study group. This means that the result of the examination in a given patient may, for example, be false-negative with the nonenhanced and true-positive with the enhanced technique, or it may shift from true-negative to false-positive. It indeed would be interesting to examine how the classification changes in an individual patient. Unfortunately, the 2 test does not answer this question, nor does the ROC analysis. ROC analysis does nothing more than plot specificity against sensitivity with different cutoff values. When two methods or criteria are compared, specificity and sensitivity are calculated separately for each method, but the diagnoses achieved in individual patients are not compared. Indeed, ROC analysis enables a more systematic and differentiated comparison of two methods, but by no means does it use the paired nature of the study group. While Dr Winehouse and colleagues may disagree, having cited the articles by Hanley and McNeil (2,3), we still believe this is true for the commonly used ROC analysis.
The method described by Hanley and McNeil (3) clearly differs from the usually applied ROC analysis. Here, the authors indeed attempt to adapt the ROC approach to data sets with different data derived from the same cases. The clear drawback is, however, that it is highly complex and in our opinion not sufficiently robust to avoid statistical artifacts. Furthermore, it will be extremely difficult to trace any artifacts back to the original data. When different methods and criteria are compared, they are commonly applied to the same samples. Although published in 1983, the method described in this article has not yet achieved the range of a standard approach for this very common problem. Thus, we believe that standard statistical approaches familiar to the reader should be used to ensure that the results are understood and also amenable to criticism.
Our main reason for not using ROC analysis was the small sample. In our case, the distribution of values could not be expected to be stable enough to be proved the same when more patients were added. Therefore, "fitting" the cutoff value to the study sample by ROC analysis would clearly run the risk that falsely high significance levels would be calculated and that the results would appear better than they actually were. Splitting the group by using simply the median as a cutoff point is the procedure of choice because of the preliminary nature of the study imposed by the small study sample and the retrospective nature of the analysis. The procedure does run the risk that the significance levels are underestimated, but it is probably more honest. If a true prospective study is planned—that is, with predefined cutoff values derived from preliminary studies—we believe it is justified then to tailor the thresholds to clinical needs.
References
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
= peak color pixel density (enhanced), 
= baseline color pixel density (nonenhanced).