| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH |
|
|
|
|||||||
|
|||||||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Breast Imaging |
1 From the Dept of Radiology (E.D.P., E.B.C., B.M.H., M.L.B., R.E.J., C.M.K.) and UNC-Lineberger Comprehensive Cancer Ctr (E.D.P., E.B.C., M.L.B.), Univ of North Carolina School of Med, 101 Manning Dr, 515 Old Infirmary, Chapel Hill, NC 27599-7510; Dept of Biostatistics, Univ of North Carolina School of Public Health, Chapel Hill (E.O.K., K.E.M.); Dept of Radiology, Christ Hosp, Cincinnati, Ohio (M.P.B.); Dept of Radiology, Wake Forest Univ School of Medicine, Winston-Salem, NC (R.I.F.); and Dept of Radiology, Duke Univ School of Med, Durham, NC (M.S.S., J.A.B., R.W.). From the 1999 RSNA scientific assembly. Received Mar 29, 2001; revision requested May 23; revision received Oct 4; accepted Nov 12. Supported in part by Department of Defense Grant 5-36845, "Evaluation of Digital Mammography Display." Address correspondence to E.D.P. (e-mail: etta_pisano@med.unc.edu).
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
MATERIALS AND METHODS: After being trained in interpretation of digital mammograms, eight radiologists interpreted 63 digital mammograms, all with old studies for comparison. All studies were interpreted by all readers in soft-copy and printed-film display, with interpretations of images in the same cases at least 1 month apart. Mammograms were interpreted in cases that included six biopsy-proved cancers and 20 biopsy-proved benign lesions, 20 cases of probably benign findings in patients who underwent 6-month follow-up, and 17 cases without apparent findings. Area under the receiver operating characteristic curve (Az), sensitivity, and specificity were calculated for soft-copy and printed-film display.
RESULTS: There was no significant difference in the speed of interpretation, but interpretations with soft-copy display were slightly faster. The differences in Az, sensitivity, and specificity were not significantly different; Az and sensitivity were slightly better for interpretations with printed film, and specificity was slightly better for interpretations with soft copy.
CONCLUSION: Interpretation with soft-copy display is likely to be useful with digital mammography and is unlikely to significantly change accuracy or speed.
© RSNA, 2002
Index terms: Breast radiography, 00.112 • Diagnostic radiology, observer performance • Images, display • Radiography, digital, 00.1215 • Screens and films, 00.112
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Currently, only technology with a cathode ray tube monitor is available for the soft-copy display of digital mammograms. Other technologies, such as liquid-crystal display, field-emission display, and organic light-emitting diode displays, should become available in the next 1–10 years (2). However, the best high-quality cathode ray tube technology is limited compared with printed-film display (2). The spatial resolution is less than one quarter of the film resolution, and the luminance range is much lower (3–7).
However, both of these factors can be mitigated. Full spatial resolution is possible with roam-and-zoom techniques. This must take place seamlessly, so that reading an image with a monitor is similar to reading mammograms on film with a magnifying glass. Reader performance can be degraded if the tool makes the task more difficult (8). Further, the luminance difference may not be that important. Findings in two studies (9,10) demonstrated that detection of mammographic features does not decrease when soft-copy display luminance ranges are used instead of mammographic lightbox ranges. However, larger-scale performance studies that include an evaluation of the effect of display characteristics on the detection and diagnosis of breast cancer are required to assure patients and physicians that printed-film and soft-copy display are equivalent. In addition, the speed of interpretation of the low-cost screening studies must not be compromised if this technology is to be implemented clinically (2).
The purpose of this study was to compare the speed and accuracy of interpretations by radiologists of digital mammograms (SenoScan; Fischer Imaging, Denver, Colo) that were displayed with two different media, printed-film display and soft-copy display. For soft-copy display, a locally developed soft-copy workstation (Mammoview; built at University of North Carolina, Chapel Hill) (11) was used. The design of this workstation was based in part on the results of previous eye-tracker studies (12).
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Digital Mammogram Selection
A total of 63 digital mammograms were identified for use in the study from the case files of a single institution. The cases included six biopsy-proved cancers (five masses and one cluster of calcifications) and 20 biopsy-proved benign lesions (18 masses and two clusters of calcifications). The remaining cases were in 20 patients who underwent 6-month follow-up for probably benign findings (14 masses and six clusters of calcifications) and 17 cases without apparent findings. In 36 of 37 patients whose mammograms were included in the study who did not undergo biopsy, follow-up screen-film mammograms were normal at 1 year after the digital mammogram was obtained. In the remaining patient, the screen-film mammogram was unchanged at 6 months after the digital mammogram was obtained.
The composition of the breast on the mammograms in the patients whose images were included was extremely dense (n = 12), heterogeneously dense (n = 14), characterized by scattered fibroglandular densities (n = 27), or almost entirely fatty (n = 10). Lesion size measured on mammograms ranged from 4 to 60 mm for the patients in whom biopsy was performed and from 4 to 40 mm for those in whom follow-up was performed.
A case in which digital mammography was performed was deemed suitable for inclusion in this study if it included only four standard digital mammograms (two craniocaudal views and two mediolateral oblique views) and if at least one prior screen-film mammogram was obtained in the previous 10–65 months for comparison. If more than one such eligible prior mammogram existed, only the most recent screen-film mammogram was used for comparison in the study. The demographic profiles of the patients whose images were included in this study are shown in Table 1.
|
The digital mammograms used in this study had a spatial resolution of 50 µ per pixel with a bit depth of 12 bits per pixel, for 40 megabytes of digital information per image.
Preparing Mammograms for Display
Printing digital mammograms.—All digital mammograms were printed by using the image-processing algorithm that was recommended by the manufacturer at the time the study began. This algorithm, manual intensity windowing without other processing, was applied, and a mammography-certified radiologic technologist with more than 20 years experience in imaging of the breast printed the images. The images were printed on film (Kodak Ektascan HN; Eastman Kodak, Rochester, NY) with a laser film printer (Kodak 2180 EktaScan; Eastman Kodak). This printer is capable of printing 12 bits per pixel. Images were printed at a 50-µm pixel size with a matrix of 4,096 x 5,120 pixels. The laser film was subsequently processed by using a medical film processor (QX-400; Konica Medical, Norcross, Ga). Images in all cases were reviewed for quality by the same technologist who printed them. No cases were excluded from the study on the basis of this review of images.
Digitizing screen-film mammograms.—To allow for soft-copy display of the prior studies for comparison, all prior comparison cases were digitized at 50-µm spatial resolution and 12 bits of contrast resolution by using a digitizer (Lumisys 100; Lumisys, Sunnyvale, Calif).
Processing of the digitized screen-film images used for comparison involved a standardization step to allow the images to appear on video monitors with an appearance that would be similar to that of a mammogram on a lightbox. A computer scientist manually adjusted the gray scale on the monitors by means of standardization of luminance response over the perceived brightness range for each of the display environments, according to digital imaging and communications in medicine, or DICOM, display standard 3.14 (13).
Soft-Copy Workstation Description
The soft-copy workstation was locally developed with funding provided by the Department of Defense and was used for soft-copy display. The soft-copy display system used in this study included two (100-foot-lambert) monitors (model 1654 High Brightness; Clinton Electronics, Loves Park, Ill), each with a display card (Md5Sun; Dome Imaging, Waltham, Mass) and run with one computer (UltraSparc model 2200; Sun Microsystems, San Jose, Calif). Both the monitors and the display cards had a display matrix size of 2,048 x 2,560 pixels.
The device was equipped with a simple user interface that allowed the reader to select preset intensity windowing options with the click of a mouse button. In addition, the system provided roam-and-zoom functions with the click of a mouse button so that all parts of the digital and digitized screen-film mammograms could be viewed at full spatial resolution. The roam-and-zoom function was used with the full monitor screen size of 2,048 x 2,560 pixels for viewing a selected portion of the 3,072 x 4,800-pixel full-resolution digital mammogram.
The hanging protocol for the images with soft-copy display was implemented according to reader preference. That is, each reader could specify the orientation that he or she preferred for the right and left images and old and new images to be displayed, and the system was adapted for each reader. The default presentation, available without any changes, was for the craniocaudal views to be displayed on the left monitor and the mediolateral views to be displayed on the right monitor, with images from the new examination located at the bottom and those from the prior examination at the top. All soft-copy presentations allowed for the display of four images per monitor, with the reader able to select and view any of the eight displayed images at full spatial resolution. Figure 1 shows the soft-copy workstation for digital mammography.
|
Participants and prior experience with digital mammograms.—A total of eight readers who were radiologists (including C.M.K., M.P.B., R.I.F., M.S.S., J.A.B., R.W.), with exclusion of the pilot-study reader, participated in the reader study. Seven of eight readers had experience in the interpretation of digital mammograms through participation in prior reader studies at the University of North Carolina. This prior experience included the interpretation of 200 mammograms with printed-film display after these readers received training with 28 printed digital mammograms with pathologically proved lesions.
One of eight readers gained experience in interpretation of digital mammograms through interpretation of 10 digital mammograms in cases that were included in the set of 28 printed digital mammograms with pathologically proved lesions that had been used to train the other seven readers, as noted previously. In this training session, this reader evaluated the cases and identified the findings, and immediately afterward, the reader was given information about the pathologic diagnoses for lesions visible on the mammograms. This set of 10 cases was different from those used in the pilot reader study.
All participating readers fulfilled U.S. Food and Drug Administration requirements for screen-film interpretation of mammograms on the basis of Mammography Quality Standards Act regulations.
Training for the study.—None of the readers who participated in this study had prior experience in interpretation of digital mammograms by using soft-copy display systems. All readers were trained specifically in the tasks of the study, including interpretation of digital mammograms with both printed-film and soft-copy display. Each reader was asked to provide his or her hanging preference so that the printed-film and soft-copy images could be displayed in the order and at the site preferred by the reader. Images with printed-film display were hung on a multipanel light box (Two-tier Desktop Mammography Illuminator; Picker, Waycross, Ga), with suitable masking of extraneous light.
Training regarding the use of the soft-copy display system was accomplished by displaying 20 digital mammograms that were obtained with the same unit used in this study and that were selected from cases that were not included in this study. A computer scientist (B.M.H.) first demonstrated how to use the soft-copy system and then assisted the readers in viewing mammograms by using the soft-copy display system in the first five cases. During this training, the readers viewed and interpreted the images and worked with a research assistant in completing data sheets regarding their interpretation of the images, just as they would in the actual study.
After the completion of review of mammograms in the first five cases, the reader was given instructive feedback regarding the presence of pathologically proved findings on the images. Mammograms in the next 15 cases were then viewed and interpreted exactly as the reader would view and interpret the images in the subsequent reader study; that is, the radiologists themselves viewed and interpreted the images, and the computer scientist was available only to answer questions about how to use the soft-copy system. Instructive feedback regarding the pathologically proved diagnoses was provided after each case was reviewed and interpreted.
If readers were not able to complete the soft-copy interpretations of images in all 63 study cases on the same day as their initial training, they were given an additional five training cases (not used in the study) with images to read and practice with at the start of each additional reading day to refamiliarize themselves with the soft-copy display system, given their lack of prior experience with soft-copy interpretation. Most readers used images in a total of 25 training cases. Some used images in 30 cases.
All readers also trained with three cases in which digital mammograms were printed to laser film so that they would become familiar with the reading and interpretation task, the dictation system, and the interaction with the research assistant by using the hard-copy presentation format.
Reader study method.—The mammograms in the 63-case set were assigned to two blocks, block A (31 mammograms) and block B (32 mammograms) for soft-copy and printed-film display. Four readers read all 63 mammograms in soft-copy display first; two readers started with the images in block A and two readers started with the images in block B, according to random assignment. Similarly, the remaining four readers read all 63 mammograms in printed-film display first; two readers began with the images in block A and two readers began with the images in block B.
At least 1 month passed before each of the two groups of four readers read the images in the other display. Again, half the readers were randomly assigned to begin with the images in block A first. The other half began with images in block B.
This counterbalancing of viewing and interpreting images in soft-copy and printed-film display was intended to mitigate the effects of learning and fatigue.
The readers were instructed to read the images as screening images, with the results dictated. A research assistant (E.B.C.) timed the readings from the moment the radiologist first viewed the images to the moment that the radiologist finished the dictation with a stopwatch. The time to display or hang the images was not included in this measurement.
After the dictation was completed for each case, the research assistant assisted the reader in filling out a data sheet to record mammographic findings and their locations, the radiologist’s belief about the probability of malignancy for each finding on the basis of five categories (definitely not malignant, probably not malignant, possibly malignant, probably malignant, definitely malignant), and the radiologist’s recommendation for follow-up or further testing.
All readings were performed in a dark environment (dimmer set to 10 lux), which was suitable for interpretation of mammograms. A magnifying glass was provided for printed-film interpretation. Reading sessions were split into 50-minute periods, with a mandatory 5-minute break per hour. Readers took additional breaks as needed.
All readings were videotaped for further analysis, as needed. Figure 2 shows the dedicated mammographic light box used for interpretation of screen-film mammograms.
|
Because there were four outcomes of interest, Bonferroni corrections were applied. Each variable was tested as follows: Az, 
= .02; sensitivity, = .01; specificity, = .01; and time, = .01. Retrospective power analysis was performed to describe the power of the study.
Since the subject in one case in which findings were classified as negative (no cancer) was lost to follow-up after 6 months of mammographic surveillance, the entire analysis was rerun after the data concerning the interpretations of that subject’s mammograms were excluded to ensure that the results did not change substantially.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
|
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
Results of this study could have possibly caused an underestimation of the benefits of soft-copy interpretation, because the only image-processing tool available to the readers was a group of preset intensity window settings that the readers could select as desired. Findings in prior work (1) suggested that other image processing algorithms might benefit radiologists for lesion detection and lesion characterization. In addition, it is possible that the availability of other soft-copy tools (ie, those not provided with the soft-copy workstation), specifically other image-processing algorithms and computer-aided diagnosis, might have improved the performance of the readers, both in terms of time and accuracy. Other variations might have caused worse reader performance as well or increased the time needed for interpretation (8). Given that the number of cases with findings was greater in this study than it usually is in a population undergoing screening, one would expect readers to interpret screening mammograms even more quickly than they did with the cases included in this study.
In fact, the results of this study showed no significant difference in time required to interpret printed-film and soft-copy digital mammograms, with interpretation of soft-copy mammograms being slightly faster than that of printed-film mammograms. The results were somewhat surprising, given that the soft-copy interpretation required the readers to roam and zoom and interact with the images to change the intensity windowing settings. We believe that the times were shorter for soft-copy display because the soft-copy workstation allowed the readers to interact with the images in an extremely comfortable and intuitive manner, and after informal conversations with the readers who participated in the study, that impression was confirmed.
Since the results were not statistically significant, there is a chance that another study with the same design might show that radiologists can interpret the printed-film images faster. The most important conclusion we can draw from this study is not that soft-copy display allowed faster or better interpretation but that the difference in time and accuracy was not large between interpretation with soft-copy display and printed-film display when the workstation was fast and user friendly.
Another factor that might have reduced the reader performance with soft-copy display was the digitization of prior mammograms for comparison. This was only required for the display of the prior studies used for the soft-copy part of the experiment. The original screen-film mammograms were used for comparison with the printed-film images. Even though the digitized images were of high quality, and the digitized images were only used for comparison with those in study cases, there may have been a difference in the information on these images compared with that on the original mammograms, and this difference might have affected the results of this study. The effect of the digitization process on the conspicuity of lesions on mammograms has never been well studied, to our knowledge, so the magnitude of this effect, if it was present, is not known.
Results of this study give some reassurance to mammographers who interpret digital mammograms with high-quality soft-copy workstations that their performance is similar to their performance with printed-film display. Radiologists in this study had extensive one-on-one hands-on training with use of the soft-copy workstation before we began the study, and the interface was adapted to each reader’s individual preferences for how the images should be displayed (ie, where on the monitors to display right and left and old and new mammograms). Similar results might not have been achievable if either of these factors were not present for this study. Readers seeking to adopt soft-copy workstations as part of their clinical environment should insist that the ergonomics of their workstations are comfortable to all readers and conducive to good-quality interpretations. In addition, we believe appropriate training is necessary. Federal regulations regarding digital mammography (15) require that radiologists have at least 8 hours of continuing medical education credits in digital mammography before they can interpret this type of image.
In this study, the reader had the ability to see the old images in soft-copy format without having to turn to a set of lightboxes set up nearby. This allowed the reader to perform the interpretation task, including the comparison with prior images, in the standard clinical manner, without any distractions. In most patients, old screen-film mammograms are available for comparison, and once digital mammography is more widely used, it is likely that images will be acquired from different digital systems in patients. We believe it is important for the manufacturers to plan for this user requirement by providing the ability to import other digital images to their display systems. This is most easily achieved if all systems are compatible with the recently released digital imaging and communications in medicine standards (16) and if they have hardware and software to allow the easy importation of images from other digital mammographic machines or film digitizers, through a disk drive or from the Internet.
|
ACKNOWLEDGMENTS |
|---|
|
FOOTNOTES |
|---|
Author contributions: Guarantor of integrity of entire study, E.D.P.; study concepts, E.D.P., E.B.C., K.E.M., B.M.H.; study design, E.D.P., K.E.M.; literature research, E.D.P.; experimental studies, E.B.C., B.M.H., C.M.K., M.P.B., R.I.F., M.S.S., J.A.B., R.W.; data acquisition, E.B.C., B.M.H.; data analysis/interpretation, E.O.K., K.E.M.; statistical analysis, E.O.K., K.E.M.; manuscript preparation, E.D.P.; manuscript definition of intellectual content, editing, revision/review, and final version approval, all authors.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODSRESULTSDISCUSSIONREFERENCES |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH |
| RADIOLOGY | RADIOGRAPHICS | RSNA JOURNALS ONLINE |
