| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
Thoracic Imaging |
1 From the Kurt Rossmann Laboratories for Radiologic Image Research, Department of Radiology, University of Chicago, 5841 S Maryland Ave, MC 2026, Chicago, IL 60637 (S.G.A., F.L., M.L.G., H.M., K.D.); and Department of Radiology, Azumi General Hospital, Nagano, Japan (S.S.). From the 2001 RSNA scientific assembly. Received August 13, 2001; revision requested October 10; revision received March 8, 2002; accepted May 7. Supported in part by United States Public Health Service grants CA83908, CA64370, and CA62525, funding from the University of Chicago Cancer Research Center, and a grant from the Grant Healthcare Foundation through the Medical Imaging and Research Foundation of the University of Chicago. Address correspondence to S.G.A. (e-mail: s-armato@uchicago.edu).
 |
ABSTRACT |
|---|
 TOP ABSTRACT INTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONREFERENCES |
|---|
MATERIALS AND METHODS: A database of 38 low-dose CT scans with 50 lung nodules was obtained from a lung cancer screening program. Thirty-eight of the nodules represented biopsy-confirmed lung cancers that had not been reported during initial clinical interpretation. A computer detection method that involved the use of gray-level thresholding techniques to identify three-dimensionally contiguous structures within the lungs was applied to the CT data. Computer-extracted volume was used to determine whether a structure became a nodule candidate. A rule-based scheme and a cascaded automated classifier were applied to the set of nodule candidates to distinguish actual nodules from areas of normal anatomy. Overall performance of the computer detection method was evaluated with free-response receiver operating characteristic (FROC) analysis.
RESULTS: At a specific operating point on the FROC curve, the method achieved a sensitivity of 80% (40 of 50 nodules), with an average of 1.0 false-positive detection per section. Missed cancers were detected by the computerized method with a sensitivity of 84% (32 of 38 nodules) and a false-positive rate of 1.0 per section.
CONCLUSION: With an automated lung nodule detection method, a large fraction (84%, 32 of 38) of missed cancers in a database of low-dose CT scans were detected correctly.
© RSNA, 2002
Index terms: Cancer screening • Computed tomography (CT), image processing • Computers, diagnostic aid • Lung neoplasms, CT, 60.12115, 60.30
|
INTRODUCTION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONREFERENCES |
|---|
The sensitivity of computed tomography (CT) in the detection of lung nodules (ie, potential lung cancers) by radiologists has been shown to surpass that of chest radiography (5–7); furthermore, the sensitivity of helical CT for lung nodule detection is significantly superior to that of conventional CT (8). Accordingly, the efficacy of low-dose helical CT protocols (9,10) has revived interest in, and demand for, lung cancer screening (11–13). The potential effect, however, of any lung cancer screening program, including screening with low-dose helical CT, remains a topic of debate in the medical community (14,15).
Although the potential camouflaging effect of overlapping anatomic structures is effectively eliminated in CT scans, identification of small lung nodules is confounded by the prominence of blood vessels on CT images. The process of interpreting CT images can lead to fatigue or distraction, especially when other abnormalities are present (16). The large amount of image data acquired during a single CT examination may quickly lead to "information overload" for the radiologists who must interpret these data. Accordingly, computerized methods for nodule detection are becoming more attractive. Various investigators have developed a number of methods for the automated detection of lung nodules in CT scans (13,17–28), including geometric modeling (23), fuzzy clustering (22), spatial filtering (13), and gray-level thresholding (17–19,21,28).
A computerized nodule detection scheme, implemented as a "second reader" in a lung cancer screening program, is expected to help radiologists focus their attention on regions that might contain lung cancer. Such a scheme could also be used to direct radiologists to suspicious lesions that would merit immediate examination with targeted full-dose thin-section CT after having been detected during an initial low-dose CT scan. Thus, the purpose of our study was to evaluate the performance of a fully automated computerized method for the detection of lung nodules in CT scans in the identification of lung cancers that may be missed during visual interpretation.
|
MATERIALS AND METHODS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONREFERENCES |
|---|
The CT examinations were performed with a mobile CT scanner (CT-W950SR; Hitachi Medical, Tokyo, Japan). The examinations in this study were performed with a low-dose protocol of 120 kVp, 25 mA (11 examinations) or 50 mA (27 examinations), 10-mm collimation, and a 10-mm reconstruction interval at a helical pitch of 2 (29). The pixel size was 0.586 mm in 33 scans and 0.684 mm in five scans. Each reconstructed CT section had an image matrix size of 512 x 512 pixels. The data from the 38 scans, as stored in the database, consisted of a total of 1,057 section images (after sections representing anatomic areas inferior to the lung bases were manually excluded from each scan); the number of sections per scan ranged from 21 to 31 (mean, 28 sections).
The image data were transferred from optical disk storage media to a research computer (SGI Onyx; Silicon Graphics, Mountain View, Calif). Contained within the larger database of CT scans from the Nagano lung cancer screening program were radiology reports and image data from 32 patients in whom lung cancers were initially missed by radiologists (30). The low-dose CT data for one of these patients, however, had been corrupted during the electronic transfer process. Accordingly, the database used in this study contained low-dose CT data for 31 patients.
Each of the 31 patients whose screening CT data were used in this study had one biopsy-confirmed "missed cancer." Since the cancers in some of these patients had been missed at multiple screening studies, data existed from a total of 38 CT scans in which these 31 lesions had been missed. Data from these 38 scans comprised the database used in the present study, and each occurrence of the 31 lesions in the database was considered a separate missed cancer for the purpose of this study.
These 38 missed cancers were retrospectively identified and had been missed because of either detection errors (n = 23) (31) or interpretation errors (n = 15) during the initial clinical reading. The cancers missed due to detection errors had not been indicated in the original radiology report, while cancers missed due to interpretation errors had initially been reported as tuberculosis (n = 6), inflammatory lesions (n = 8), or pleural thickening (n = 1). All 38 lesions were subsequently confirmed to be lung cancers on the basis of results of biopsies motivated by the radiologic appearance of the lesions in the screening examination performed during a subsequent year. The radiologic pattern of these 38 missed cancers was that of a solid nodule in 12, pure ground-glass opacity (GGO) in 10, and mixed GGO in 16 (30).
In five patients, the cancer was missed for 2 years before biopsy was performed after the third annual screening study; consequently, scans from two studies of each of these five patients were included in the database. In one patient, the cancer was missed during 3 successive years; scans from three studies of this patient were included in the database. The scans from the 38 studies (of 31 patients) that comprise the present database were acquired during years in which the cancers were missed. Although some patients had scans from multiple studies in the database, scans were grouped by patient for the training and evaluation of the automated nodule detection method.
The clinical, radiologic, and pathologic characteristics of the missed lung cancers were evaluated to determine the attributes of malignant lesions that may result in oversight or misinterpretation on the part of human observers (30).
In addition to the missed cancers, one other nodule was observed in each of five scans, two other nodules were observed in each of two scans, and three other nodules were observed in one scan, resulting in a total of 50 lung nodules among the 38 scans. The presence of these 12 additional nodules was confirmed at thin-section CT performed during the time period of the screening program, and the nodules were classified as "confirmed benign" (n = 8), "suspected benign" (n = 3), or "suspected malignant" (n = 1) (Table 1). The confirmed benign nodules either decreased in size at repeat low-dose CT or exhibited no change in size over a 2-year period (32), while the suspected benign nodules demonstrated no change in size during a follow-up period that spanned less than 2 years. The lone nodule that was thought to be malignant was classified on the basis of results of a number of follow-up diagnostic CT studies; no biopsy results were available for this nodule.
|
Figure 1 presents the distributions of effective diameters of the missed cancers. Effective diameter was computed as the mean dimension of the bounding region (ie, one-half of the sum of the short and long axis lengths). When a nodule was depicted on more than one section, the bounding region with the greatest area was used for the calculation of effective diameter. The geometric center of mass of each bounding region and the location of the maximum gray-level pixel within each bounding region were used to determine whether the automated nodule detection method correctly identified a lung nodule.
|
|
A multiple-gray-level thresholding technique was applied to the segmented 3D lung volume. At each of 36 gray-level thresholds, individual structures were identified by means of grouping spatially contiguous pixels that remained in the volume at each threshold level. Since a nodule is defined radiologically as any well-demarcated, soft-tissue focal opacity with a diameter less than 3 cm (34), the automated method designated a structure as a nodule candidate if its volume was less than that of a model sphere 3 cm in diameter. The rationale for this approach is that structures too large to become nodule candidates at initial gray-level thresholds will, at higher thresholds, decompose into multiple smaller structures, many of which will qualify as nodule candidates (19). With this process, nodules with a wide range of attenuation values may be detected by the automated method.
The automatic categorization of nodule candidates as "nodule" or "nonnodule" (ie, as normal anatomy or as other disease with a nonnodular appearance) was based on a combination of rule-based and linear discriminant (35) classifiers applied to a set of nine 2D and 3D features extracted from each nodule candidate (18,19,21,33). The 2D features used were maximum eccentricity, maximum circularity, and maximum compactness, and the 3D features were volume, sphericity, radius of the equivalent sphere, mean pixel value, pixel value SD, and the gray-level threshold at which the structure first became a nodule candidate.
A round-robin, leave-one-out-by-patient procedure was performed in which nodule candidates identified in the CT scans of 30 of the 31 patients were used to train the linear discriminant classifier, which was then applied to the nodule candidates identified in the CT scan or scans of the remaining patient. This process was repeated until each of the 31 patients whose CT data comprised the database was the "left out" patient. Use of the leave-one-out-by-patient scheme was necessary to eliminate bias caused by the existence of data from multiple CT scans for six patients. (The database contained a single scan for each of 25 patients, two scans for each of five patients, and three scans for another patient, for a total of 38 scans.)
To further eliminate false-positive detections, a second linear discriminant classifier was applied in a leave-one-out-by-patient manner to nodule candidates that remained after application of the initial linear discriminant classifier at a particular output threshold. The ability of the classifiers to differentiate nodule candidates that corresponded to actual nodules from those that corresponded to areas of normal anatomy was evaluated by using receiver operating characteristic (ROC) analysis (36). Performance of the automated nodule detection method was evaluated by using free-response receiver operating characteristic (FROC) curves (37), which were generated by plotting overall nodule detection sensitivity as a function of the number of false-positive structures identified per section.
|
RESULTS |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONREFERENCES |
|---|
Figure 3 presents the resulting ROC curves and the corresponding areas under the ROC curves (Az) (38) for three combinations of nodule candidate features merged through the initial linear discriminant classifier and evaluated by using the leave-one-out-by-patient analysis. These ROC curves represent the ability of the linear discriminant classifier to differentiate between nodule candidates that correspond to actual nodules and those that do not (ie, false-positive detections) on the basis of the corresponding combinations of candidate features.
|
FROC curves representing overall nodule detection performance after the first linear discriminant analysis are shown in Figure 4. These curves were obtained by incrementally altering the operating point selected along the upper ROC curve (for all nine features) in Figure 3 and by accounting for the fact that not all actual nodules became nodule candidates initially. The solid curve in Figure 4 represents the performance of the automated method with respect to all 50 lung nodules in the 38-scan database, while the dashed curve indicates the performance of the automated method in the detection of the 38 missed cancers in the database.
|
|
Examples of CT images annotated with results from the automated method are shown in Figures 5 and 6. Suspicious regions identified by the computer are indicated by circles on the images; the level of performance represented by these images is consistent with that described in Table 2. Figure 5 presents sections from two different scans; depicted in each section is a cancer that was missed by the radiologist due to detection error. The lesion shown in Figure 5a was detected by the automated method (along with two false-positive detections), while the lesion in Figure 5b was not detected by the computer (resulting in one false-negative detection and one false-positive detection). Figure 6 presents sections from two different scans; depicted in each section is a cancer that was missed by the radiologist due to interpretation error. The lesion in Figure 6a was detected by the automated method (with no false-positive detections), while the lesion in Figure 6b was not detected by the computer (resulting in one false-negative detection and two false-positive detections).
|
|
|
|
The automated method presented in this study has the potential to identify any focal opacity that may generally be regarded as a lung nodule. While lung cancers represent the most clinically important subset of such focal opacities, the automated identification of any focal opacity for consideration by radiologists may be useful. In fact, a number of detections that were scored as false-positive detections in our study actually represented nodular abnormalities, although these would not be recorded as "lung nodules" by a radiologist. Figure 7 shows five detections that were annotated by the computerized method on one CT section. Although all five were considered false-positive detections, each of these "false-positive" detections actually corresponded to an area of focal scarring. In clinical practice, automated nodule detection would be integrated with automated classification, which would involve methods of distinguishing between malignant and benign nodules.
|
|
DISCUSSION |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONREFERENCES |
|---|
The extent to which a computerized nodule detection method will be used to assist radiologists in the interpretation of CT images ultimately depends on the degree to which radiologists understand and trust the computer results. This trust, in turn, develops from the performance of the method in terms of both an appropriately high sensitivity for nodule detection and a rate of false-positive detections that is low enough to result in few distractions that may adversely affect the overall usefulness of the method. Too many false-positive detections by the computer, for example, may overwhelm the observer so that true-positive detections indicated by the computer are overlooked, or, conversely, may result in overinterpretation of false-positive results and the performance of unnecessary follow-up procedures. The ability to detect a relatively high proportion of a clinically important subset of lung nodules (ie, missed cancers) by using a computer detection method has been demonstrated in this study, and current efforts are focused on methods to further reduce the number of false-positive detections. Ultimately, the confounding effect of false-positive detections, as a function of the false-positive rate, will be assessed through an observer study.
According to the categories of Kundel et al (31), the cancers in the database that were missed due to detection error (n = 23) and interpretation error (n = 15) would correspond to cancers missed due to search error and decision-making error, respectively. In our study, the fact that a cancer had been missed due to an interpretation error was established on the basis of the clinical radiology report. Cancers missed due to detection error, however, were so designated on the basis of the lack of corresponding information in the radiology reports. Accordingly, these cancers were assumed (rather than known) to have been missed due to detection error (ie, search error). Search errors occur "if a nodule-containing area is not scanned by the useful visual field" (31); it is likely, however, that some of the detection-error misses in our study were in fact the result of recognition error, in which the nodule was properly scanned but "the potential abnormal feature is not recognized" (31), and, consequently, no indication of an abnormality was noted in the radiology report. Regardless of whether these 23 cancers were missed by radiologists due to search error or to recognition error, the fact that 18 (78%) of these lesions were detected by the computer has important implications for the clinical utility of computerized nodule detection methods and the effect of these methods on patient care.
The performance of an automated lung nodule detection method, as applied to a database of low-dose screening CT scans in which lung cancers had initially been missed by radiologists, was evaluated; this method achieved a detection sensitivity of 84% (32 of 38 cancerous nodules), with an average of 1.0 false-positive detection per section. These results warrant testing of this method on larger databases of cases to evaluate its generalizability.
The motivation for research into computer-aided detection is the prospect that computerized methods will become a routine part of radiologists’ medical decision-making processes and will facilitate the earlier detection of disease at a stage that may allow for a more favorable patient prognosis. The present study demonstrates the reality behind this motivation: Radiologists may overlook potentially important lesions, and computerized methods incorporate image-processing, mathematic, and statistical techniques that provide an objective complement to the experience of radiologists. Future research must address whether the performance of radiologists is positively affected by the use of computerized nodule detection.
|
ACKNOWLEDGMENTS |
|---|
|
FOOTNOTES |
|---|
Abbreviations: Az = area under the ROC curve, FROC = free-response ROC, GGO = ground-glass opacity, ROC = receiver operating characteristic, 3D = three-dimensional, 2D = two-dimensional
Author contributions: Guarantor of integrity of entire study, S.G.A.; study concepts, S.G.A., H.M., K.D.; study design, S.G.A., M.L.G., K.D.; literature research, S.G.A., F.L.; clinical studies, S.S., F.L., H.M.; experimental studies, S.G.A., F.L.; data acquisition, S.G.A., S.S., F.L.; data analysis/interpretation, S.G.A., M.L.G., K.D.; statistical analysis, S.G.A., M.L.G.; manuscript preparation, S.G.A., F.L.; manuscript definition of intellectual content and editing, S.G.A.; manuscript revision/review and final version approval, all authors.
|
REFERENCES |
|---|
|
TOPABSTRACTINTRODUCTIONMATERIALS AND METHODS RESULTSDISCUSSIONREFERENCES |
|---|
This article has been cited by other articles:
|
|
 |
|
  C. Beigelman-Aubry, P. Raffy, W. Yang, R. A. Castellino, and P. A. Grenier Computer-Aided Detection of Solid Lung Nodules on Follow-Up MDCT Screening: Evaluation of Detection, Tracking, and Reading Time Am. J. Roentgenol., October 1, 2007; 189(4): 948 - 955. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. Peloschek, J. Sailer, M. Weber, C. J. Herold, M. Prokop, and C. Schaefer-Prokop Pulmonary Nodules: Sensitivity of Maximum Intensity Projection versus That of Volume Rendering of 3D Multidetector CT Data Radiology, May 1, 2007; 243(2): 561 - 569. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y. J. Jeong, C. A. Yi, and K. S. Lee Solitary Pulmonary Nodules: Detection, Characterization, and Guidance for Further Diagnostic Workup and Treatment Am. J. Roentgenol., January 1, 2007; 188(1): 57 - 68. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Das, G. Muhlenbruch, A. H. Mahnken, T. G. Flohr, L. Gundel, S. Stanzel, T. Kraus, R. W. Gunther, and J. E. Wildberger Small Pulmonary Nodules: Effect of Two Computer-aided Detection Systems on Radiologist Performance. Radiology, November 1, 2006; 241(2): 564 - 571. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. C. Jaffe Measures of Response: RECIST, WHO, and New Alternatives J. Clin. Oncol., July 10, 2006; 24(20): 3245 - 3251. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Yuan, P. M. Vos, and P. L. Cooperberg Computer-Aided Detection in Screening CT for Pulmonary Nodules. Am. J. Roentgenol., May 1, 2006; 186(5): 1280 - 1287. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Awai, K. Murao, A. Ozawa, Y. Nakayama, T. Nakaura, D. Liu, K. Kawanaka, Y. Funama, S. Morishita, and Y. Yamashita Pulmonary Nodules: Estimation of Malignancy at Thin-Section Helical CT--Effect of Computer-aided Diagnosis on Performance of Radiologists Radiology, April 1, 2006; 239(1): 276 - 284. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Gur Lung Cancer Screening: Radiology's Opportunity Here and Now Radiology, February 1, 2006; 238(2): 395 - 397. [Full Text] [PDF]
|
|
|
|
|
|
F. Li, H. Arimura, K. Suzuki, J. Shiraishi, Q. Li, H. Abe, R. Engelmann, S. Sone, H. MacMahon, and K. Doi Computer-aided Detection of Peripheral Lung Cancers Missed at CT: ROC Analyses without and with Localization Radiology, November 1, 2005; 237(2): 684 - 690. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. G. Kim, J. M. Goo, J. H. Kim, H. J. Lee, B. G. Min, K. T. Bae, and J.-G. Im Computer-aided Diagnosis of Localized Ground-Glass Opacity in the Lung at CT: Initial Experience Radiology, November 1, 2005; 237(2): 657 - 661. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. K. Leader, T. E. Warfel, C. R. Fuhrman, S. K. Golla, J. L. Weissfeld, R. S. Avila, W. D. Turner, and B. Zheng Pulmonary Nodule Detection with Low-Dose CT of the Lung: Agreement Among Radiologists Am. J. Roentgenol., October 1, 2005; 185(4): 973 - 978. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. Peldschus, P. Herzog, S. A. Wood, J. I. Cheema, P. Costello, and U. J. Schoepf Computer-Aided Diagnosis as a Second Reader: Spectrum of Findings in CT Studies of the Chest Interpreted as Normal Chest, September 1, 2005; 128(3): 1517 - 1523. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. M. Goo, T. Tongdee, R. Tongdee, K. Yeo, C. F. Hildebolt, and K. T. Bae Volumetric Measurement of Synthetic Lung Nodules with Multi-Detector Row CT: Effect of Various Image Reconstruction Parameters and Segmentation Thresholds on Measurement Accuracy Radiology, June 1, 2005; 235(3): 850 - 856. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K Doi Current status and future potential of computer-aided diagnosis in medical imaging Br. J. Radiol., January 1, 2005; 78(suppl_1): S3 - s19. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. D. Rubin, J. K. Lyo, D. S. Paik, A. J. Sherbondy, L. C. Chow, A. N. Leung, R. Mindelzun, P. K. Schraedley-Desmond, S. E. Zinck, D. P. Naidich, et al. Pulmonary Nodules on Multi-Detector Row CT Scans: Performance Comparison of Radiologists and Computer-aided Detection Radiology, January 1, 2005; 234(1): 274 - 283. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. F. Patz Jr, S. J. Swensen, and J. E. Herndon II Estimate of Lung Cancer Mortality From Low-Dose Spiral Computed Tomography Screening Trials: Implications for Current Mass Screening Recommendations J. Clin. Oncol., June 1, 2004; 22(11): 2202 - 2206. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. N. Bryan The Digital rEvolution: The Millennial Change in Medical Imaging Radiology, November 1, 2003; 229(2): 299 - 304. [Full Text] [PDF]
|
|
|
|
|
|
C. D. Johnson Hard- versus Soft-Copy Interpretation Radiology, June 1, 2003; 227(3): 629 - 630. [Full Text] [PDF]
|
|
|
|
|
|
F. Li, S. Sone, H. Abe, H. MacMahon, S. G. Armato III, and K. Doi Lung Cancers Missed at Low-Dose Helical CT Screening in a General Population: Comparison of Clinical, Histopathologic, and Imaging Findings Radiology, December 1, 2002; 225(3): 673 - 683. [Abstract] [Full Text] [PDF] |
). All of these "false-positive" detections, however, actually correspond to focal areas of interstitial disease.