Radiology
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

DOI: 10.1148/radiol.2412051139
This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Das, M.
Right arrow Articles by Wildberger, J. E.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Das, M.
Right arrow Articles by Wildberger, J. E.
(Radiology 2006;241:564-571.)
© RSNA, 2006

Thoracic Imaging

Small Pulmonary Nodules: Effect of Two Computer-aided Detection Systems on Radiologist Performance1

Marco Das, MD, Georg Mühlenbruch, MD, Andreas H. Mahnken, MD, Thomas G. Flohr, PhD, Lutz Gündel, PhD, Sven Stanzel, PhD, Thomas Kraus, MD, Rolf W. Günther, MD and Joachim E. Wildberger, MD

1 From the Department of Diagnostic Radiology (M.D., G.M., A.H.M., R.W.G., J.E.W.), Institute of Medical Statistics (S.S.), and Department of Occupational Health (T.K.), Rheinisch-Westfâlische Technische Hochschule Aachen University, Pauwelsstrasse 30, D-52074 Aachen, Germany; and Department of Computed Tomography, Siemens Medical Solutions, Forchheim, Germany (T.G.F., L.G.). Received July 6, 2005; revision requested September 6; revision received December 15; accepted January 11, 2006; final version accepted February 26. Address correspondence to M.D. (e-mail: das{at}rad.rwth-aachen.de).


   ABSTRACT
TOP
 ABSTRACT
INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 
Purpose: To prospectively compare the effects of two computer-aided detection (CAD) systems on the detection of small pulmonary nodules at multi–detector row computed tomography (CT) by using a consensus panel decision as the reference standard.

Materials and Methods: Institutional review board approval and informed consent were obtained. Multi–detector row CT scans were randomly chosen and prospectively evaluated in 25 patients. Two dedicated CAD systems—ImageChecker CT (R2 Technologies, Sunnyvale, Calif) and Nodule Enhanced Viewing (NEV) (Siemens Medical Solutions, Forchheim, Germany)—were used. Results were interpreted by three radiologists with 1, 3, and 6 years of experience. Images were evaluated without and with CAD software. The reference standard was assessed by a consensus panel consisting of all three radiologists and an adjudicator with 8 years of experience.

Results: A total of 116 pulmonary nodules (average diameter, 3.4 mm; average volume, 32.05 mm3) were found in all data sets during consensus interpretation, which included findings from the CAD software and all radiologists. Overall sensitivity was 73% with ImageChecker CT and 75% with NEV. Overall sensitivity without CAD was 68% for radiologist 1, 78% for radiologist 2, and 82% for radiologist 3. With ImageChecker CT, sensitivity increased to 79% for radiologist 1, 90% for radiologist 2, and 84% for radiologist 3. With NEV, sensitivity increased to 79% for radiologist 1, 90% for radiologist 2, and 86% for radiologist 3. The average number of false-positive findings was six (range, 0–14) with ImageChecker CT and eight (range, 0–22) with NEV.

Conclusion: Radiologist performance in the interpretation of multi–detector row CT scans can be improved by using CAD systems, with a reduction in the number of false-negative diagnoses. No statistically significant difference in sensitivity was found between the two CAD systems.

© RSNA, 2006


   INTRODUCTION
TOP
 ABSTRACT
 INTRODUCTION
MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 
The detection of pulmonary nodules is essential for the diagnosis various pathologic conditions. A false-negative diagnosis due to perceptual error or misinterpretation is a very common and well-known problem (1–5). With multi–detector row computed tomography (CT), high-spatial-resolution images are acquired within one breath hold. This enables the detection of small (1–2-mm-diameter) pulmonary nodules that are not visible on images obtained with lower spatial resolution or thicker sections (6). Several screening studies have been performed for the early detection of lung cancer, and the results of these studies suggest that a low-dose thin-section CT technique can be used for the detection of pulmonary nodules (7–12). It has been shown that, although low-dose screening is effective in facilitating the detection of nodules, it may be more difficult for the radiologist to interpret images with increased image noise.

To reduce the number of false-negative diagnoses, double reading has been proposed. However, double reading does not usually occur in the clinical routine setting. First introduced for mammography, double reading helped significantly improve sensitivity (13–15). In chest CT, double reading showed similar results, with a significant improvement in the detection rate of pulmonary nodules (16). The use of double reading is not always possible in routine clinical settings—especially because of limited human resources and cost effectiveness. Thus, the use of a computer-aided detection (CAD) system as a second reader may provide a solution to this problem.

The sensitivity of CAD systems alone has been previously assessed (2,17–20), with reported sensitivities of between 38% and 84%. The effect of a single system on the performance of a radiologist with regard to the detection of pulmonary nodules has been described in several studies (17,19,21–24). Recently, Rubin et al (21) compared the sensitivity of radiologists alone with the sensitivity of double reading and with that of a radiologist using CAD software as a second reader. Rubin et al showed a significant improvement in sensitivity—from an average of 50% for radiologists alone to 76% for radiologists using CAD software. Marten et al (19) also tested the performance of radiologists with and without the use of a CAD system. In this study, they took into account the radiologist's experience and found that experienced radiologists benefited more from CAD schemes. Although inexperienced radiologists benefit from CAD systems as well, experience is crucial for the differentiation of true-positive findings from false-positive findings.

To our knowledge, no study has been performed to test two different commercially available systems head to head with regard to radiologist performance. Thus, the purpose of our study was to prospectively compare the effects of two CAD systems on the detection of small pulmonary nodules at multi–detector row CT by using a consensus panel decision as the reference standard.


   MATERIALS AND METHODS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 
Because a Siemens CAD unit was evaluated in this study, full control of the data and information presented for publication was maintained by those authors who were not employees of Siemens Medical Solutions.

Patient Selection and Imaging
Institutional review board approval and informed consent were obtained. Twenty-five patients (five women, 20 men; age range, 43–74 years; mean age, 54 years ± 8 [standard deviation]) who had undergone multi–detector row CT of the chest (12 patients [five women, seven men]; age range, 43–67 years; mean age, 52 years) or who were included in a low-dose lung cancer screening study (13 men; mean age, 67 years ± 9; age range, 57–74 years) were randomly chosen. In the first group, eight patients were scanned for metastasis with underlying malignant disease (four kidney cancers, three prostate cancers, and one breast cancer), and four patients were scanned for tumor search. In the second group, the results of the screening examinations were obtained from a large lung cancer screening trial in which workers who had been exposed to asbestos underwent low-dose multi–detector row CT at yearly intervals (inclusion criteria: minimum age, 45 years; maximum age, 75 years; asbestos exposure, 16–45 years). All patients were scanned with a 16-section CT scanner (Somatom Sensation 16; Siemens Medical Solutions, Forchheim, Germany).

Imaging parameters for the first patient group (routine protocol) were as follows: 120 kV, effective milliampere-second according to patient weight (eg, 60 kg = 60 mAseff), 0.5-second rotation time, 16 x 1.5-mm collimation, 24-mm table feed per rotation, 2-mm section thickness, 1.5-mm reconstruction interval, and sharp reconstruction kernel (B50f; Siemens Medical Solutions).

Imaging parameters for the second patient group (low-dose screening protocol) were as follows: 120 kV, effective milliampere-second of 10 for patients who weighed less than 80 kg and 20 for patients who weighed more than 80 kg, 0.5-second rotation, 16 x 0.75-mm collimation, 18-mm table feed per rotation, 1-mm section thickness, 0.5-mm reconstruction intervals, and sharp reconstruction kernel (B50f; Siemens Medical Solutions).

The images were stored on a local network drive and were distributed to the two CAD workstations for further analysis.

Image Interpretation
Three radiologists with different levels of experience were chosen from the department of radiology. Radiologist 1 (G.M.) had 1 year of experience in interpreting CT scans of the chest, radiologist 2 (M.D.) had 3 years of experience in interpreting CT scans of the chest, and radiologist 3 (A.H.M.) had 6 years of experience in interpreting CT scans of the chest. All three radiologists independently interpreted all images without the aid of either of the two CAD systems. Analysis was performed in a room where no distractions could occur, and radiologists were given as much reading time as they wanted. Reading time was measured by the radiologists themselves with a standard stopwatch. Time started when the patient data were opened and stopped when the patient data were closed.

During analysis, each nodule was marked, and the diameter, volume, attenuation, and location were reported. Images were displayed with a standard lung window (window level, –700 HU; window width, 2000 HU). Each radiologist used cine viewing with thinMips, which is a stack of images presented as a 5-mm-thick maximum intensity projection, for interpretation. In addition, multiplanar reconstructions or three-dimensional renderings were allowed.

In the second step, which was performed 6 weeks to 3 months later to avoid recall bias, the data sets were put in random order and were interpreted independently by the radiologists with use of ImageChecker CT (R2 Technologies, Sunnyvale, Calif). Radiologists were blinded to previous results. The ImageChecker CT software preprocesses the data set and immediately presents the potential nodules ("candidate nodules") to the radiologist. All results that were found by the CAD software were stored. The candidate nodules were then evaluated by the radiologist and were either added as true-positive findings or dismissed as false-positive findings.

In the third step, which was performed within the next 6 weeks to 3 months after the final reading, the data sets were interpreted independently by all three radiologists with use of the Nodule Enhanced Viewing (NEV) system (Siemens Medical Solutions). Again, the radiologists were blinded to previous results. The NEV software begins processing the data set while the radiologist reviews images from the entire examination. CAD results are presented within a few minutes (average, 3.1 minutes per examination ± 2.9). All results obtained with the CAD software were stored, and the candidate nodules were either added as true-positive findings or dismissed as false-positive findings. In addition, original images were available for comparison with either system.

The times needed for reviewing the images with the two second-reader approaches were also noted with a stopwatch by one author (M.D.) and were compared.

CAD and Reference Standard
To assess the sensitivity and false-positive rate of the two systems individually, both CAD software systems were used to analyzed all images. Candidate lesions were stored and classified as true-positive or false-positive findings by the consensus panel, which included all three radiologists and an adjudicator (J.E.W.), as needed, with 8 years of experience.

The reference standard was assessed by using all reported findings after the single reading was performed (radiologist alone, radiologist with CAD software, CAD software alone, and radiologists and adjudicator, as needed). All findings were accepted or dismissed by the consensus panel.

Statistical Analysis
Nodule diameter, nodule volume, and reading times are given as arithmetic means and corresponding standard deviations; the number of detected nodules is given as the range, median, and corresponding interquartile range (the difference between the upper and the lower quartile). Observed false-positive rates with the two CAD systems are given as medians and corresponding interquartile ranges and are represented in a bar graph.

Observed sensitivity values are given as arithmetic means, medians, and interquartile ranges separately for the different analysis protocols (CAD system alone, radiologist alone, and radiologist with each of the two CAD systems) and are displayed graphically by using parallel box plots.

The nonparametric unpaired Wilcoxon test was used for overall comparison of sensitivity values obtained for all nodules with respect to the two acquisition protocols; comparisons were made separately for each of the two CAD systems. In case of a statistically significant test result with at least one of these two tests, further statistical analysis was conducted separately for the two acquisition protocols; otherwise, the 25 patients were treated as one study group.

First, sensitivity was assessed for all nodule sizes for the CAD software alone, for the three radiologists alone, and for the three radiologists with the CAD software.

The nonparametric paired Wilcoxon test was used for overall comparison of sensitivity values between the two software tools and for pairwise comparison of sensitivity values among the three different analysis protocols (radiologist alone, radiologist with ImageChecker CT, and radiologist with NEV); comparisons were made individually for each of the three radiologists.

A {kappa} coefficient was calculated to evaluate the degree of agreement between the three radiologists and between the three radiologists using the two CAD software tools. To simplify the computation of {kappa} coefficients, the number of nodules diagnosed per patient was classified into three categories as follows: 0, no nodules found; 1, 1–3 nodules found; and 2, more than 3 nodules found.

Moreover, the same statistical analysis scheme (ie, paired Wilcoxon tests and {kappa} coefficients) was applied to the subgroup of nodules that were smaller than 5 mm, thus ignoring nodules with diameters of 5 mm or more.

An {alpha} level of .05 was chosen for all statistical test procedures. In our opinion, a confirmative statement seemed plausible only for overall comparison between the two software tools during the analysis of all nodules. Thus, all additional statistical tests were performed in a clearly explorative sense. Therefore, no {alpha} adjustment was made. Hence, P values of less than or equal to .05 could be interpreted as indicating a statistically significant result.

All statistical analyses were performed by using a statistical software package (SAS for Windows, release 8.02, TS level 02MO; SAS Institute, Cary, NC) and the R open source statistical analysis system (www.R-project.org).


   RESULTS
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 
Reference Standard
The consensus panel found 116 pulmonary nodules in all patients (median, four nodules per patient; interquartile range, 2; range, 0–12). The average nodule diameter was 3.4 mm ± 5.9, and the average volume was 32.05 mm3 ± 85.4. Ten nodules were calcified.

Nodule Size
Of the 116 pulmonary nodules (Fig 1), 89 were smaller than 5 mm in diameter. The average diameter of these smaller nodules was 2.9 mm ± 1.2, and the average volume was 27.3 mm3 ± 25.4. No statistically significant difference in sensitivity was found between the two acquisition protocols by using ImageChecker CT (P = .323) or NEV (P = .129); therefore, further statistical analysis was performed by using all 25 patients, without differentiation between the two protocols.


Figure 1
View larger version (20K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 1: Bar graph shows the distribution of nodules per patient for all nodules (total) and for those smaller than 5 mm (small).

 
In the first patient group (routine protocol), 15 nodules had a diameter of 5–10 mm. Twelve of these 15 nodules were found in patients who were scanned for metastasis, and three were found in patients who were scanned for tumor search. In the second patient group (low-dose screening protocol), one nodule was larger than 10 mm, and small cell lung cancer was diagnosed in this nodule at histologic examination. Ten nodules were between 5 and 10 mm in diameter, and patients underwent short-term follow-up (3 months). Thus far, the nodules have not shown any growth. No histologic specimens were assessed for the 89 nodules that were smaller than 5 mm owing to the size of the nodules.

CAD Comparisons
ImageChecker CT detected 85 of 116 nodules, leading to an overall sensitivity of 73% for nodules of all sizes. The average sensitivity per examination was 75% (median, 80%; interquartile range, 25%). The average nodule diameter was 2.9 mm ± 7.2, and the average volume was 27.23 mm3 ± 89.43. The mean number of false-positive findings was six (range, 0–14) (Fig 2). The ImageChecker CT system found 68 of 89 nodules that were smaller than 5 mm, for a mean sensitivity of 76% (median, 83%; interquartile range, 25%).


Figure 2
View larger version (16K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 2: Bar graph shows the number of false-positive findings per patient for ImageChecker CT, (IC) and NEV.

 
The NEV system detected 87 of 116 nodules, for an overall sensitivity of 75%. The average sensitivity per examination was 78% (median, 75%; interquartile range, 10%). The average nodule diameter was 3.0 mm ± 5.6, and the average volume was 28.59 mm3 ± 57.30. The average number of false-positive findings was eight (range, 0–22) (Fig 2). The NEV system found 71 of 89 nodules that were smaller than 5 mm, for a mean sensitivity of 80%.

The paired Wilcoxon test for overall comparisons between the two CAD systems gave a P value of less than .129, which is not indicative of a statistically significant difference between the two CAD systems regarding sensitivity for all nodule sizes.

The average reading time, as measured with a stopwatch, was 5.03 minutes ± 3.43 without CAD, 5.25 minutes ± 6.45 with ImageChecker CT, and 6.35 minutes ± 4.35 with NEV.

All Nodules
Sensitivity data for the detection of all nodules are shown in Figure 3. For all nodule sizes, radiologist 1 detected 79 of 116 nodules (overall sensitivity, 68%; average sensitivity per examination, 72% [median, 75%; interquartile range, 34%]). With ImageChecker CT, the number of detected nodules increased to 92 (overall sensitivity, 79%; average sensitivity per examination, 80% [median, 100%; interquartile range, 34%]). With NEV, the number of detected nodules increased to 92 (overall sensitivity, 79%; average sensitivity per examination, 81% [median, 83%; interquartile range, 34%]). If we compare the sensitivity values of the different analysis strategies, no statistically significant difference is found between the two CAD systems (P = .100). ImageChecker CT (P = .005) helped to significantly increase sensitivity relative to diagnosis without the aid of a CAD system, but no statistically significant increase was found with NEV (P = .116).


Figure 3
View larger version (16K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 3: Graphs show sensitivity for the radiologists and CAD systems in the detection of all nodule sizes. Graphs are shown for the two CAD systems alone and for all radiologists alone and in combination with CAD. IC = ImageChecker CT, {circ} = outliers.

 
For all nodule sizes, radiologist 2 detected 91 of 116 nodules, for an overall sensitivity of 78% (average sensitivity per examination, 79% [median, 83%; interquartile range, 34%]). With ImageChecker CT, the number of detected nodules increased to 104 (overall sensitivity, 90%; average sensitivity per examination, 91% [median, 100%; interquartile range, 10%]). With NEV, the number of detected nodules increased to 104 (overall sensitivity 90%; average sensitivity per examination, 92% [median, 100%; interquartile range, 20%]). If we compare the sensitivity values of the three analysis strategies, a statistically significant difference is found between the two CAD systems (P = .005). No statistically significant increase relative to diagnosis without the aid of a CAD system was found with ImageChecker CT (P = .081), but a statistically significant increase was found with NEV (P = .032).

For all nodule sizes, radiologist 3 detected 95 of 116 nodules (overall sensitivity, 82%; average sensitivity per examination, 84% [median, 83%; interquartile range, 25%]). With ImageChecker CT, the number of detected nodules increased to 98 (overall sensitivity, 84%; average sensitivity per examination, 84% [median, 100%; interquartile range, 25%]). With NEV, the number of detected nodules increased to 100 (overall sensitivity, 86%; average sensitivity per examination, 89% [median, 100%; interquartile range, 20%]). No statistically significant difference in sensitivity was seen between the two CAD systems (P = .893) nor was any statistically significant difference seen with ImageChecker CT (P = .123) and NEV (P = .161) relative to diagnosis without the use of a CAD system.

Calculation of {kappa} coefficients (Table 1) showed an increased degree of agreement between any pair of radiologists with use of the CAD software.


View this table:
[in this window]
[in a new window]

  		Table 1. {kappa} Coefficient for All Nodules

 
Small Nodules
Sensitivity data for the detection of small nodules are shown in Figure 4. Radiologist 1 detected 49 of 89 nodules that were smaller than 5 mm (overall sensitivity, 55%; average sensitivity per examination, 62% [median, 50%; interquartile range, 25%]). With ImageChecker CT, the number of detected nodules increased to 61 (overall sensitivity, 68%; average sensitivity per examination, 71% [median, 66%; interquartile range, 17%]). With NEV, the number of detected nodules increased to 63 (overall sensitivity, 71%; average sensitivity per examination, 74% [median, 75%; interquartile range, 17%]).


Figure 4
View larger version (17K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 4: Graphs show sensitivity for the radiologists and CAD systems in the detection of nodules smaller than 5 mm in diameter. Graphs are shown for the two CAD systems alone and for all radiologists alone and in combination with CAD. IC = ImageChecker CT, {circ} = outliers.

 
Results of the paired Wilcoxon test showed a statistically significant difference between sensitivity values obtained for radiologist 1 alone and those obtained for radiologist 1 with ImageChecker CT (P = .007) and NEV (P = .007).There was, however, no statistically significant difference between the two CAD systems (P = .140).

Radiologist 2 detected 63 of 89 nodules that were smaller than 5 mm (overall sensitivity, 71%; average sensitivity per examination, 72% [median, 75%; interquartile range, 34%]). With ImageChecker CT, the number of detected nodules increased to 72 (sensitivity, 81%; average sensitivity per examination, 80% [median, 75%; interquartile range, 34%]). With NEV, the number of detected nodules increased to 75 (overall sensitivity, 84%; average sensitivity per examination, 83% [median, 83%; interquartile range, 25%]).

The results of pairwise comparison of the three analysis strategies with respect to sensitivity showed a statistically significant difference between the sensitivity values obtained for radiologist 2 alone and those obtained for radiologist 2 with ImageChecker CT (P = .027) and NEV (P = .011). There was, however, no statistically significant difference between the two CAD systems (P = .109).

Radiologist 3 detected 56 of 89 nodules that were smaller than 5 mm (overall sensitivity, 63%; average sensitivity per examination, 69% [median, 66%; interquartile range, 50%]). With ImageChecker CT, the number of detected nodules increased to 66 (overall sensitivity, 74%; average sensitivity per examination, 76% [median, 75%; interquartile range, 23%]). With NEV, the number of detected nodules increased to 71 (overall sensitivity, 80%; average sensitivity per examination, 80% [median, 75%; interquartile range, 34%]).

The results of pairwise comparison of the three analysis strategies with respect to sensitivity showed no statistically significant difference between sensitivity values obtained for radiologist 3 alone and those obtained for radiologist 3 with ImageChecker CT (P = .139). However, a statistically significant difference was found between sensitivity values obtained for radiologist 3 alone and those obtained with for radiologist 3 with NEV (P = .041). There was no statistically significant difference between the two CAD systems (P = .109).

Calculation of {kappa} coefficients (Table 2) showed an increased degree of agreement between any pair of radiologists with use of CAD software.


View this table:
[in this window]
[in a new window]

  		Table 2. {kappa} Coefficient for Nodules Smaller than 5 mm

 

   DISCUSSION
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
ADVANCES IN KNOWLEDGE
 References
 
CAD software, which functions as a second reader, has been developed to increase the detection performance of radiologists (25–29). We tested the effect of two commercially available software tools on the sensitivity of radiologists with different levels of experience.

NEV versus ImageChecker CT
Although both tested CAD systems function in a different way, no statistically significant difference could be detected. Although the overall sensitivity of NEV was slightly higher than that of ImageChecker CT, there were fewer false-positive findings with ImageChecker CT. As mentioned earlier, however, there was no statistically significant difference between the two software systems. Although the sensitivity rates obtained in this study were higher than those reported in the literature (range, 38%–86%) (2,17–19), neither software system is ideal because they both miss 20%–25% of all nodules. A further increase in sensitivity, however, can be expected with the ongoing development of algorithms. Thus, the sensitivity of the CAD systems is not perfect, and although CAD software should not be used as a first reader, it could have value as a second reader. Although it is not yet clear why some nodules are missed by the software system, the CAD software tends to work better for smaller nodules than for larger nodules.

Usually, the number of false-positive findings is a major drawback of CAD systems. In our study, both CAD systems had similar false-positive rates, with ImageChecker CT having fewer false-positive findings than NEV. Often, both software tools marked vessel bifurcations or small consolidations, which could be easily dismissed by the radiologists. All candidate nodules must be verified as true-positive or false-positive findings, and this can be challenging depending on the experience of the radiologist, as was demonstrated by Marten et al (19). In our study, the sensitivity of all three radiologists increased with use of the software tools; however, the least experienced radiologist still had the lowest sensitivity. This can be explained by this radiologist's lack of experience in the interpretation of CAD findings.

Nodule Detection
All radiologists had an increase in sensitivity with use of the software tools. Although there was not a statistically significant difference in the detection of all nodule sizes, the software tools were especially helpful in the detection of nodules that were smaller than 5 mm. There was a statistically significant increase in sensitivity with both software tools relative to the assessment of the radiologist alone. This result holds for all three radiologists, with the only exception being that the comparison between radiologist 3 with NEV and radiologist 3 alone did not result in a statistically significant difference in sensitivity. Rubin et al (21) showed that CAD software provides complementary nodule findings, making such findings superior to those of a second radiologist, who usually finds similar nodules as the first radiologist. In our study, an increased degree of agreement among the three radiologists was shown with CAD. Thus, CAD software tools have the potential to reduce variability among different radiologists and may lead to a less variable diagnosis.

Study Limitations
There were several limitations to this study. One might argue that there is a recall bias when looking at the examination results again. To avoid this bias, we waited six weeks to 3 months to reinterpret the images. Another bias could be the order of use for both systems; randomization of reading order would have reduced this bias. Third, the examinations were chosen at random and might not reflect a perfect average type of routine examination. Fourth, the number of examinations could have been increased. However, with three radiologists interpreting all examinations four times, the task was already time-consuming. In addition, more images without nodules could have been added to evaluate the performance of the CAD systems alone. Our study also contained a large number of very small nodules, for which clinical relevance is not yet clear.

In summary, the sensitivity of all three radiologists in the detection of pulmonary nodules increased with both CAD systems, and there was an increased degree of agreement among radiologists. These findings indicate the potential value of CAD, which leads to fewer false-negative diagnoses and a higher degree of interobserver agreement in clinical practice.


   ADVANCES IN KNOWLEDGE
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
References
 


   FOOTNOTES
 

Abbreviations: CAD = computer-aided detection • NEV = Nodule Enhanced Viewing

See Materials and Methods for pertinent disclosures.

Author contributions: Guarantor of integrity of entire study, M.D.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; manuscript final version approval, all authors; literature research, M.D., A.H.M., J.E.W.; clinical studies, M.D., G.M., L.G., S.S., J.E.W.; experimental studies, all authors; statistical analysis, M.D., A.H.M., S.S., J.E.W.; and manuscript editing, M.D., G.M., T.G.F., L.G., S.S., T.K., R.W.G., J.E.W.


   References
TOP
 ABSTRACT
 INTRODUCTION
 MATERIALS AND METHODS
 RESULTS
 DISCUSSION
 ADVANCES IN KNOWLEDGE
 References
 

  1. Li F, Sone S, Abe H, MacMahon H, Armato SG III, Doi K. Lung cancers missed at low-dose helical CT screening in a general population: comparison of clinical, histopathologic, and imaging findings. Radiology 2002;225:673–683.[Abstract/Free Full Text]
  2. Armato SG III, Li F, Giger ML, MacMahon H, Sone S, Doi K. Lung cancer: performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology 2002;225:685–692.[Abstract/Free Full Text]
  3. Kakinuma R, Ohmatsu H, Kaneko M, et al. Detection failures in spiral CT screening for lung cancer: analysis of CT findings. Radiology 1999;212:61–66.[Abstract/Free Full Text]
  4. Gurney JW. Missed lung cancer at CT: imaging findings in nine patients. Radiology 1996;199:117–122.[Abstract/Free Full Text]
  5. Shah PK, Austin JH, White CS, et al. Missed non–small cell lung cancer: radiographic findings of potentially resectable lesions evident only in retrospect. Radiology 2003;226:235–241.[Abstract/Free Full Text]
  6. Fischbach F, Knollmann F, Griesshaber V, Freund T, Akkol E, Felix R. Detection of pulmonary nodules by multislice computed tomography: improved detection rate with reduced slice thickness. Eur Radiol 2003;13:2378–2383.[CrossRef][Medline]
  7. Henschke CI, McCauley DI, Yankelevitz DF, et al. Early Lung Cancer Action Project: overall design and findings from baseline screening. Lancet 1999;354:99–105.[CrossRef][Medline]
  8. Diederich S, Wormanns D, Semik M, et al. Screening for early lung cancer with low-dose spiral CT: prevalence in 817 asymptomatic smokers. Radiology 2002;222:773–781.[Abstract/Free Full Text]
  9. Hartman TE, Swensen SJ. Lung cancer screening with low-dose computed tomography. Semin Roentgenol 2003;38:34–38.[CrossRef][Medline]
  10. Swensen SJ, Jett JR, Hartman TE, et al. Lung cancer screening with CT: Mayo Clinic experience. Radiology 2003;226:756–761.[Abstract/Free Full Text]
  11. International Early Lung Cancer Program Investigators. An update of CT screening for lung cancer. Semin Ultrasound CT MR 2005;26:348–356.[CrossRef][Medline]
  12. Swensen SJ, Jett JR, Hartman TE, et al. CT screening for lung cancer: five-year prospective experience. Radiology 2005;235:259–265.[Abstract/Free Full Text]
  13. Karssemeijer N, Otten JD, Verbeek AL, et al. Computer-aided detection versus independent double reading of masses on mammograms. Radiology 2003;227:192–200.[Abstract/Free Full Text]
  14. Thurfjell E. Mammography screening: one versus two views and independent double reading. Acta Radiol 1994;35:345–350.[Medline]
  15. Thurfjell EL, Lernevall KA, Taube AA. Benefit of independent double reading in a population-based mammography screening program. Radiology 1994;191:241–244.[Abstract/Free Full Text]
  16. Wormanns D, Ludwig K, Beyer F, Heindel W, Diederich S. Detection of pulmonary nodules at multirow-detector CT: effectiveness of double reading to improve sensitivity at standard-dose and low-dose chest CT. Eur Radiol 2005;15(1):14–22.[CrossRef][Medline]
  17. Wormanns D, Beyer F, Diederich S, Ludwig K, Heindel W. Diagnostic performance of a commercially available computer-aided diagnosis system for automatic detection of pulmonary nodules: comparison with single and double reading. Rofo 2004;176:953–958.[Medline]
  18. Wormanns D, Fiebich M, Saidi M, Diederich S, Heindel W. Automatic detection of pulmonary nodules at spiral CT: clinical application of a computer-aided diagnosis system. Eur Radiol 2002;12:1052–1057.[CrossRef][Medline]
  19. Marten K, Seyfarth T, Auer F, et al. Computer-assisted detection of pulmonary nodules: performance evaluation of an expert knowledge-based detection system in consensus reading with experienced and inexperienced chest radiologists. Eur Radiol 2004;14(10):1930–1938.[Medline]
  20. Arimura H, Katsuragawa S, Suzuki K, et al. Computerized scheme for automated detection of lung nodules in low-dose computed tomography images for lung cancer screening. Acad Radiol 2004;11:617–629.[CrossRef][Medline]
  21. Rubin GD, Lyo JK, Paik DS, et al. Pulmonary nodules on multi-detector row CT scans: performance comparison of radiologists and computer-aided detection. Radiology 2005;234:274–283.[Abstract/Free Full Text]
  22. Lee JW, Goo JM, Lee HJ, Kim JH, Kim S, Kim YT. The potential contribution of a computer-aided detection system for lung nodule detection in multidetector row computed tomography. Invest Radiol 2004;39:649–655.[CrossRef][Medline]
  23. Kakeda S, Moriya J, Sato H, et al. Improved detection of lung nodules on chest radiographs using a commercial computer-aided diagnosis system. AJR Am J Roentgenol 2004;182:505–510.[Abstract/Free Full Text]
  24. Awai K, Murao K, Ozawa A, et al. Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance. Radiology 2004;230:347–352.[Abstract/Free Full Text]
  25. Giger ML, Bae KT, MacMahon H. Computerized detection of pulmonary nodules in computed tomography images. Invest Radiol 1994;29:459–465.[CrossRef][Medline]
  26. Kanazawa K, Kawata Y, Niki N, et al. Computer-aided diagnosis for pulmonary nodules based on helical CT images. Comput Med Imaging Graph 1998;22:157–167.[CrossRef][Medline]
  27. Lee Y, Hara T, Fujita H, Itoh S, Ishigaki T. Automated detection of pulmonary nodules in helical CT images based on an improved template-matching technique. IEEE Trans Med Imaging 2001;20:595–604.[CrossRef][Medline]
  28. Gurcan MN, Sahiner B, Petrick N, et al. Lung nodule detection on thoracic computed tomography images: preliminary evaluation of a computer-aided diagnosis system. Med Phys 2002;29:2552–2558.[CrossRef][Medline]
  29. Brown MS, Goldin JG, Suh RD, McNitt-Gray MF, Sayre JW, Aberle DR. Lung micronodules: automated method for detection at thin-section CT—initial experience. Radiology 2003;226:256–262.[Abstract/Free Full Text]



This article has been cited by other articles:


Home page
 Am. J. Roentgenol.Home page
N. Kawel, B. Seifert, M. Luetolf, and T. Boehm
Effect of Slab Thickness on the CT Detection of Pulmonary Nodules: Use of Sliding Thin-Slab Maximum Intensity Projection and Volume Rendering
Am. J. Roentgenol., May 1, 2009; 192(5): 1324 - 1329.
[Abstract] [Full Text] [PDF]

Home page
 Am. J. Roentgenol.Home page
F. Girvin and J. P. Ko
Pulmonary Nodules: Detection, Assessment, and CAD
Am. J. Roentgenol., October 1, 2008; 191(4): 1057 - 1069.
[Abstract] [Full Text] [PDF]

Home page
 NEJMHome page
J. J. Fenton, S. H. Taplin, P. A. Carney, L. Abraham, E. A. Sickles, C. D'Orsi, E. A. Berns, G. Cutter, R. E. Hendrick, W. E. Barlow, et al.
Influence of Computer-Aided Detection on Performance of Screening Mammography
N. Engl. J. Med., April 5, 2007; 356(14): 1399 - 1409.
[Abstract] [Full Text] [PDF]

This Article
Right arrow Abstract Freely available
Right arrow Figures Only
Right arrow Full Text (PDF)
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via HighWire
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Das, M.
Right arrow Articles by Wildberger, J. E.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Das, M.
Right arrow Articles by Wildberger, J. E.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
RADIOLOGY RADIOGRAPHICS RSNA JOURNALS ONLINE