HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS		QUICK SEARCH:   [advanced] Author: Keyword(s): Year:  Vol:  Page:

This Article Abstract Figures Only Full Text (PDF) All Versions of this Article: 2372041734v1 237/2/605    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Hirai, T. Articles by Doi, K. Search for Related Content PubMed PubMed Citation Articles by Hirai, T. Articles by Doi, K.

Intracranial Aneurysms at MR Angiography: Effect of Computer-aided Diagnosis on Radiologists' Detection Performance¹

¹ From the Department of Diagnostic Radiology, Graduate School of Medical Sciences, Kumamoto University 1-1-1 Honjo, Kumamoto 860-8556, Japan (T.H., S.K., M.K., M.Y., Y.Y.); Department of Radiology, University of Occupational and Environmental Health, School of Medicine, Kitakyushu, Japan (Y.K.); Department of Health Sciences, Kyushu University, Fukuoka, Japan (H.A.); and Department of Radiology, University of Chicago, Chicago, Ill (K.D.). Received October 9, 2004; revision requested December 20; revision received January 6, 2005; accepted January 25. Address correspondence to T.H. (e-mail: toshinor{at}beige.ocn.ne.jp).

	ABSTRACT

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 
PURPOSE: To retrospectively evaluate the effect of computer-aided detection (CAD) on radiologists' performance in detection of intracranial aneurysms with magnetic resonance (MR) angiography.

MATERIALS AND METHODS: The institutional review board approved this study and did not require patient informed consent. Fifty maximum intensity projection MR angiograms in 50 patients were used for observer performance study. The group included 22 patients (age range, 43–86 years; mean, 60.2 years; 6 men and 16 women) with intracranial aneurysms and 28 patients (age range, 32–80 years; mean, 58.8 years; 10 men and 18 women) without aneurysms. The MR angiograms were obtained with three-dimensional time-of-flight 1.5-T MR imaging. Fifteen radiologists, including eight neuroradiologists and seven general radiologists, participated in the observer performance test. They interpreted the angiograms first without and then with the aid of the computer output by using an automated computerized scheme. The observers' performance without and with the computer output was evaluated with receiver operating characteristic analysis.

RESULTS: For all 15 observers, average area under the receiver operating characteristic curve (A_z) value for detection of aneurysms was increased significantly from 0.931 to 0.983 (P = .001) when they used the computer output. A_z values for general radiologists and neuroradiologists increased from 0.894 to 0.983 (P = .022) and from 0.963 to 0.984 (P = .014), respectively. Improvement in the performance of general radiologists in terms of the A_z value was much greater than that of neuroradiologists. Performance of general radiologists with CAD (A_z = 0.983) slightly exceeded that of neuroradiologists without CAD (A_z = 0.963) (P = .048).

CONCLUSION: CAD improved neuroradiologists' and general radiologists' performance for detection of intracranial aneurysms with MR angiography; improvement was greater for general radiologists than it was for neuroradiologists.

© RSNA, 2005

	INTRODUCTION

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 
The natural history of unruptured intracranial aneurysms is not completely understood. Findings at autopsy and angiographic studies showed that between 3.6% and 6% of the general population harbor an unruptured intracranial aneurysm (1). The analytic results of the International Study of Unruptured Intracranial Aneurysms indicated that the risk of rupture is related to the site and size of an aneurysm and to a history of subarachnoid hemorrhage (2). With regard to treatment for unruptured intracranial aneurysms, benefits of surgical treatment or of endovascular intervention have been suggested in some studies (3,4). Recently, the investigators of the International Study of Unruptured Intracranial Aneurysms proposed that the specific risks related to the natural history of intracranial aneurysms should be considered in the treatment of each patient (5).

Magnetic resonance (MR) angiography is a widely available and noninvasive imaging modality that has high sensitivity for the detection of intracranial aneurysms. The accuracy of MR angiography in the detection of intracranial aneurysms is about 90% (6). The sensitivity of MR angiography was greater for detection of aneurysms larger than 3 mm than it was for detection of aneurysms that were 3 mm or smaller, that is, 94% versus 38% (6).

During the past 10 years, screening of unruptured intracranial aneurysms with MR angiography has gained attention, for example, for examining asymptomatic patients who are at risk of having an aneurysm because of a strong family history of aneurysmal subarachnoid hemorrhage or because of autosomal dominant polycystic kidney disease (7–9). Although screening of asymptomatic individuals without such risk factors is still controversial, MR angiography is used for screening of unruptured aneurysms in the general population in Japan (10).

To date, MR angiography is widely accepted in screening for intracranial aneurysms, and the demand is increasing. In screening for unruptured intracranial aneurysms with MR angiography, however, radiologists need to analyze large amounts of data that include multiple image projections per examination and source images obtained with MR angiography. It is time consuming and sometimes difficult for radiologists to find small aneurysms, or it may not be easy to detect medium aneurysms on maximum intensity projection (MIP) images, because of overlap with adjacent vessels and because of unusual locations (11,12). Thus, there is always the risk of missing an aneurysm. There are some methods that help avoid missing an intracranial aneurysm, such as the use of independent readings by two or more radiologists (11). Independent readings by two or more radiologists, however, increase the workload of radiologists in a routine clinical practice.

In regard to other organs, the concept of computer-aided detection (CAD) for screening of breast cancer with mammography and for screening of lung cancer with chest radiography and/or computed tomography (CT) has been applied, and the usefulness of CAD systems for the detection of cancers has been reported (13–17). To our knowledge, however, a CAD scheme has not been applied to the detection of intracranial aneurysms with MR angiography. We have been developing a CAD system for detection of intracranial aneurysms in screening with MR angiography (18). The purpose of our study, therefore, was to retrospectively evaluate the effect of CAD on radiologists' performance in the detection of intracranial aneurysms with MR angiography.

	MATERIALS AND METHODS

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 
Patient Selection
One neuroradiologist (T.H.) retrospectively selected 60 consecutive patients with unruptured intracranial aneurysms who underwent MR angiography at the Kumamoto University Hospital, Kumamoto, Japan, to evaluate possible intracranial vascular lesions between October 1999 and August 2003. The patients' data were derived from the MR imaging database of our institution. From the same database, 60 age-matched patients without intracranial aneurysms were also recruited by the same neuroradiologist. MR angiograms obtained in the 120 patients were reviewed in consensus by two experienced neuroradiologists (T.H. and Y.K., with 16 and 24 years of experience, respectively) who did not participate in the observer performance study. Our institutional review board approved this study and did not require patient informed consent.

MR Imaging
All MR imaging examinations were performed with a 1.5-T imager (Magnetom Vision; Siemens Medical Systems, Erlangen, Germany) by using a circular polarized head coil. Three-dimensional time-of-flight MR angiograms were obtained with the following parameters: 32/6 (repetition time msec/echo time msec), one signal acquired, 20° flip angle, 64-mm slab thickness, 64 partitions, 20-cm field of view, and 256 x 512 matrix. The acquisition data were converted into a 512 x 512 matrix, with a pixel size of 0.39 x 0.39 mm, by using an interpolation technique. Tilted optimized nonsaturating excitation and magnetization transfer pulses were applied for enhancement of the signals of blood flow. The slabs were placed to include the structures from the intracranial vertebral artery to the A2 branch of the anterior cerebral artery. Saturation pulses were applied cephalad so that venous blood signals were eliminated.

Selection for Observer Performance Study
Twenty-two of the 60 patients with unruptured intracranial aneurysms were further selected for an observer performance study by the same two neuroradiologists who performed the patient selection. The inclusion criteria for the patient selection were as follows: (a) patients with only one aneurysm were chosen, (b) location and size of the aneurysm were similar to the distributions in a clinical environment, and (c) high-quality MR angiograms were available for interpretation. There were six men and 16 women, with an age range of 43–86 years (mean age, 60.2 years). The locations and sizes of the aneurysms are listed in Table 1. The sizes of the aneurysm ranged from 3 to 26 mm in maximum diameter (mean, 7.1 mm). According to the maximum diameter, the aneurysms were classified into three groups: eight small (<5 mm), 11 medium (5–12 mm), and three large (>12 mm). The aneurysms were located at the internal carotid artery (n = 10), middle cerebral artery (n = 4), anterior cerebral artery (n = 3), basilar artery (n = 3), posterior cerebral artery (n = 1), and vertebral artery (n = 1). The 22 lesions included 20 saccular aneurysms and two fusiform aneurysms. The diagnosis was proved at CT angiography (n = 12) and conventional digital subtraction angiography (n = 3); the remaining seven aneurysms were confirmed with consensus reading of MR angiograms by the same two experienced neuroradiologists mentioned before. Source images obtained with MR angiography were also used for the confirmation. Eight of the 22 patients had old brain infarctions; intracranial steno-occlusive disease was present in two patients. An azygous anterior cerebral artery was seen in one patient with a distal anterior cerebral artery aneurysm. Meningioma and pituitary microadenoma were seen on conventional MR images in one case each. These additional lesions did not affect the interpretation of MR angiograms.

Computerized Scheme for Automated Detection of Intracranial Aneurysms
Details of the CAD scheme used in this study are shown elsewhere (18). First, the isotropic images from three-dimensional MR angiography were processed by using three selective filters for enhancement of aneurysms, vessels, and vessel walls. The initial candidates were identified by using a multiple gray-level threshold technique on the three-dimensional dot-enhanced images within the search area, which was determined with dilatation of major vessels, because most aneurysms appear in specific vessels. Candidate regions were segmented by using a region-growing technique with monitoring of some image features. In the next step, all candidates were classified into four types according to the size of the aneurysms and the local structures involved. In each group, a number of nonaneurysms were removed by using rules that were based on localized image features related to gray levels and morphologic characteristics. Finally, linear discriminant analysis was employed for further removal of some false-positive findings. Our scheme achieved a sensitivity of 100% (36 of 36), with 0.55 false-positive finding per patient in a consistency test (18).

Observer Performance Study
A total of 15 observers, including eight board-certified radiologists who specialized in neuroradiology (neuroradiologists) (M.K., M.Y.) and seven board-certified radiologists who did not specialize in neuroradiology (general radiologists), took part in the observer performance study. All observers were blinded to the results of the diagnosis obtained from the consensus of the two experienced neuroradiologists (T.H., Y.K.). The neuroradiologists had 8–17 years of experience (mean, 13.0 years), and the general radiologists had 7–20 years of experience (mean, 13.3 years). The sequential test method (19) was used in the observer performance study. Each observer read the MR angiograms displayed on a monitor first without the computer output and rated his or her confidence level in determining the presence or absence of an aneurysm. Next, the computer output, marked by circles that indicated potential aneurysms, was superimposed on the MR angiograms. The observer then viewed the image with the computer output and rated it again. An interface program was created for an image display without and with the computer output for the observer performance test. The 15 observers viewed the MIP MR angiograms displayed on a gray-scale monitor (Precision 650; Dell, Round Rock, Tex) with a spatial resolution of 1600 x 1200. The monitor screen was divided into two areas: one area in which MR angiograms were displayed in z-axis, x-axis, and y-axis rotations and another area in which observers indicated their confidence levels in the observer performance study. On the monitor, observers could see 19 MIP MR angiograms ranging from –90° to +90° (10° intervals) in the z-axis rotation and five MIP MR angiograms ranging from –20° to +20° (10° intervals) in the x-axis and y-axis rotations. The observers were allowed to select the direction of the MR angiograms and the magnification of the images on the monitor. They were also permitted to use the cine mode for displaying the images.

Images in all of the 50 patients (22 with aneurysms and 28 without aneurysms) selected for the observer performance study were presented in the same randomized order to the observers. The observers were provided with the following information before the observer performance test: (a) the sequential test method that was used, (b) there was only one aneurysm in each patient, and (c) the type of aneurysm was either saccular or fusiform. The observers were blinded to the number of patients with aneurysms and the performance level of the CAD scheme. There was no limit on the reading time.

Before the observer performance test, each observer underwent a training session with four training cases to become familiar with the computer output and the test procedure. The four training cases were not included with the images from the 50 selected patients used in the observer performance study. Each observer used a continuous rating scale of a line-marking method to rate his or her confidence level on the monitor. At the left end of the line, a confidence level that an aneurysm was definitely absent was indicated, whereas at the right end, a confidence level that an aneurysm was definitely present was indicated. Intermediate levels of confidence were indicated by the different positions on the line between the two ends, and positions close to the right and left ends indicated, respectively, greater and lesser degrees of confidence in regard to the presence of an aneurysm. The distance between the left end and the marked point was automatically determined in the computer and was converted to a confidence level that could range from 0 to 100.

To investigate the effect of CAD further, we also determined the difference between the confidence levels without and with the computer output. We assumed that a clinically relevant change in confidence levels occurred only when the difference was greater than 20 units on a 0–100 confidence rating scale. A shift of more than 20 units in the direction toward a correct diagnosis implied that the use of the CAD was beneficial. Similarly, a shift of more than 20 units in the direction toward an incorrect diagnosis implied that the use of CAD was detrimental. The number of cases with beneficial effects and the number of those with detrimental effects, according to each observer, were assessed.

Statistical Analysis
Observer performance was evaluated by using receiver operating characteristic analysis with the LABROC program, according to Metz et al (20). The receiver operating characteristic curves for each observer without and with the computer output indicated the true-positive fraction to the false-positive fraction at each confidence level. The area under the receiver operating characteristic curve (A_z) was used for comparing the observers' performance in the detection of intracranial aneurysms.

To investigate the effect of CAD for all observers, the significance of the difference between the A_z values obtained without and with CAD was evaluated by using the Wilcoxon matched-pairs signed rank test. To investigate the effect of CAD for each radiologist group, the significance of the difference in A_z values between neuroradiologists and general radiologists was evaluated with a Mann-Whitney U test. In addition, the difference between the average numbers of cases affected beneficially and detrimentally because of CAD was analyzed by using the Wilcoxon matched-pairs signed rank test. In all the analyses, a P value of less than .05 was considered to indicate a significant difference.

	RESULTS

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 
Observer Performance
For all observers, the mean A_z values (Fig 1, Table 2) obtained without and with CAD were 0.931 ± 0.04 (standard deviation) and 0.983 ± 0.01, respectively, which indicated a significant difference (P = .001). In the neuroradiologist group (Fig 2), the mean A_z values obtained without and with CAD were 0.963 ± 0.02 and 0.984 ± 0.01, respectively, which indicated a significant difference (P = .014). In the general radiologist group (Fig 3), the mean A_z values obtained without and with CAD were 0.894 ± 0.02 and 0.983 ± 0.01, respectively, which indicated a significant difference (P = .022). There was a significant difference in the mean A_z values obtained without CAD between the two groups of radiologists (P = .001). There was no significant difference, however, in the mean A_z values obtained with CAD between the two groups of radiologists (P = .522). The performance of general radiologists (A_z = 0.983) with CAD slightly exceeded that of neuroradiologists (A_z = 0.963) without CAD. There was a significant difference in the mean A_z values between the general radiologist group with CAD and the neuroradiologist group without CAD (P = .048).

View larger version (34K): [in this window] [in a new window] [Download PPT slide]   Figure 1. Graph shows average receiver operating characteristic curves for all observers in the detection of intracranial aneurysms without and with CAD output. The average Az value in the detection of aneurysms was significantly improved from 0.931 to 0.983 when observers used the computer output (P = .001).

View larger version (34K): [in this window] [in a new window] [Download PPT slide]   Figure 2. Graph shows average receiver operating characteristic curves for eight neuroradiologists in the detection of intracranial aneurysms. The average Az value in detection of aneurysms was improved significantly from 0.963 to 0.984 when they used the computer output (P = .014).

View larger version (36K): [in this window] [in a new window] [Download PPT slide]   Figure 3. Graph shows average receiver operating characteristic curves for seven general radiologists in the detection of intracranial aneurysms. The average Az value in detection of aneurysms was improved significantly from 0.894 to 0.983 when they used the computer output (P = .022).

View larger version (26K): [in this window] [in a new window] [Download PPT slide]   Figure 4. Graph shows number of cases (>20%) affected by the CAD output in confidence level with regard to patients with aneurysms. There were no cases affected detrimentally. The numbers 1–8 on x-axis represent the eight neuroradiologists and 9–15 represent the seven general radiologists.

View larger version (27K): [in this window] [in a new window] [Download PPT slide]   Figure 5. Graph shows number of cases (>20%) affected by CAD output in confidence level with regard to patients without aneurysms. The numbers 1–8 on x-axis represent the eight neuroradiologists, and 9–15 represent the seven general radiologists.

	DISCUSSION

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

The knowledge required for detection of intracranial aneurysms with MR angiography includes the anatomy of the intracranial vessels, common locations of intracranial aneurysms, etiology and classification of the aneurysms, diagnostic accuracy of MR angiography for intracranial aneurysms, and artifacts and pitfalls for MR angiography. All of the general radiologists who took part in the observer study had limited experience, of 1 year or less, in neuroradiology, and were considered to be unfamiliar with findings of intracranial aneurysms. This factor presumably accounts for the finding that general radiologists benefited much more from the use of CAD. In other words, the radiologist's performance in detection of intracranial aneurysms probably depends on his or her experience and knowledge as a neuroradiologist. The fact that the effect of using CAD for neuroradiologists was also beneficial, however, may indicate that the use of CAD can improve radiologists' detection performance irrespective of their knowledge and experience.

In our study, 10% of cases in patients with aneurysms were affected beneficially by the computer output. This result was caused by an increase in the observer's confidence in diagnosing the presence of aneurysms in cases in patients with aneurysms that were detected with the use of CAD. It is sometimes difficult for radiologists to find small aneurysms, or it may not be easy to detect medium aneurysms on MIP images because of overlap with adjacent vessels and because of unusual locations (11,12). Our results without use of CAD showed a similar tendency for the false-negative findings with MIP MR angiograms. We also found that large aneurysms might be missed by general radiologists. Because general radiologists are not familiar with the unusual signal intensity of large aneurysms on MR angiograms, they might have missed these aneurysms.

Our observer performance test results indicated that the use of CAD was clearly beneficial, even in cases in patients without aneurysms. About 8% of cases in patients without aneurysms were affected beneficially because of the increase in the observer's confidence in diagnosing the absence of aneurysms in cases in patients without aneurysms. On the other hand, only 1% of cases in patients without aneurysms were affected detrimentally because of an increase in the confidence level in favor of the presence of an aneurysm. The detrimental effects were significantly smaller than were the beneficial effects. Unruptured intracranial aneurysms are being diagnosed with greater frequency as imaging technologies improve. Since the workload of radiologists will increase in the near future, the use of CAD may be necessary to increase true-positive findings and decrease false-positive findings as interpreted by radiologists.

Our study had limitations. First, source images obtained with MR angiography were not included in this observer performance study. It is likely that source images obtained with MR angiography would have been helpful in the detection of aneurysms. Reading source images obtained with MR angiography, however, may be time consuming for radiologists, especially for general radiologists. Because the fatigue in reading many images for observer performance testing may affect the results, we used only MIP images obtained with MR angiography in this study. We provided multiple views of MIP images to observers. Therefore, we believe that this observer performance test was adequate for assessment of observer performance in the detection of aneurysms. Second, this study might have had a bias in patient selection. We selected 22 patients with aneurysms and 28 control patients. The aneurysms were of various sizes, but most were small or medium, and nearly half were small. In addition, some aneurysms were located at unusual sites. Patients with steno-occlusive disease were also included to simulate patients in our routine practice. Thus, we believe that the patients we selected reflected patients seen in routine clinical practice. Third, there was a difference between the observer performance study and a clinical environment. We provided observers with the information in regard to the number of aneurysms in patients with aneurysms. Provision of such information may have affected observer performance. Although readers considered that multiple aneurysms may have existed on a given image, they indicated one aneurysm with confidence, because they knew that only one aneurysm was present in a patient with an aneurysm in the observer performance study. Thus, potential false-positive reports might have been artificially excluded.

In conclusion, the use of our CAD system helped to improve both neuroradiologists' and general radiologists' performance in the detection of intracranial aneurysms with MR angiography, and this improvement was more marked for general radiologists than it was for neuroradiologists. Further investigations about the usefulness of the CAD scheme, as applied to various MR angiograms with various MR imaging units, are needed for application in routine clinical practice.

	ACKNOWLEDGMENTS

 
We are grateful to the following personnel for their help: Shoji Morishita, MD, Yoshiko Hayashida, MD, Ryuji Murakami, MD, Ichiro Ikushima, MD, Yoshinori Shigematsu, MD, Takeshi Sugahara, MD, Tomoko Okuda, MD, Shingo Kakeda, MD, Takatoshi Aoki, MD, Koichi Kawanaka, MD, Yoshiharu Nakayama, MD, Masanori Imuta, MD, and Ichiro Ogata, MD, for their participation in this observer study; Junji Shiraishi, PhD, for receiver operating characteristic analysis; Roger Engelmann, MS, for development of the software used in this observer study; Ryuji Ikeda for data management; and Elisabeth Lanzl for English wording.

	FOOTNOTES

 

Abbreviations: A_z = area under receiver operating characteristic curve • CAD = computer-aided detection • MIP = maximum intensity projection

Authors stated no financial relationship to disclose.

Author contributions: Guarantors of integrity of entire study, T.H., Y.K., H.A., S.K., K.D.; study concepts/study design or data acquisition or data analysis/interpretation, all authors; manuscript drafting or manuscript revision for important intellectual content, all authors; approval of final version of submitted manuscript, all authors; literature research, T.H.; clinical studies, H.A., S.K., M.K., M.Y.; statistical analysis, T.H., S.K.; and manuscript editing, T.H., Y.K., H.A., S.K., Y.Y., K.D.

	References

TOP ABSTRACT INTRODUCTION MATERIALS AND METHODS RESULTS DISCUSSION References

 

1. Wardlaw JM, White PM. The detection and management of unruptured intracranial aneurysms. Brain 2000;123:205–221.[Abstract/Free Full Text]
2. International Study of Unruptured Intracranial Aneurysms Investigators. Unruptured intracranial aneurysms: risk of rupture and risks of surgical intervention. N Engl J Med 1998; 339:1725–1733.[Abstract/Free Full Text]
3. Chang HS, Kirino T. Quantification of operative benefit for unruptured cerebral aneurysms: a theoretical approach. J Neurosurg 1995;83:413–420.[Medline]
4. Leblanc R, Worsley KJ. Surgery of unruptured, asymptomatic aneurysms: a decision analysis. Can J Neurol Sci 1995;22:30–35.[Medline]
5. International Study of Unruptured Intracranial Aneurysms Investigators. Unruptured intracranial aneurysms: natural history, clinical outcome, and risks of surgical and endovascular treatment. Lancet 2003; 362:103–110.[CrossRef][Medline]
6. White PM, Wardlaw JM, Easton V. Can noninvasive imaging accurately depict intracranial aneurysms? a systematic review. Radiology 2000;217:361–370.[Abstract/Free Full Text]
7. Ruggieri PM, Poulos N, Masaryk TJ, et al. Occult intracranial aneurysms in polycystic kidney disease: screening with MR angiography. Radiology 1994;191:33–39.[Abstract/Free Full Text]
8. Ronkainen A, Puranen MI, Hernesniemi JA, et al. Intracranial aneurysms: MR angiographic screening in 400 asymptomatic individuals with increased familial risk. Radiology 1995;195:35–40.[Abstract/Free Full Text]
9. Raaymakers TW, Buys PC, Verbeeten B Jr, et al. MR angiography as a screening tool for intracranial aneurysms: feasibility, test characteristics, and interobserver agreement. AJR Am J Roentgenol 1999;173:1469–1475.[Abstract]
10. Baba Y, Takahashi M, Korogi Y. Cost-effectiveness of screening unruptured cerebral aneurysms in Japan. Eur Radiol 2000; 10:S362–S365.
11. Korogi Y, Takahashi M, Mabuchi N, et al. Intracranial aneurysms: diagnostic accuracy of three-dimensional, Fourier transform, time-of-flight MR angiography. Radiology 1994;193:181–186.[Abstract/Free Full Text]
12. Korogi Y, Takahashi M, Mabuchi N, et al. Intracranial aneurysms: diagnostic accuracy of MR angiography with evaluation of maximum intensity projection and source images. Radiology 1996;199:199–207.[Abstract/Free Full Text]
13. Freer TW, Ulissey MJ. Screening mammography with computer-aided detection: prospective study of 12,860 patients in a community breast center. Radiology 2001;220:781–786.[Abstract/Free Full Text]
14. Destounis SV, DiNitto P, Logan-Young W, Bonaccio E, Zuley ML, Willison KM. Can computer-aided detection with double reading of screening mammograms help decrease the false-negative rate? initial experience. Radiology 2004;232:578–584.[Abstract/Free Full Text]
15. MacMahon H, Engelmann R, Behlen FM, et al. Computer-aided diagnosis of pulmonary nodules: results of a large-scale observer test. Radiology 1999;213:723–726.[Abstract/Free Full Text]
16. Shiraishi J, Abe H, Engelmann R, Aoyama M, MacMahon H, Doi K. Computer-aided diagnosis to distinguish benign from malignant solitary pulmonary nodules on radiographs: ROC analysis of radiologists' performance—initial experience. Radiology 2003;227:469–474.[Abstract/Free Full Text]
17. Awai K, Murao K, Ozawa A, et al. Pulmonary nodules at chest CT: effect of computer-aided diagnosis on radiologists' detection performance. Radiology 2004;230:347–352.[Abstract/Free Full Text]
18. Arimura H, Li Q, Korogi Y, et al. Automated computerized scheme for detection of unruptured intracranial aneurysms in three-dimensional MRA. Acad Radiol 2004;11:1093–1104.[CrossRef][Medline]
19. Kobayashi T, Xu XW, MacMahon H, Metz CE, Doi K. Effect of a computer-aided diagnosis scheme on radiologists' performance in detection of lung nodules on radiographs. Radiology 1996;199:843–848.[Abstract/Free Full Text]
20. Metz CE, Herman BA, Shen JH. Maximum-likelihood estimation of receiver operating (ROC) characteristic curves from continuously-distributed data. Stat Med 1998;17:1033–1053.[CrossRef][Medline]

This article has been cited by other articles:

				S. Kasai, F. Li, J. Shiraishi, and K. Doi Usefulness of Computer-Aided Diagnosis Schemes for Vertebral Fractures and Lung Nodules on Chest Radiographs Am. J. Roentgenol., July 1, 2008; 191(1): 260 - 265. [Abstract] [Full Text] [PDF]

				S. Kakeda, Y. Korogi, H. Arimura, T. Hirai, S. Katsuragawa, T. Aoki, and K. Doi Diagnostic Accuracy and Reading Time to Detect Intracranial Aneurysms on MR Angiography Using a Computer-Aided Diagnosis System Am. J. Roentgenol., February 1, 2008; 190(2): 459 - 465. [Abstract] [Full Text] [PDF]

				R. YOKOYAMA, X. ZHANG, Y. UCHIYAMA, H. FUJITA, T. HARA, X. ZHOU, M. KANEMATSU, T. ASANO, H. KONDO, S. GOSHIMA, et al. Development of an Automated Method for the Detection of Chronic Lacunar Infarct Regions in Brain MR Images IEICE Trans D: Information, June 1, 2007; E90-D(6): 943 - 954. [Abstract] [PDF]

This Article Abstract Figures Only Full Text (PDF) All Versions of this Article: 2372041734v1 237/2/605    most recent Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Hirai, T. Articles by Doi, K. Search for Related Content PubMed PubMed Citation Articles by Hirai, T. Articles by Doi, K.