DOI: 10.1148/radiol.2441061192
(Radiology 2007;244:316-317.)
© RSNA, 2007
Polyp Detection at CT Colonography: Inadequate Primary 3D Endoluminal Reference Standard Precludes Meaningful Comparison
Perry J. Pickhardt, MD
Department of Radiology, University of Wisconsin Medical School, E3/311 Clinical Science Center, 600 Highland Avenue, Madison, WI 53792-3252
e-mail: pj.pickhardt{at}hosp.wisc.edu
Editor:
I read with interest the article by Dr Taylor and colleagues (1) in the June 2006 issue of Radiology. In this retrospective study of 14 patients, primary three-dimensional (3D) endoluminal analysis was compared with primary two-dimensional (2D) analysis supplemented by computer-assisted reader (CAR) software. The authors found that the 2D CAR approach was as sensitive as 3D endoluminal analysis for computed tomographic (CT) polyp detection.
I have no real issue with the performance findings reported for the primary 2D CAR method in this study. My main concern surrounds the use of Vitrea 2 (Vital Images, Plymouth, Minn) for primary 3D endoluminal evaluation, which results in an inadequate 3D reference standard that negates any useful comparison with the 2D CAR approach. Given the results obtained with CT colonography systems capable of effective primary 3D evaluation (2), I am curious as to why the authors chose instead to use a primary 2D CT colonography system widely regarded to have substandard primary 3D capabilities (3). The predictably poor results seen with primary 3D evaluation by using Vitrea 2, including a sensitivity for large polyps (
10 mm) of only 67% (33% by one of the readers!) should have been expected.
The simple truth is that, although standard 2D multiplanar reformation capabilities between the various commercial CT colonography systems are all fairly comparable and interchangeable, the 3D capabilities are extremely variable and highly system-specific. As such, the findings of the "primary 3D endoluminal analysis" in this study have little to do with current state-of-the-art 3D CT colonography and therefore the conclusions are not justified. In practice, it should be noted that only the primary 3D approach has been validated for screening detection of polyps in a low-prevalence setting (2), whereas primary 2D approaches, to date, have largely failed in this regard (4–6). In our experience, polyp detection by using an effective 3D tool is not only less fatiguing, since it demands less intense concentration compared with 2D polyp detection, but it also requires less training for general radiologists to consistently attain "expert" levels of performance (2,7).
The author serves on the medical advisory boards for Viatronix (Stony Brook, NY) and Medicsight (London, England).
 |
References
|
|---|
- Taylor SA, Halligan S, Slater A, et al. Polyp detection with CT colonography: primary 3D endoluminal analysis versus primary 2D transverse analysis with computer-assisted reader software. Radiology 2006;239:759–767.[Abstract/Free Full Text]
- Pickhardt PJ, Choi JR, Hwang I, et al. Computed tomographic virtual colonoscopy to screen for colorectal neoplasia in asymptomatic adults. N Engl J Med 2003;349:2191–2200.[Abstract/Free Full Text]
- Pickhardt PJ. Three-dimensional endoluminal CT colonography (virtual colonoscopy): comparison of three commercially available systems. AJR Am J Roentgenol 2003;181:1599–1606.[Abstract/Free Full Text]
- Johnson CD, Harmsen WS, Wilson LA, et al. Prospective blinded evaluation of computed tomographic colonography for screen detection of colorectal polyps. Gastroenterology 2003;125:311–319.[CrossRef][Medline]
- Cotton PB, Durkalski VL, Pineau BC, et al. Computed tomographic colonography (virtual colonoscopy): a multicenter comparison with standard colonoscopy for detection of colorectal neoplasia. JAMA 2004;291:1713–1719.[Abstract/Free Full Text]
- Rockey DC, Paulsen EK, Niedzwiecki D, et al. Analysis of air contrast barium enema, computed tomographic colonography, and colonoscopy: prospective comparison. Lancet 2005;365:305–311.[Medline]
- Cash BD, Kim C, Cullen P, et al. Accuracy of computed tomographic colonography for colorectal cancer screening in asymptomatic individuals [abstr]. In: Digestive Disease Week 2006 annual meeting program. Los Angeles, Calif: Digestive Disease Week, 2006; 473.
- Summers RM, Yao J, Pickhardt PJ, et al. Computed tomographic virtual colonoscopy computer-aided polyp detection in a screening population. Gastroenterology 2005;129:1832–1844.[CrossRef][Medline]
- Pickhardt PJ, Taylor AJ, Kim DH, Reichelderfer M, Gopal DV, Pfau PR. Screening for colorectal neoplasia with CT colonography: initial experience from the 1st year of coverage by third-party payers. Radiology 2006;241:417–425.[Abstract/Free Full Text]
Response
Stuart A. Taylor, BSc, MD, MRCP, FRCR and
Steve Halligan, MD, FRCP, FRCR
Department of Specialist X-Ray, Level 2 Podium, University College Hospital, 235 Euston Road, London NW1 2BU, England
e-mail: csytaylor{at}yahoo.co.uk
Dr Pickhardt objects to the software used for the 3D arm of our comparison (1), claiming that its performance is so poor as to negate meaningful analysis. The clear implication from his cited references is that only a Viatronix system would suffice. However, this assertion is speculative and is predicated by a comparative study (for which Dr Pickhardt is sole author) that did not investigate polyp detection (2). Rather it was focused on subjective assessments of the conspicuity of eight polyps. Conversely, our study was powered at 80% to detect proportional differences in detection rates between platforms, and one of our readers achieved 100% sensitivity for 3D detection of 10-mm polyps, which suggests that the Vitrea 2 system was adequate for depiction. We take no issue with the assertion that primary 3D analysis is more sensitive than primary 2D—this assumption underpinned our prestudy power calculation.
One of us (S.H.) uses the Viatronix system daily, and while we would readily acknowledge its attractions, we disagree that other systems are "inadequate." Vitrea 2 has regulatory approval, is commercially available, and is widely used both by respected researchers and in clinical practice. The iteration we used incorporated a semiautomated full 3D endoluminal flythrough capability that is much improved since Dr Pickhardt's evaluation (2), so that comparison is invalid even if we ignore other confounders, notably reader experience. Well-performed comparative studies of other platforms have revealed no differences attributable to software, whereas experience has a substantial effect on performance (3). Furthermore, findings of studies using Viatronix reveal that it is also not immune to these factors (4). Additional evidence that Vitrea 2 is far from "inadequate" is provided by its adoption in the ACRIN (American College of Radiology Imaging Network) protocol 6664 study, where it is used both for primary 2D and 3D analysis by radiologists with exemplary "track records" in CT colonography research (5). It seems unlikely to us that these investigators would use a demonstrably inferior platform. We can find no data showing that any specific 3D visualization software significantly influences detection, with good results obtained from several vendors, including Vitrea (6,7).
We believe that colonography will have clinical utility only if good results are proved generalizable. For example, investigators in the UK SIGGAR (Special Interest Group in Gastrointestinal and Abdominal Radiology) study use multiple platforms, including Viatronix, according to personal preference (reflecting normal clinical practice) because generalizability is a prime objective for the Department of Health (8). Reliance on a specific vendor militates against this.
|
References
|
|---|
- Taylor SA, Halligan S, Slater A, et al. Polyp detection with CT colonography: primary 3D endoluminal analysis versus primary 2D transverse analysis with computer-assisted reader software. Radiology 2006;239:759–767.[Abstract/Free Full Text]
- Pickhardt PJ. Three dimensional endoluminal CT colonography (virtual colonoscopy): comparison of three commercially available systems. AJR Am J Roentgenol 2003;181:1599–1606.[Abstract/Free Full Text]
- Johnson CD, Toledano AY, Herman BA, et al. Computerized tomographic colonography: performance evaluation in a retrospective multicenter setting. Gastroenterology 2003;125:688–695.[CrossRef][Medline]
- Iannaccone R, Catalano C, Marin D, et al. Primary two-dimensional (2D) versus primary three-dimensional (3D) reading: performance of abdominal radiologists with limited training in CT colonography [abstr]. In: Radiological Society of North America Scientific Assembly and Annual Meeting Program. Oak Brook, Ill: Radiological Society of North America, 2005;186.
- The national CT colonography trial: ACRIN protocol 6664. American College of Radiology Imaging Network Web site. http://www.acrin.org/6664_protocol.html. Accessed July 28, 2006.
- Laghi A, Iannaccone R, Carbone I, et al. Detection of colorectal lesions with virtual computed tomographic colonography. Am J Surg 2002;183:124–131.[CrossRef][Medline]
- Yasumoto T, Murakami T, Yamamoto H, et al. Assessment of two 3D MDCT colonography protocols for observation of colorectal polyps. AJR Am J Roentgenol 2006;186:85–89.[Abstract/Free Full Text]
- CT colonography, colonoscopy or barium enema for diagnosis of colorectal cancer in older symptomatic patients: the SIGGAR trial. NHS Health Technology Assessment Programme Web site. http://www.hta.nhsweb.nhs.uk//project/1366.asp. Accessed July 28, 2006.
TOP
References
References 
